Hello again! I was originally planning to cover implementing a search system, adding more feeds, and tidying up the front end in this instalment. However, I felt it was time for a change of pace, so instead, we're going to look at:

• Checking code coverage and getting it to 100%
• Using continuous integration
• Deploying to Heroku

Don't worry, the original lesson isn't going anywhere. We'll still be implementing all of that later on, but today is the day we get your Django blog up and running on the web. That way, you can get a better idea of how Django works in the wild, and you have something concrete to show for your efforts.

Continuous integration

If you're not familiar with continuous integration, it's basically a process that carries out some tasks automatically when you push a new commit to the repository. These tasks may include running unit tests, linting your code, and checking it to see what percentage of the code base is covered by the tests (after all, if your tests don't actually cover every scenario, then there's more of a chance that something might slip through the net. It's also possible to implement hooks to automatically deploy your application only if the tests pass.

Typically, you will have a continuous integration server running somewhere that regularly polls your Git repository for changes, and when it finds a new commit, will check it out and run the tests (or whatever other task you configure it to). One of the most popular continuous integration servers around is Jenkins - I use it at work and can highly recommend it. However, we aren't going to cover using Jenkins here because setting it up is quite a big deal and it really is best kept on a server of its own (although feel free to use it if you prefer). Instead, we're going to use Travis CI, which integrates nicely with GitHub and is free for open source projects. If you don't mind your code being publicly available on GitHub, then Travis is a really great way to dip your toe into continuous integration.

NB: You don't have to use continuous integration at all if you don't want to - this isn't a big project so you don't really need it. If you don't want to put your code on GitHub, then feel free to just follow along and not bother pushing your code to GitHub and configuring Travis.

Code coverage

As mentioned above, code coverage is a measure of the percentage of the code that is covered by tests. While not infallible, it's a fairly good guide to how comprehensive your tests are. If you have 100% code coverage, you can be fairly confident that your tests are comprehensive enough to catch most errors, so it's a good rule of thumb to aim for 100% test coverage on a project.

So how do we check our test coverage? The coverage Python module is the most common tool for this. There's also a handy Django module called django-jenkins, which is designed to work with Jenkins, but can be used with any continuous integration server, that can not only run your tests, but also check code coverage at the same time. So, make sure your virtualenv is up and running:

$ source venv/bin/activate

Then, run the following command:

$ pip install coverage django-jenkins

Once that's done, add these to our requirements file:

$ pip freeze > requirements.txt

We now need to configure our Django project to use django-jenkins. Add the following to the bottom of the settings file:

1INSTALLED_APPS += ('django_jenkins',)
2JENKINS_TASKS = (
3    'django_jenkins.tasks.run_pylint',
4    'django_jenkins.tasks.with_coverage',
5)
6PROJECT_APPS = ['blogengine']

This adds django-jenkins to our installed apps and tells it to include two additional tasks, besides running the tests. The first task runs Pylint to check our code quality (but we aren't really concerned about that at this point). The second checks the coverage. Finally, we tell django-jenkins that the blogengine app is the only one to be tested.

You'll also want to add the following lines to your .gitignore:

1reports/
2htmlcov/

These are the reports generated by django-jenkins, and should not be kept under version control. With that done, it's time to commit:

1$ git add .gitignore django_tutorial_blog_ng/ requirements.txt
2$ git commit -m 'Added coverage checking using django-jenkins'

Now, let's run our tests. From now on, you'll use the following command to run your tests:

$ python manage.py jenkins

This ensures we check the coverage at the same time. Now, you'll notice that the reports folder has been created, and it will contain three files, including one called coverage.xml. However, XML isn't a very friendly format. Happily, we can easily generate reports in HTML instead:

$ python manage.py jenkins --coverage-html-report=htmlcov

Running this command will create another folder called htmlcov/, and in here you will find your report, nicely formatted as HTML. Open up index.html in your web browser and you should see a file-by-file breakdown of your code coverage. Nice, huh?

Now, if your code so far is largely identical to mine, you'll notice that the model and view files don't have 100% coverage yet. If you click on each one, you'll see a handy line-by-line breakdown of the test coverage for each file. You'll notice that in the views file, the areas of the code for when tags and categories don't exist are highlighted in pink - this tells you that these lines of code are never executed during the tests. So let's fix that.

First, our template needs to be able to handle empty lists.

1{% extends "blogengine/includes/base.html" %}
2
3    {% load custom_markdown %}
4
5    {% block content %}
6        {% if object_list %}
7            {% for post in object_list %}
8            <div class="post">
9            <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1>
10            <h3>{{ post.pub_date }}</h3>
11            {{ post.text|custom_markdown }}
12            </div>
13            <a href="{{ post.category.get_absolute_url }}">{{ post.category.name }}</a>
14            {% for tag in post.tags.all %}
15            <a href="{{ tag.get_absolute_url }}">{{ tag.name }}</a>
16            {% endfor %}
17            {% endfor %}
18        {% else %}
19            <p>No posts found</p>
20        {% endif %}
21
22        {% if page_obj.has_previous %}
23        <a href="/{{ page_obj.previous_page_number }}/">Previous Page</a>
24        {% endif %}
25        {% if page_obj.has_next %}
26        <a href="/{{ page_obj.next_page_number }}/">Next Page</a>
27        {% endif %}
28
29    {% endblock %}

Let's commit the changes:

1$ git add templates/blogengine/post_list.html
2$ git commit -m 'Added a "No posts found" message to post list template'

Next, we need to write tests to check that we get a "No posts found" message when we view a tag or category that does not exist. Add the following methods to the class PostViewTest():

1    def test_nonexistent_category_page(self):
2        category_url = '/category/blah/'
3        response = self.client.get(category_url)
4        self.assertEquals(response.status_code, 200)
5        self.assertTrue('No posts found' in response.content)
6
7    def test_nonexistent_tag_page(self):
8        tag_url = '/tag/blah/'
9        response = self.client.get(tag_url)
10        self.assertEquals(response.status_code, 200)
11        self.assertTrue('No posts found' in response.content)

Now, let's run our tests again:

$ python manage.py jenkins --coverage-html-report=htmlcov

Assuming they all pass as expected, then your coverage reports will be regenerated. If you reload the coverage report, you should see that your views now have 100% test coverage. Let's commit again:

1$ git add blogengine/tests.py
2$ git commit -m 'Views file now has 100% coverage'

Now, our models still don't have 100% coverage yet. If you look at the breakdown for models.py, you'll see that the line if not self.slug: has only partial coverage, because we missed out setting a slug for the categories and tags in our test. So, let's fix that. In PostTest(), amend the test_create_category() and test_create_tag() methods as follows:

1    def test_create_category(self):
2        # Create the category
3        category = Category()
4
5        # Add attributes
6        category.name = 'python'
7        category.description = 'The Python programming language'
8        category.slug = 'python'
9
10        # Save it
11        category.save()
12
13        # Check we can find it
14        all_categories = Category.objects.all()
15        self.assertEquals(len(all_categories), 1)
16        only_category = all_categories[0]
17        self.assertEquals(only_category, category)
18
19        # Check attributes
20        self.assertEquals(only_category.name, 'python')
21        self.assertEquals(only_category.description, 'The Python programming language')
22        self.assertEquals(only_category.slug, 'python')
23
24    def test_create_tag(self):
25        # Create the tag
26        tag = Tag()
27
28        # Add attributes
29        tag.name = 'python'
30        tag.description = 'The Python programming language'
31        tag.slug = 'python'
32
33        # Save it
34        tag.save()
35
36        # Check we can find it
37        all_tags = Tag.objects.all()
38        self.assertEquals(len(all_tags), 1)
39        only_tag = all_tags[0]
40        self.assertEquals(only_tag, tag)
41
42        # Check attributes
43        self.assertEquals(only_tag.name, 'python')
44        self.assertEquals(only_tag.description, 'The Python programming language')
45        self.assertEquals(only_tag.slug, 'python')

Run the tests again:

$ python manage.py jenkins --coverage-html-report=htmlcov

Then refresh your coverage page, and we should have hit 100% coverage. Excellent news! This means that we can be confident that if any problems get introduced in future, we can pick them up easily. To demonstrate this, let's upgrade our Django install to the latest version and check everything still works as expected:

$ pip install Django --upgrade

This will upgrade the copy of Django in our virtualenv to the latest version. Then we can run our tests again:

$ python manage.py jenkins --coverage-html-report=htmlcov

Our tests should still pass, indicating that the upgrade to our Django version does not appear to have broken any functionality. Let's update our requirements.txt:

1$ pip freeze > requirements.txt
2$ git add requirements.txt
3$ git commit -m 'Upgraded Django version'

Preparing our web app for deployment to Heroku

As mentioned previously, I'm going to assume you plan to deploy your site on Heroku. It has good Django support, and you can quite happily host a blog there using their free tariff. If you'd prefer to use another hosting provider, then you should be able to adapt these instructions accordingly.

Now, so far we've used SQLite as our database for development purposes. However, SQLite isn't really suitable for production purposes. Heroku provide a Postgresql database for each web app, so we will use that. To configure it, open up settings.py and amend the database configuration section to look like this:

1DATABASES = {
2    'default': {
3        'ENGINE': 'django.db.backends.',
4        'NAME': '',
5    }
6}

Then, add the following at the end of the file:

1
2# Heroku config
3# Parse database configuration from $DATABASE_URL
4import dj_database_url
5DATABASES['default'] =  dj_database_url.config(default="sqlite:///db.sqlite3")
6
7# Honor the 'X-Forwarded-Proto' header for request.is_secure()
8SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
9
10# Allow all host headers
11ALLOWED_HOSTS = ['*']
12
13# Static asset configuration
14STATIC_ROOT = 'staticfiles'
15
16STATICFILES_DIRS = (
17    os.path.join(BASE_DIR, 'static'),
18)

A little explanation is called for. Remember when we first started out, we installed the django-toolbelt package, which included dj-database-url? Well, here we use the dj_database_url module to get the database from an environment variable set on Heroku. We set a default value so as to fall back to SQLite when that variable is not set. The other settings are required by Heroku.

You may want to run your tests again to ensure everything is still working before committing:

1$ git add django_tutorial_blog_ng/settings.py
2$ git commit -m 'Amended settings to work on desktop and Heroku'

Setting up Continuous Integration and coverage

Now, as mentioned previously, I'll be demonstrating how to set up Travis CI for our project. Travis CI is only free for open-source projects, so if you don't want to make your code publicly accessible, you may want to use Jenkins instead. I'll leave setting that up and running it as an exercise for the reader, but I recommend a plugin called Cobertura which allows you to publish details of your code's test coverage.

Unfortunately, Travis CI doesn't have the capability to publish your coverage results. Fortunately, you can use Coveralls.io to pick up the slack. Like Travis, it's free for open-source projects, and if you host your code on GitHub, it's pretty easy to use.

You can find instructions on setting up a project on Travis CI here. Once you've configured it, you'll need to set up your .travis.yml file. This is a simple text file that tells Travis CI how to run your tests. Put the following content in the file:

1language: python
2python:
3- "2.7"
4# command to install dependencies
5install: "pip install -r requirements.txt"
6# command to run tests
7script: coverage run --include="blogengine/*" --omit="blogengine/migrations/*" manage.py test blogengine
8after_success:
9    coveralls

Now, this tells Travis that this is a Python application, and we should be using Python 2.7. Please note you can test against multiple versions of Python if you wish - just add another line with the Python version you want to test against.

Then, we see that we use Pip to install our requirements. Finally we run our tests with Coverage, in order to generate coverage data, and afterwards call coveralls to pass the coverage data to Coveralls.io.

We also need to install the coveralls module:

1$ pip install coveralls
2$ pip freeze > requirements.txt

And we need to keep our coverage data out of version control:

1env/
2*.pyc
3db.sqlite3
4blogengine/static/bower_components/
5reports/
6htmlcov/
7.coverage

You'll also want to set up your project on Coveralls, as described here - please note that this requires your project be in a public GitHub repository.

With that done, let's commit our changes:

1$ git add .gitignore .travis.yml requirements.txt
2$ git commit -m 'Added Travis config file and Coveralls support'

With that done, assuming you have Travis CI and Coveralls configured, and your code is already hosted on GitHub, then you should be able to just push your code up to trigger the build:

$ git push origin master

If you keep an eye on Travis in your browser, you can watch what happens as your tests are run. If for any reason the build fails, then it shouldn't be too hard to figure out what has gone wrong using the documentation for Travis and Coveralls - both services are pretty easy to use.

Congratulations - you're now using Continuous Integration! That wasn't so hard, was it? Now, every time you push to GitHub, Travis will run your tests, and you'll get an email if they fail, giving you early warning of any problems, and you can check your coverage at the same time. Both Travis and Coveralls offer badges that you can place in your README file on GitHub to show off your coverage and build status - feel free to add these to your repo.

You may want to try making a change that breaks your tests and committing it, then pushing it up, so that you can see what happens when the build breaks.

Deploying to Heroku

Our final task today is deploying our blog to Heroku so we can see it in action. First of all, if you don't already have an account with Heroku, you'll need to sign up here. You should also be prompted to install the Heroku toolbelt. Once that's done, run the following command:

$ heroku login

You'll be prompted for your credentials - enter these and you should be able to log in successfully.

Now, in order to run our Django app on Heroku, we'll need to add a Procfile to tell Heroku what command to run in order to start your app. In this case, we will be using Gunicorn as our web server. Assuming our project is called django_tutorial_blog_ng, this is what you need to put in this file:

web: gunicorn django_tutorial_blog_ng.wsgi

That tells Heroku that the file we need to run for this application is django_tutorial_blog_ng/wsgi.py. To test it, run the following command:

$ foreman start

That will start our web server on port 5000, using Gunicorn rather than the Django development server. You should be able to see it in action here, but you'll notice a very serious issue - namely that the static files aren't being served. Now, Django has a command called collectstatic that collects all the static files and drops them into one convenient folder. Heroku will run this command automatically, so we need to ensure our static files are available. Amend your .gitignore file so it no longer excludes our static files:

1venv/
2*.pyc
3db.sqlite3
4reports/
5htmlcov/
6.coverage

We also need to amend our wsgi.py to serve static files:

1import os
2os.environ.setdefault("DJANGO_SETTINGS_MODULE", "django_tutorial_blog_ng.settings")
3
4from django.core.wsgi import get_wsgi_application
5from dj_static import Cling
6
7application = Cling(get_wsgi_application())

This should solve our problem. Let's commit our changes:

1$ git add django_tutorial_blog_ng/wsgi.py Procfile blogengine/static/ .gitignore
2$ git commit -m 'Configured app for deployment on Heroku'

Now we're ready to deploy our app!

Deployment

Every Heroku app needs a unique name. If you don't specify one, then Heroku will generate one for you. Your app will have the domain name appname.herokuapp.com - however, if you already have a domain name lined up for your blog, you can point that domain name at the Heroku app if you wish. I'm going to deploy mine with the name blog-shellshocked-info.

You also need to consider where you want to deploy it. Heroku has two regions - North America and EU. By default it will deploy to North America, but as I'm in the EU, that's where I want to deploy my app to - obviously if you're in the Americas, you may be better off sticking with North America.

So, let's create our app. Here's the command you need to run:

$ heroku apps:create blog-shellshocked-info --region eu

You will want to change the app name, or remove it entirely if you're happy for Heroku to generate one for you. If you want to host it in North America, drop the --region eu section.

Once completed, this will have created your new app, and added a Git remote for it, but will not have deployed it. To deploy your app, run this command:

$ git push heroku master

That will push your code up to Heroku. Please note that building the app may take a little while. Once it's done, you can run heroku open to open it in your web browser. You should see an error message stating that the relation blogengine_post does not exist. That's because we need to create our database structure. Heroku allows you to easily run commands on your app with heroku run, so let's create our database and run our migrations:

1$ heroku run python manage.py syncdb
2$ heroku run python manage.py migrate

These are exactly the same commands you would run locally to create your database, but prefaced with heroku run so that they get run by Heroku. As usual, you will be prompted to create a superuser - you'll want to do this so you can log into the admin.

That's all for today! We've finally got our site up and running on Heroku, and set up continuous integration so our tests will get run for us. You can see an example of the site working here. As usual, you can check out the latest version of the code with git checkout lesson-5. If you'd like a homework assignment, then take a look at automating deployment to Heroku on successful builds and see if you can get it set up successfully.

Next time around, we'll get back to implementing our search and tidying up the front end. See you then!