28th September 2014 7:53 pm
Recently I had the occasion to reformat a load of dates in Vim from DD/MM/YYYY to YYYY-MM-DD. In Vim, this is quite simple:
:%s/\(\d\{2}\)\/\(\d\{2}\)\/\(\d\{4}\)/\3-\2-\1/gThis should be easy to adapt to reformatting other date formats.
31st August 2014 10:00 pm
Hello again! In our final instalment, we’ll wrap up our blog by:
I’ll also cover development tools and practices that can make using Django easier. But first there’s a few housekeeping tasks that need doing…
Don’t forget to activate your virtualenv - you should know how to do this off by heart by now!
At the time of writing, Django 1.7 is due any day now, but it’s not out yet so I won’t cover it. The biggest change is the addition of a built-in migration system, but switching from South to this is well-documented. When Django 1.7 comes out, it shouldn’t be difficult to upgrade to it - because we have good test coverage, we shouldn’t have much trouble catching errors.
However, Django 1.6.6 was recently released, and we need to upgrade to it. Just enter the following command to upgrade:
$ pip install Django --upgradeThen add it to your requirements.txt:
$ pip freeze > requirements.txtThen commit your changes:
$ git add requirements.txt $ git commit -m 'Upgraded Django version'
Creating a sitemap for your blog is a good idea - it can be submitted to search engines, so that they can easily find your content. With Django, it’s pretty straightforward too.
First, let’s create a test for our sitemap. Add the following code at the end of tests.py:
class SitemapTest(BaseAcceptanceTest): def test_sitemap(self): # Create a post post = PostFactory() # Create a flat page page = FlatPageFactory() # Get sitemap response = self.client.get('/sitemap.xml') self.assertEquals(response.status_code, 200) # Check post is present in sitemap self.assertTrue('my-first-post' in response.content) # Check page is present in sitemap self.assertTrue('/about/' in response.content)
Run it, and you should see the test fail:
$ python manage.py test blogengine Creating test database for alias 'default'... ...........................F ====================================================================== FAIL: test_sitemap (blogengine.tests.SitemapTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 847, in test_sitemap self.assertEquals(response.status_code, 200) AssertionError: 404 != 200 ---------------------------------------------------------------------- Ran 28 tests in 6.873s FAILED (failures=1) Destroying test database for alias 'default'...
Now, let’s implement our sitemap. The sitemap application comes with Django, and needs to be activated in your settings file, under INSTALLED_APPS:
'django.contrib.sitemaps',Next, let’s think about what content we want to include in the sitemap. We want to index our flat pages and our blog posts, so our sitemap should reflect that. Create a new file at blogengine/sitemap.py and enter the following text:
from django.contrib.sitemaps import Sitemap from django.contrib.flatpages.models import FlatPage from blogengine.models import Post class PostSitemap(Sitemap): changefreq = "always" priority = 0.5 def items(self): return Post.objects.all() def lastmod(self, obj): return obj.pub_date class FlatpageSitemap(Sitemap): changefreq = "always" priority = 0.5 def items(self): return FlatPage.objects.all()
We define two sitemaps, one for all the posts, and the other for all the flat pages. Note that this works in a very similar way to the syndication framework.
Next, we amend our URLs. Add the following text after the existing imports in your URL file:
from django.contrib.sitemaps.views import sitemap from blogengine.sitemap import PostSitemap, FlatpageSitemap # Define sitemaps sitemaps = { 'posts': PostSitemap, 'pages': FlatpageSitemap }
Then add the following after the existing routes:
# Sitemap url(r'^sitemap\.xml$', sitemap, {'sitemaps': sitemaps}, name='django.contrib.sitemaps.views.sitemap'),
Here we define what sitemaps we’re going to use, and we define a URL for them. It’s pretty straightforward to use.
Let’s run our tests:
$ python manage.py test blogengine Creating test database for alias 'default'... ............................ ---------------------------------------------------------------------- Ran 28 tests in 6.863s OK Destroying test database for alias 'default'...
And done! Let’s commit our changes:
$ git add blogengine/ django_tutorial_blog_ng/settings.py $ git commit -m 'Implemented a sitemap'
Our blog is now feature-complete, but there are a few gaps in test coverage, so we’ll fix them. If, like me, you’re using Coveralls.io, you can easily see via their web interface where there are gaps in the coverage.
Now, our gaps are all in our view file - if you take a look at my build, you can easily identify the gaps as they’re marked in red.
The first gap is where a tag does not exist. Interestingly, if we look at the code in the view, we can see that some of it is redundant:
class TagPostsFeed(PostsFeed): def get_object(self, request, slug): return get_object_or_404(Tag, slug=slug) def title(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def link(self, obj): return obj.get_absolute_url() def description(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def items(self, obj): try: tag = Tag.objects.get(slug=obj.slug) return tag.post_set.all() except Tag.DoesNotExist: return Post.objects.none()
Under the items function, we check to see if the tag exists. However, under get_object we can see that if the object didn’t exist, it would already have returned a 404 error. We can therefore safely amend items to not check, since that try statement will never fail:
class TagPostsFeed(PostsFeed): def get_object(self, request, slug): return get_object_or_404(Tag, slug=slug) def title(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def link(self, obj): return obj.get_absolute_url() def description(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def items(self, obj): tag = Tag.objects.get(slug=obj.slug) return tag.post_set.all()
The other two gaps are in our search view - we never get an empty result for the search in the following section:
def getSearchResults(request): """ Search for a post by title or text """ # Get the query data query = request.GET.get('q', '') page = request.GET.get('page', 1) # Query the database if query: results = Post.objects.filter(Q(text__icontains=query) | Q(title__icontains=query)) else: results = None # Add pagination pages = Paginator(results, 5) # Get specified page try: returned_page = pages.page(page) except EmptyPage: returned_page = pages.page(pages.num_pages) # Display the search results return render_to_response('blogengine/search_post_list.html', {'page_obj': returned_page, 'object_list': returned_page.object_list, 'search': query})
So replace it with this:
def getSearchResults(request): """ Search for a post by title or text """ # Get the query data query = request.GET.get('q', '') page = request.GET.get('page', 1) # Query the database results = Post.objects.filter(Q(text__icontains=query) | Q(title__icontains=query)) # Add pagination pages = Paginator(results, 5) # Get specified page try: returned_page = pages.page(page) except EmptyPage: returned_page = pages.page(pages.num_pages) # Display the search results return render_to_response('blogengine/search_post_list.html', {'page_obj': returned_page, 'object_list': returned_page.object_list, 'search': query})
We don’t need to check whether query is defined because if q is left blank, the value of query will be an empty string, so we may as well pull out the redundant code.
Finally, the other gap is for when a user tries to get an empty search page (eg, page two of something with five or less results). So let’s add another test to our SearchViewTest class:
def test_failing_search(self): # Search for something that is not present response = self.client.get('/search?q=wibble') self.assertEquals(response.status_code, 200) self.assertTrue('No posts found' in response.content) # Try to get nonexistent second page response = self.client.get('/search?q=wibble&page=2') self.assertEquals(response.status_code, 200) self.assertTrue('No posts found' in response.content)
Run our tests and check the coverage:
$ coverage run --include="blogengine/*" --omit="blogengine/migrations/*" manage.py test blogengine $ coverage html
If you open htmlcov/index.html in your browser, you should see that the test coverage is back up to 100%. With that done, it’s time to commit again:
$ git add blogengine/ $ git commit -m 'Fixed gaps in coverage'
Remember, it’s not always possible to achieve 100% test coverage, and you shouldn’t worry too much about it if it’s not possible - it’s possible to ignore code if necessary. However, it’s a good idea to aim for 100%.
Next we’ll cover using Fabric, a handy tool for deploying your changes (any pretty much any other task you want to automate). First, you need to install it:
$ pip install FabricIf you have any problems installing it, you should be able to resolve them via Google - most of them are likely to be absent libraries that Fabric depends upon. Once it’s installed, add it to your requirements.tzt:
$ pip freeze > requirements.txtNext, create a file called fabfile.py and enter the following text:
#!/usr/bin/env python from fabric.api import local def deploy(): """ Deploy the latest version to Heroku """ # Push changes to master local("git push origin master") # Push changes to Heroku local("git push heroku master") # Run migrations on Heroku local("heroku run python manage.py migrate")
Now, all this file does is push our changes to Github (or wherever else your repository is hosted) and to Heroku, and runs your migrations. It’s not a terribly big task anyway, but it’s handy to have it in place. Let’s commit our changes:
$ git add fabfile.py requirements.txt $ git commit -m 'Added Fabric task for deployment'
Then, let’s try it out:
$ fab deployThere, wasn’t that more convenient? Fabric is much more powerful than this simple demonstration indicates, and can run tasks on remote servers via SSH easily. I recommend you take a look at the documentation to see what else you can do with it. If you’re hosting your site on a VPS, you will probably find Fabric indispensable, as you will need to restart the application every time you push up a new revision.
We want our blog application to play nicely with other Django apps. For instance, say you’re working on a new site that includes a blogging engine. Wouldn’t it make sense to just be able to drop in this blogging engine and have it work immediately? At the moment, some of our URL’s are hard-coded, so we may have problems in doing so. Let’s fix that.
First we’ll amend our tests. Add this at the top of the tests file:
from django.core.urlresolvers import reverseNext, replace every instance of this:
response = self.client.get('/')with this:
response = self.client.get(reverse('blogengine:index'))Then, rewrite the calls to the search route. For instance, this:
response = self.client.get('/search?q=first')should become this:
response = self.client.get(reverse('blogengine:search') + '?q=first')I’ll leave changing these as an exercise for the reader, but check the repository if you get stuck.
Next, we need to assign a namespace to our app’s routes:
from django.conf.urls import patterns, include, url from django.contrib import admin admin.autodiscover() urlpatterns = patterns('', # Examples: # url(r'^$', 'django_tutorial_blog_ng.views.home', name='home'), # url(r'^blog/', include('blog.urls')), url(r'^admin/', include(admin.site.urls)), # Blog URLs url(r'', include('blogengine.urls', namespace="blogengine")), # Flat pages url(r'', include('django.contrib.flatpages.urls')), )
We then assign names to our routes in the app’s urls.py:
from django.conf.urls import patterns, url from django.views.generic import ListView, DetailView from blogengine.models import Post, Category, Tag from blogengine.views import CategoryListView, TagListView, PostsFeed, CategoryPostsFeed, TagPostsFeed, getSearchResults from django.contrib.sitemaps.views import sitemap from blogengine.sitemap import PostSitemap, FlatpageSitemap # Define sitemaps sitemaps = { 'posts': PostSitemap, 'pages': FlatpageSitemap } urlpatterns = patterns('', # Index url(r'^(?P<page>\d+)?/?$', ListView.as_view( model=Post, paginate_by=5, ), name='index' ), # Individual posts url(r'^(?P<pub_date__year>\d{4})/(?P<pub_date__month>\d{1,2})/(?P<slug>[a-zA-Z0-9-]+)/?$', DetailView.as_view( model=Post, ), name='post' ), # Categories url(r'^category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryListView.as_view( paginate_by=5, model=Category, ), name='category' ), # Tags url(r'^tag/(?P<slug>[a-zA-Z0-9-]+)/?$', TagListView.as_view( paginate_by=5, model=Tag, ), name='tag' ), # Post RSS feed url(r'^feeds/posts/$', PostsFeed()), # Category RSS feed url(r'^feeds/posts/category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryPostsFeed()), # Tag RSS feed url(r'^feeds/posts/tag/(?P<slug>[a-zA-Z0-9-]+)/?$', TagPostsFeed()), # Search posts url(r'^search', getSearchResults, name='search'), # Sitemap url(r'^sitemap\.xml$', sitemap, {'sitemaps': sitemaps}, name='django.contrib.sitemaps.views.sitemap'), )
You also need to amend two of your templates:
<!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>{% block title %}My Django Blog{% endblock %}</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="alternate" type="application/rss+xml" title="Blog posts" href="/feeds/posts/" > <!-- Place favicon.ico and apple-touch-icon.png in the root directory --> {% load staticfiles %} <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/normalize.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/main.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap.min.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap-theme.min.css' %}"> <link rel="stylesheet" href="{% static 'css/main.css' %}"> <link rel="stylesheet" href="{% static 'css/code.css' %}"> <script src="{% static 'bower_components/html5-boilerplate/js/vendor/modernizr-2.6.2.min.js' %}"></script> </head> <body> <!--[if lt IE 7]> <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p> <![endif]--> <!-- Add your site or application content here --> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_GB/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <div class="navbar navbar-static-top navbar-inverse"> <div class="container-fluid"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#header-nav"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="{% url 'blogengine:index' %}">My Django Blog</a> </div> <div class="collapse navbar-collapse" id="header-nav"> <ul class="nav navbar-nav"> {% load flatpages %} {% get_flatpages as flatpages %} {% for flatpage in flatpages %} <li><a href="{{ flatpage.url }}">{{ flatpage.title }}</a></li> {% endfor %} <li><a href="/feeds/posts/">RSS feed</a></li> <form action="/search" method="GET" class="navbar-form navbar-left"> <div class="form-group"> <input type="text" name="q" placeholder="Search..." class="form-control"></input> </div> <button type="submit" class="btn btn-default">Search</button> </form> </ul> </div> </div> </div> <div class="container"> {% block header %} <div class="page-header"> <h1>My Django Blog</h1> </div> {% endblock %} <div class="row"> {% block content %}{% endblock %} </div> </div> <div class="container footer"> <div class="row"> <div class="span12"> <p>Copyright © {% now "Y" %}</p> </div> </div> </div> <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="{% static 'bower_components/html5-boilerplate/js/vendor/jquery-1.10.2.min.js' %}</script> <script src="{% static 'bower_components/html5-boilerplate/js/plugins.js' %}"></script> <script src="{% static 'bower_components/bootstrap/dist/js/bootstrap.min.js' %}"></script> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. --> <script> (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]= function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date; e=o.createElement(i);r=o.getElementsByTagName(i)[0]; e.src='//www.google-analytics.com/analytics.js'; r.parentNode.insertBefore(e,r)}(window,document,'script','ga')); ga('create','UA-XXXXX-X');ga('send','pageview'); </script> </body> </html>
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post col-md-12"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> {% if post.category %} <div class="col-md-12"> <a href="{{ post.category.get_absolute_url }}"><span class="label label-primary">{{ post.category.name }}</span></a> </div> {% endif %} {% if post.tags %} <div class="col-md-12"> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} <ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="{% url 'blogengine:search' %}?page={{ page_obj.previous_page_number }}&q={{ search }}">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="{% url 'blogengine:search' %}?page={{ page_obj.next_page_number }}&q={{ search }}">Next Page</a></li> {% endif %} </ul> {% endblock %}
Let’s run our tests:
$ python manage.py test blogengine/ Creating test database for alias 'default'... ............................. ---------------------------------------------------------------------- Ran 29 tests in 10.456s OK Destroying test database for alias 'default'...
And commit our changes:
$ git add . $ git commit -m 'Now use named routes'
There are a number of handy ways to debug Django applications. One of the simplest is to use the Python debugger. To use it, just enter the following lines at the point you want to break at:
import pdb pdb.set_trace()
Now, whenever that line of code is run, you’ll be dropped into an interactive shell that lets you play around to find out what’s going wrong. However, it doesn’t offer auto-completion, so we’ll install ipdb, which is an improved version:
$ pip install ipdb $ pip freeze > requirements.txt
Now you can use ipdb in much the same way as you would use pdb:
import ipdb ipdb.set_trace()
Now, ipdb is very useful, but it isn’t much help for profiling your application. For that you need the Django Debug Toolbar. Run the following commands:
$ pip install django-debug-toolbar $ pip freeze > requirements.txt
Then add the following line to INSTALLED_APPS in your settings file:
'debug_toolbar',Then, try running the development server, and you’ll see a toolbar on the right-hand side of the screen that allows you to view some useful data about your page. For instance, you’ll notice a field called SQL - this contains details of the queries carried out when building the page. To actually see the queries carried out, you’ll want to disable caching in your settings file by commenting out all the constants that start with CACHE.
We won’t go into using the toolbar to optimise queries, but using this, you can easily see what queries are being executed on a specific page, how long they take, and the values they return. Sometimes, you may need to optimise a slow query - in this case, Django allows you to drop down to writing raw SQL if necessary.
Note that if you’re running Django in production, you should set DEBUG to False as otherwise it gives rather too much information to potential attackers, and with Django Debug Toolbar installed, that’s even more important.
Please also note that when you disable debug mode, Django no longer handles static files automatically, so you’ll need to run python manage.py collectstatic and commit the staticfiles directory.
Once you’ve disabled debug mode, collected the static files, and re-enables caching, you can commit your changes:
$ git add . $ git commit -m 'Installed debugging tools'
We want our blog to get the best SEO results it can, so making it fast is essential. One of the simplest things you can do is to concatenate and minify static assets such as CSS and JavaScript. There are numerous ways to do this, but I generally use Grunt. Let’s set up a Grunt config to concatenate and minify our CSS and JavaScript.
You’ll need to have Node.js installed on your development machine for this. Then, you need to install the Grunt command-line interface:
$ sudo npm install -g grunt-cliWith that done, we need to create a package.json file. You can create one using the command npm init. Here’s mine:
{ "name": "django_tutorial_blog_ng", "version": "1.0.0", "description": "Django Tutorial Blog NG =======================", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "repository": { "type": "git", "url": "https://github.com/matthewbdaly/django_tutorial_blog_ng.git" }, "author": "Matthew Daly <matthew@matthewdaly.co.uk> (http://matthewdaly.co.uk/)", "license": "ISC", "bugs": { "url": "https://github.com/matthewbdaly/django_tutorial_blog_ng/issues" }, "homepage": "https://github.com/matthewbdaly/django_tutorial_blog_ng" }
Feel free to amend it as you see fit.
Next we install Grunt and the required plugins:
$ npm install grunt grunt-contrib-cssmin grunt-contrib-concat grunt-contrib-uglify --save-devWe now need to create a Gruntfile for our tasks:
module.exports = function (grunt) { 'use strict'; grunt.initConfig({ concat: { dist: { src: [ 'blogengine/static/bower_components/bootstrap/dist/css/bootstrap.css', 'blogengine/static/bower_components/bootstrap/dist/css/bootstrap-theme.css', 'blogengine/static/css/code.css', 'blogengine/static/css/main.css', ], dest: 'blogengine/static/css/style.css' } }, uglify: { dist: { src: [ 'blogengine/static/bower_components/jquery/jquery.js', 'blogengine/static/bower_components/bootstrap/dist/js/bootstrap.js' ], dest: 'blogengine/static/js/all.min.js' } }, cssmin: { dist: { src: 'blogengine/static/css/style.css', dest: 'blogengine/static/css/style.min.css' } } }); grunt.loadNpmTasks('grunt-contrib-concat'); grunt.loadNpmTasks('grunt-contrib-uglify'); grunt.loadNpmTasks('grunt-contrib-cssmin'); grunt.registerTask('default', ['concat', 'uglify', 'cssmin']); };
You’ll also need to change the paths in your base HTML file to point to the minified versions:
<!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>{% block title %}My Django Blog{% endblock %}</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="alternate" type="application/rss+xml" title="Blog posts" href="/feeds/posts/" > <!-- Place favicon.ico and apple-touch-icon.png in the root directory --> {% load staticfiles %} <link rel="stylesheet" href="{% static 'css/style.min.css' %}"> </head> <body> <!--[if lt IE 7]> <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p> <![endif]--> <!-- Add your site or application content here --> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_GB/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <div class="navbar navbar-static-top navbar-inverse"> <div class="container-fluid"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#header-nav"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="{% url 'blogengine:index' %}">My Django Blog</a> </div> <div class="collapse navbar-collapse" id="header-nav"> <ul class="nav navbar-nav"> {% load flatpages %} {% get_flatpages as flatpages %} {% for flatpage in flatpages %} <li><a href="{{ flatpage.url }}">{{ flatpage.title }}</a></li> {% endfor %} <li><a href="/feeds/posts/">RSS feed</a></li> <form action="/search" method="GET" class="navbar-form navbar-left"> <div class="form-group"> <input type="text" name="q" placeholder="Search..." class="form-control"></input> </div> <button type="submit" class="btn btn-default">Search</button> </form> </ul> </div> </div> </div> <div class="container"> {% block header %} <div class="page-header"> <h1>My Django Blog</h1> </div> {% endblock %} <div class="row"> {% block content %}{% endblock %} </div> </div> <div class="container footer"> <div class="row"> <div class="span12"> <p>Copyright © {% now "Y" %}</p> </div> </div> </div> <script src="{% static 'js/all.min.js' %}"></script> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. --> <script> (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]= function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date; e=o.createElement(i);r=o.getElementsByTagName(i)[0]; e.src='//www.google-analytics.com/analytics.js'; r.parentNode.insertBefore(e,r)}(window,document,'script','ga')); ga('create','UA-XXXXX-X');ga('send','pageview'); </script> </body> </html>
Now, run the Grunt task:
$ gruntAnd collect the static files:
$ python manage.py collectstaticYou’ll also want to add your node_modules folder to your gitignore:
venv/ *.pyc db.sqlite3 reports/ htmlcov/ .coverage node_modules/
Then commit your changes:
$ git add . $ git commit -m 'Optimised static assets'
Now, our package.json will cause a problem - it will mean that this app is mistakenly identified as a Node.js app. To prevent this, create the .slugignore file:
package.jsonThen commit your changes and push them up:
$ git add .slugignore $ git commit -m 'Added slugignore' $ fab deploy
If you check, your site should now be loading the minified versions of the static files.
That’s our site done! As usual I’ve tagged the final commit with lesson-8.
Sadly, that’s our final instalment over with! I hope you’ve enjoyed these tutorials, and I look forward to seeing what you create with them.
25th August 2014 5:15 pm
Hello once again! In this instalment we’ll cover:
Don’t forget to activate your virtualenv:
$ source venv/bin/activateNow let’s get started!
If you frequent (or used to frequent) social media sites like Reddit, Slashdot or Digg, you may be familiar with something called variously the Digg or Slashdot effect, whereby if a page gets submitted to a social media site, and subsequently becomes popular, it can be hit by a huge number of HTTP requests in a very short period of time, slowing it down or even taking the server down completely.
Now, as a general rule of thumb, for most dynamic websites such as blogs, the bottleneck is not the web server or the programming language, but the database. If you have a lot of people hitting the same page over and over again in quick succession, then you’re essentially running the same query over and over again and getting the same result each time, which is expensive in terms of processing power. What you need to be able to do is cache the results of the query in memory for a given period of time so that the number of queries is reduced.
That’s where Memcached comes in. It’s a simple key-value store that allows you to store values in memory for a given period of time so that they can be retrieved without having to query the database. Memcached is a very common choice for caching, and is by far the fastest and most efficient type of cache available for Django. It’s also available on Heroku’s free tier.
Django has a very powerful caching framework that supports numerous types of cache in addition to Memcached, such as:
There are also third-party backends for using other caching systems such as Redis.
Now, the cache can be used in a number of different ways. You can cache only certain parts of your site if you wish. However, because our site is heavily content-driven, we should be pretty safe to use the per-site cache, which is the simplest way to set up caching.
In order to set up Memcached, there’s a couple of Python libraries we’ll need. If you want to install them locally, however, you’ll need to install both memcached and libmemcached (on Ubuntu, the packages you need are called memcached and libmemcached-dev) on your development machine. If you don’t want to do this, then just copy and paste these lines into requirements.txt instead:
django-pylibmc-sasl==0.2.4 pylibmc==1.3.0
If you are happy to install these dependencies locally, then run this command once memcached and libmemcached are installed:
$ pip install pylibmc django-pylibmc-saslWith that done let’s configure Memcached. Open up the settings file and add the following at the bottom:
def get_cache(): import os try: os.environ['MEMCACHE_SERVERS'] = os.environ['MEMCACHIER_SERVERS'].replace(',', ';') os.environ['MEMCACHE_USERNAME'] = os.environ['MEMCACHIER_USERNAME'] os.environ['MEMCACHE_PASSWORD'] = os.environ['MEMCACHIER_PASSWORD'] return { 'default': { 'BACKEND': 'django_pylibmc.memcached.PyLibMCCache', 'TIMEOUT': 300, 'BINARY': True, 'OPTIONS': { 'tcp_nodelay': True } } } except: return { 'default': { 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache' } } CACHES = get_cache() CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_SECONDS = 300 CACHE_MIDDLEWARE_KEY_PREFIX = ''
Then add the following to MIDDLEWARE_CLASSES:
'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware',
That’s it! The first section configures the application to use Memcached to cache the content when running on Heroku, and sets some configuration parameters, while the second section tells Django to use the per-site cache in order to cache all the site content.
Now, Heroku doesn’t include Memcached by default - instead it’s available as an add-on called Memcachier. To use add-ons you need to set up a credit card for billing. We will set it up to use the free developer plan, but if you outgrow this you can easily switch to a paid plan. To add Memcachier, run this command:
$ heroku addons:add memcachier:devPlease note that Memcachier can take a few minutes to get set up, so you may want to leave it a little while between adding it and pushing up your changes. Now we’ll commit our changes:
$ git add requirements.txt django_tutorial_blog_ng/settings.py $ git commit -m 'Implemented caching with Memcached'
Then we’ll push them up to our remote repository and to Heroku:
$ git push origin master $ git push heroku master
And that’s all you need to do to set up Memcached. In addition to storing your query results in Memcached, enabling the caching framework in Django will also set various HTTP headers to enable web proxies and browsers to cache content for an appropriate length of time. If you open up your browser’s developer tools and compare the response headers for your homepage on the latest version of the code with the previous version, you’ll notice that a number of additional headers appear, including Cache-Control, Expires and Last-Modified. These tell web browsers and web proxies how often to request the latest version of the HTML document, in order to help you reduce the bandwidth used.
As you can see, for a site like this where you are the only person adding content, it’s really easy to implement caching with Django, and for a blog there’s very little reason not to do it. If you’re not using Heroku and are instead hosting your site on a VPS, then the configuration will be somewhat different - see here for details. You can also find information on using other cache backends on the same page.
That isn’t all you can do to speed up your site. Heroku doesn’t seem to be very good for serving static files, and if your site is attracting a lot of traffic you might want to host your static files elsewhere, such as on Amazon’s S3 service. Doing so is outside the scope of this tutorial, but for that use case, you should check out django-storages.
There is one issue with this implementation. As it is right now, if you view the home page, add a post, then reload the page, you may not see the new post immediately because the cache will continue serving the old version until it has expired. That behaviour is less than ideal - we would like the cache to be cleared automatically when a new post gets added so that users will see the new version immediately. That response will still be cached afterwards, so it only means one extra query.
This is the ideal place to introduce signals. Signals are a way to carry out a given action when an event takes place. In our case, we plan to clear the cache when a post is saved (either created or updated).
Note that as we’ll be testing the behaviour of the cache at this point, you’ll need to install Memcached on your local machine, and we’ll need to change the settings to fall back to our local Memcached instance:
def get_cache(): import os try: os.environ['MEMCACHE_SERVERS'] = os.environ['MEMCACHIER_SERVERS'].replace(',', ';') os.environ['MEMCACHE_USERNAME'] = os.environ['MEMCACHIER_USERNAME'] os.environ['MEMCACHE_PASSWORD'] = os.environ['MEMCACHIER_PASSWORD'] return { 'default': { 'BACKEND': 'django_pylibmc.memcached.PyLibMCCache', 'TIMEOUT': 300, 'BINARY': True, 'OPTIONS': { 'tcp_nodelay': True } } } except: return { 'default': { 'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache', 'LOCATION': '127.0.0.1:11211' } } CACHES = get_cache() CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_SECONDS = 300 CACHE_MIDDLEWARE_KEY_PREFIX = ''
If you don’t want to install Memcached locally, you can skip this step, but be aware that the test we write for clearing the cache will always pass if you do skip it.
Then we’ll run our tests to make sure nothing has been broken:
$ python manage.py jenkins blogengine Creating test database for alias 'default'... ....................... ---------------------------------------------------------------------- Ran 23 tests in 6.164s OK
Let’s commit:
$ git add django_tutorial_blog_ng/settings.py $ git commit -m 'Now use Memcached in development'
Now we’ll add a test for clearing the cache to PostViewTest:
def test_clear_cache(self): # Create the category category = Category() category.name = 'python' category.description = 'The Python programming language' category.save() # Create the tag tag = Tag() tag.name = 'perl' tag.description = 'The Perl programming language' tag.save() # Create the author author = User.objects.create_user('testuser', 'user@example.com', 'password') author.save() # Create the site site = Site() site.name = 'example.com' site.domain = 'example.com' site.save() # Create the first post post = Post() post.title = 'My first post' post.text = 'This is [my first blog post](http://127.0.0.1:8000/)' post.slug = 'my-first-post' post.pub_date = timezone.now() post.author = author post.site = site post.category = category post.save() post.tags.add(tag) # Check new post saved all_posts = Post.objects.all() self.assertEquals(len(all_posts), 1) # Fetch the index response = self.client.get('/') self.assertEquals(response.status_code, 200) # Create the second post post = Post() post.title = 'My second post' post.text = 'This is [my second blog post](http://127.0.0.1:8000/)' post.slug = 'my-second-post' post.pub_date = timezone.now() post.author = author post.site = site post.category = category post.save() post.tags.add(tag) # Fetch the index again response = self.client.get('/') # Check second post present self.assertTrue('my second blog post' in response.content)
This should be fairly self-explanatory. We create one post, and request the index page. We then add a second post, request the index page again, and check for the second post. The test should fail because the cached version is returned, rather than the version in the database.
Now we have a test in place, we can implement a fix. First, add this to the top of your models.py:
from django.db.models.signals import post_save from django.core.cache import cache
Then add the following at the bottom of the file:
# Define signals def new_post(sender, instance, created, **kwargs): cache.clear() # Set up signals post_save.connect(new_post, sender=Post)
This is fairly straightforward. What we’re doing is first defining a function called new_post that is called when a new post is created. We then connect it to the post_save signal. When a post is saved, it calls new_post, which clears the cache, making sure users are seeing the latest and greatest version of your site immediately.
Let’s test it:
$ python manage.py jenkins blogengine Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 8.473s OK Destroying test database for alias 'default'...
There are a number of signals available, and when you create one, you have access to the created object via the instance parameter. Using signals you can implement all kinds of functionality. For instance, you could implement the functionality to send an email when a new post is published.
If you’re using Travis CI, you’ll also need to update the config file:
language: python python: - "2.7" services: memcached before_install: - sudo apt-get install -y libmemcached-dev # command to install dependencies install: "pip install -r requirements.txt" # command to run tests script: coverage run --include="blogengine/*" --omit="blogengine/migrations/*" manage.py test blogengine after_success: coveralls
Time to commit:
$ git add blogengine/ .travis.yml $ git commit -m 'Now clear cache when post added'
Now, we want to offer more than one option for RSS feeds. For instance, if your blog is aggregated on a site such as Planet Python, but you also blog about JavaScript, you may want to be able to provide a feed for posts in the python category only.
If you have written any posts that use any of Markdown’s custom formatting, you may notice that if you load your RSS feed in a reader, it isn’t formatted as Markdown. Let’s fix that. First we’ll amend our test:
class FeedTest(BaseAcceptanceTest): def test_all_post_feed(self): # Create the category category = Category() category.name = 'python' category.description = 'The Python programming language' category.save() # Create the tag tag = Tag() tag.name = 'python' tag.description = 'The Python programming language' tag.save() # Create the author author = User.objects.create_user('testuser', 'user@example.com', 'password') author.save() # Create the site site = Site() site.name = 'example.com' site.domain = 'example.com' site.save() # Create a post post = Post() post.title = 'My first post' post.text = 'This is my *first* blog post' post.slug = 'my-first-post' post.pub_date = timezone.now() post.author = author post.site = site post.category = category # Save it post.save() # Add the tag post.tags.add(tag) post.save() # Check we can find it all_posts = Post.objects.all() self.assertEquals(len(all_posts), 1) only_post = all_posts[0] self.assertEquals(only_post, post) # Fetch the feed response = self.client.get('/feeds/posts/') self.assertEquals(response.status_code, 200) # Parse the feed feed = feedparser.parse(response.content) # Check length self.assertEquals(len(feed.entries), 1) # Check post retrieved is the correct one feed_post = feed.entries[0] self.assertEquals(feed_post.title, post.title) self.assertTrue('This is my <em>first</em> blog post' in feed_post.description)
Don’t forget to run the tests to make sure they fail. Now, let’s fix it:
from django.shortcuts import render from django.views.generic import ListView from blogengine.models import Category, Post, Tag from django.contrib.syndication.views import Feed from django.utils.encoding import force_unicode from django.utils.safestring import mark_safe import markdown2 # Create your views here. class CategoryListView(ListView): def get_queryset(self): slug = self.kwargs['slug'] try: category = Category.objects.get(slug=slug) return Post.objects.filter(category=category) except Category.DoesNotExist: return Post.objects.none() class TagListView(ListView): def get_queryset(self): slug = self.kwargs['slug'] try: tag = Tag.objects.get(slug=slug) return tag.post_set.all() except Tag.DoesNotExist: return Post.objects.none() class PostsFeed(Feed): title = "RSS feed - posts" link = "feeds/posts/" description = "RSS feed - blog posts" def items(self): return Post.objects.order_by('-pub_date') def item_title(self, item): return item.title def item_description(self, item): extras = ["fenced-code-blocks"] content = mark_safe(markdown2.markdown(force_unicode(item.text), extras = extras)) return content
All we’re doing here is amending the item_description method of PostsFeed to render it as Markdown. Now let’s run our tests again:
$ python manage.py jenkins Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 9.370s OK Destroying test database for alias 'default'...
With that done, we’ll commit our changes:
$ git add blogengine/ $ git commit -m 'Fixed rendering for post feed'
Now, before we get into implementing the feed, our tests are a bit verbose. We create a lot of items over and over again - let’s sort that out. Factory Boy is a handy Python module that allows you to create easy-to-use factories for creating objects over and over again in tests. Let’s install it:
$ pip install factory_boy $ pip freeze > requirements.txt $ git add requirements.txt $ git commit -m 'Installed Factory Boy'
Now let’s set up a factory for creating posts. Add this at the top of the test file:
import factory.djangoThen, before your actual tests, insert the following:
# Factories class SiteFactory(factory.django.DjangoModelFactory): class Meta: model = Site django_get_or_create = ( 'name', 'domain' ) name = 'example.com' domain = 'example.com'
Now, wherever you call Site(), add its attributes, and save it, replace those lines with the following:
site = SiteFactory()Much simpler and more concise, I’m sure you’ll agree! Now, let’s run the tests to make sure they aren’t broken:
$ python manage.py test blogengine Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 7.482s OK Destroying test database for alias 'default'...
Let’s commit again:
$ git add blogengine/tests.py $ git commit -m 'Now use Factory Boy for site objects'
Let’s do the same thing with Category objects:
class CategoryFactory(factory.django.DjangoModelFactory): class Meta: model = Category django_get_or_create = ( 'name', 'description', 'slug' ) name = 'python' description = 'The Python programming language' slug = 'python'
Again, just find every time we call Category() and replace it with the following:
category = CategoryFactory()Now if we run our tests, we’ll notice a serious error:
$ python manage.py test blogengine Creating test database for alias 'default'... EE..EE.EE.EE...E.EEE..E. ====================================================================== ERROR: test_create_category (blogengine.tests.PostTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 42, in test_create_category category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_create_post (blogengine.tests.PostTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 80, in test_create_post category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_create_post (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 339, in test_create_post category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_create_post_without_tag (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 378, in test_create_post_without_tag category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_delete_category (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 245, in test_delete_category category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_delete_post (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 467, in test_delete_post category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_edit_category (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 221, in test_edit_category category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_edit_post (blogengine.tests.AdminTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 410, in test_edit_post category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_all_post_feed (blogengine.tests.FeedTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 810, in test_all_post_feed category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_category_page (blogengine.tests.PostViewTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 640, in test_category_page category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_clear_cache (blogengine.tests.PostViewTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 753, in test_clear_cache category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_index (blogengine.tests.PostViewTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 518, in test_index category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ====================================================================== ERROR: test_post_page (blogengine.tests.PostViewTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 576, in test_post_page category = CategoryFactory() File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 82, in __call__ return cls.create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 585, in create return cls._generate(True, attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 510, in _generate obj = cls._prepare(create, **attrs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/base.py", line 485, in _prepare return cls._create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 151, in _create return cls._get_or_create(model_class, *args, **kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/factory/django.py", line 142, in _get_or_create obj, _created = manager.get_or_create(*args, **key_fields) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create return self.get_queryset().get_or_create(**kwargs) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/venv/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create obj.save(force_insert=True, using=self.db) TypeError: save() got an unexpected keyword argument 'force_insert' ---------------------------------------------------------------------- Ran 24 tests in 5.162s FAILED (errors=13) Destroying test database for alias 'default'...
Thankfully, this is easy to fix. We just need to amend the custom save() method of the Category model:
def save(self, *args, **kwargs): if not self.slug: self.slug = slugify(unicode(self.name)) super(Category, self).save(*args, **kwargs)
That should resolve the issue:
$ python manage.py jenkins Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 7.749s OK Destroying test database for alias 'default'...
Let’s commit again:
$ git add blogengine/ $ git commit -m 'Category now uses Factory Boy'
Now let’s do the same thing for tags:
class TagFactory(factory.django.DjangoModelFactory): class Meta: model = Tag django_get_or_create = ( 'name', 'description', 'slug' ) name = 'python' description = 'The Python programming language' slug = 'python'
And replace the sections where we create new Tag objects:
tag = TagFactory()Note that some tags have different values. We can easily pass different values to our TagFactory() to override the default values:
tag = TagFactory(name='perl', description='The Perl programming language')The Tag model has the same issue as the Category one did, so let’s fix that:
def save(self, *args, **kwargs): if not self.slug: self.slug = slugify(unicode(self.name)) super(Tag, self).save(*args, **kwargs)
We run our tests again:
$ python manage.py test blogengine/ Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 7.153s OK Destroying test database for alias 'default'...
Time to commit again:
$ git add blogengine/ $ git commit -m 'Now use Factory Boy for tags'
Next we’ll create a factory for adding users. Note that the factory name doesn’t have to match the object name, so you can create factories for different types of users. Here we create a factory for authors - you could, for instance, create a separate factory for subscribers if you wanted:
class AuthorFactory(factory.django.DjangoModelFactory): class Meta: model = User django_get_or_create = ('username','email', 'password',) username = 'testuser' email = 'user@example.com' password = 'password'
And as before, replace those sections where we create users with the following:
author = AuthorFactory()Run the tests again:
$ python manage.py test blogengine Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 5.808s OK Destroying test database for alias 'default'...
We commit our changes:
$ git add blogengine/ $ git commit -m 'Now use Factory Boy for creating authors'
Now we’ll create a flat page factory:
class FlatPageFactory(factory.django.DjangoModelFactory): class Meta: model = FlatPage django_get_or_create = ( 'url', 'title', 'content' ) url = '/about/' title = 'About me' content = 'All about me'
And use it for our flat page test:
page = FlatPageFactory()Check the tests pass:
$ python manage.py test blogengine Creating test database for alias 'default'... ........................ ---------------------------------------------------------------------- Ran 24 tests in 5.796s OK Destroying test database for alias 'default'...
And commit again:
$ git add blogengine/ $ git commit -m 'Now use Factory Boy for flat page test'
Now we’ll create a final factory for posts:
class PostFactory(factory.django.DjangoModelFactory): class Meta: model = Post django_get_or_create = ( 'title', 'text', 'slug', 'pub_date' ) title = 'My first post' text = 'This is my first blog post' slug = 'my-first-post' pub_date = timezone.now() author = factory.SubFactory(AuthorFactory) site = factory.SubFactory(SiteFactory) category = factory.SubFactory(CategoryFactory)
This factory is a little bit different. Because our Post model depends on several others, we need to be able to create those additional objects on demand. By designating them as subfactories, we can easily create the associated objects for our Post object.
That means that not only can we get rid of our Post() calls, but we can also get rid of the factory calls to create the associated objects for Post models. Again, I’ll leave actually doing this as an exercise for the reader, but you can always refer to the GitHub repository if you’re not too sure.
Make sure your tests still pass, then commit the changes:
$ git add blogengine/ $ git commit -m 'Now use Factory Boy for testing posts'
Using Factory Boy made a big difference to the size of the test file - I was able to cut it down by over 200 lines of code. As your application gets bigger, it gets harder to maintain, so do what you can to keep the size down.
Now, let’s implement our additional RSS feeds. First, we’ll write a test for the category feed. Add this to the FeedTest class:
def test_category_feed(self): # Create a post post = PostFactory(text='This is my *first* blog post') # Create another post in a different category category = CategoryFactory(name='perl', description='The Perl programming language', slug='perl') post2 = PostFactory(text='This is my *second* blog post', title='My second post', slug='my-second-post', category=category) # Fetch the feed response = self.client.get('/feeds/posts/category/python/') self.assertEquals(response.status_code, 200) # Parse the feed feed = feedparser.parse(response.content) # Check length self.assertEquals(len(feed.entries), 1) # Check post retrieved is the correct one feed_post = feed.entries[0] self.assertEquals(feed_post.title, post.title) self.assertTrue('This is my <em>first</em> blog post' in feed_post.description) # Check other post is not in this feed self.assertTrue('This is my <em>second</em> blog post' not in response.content)
Here we create two posts in different categories (note that we create a new category and override the post category for it). We then fetch /feeds/posts/category/python/ and assert that it contains only one post, with the content of the first post and not the content of the second.
Run the tests and they should fail:
$ python manage.py test blogengine Creating test database for alias 'default'... ................F........ ====================================================================== FAIL: test_category_feed (blogengine.tests.FeedTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 716, in test_category_feed self.assertEquals(response.status_code, 200) AssertionError: 404 != 200 ---------------------------------------------------------------------- Ran 25 tests in 5.804s FAILED (failures=1) Destroying test database for alias 'default'...
Because we haven’t yet implemented that route, we get a 404 error. So let’s create a route for this:
from django.conf.urls import patterns, url from django.views.generic import ListView, DetailView from blogengine.models import Post, Category, Tag from blogengine.views import CategoryListView, TagListView, PostsFeed, CategoryPostsFeed urlpatterns = patterns('', # Index url(r'^(?P<page>\d+)?/?$', ListView.as_view( model=Post, paginate_by=5, )), # Individual posts url(r'^(?P<pub_date__year>\d{4})/(?P<pub_date__month>\d{1,2})/(?P<slug>[a-zA-Z0-9-]+)/?$', DetailView.as_view( model=Post, )), # Categories url(r'^category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryListView.as_view( paginate_by=5, model=Category, )), # Tags url(r'^tag/(?P<slug>[a-zA-Z0-9-]+)/?$', TagListView.as_view( paginate_by=5, model=Tag, )), # Post RSS feed url(r'^feeds/posts/$', PostsFeed()), # Category RSS feed url(r'^feeds/posts/category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryPostsFeed()), )
Note that the category RSS feed route is similar to the post RSS feed route, but accepts a slug parameter. We will use this to pass through the slug for the category in question. Also note we import the CategoryPostsFeed view. Now, we need to create that view. Fortunately, because it’s written as a Python class, we can extend the existing PostsFeed class. Open up your views file and amend it to look like this:
from django.shortcuts import get_object_or_404 from django.views.generic import ListView from blogengine.models import Category, Post, Tag from django.contrib.syndication.views import Feed from django.utils.encoding import force_unicode from django.utils.safestring import mark_safe import markdown2 # Create your views here. class CategoryListView(ListView): def get_queryset(self): slug = self.kwargs['slug'] try: category = Category.objects.get(slug=slug) return Post.objects.filter(category=category) except Category.DoesNotExist: return Post.objects.none() class TagListView(ListView): def get_queryset(self): slug = self.kwargs['slug'] try: tag = Tag.objects.get(slug=slug) return tag.post_set.all() except Tag.DoesNotExist: return Post.objects.none() class PostsFeed(Feed): title = "RSS feed - posts" description = "RSS feed - blog posts" link = '/' def items(self): return Post.objects.order_by('-pub_date') def item_title(self, item): return item.title def item_description(self, item): extras = ["fenced-code-blocks"] content = mark_safe(markdown2.markdown(force_unicode(item.text), extras = extras)) return content class CategoryPostsFeed(PostsFeed): def get_object(self, request, slug): return get_object_or_404(Category, slug=slug) def title(self, obj): return "RSS feed - blog posts in category %s" % obj.name def link(self, obj): return obj.get_absolute_url() def description(self, obj): return "RSS feed - blog posts in category %s" % obj.name def items(self, obj): return Post.objects.filter(category=obj).order_by('-pub_date')
Note that many of our fields don’t have to be explicitly defined as they are inherited from PostsFeed. We can’t hard-code the title, link or description because they depend on the category, so we instead define methods to return the appropriate text.
Also note get_object() - we define this so that we can ensure the category exists. If it doesn’t exist, then it returns a 404 error rather than showing an empty feed.
We also override items() to filter it to just those posts that are in the given category.
If you run the tests again, they should now pass:
$ python manage.py test blogengine Creating test database for alias 'default'... ......................... ---------------------------------------------------------------------- Ran 25 tests in 5.867s OK Destroying test database for alias 'default'...
Let’s commit our changes:
$ git add blogengine/ $ git commit -m 'Implemented category RSS feed'
Now, we can get our category RSS feed, but how do we navigate to it? Let’s add a link to each category page that directs a user to its RSS feed. To do so, we’ll need to create a new template for category pages. First, let’s add some code to our tests to ensure that the right template is used at all times. Add the following to the end of the test_index method of PostViewTest:
# Check the correct template was used self.assertTemplateUsed(response, 'blogengine/post_list.html')
Then, add this to the end of test_post_page:
# Check the correct template was used self.assertTemplateUsed(response, 'blogengine/post_detail.html')
Finally, add this to the end of test_category_page:
# Check the correct template was used self.assertTemplateUsed(response, 'blogengine/category_post_list.html')
These assertions confirm which template was used to generate which request.
Next, we head into our views file:
class CategoryListView(ListView): template_name = 'blogengine/category_post_list.html' def get_queryset(self): slug = self.kwargs['slug'] try: category = Category.objects.get(slug=slug) return Post.objects.filter(category=category) except Category.DoesNotExist: return Post.objects.none() def get_context_data(self, **kwargs): context = super(CategoryListView, self).get_context_data(**kwargs) slug = self.kwargs['slug'] try: context['category'] = Category.objects.get(slug=slug) except Category.DoesNotExist: context['category'] = None return context
Note that we first of all change the template used by this view. Then, we override get_context_data to add in additional data. What we’re doing is getting the slug that was passed through, looking up any category for which it is the slug, and returning it as additional context data. Using this method, you can easily add additional data that you may wish to render in your Django templates.
Finally, we create our new template:
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post col-md-12"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> {% if post.category %} <div class="col-md-12"> <a href="{{ post.category.get_absolute_url }}"><span class="label label-primary">{{ post.category.name }}</span></a> </div> {% endif %} {% if post.tags %} <div class="col-md-12"> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} <ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="/{{ page_obj.previous_page_number }}/">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="/{{ page_obj.next_page_number }}/">Next Page</a></li> {% endif %} </ul> <a href="/feeds/posts/category/{{ category.slug }}/">RSS feed for category {{ category.name }}</a> {% endblock %}
Note that the category has been passed through to the template and is now accessible. If you run the tests, they should now pass:
$ python manage.py jenkins Creating test database for alias 'default'... ......................... ---------------------------------------------------------------------- Ran 25 tests in 7.232s OK Destroying test database for alias 'default'...
With that done. we can commit our changes:
$ git add templates/ blogengine/ $ git commit -m 'Added link to RSS feed from category page'
Next up, let’s implement another RSS feed for tags. First, we’ll implement our test:
def test_tag_feed(self): # Create a post post = PostFactory(text='This is my *first* blog post') tag = TagFactory() post.tags.add(tag) post.save() # Create another post with a different tag tag2 = TagFactory(name='perl', description='The Perl programming language', slug='perl') post2 = PostFactory(text='This is my *second* blog post', title='My second post', slug='my-second-post') post2.tags.add(tag2) post2.save() # Fetch the feed response = self.client.get('/feeds/posts/tag/python/') self.assertEquals(response.status_code, 200) # Parse the feed feed = feedparser.parse(response.content) # Check length self.assertEquals(len(feed.entries), 1) # Check post retrieved is the correct one feed_post = feed.entries[0] self.assertEquals(feed_post.title, post.title) self.assertTrue('This is my <em>first</em> blog post' in feed_post.description) # Check other post is not in this feed self.assertTrue('This is my <em>second</em> blog post' not in response.content)
This is virtually identical to the test for the category feed, but we adjust it to work with the Tag attribute and change the URL. Let’s check that our test fails:
$ python manage.py test blogengine Creating test database for alias 'default'... .................F........ ====================================================================== FAIL: test_tag_feed (blogengine.tests.FeedTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 757, in test_tag_feed self.assertEquals(response.status_code, 200) AssertionError: 404 != 200 ---------------------------------------------------------------------- Ran 26 tests in 5.760s FAILED (failures=1) Destroying test database for alias 'default'...
As before, we create a route for this:
from django.conf.urls import patterns, url from django.views.generic import ListView, DetailView from blogengine.models import Post, Category, Tag from blogengine.views import CategoryListView, TagListView, PostsFeed, CategoryPostsFeed, TagPostsFeed urlpatterns = patterns('', # Index url(r'^(?P<page>\d+)?/?$', ListView.as_view( model=Post, paginate_by=5, )), # Individual posts url(r'^(?P<pub_date__year>\d{4})/(?P<pub_date__month>\d{1,2})/(?P<slug>[a-zA-Z0-9-]+)/?$', DetailView.as_view( model=Post, )), # Categories url(r'^category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryListView.as_view( paginate_by=5, model=Category, )), # Tags url(r'^tag/(?P<slug>[a-zA-Z0-9-]+)/?$', TagListView.as_view( paginate_by=5, model=Tag, )), # Post RSS feed url(r'^feeds/posts/$', PostsFeed()), # Category RSS feed url(r'^feeds/posts/category/(?P<slug>[a-zA-Z0-9-]+)/?$', CategoryPostsFeed()), # Tag RSS feed url(r'^feeds/posts/tag/(?P<slug>[a-zA-Z0-9-]+)/?$', TagPostsFeed()), )
Next, we create our view:
class TagPostsFeed(PostsFeed): def get_object(self, request, slug): return get_object_or_404(Tag, slug=slug) def title(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def link(self, obj): return obj.get_absolute_url() def description(self, obj): return "RSS feed - blog posts tagged %s" % obj.name def items(self, obj): try: tag = Tag.objects.get(slug=obj.slug) return tag.post_set.all() except Tag.DoesNotExist: return Post.objects.none()
Again, this inherits from PostsFeed, but the syntax for getting posts matching a tag is slightly different because they use a many-to-many relationship.
We also need a template for the tag pages. Add this to the end of the test_tag_page method:
# Check the correct template was used self.assertTemplateUsed(response, 'blogengine/tag_post_list.html')
Let’s create that template:
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post col-md-12"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> {% if post.category %} <div class="col-md-12"> <a href="{{ post.category.get_absolute_url }}"><span class="label label-primary">{{ post.category.name }}</span></a> </div> {% endif %} {% if post.tags %} <div class="col-md-12"> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} <ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="/{{ page_obj.previous_page_number }}/">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="/{{ page_obj.next_page_number }}/">Next Page</a></li> {% endif %} </ul> <a href="/feeds/posts/tag/{{ tag.slug }}/">RSS feed for tag {{ tag.name }}</a> {% endblock %}
This is virtually identical to the category template. You’ll also need to apply this template in the view for the tag list, and pass the tag name through as context data:
class TagListView(ListView): template_name = 'blogengine/tag_post_list.html' def get_queryset(self): slug = self.kwargs['slug'] try: tag = Tag.objects.get(slug=slug) return tag.post_set.all() except Tag.DoesNotExist: return Post.objects.none() def get_context_data(self, **kwargs): context = super(TagListView, self).get_context_data(**kwargs) slug = self.kwargs['slug'] try: context['tag'] = Tag.objects.get(slug=slug) except Tag.DoesNotExist: context['tag'] = None return context
Let’s run our tests:
$ python manage.py test blogengine Creating test database for alias 'default'... .......................... ---------------------------------------------------------------------- Ran 26 tests in 5.770s OK Destroying test database for alias 'default'...
You may want to do a quick check to ensure your tag feed link works as expected. Time to commit:
$ git add blogengine templates $ git commit -m 'Implemented tag feeds'
Before we crack on with implementing search, there’s one more piece of housekeeping. In Django, templates can be applied at project level or at app level. So far, we’ve been storing them in the project, but we would like our app to be as self-contained as possible so it can just be dropped into future projects where we need a blog. That way, it can be easily overridden for specific projects. You can move the folders and update the Git repository at the same time with this command:
$ git mv templates/ blogengine/We run the tests to make sure nothing untoward has happened:
$ python manage.py test Creating test database for alias 'default'... .......................... ---------------------------------------------------------------------- Ran 26 tests in 5.847s OK Destroying test database for alias 'default'...
And we commit:
$ git commit -m 'Moved templates'Note that git mv updates Git and moves the files, so you don’t need to call git add.
For our final task today, we will be implementing a very simple search engine. Our requirements are:
First, we’ll write our tests:
class SearchViewTest(BaseAcceptanceTest): def test_search(self): # Create a post post = PostFactory() # Create another post post2 = PostFactory(text='This is my *second* blog post', title='My second post', slug='my-second-post') # Search for first post response = self.client.get('/search?q=first') self.assertEquals(response.status_code, 200) # Check the first post is contained in the results self.assertTrue('My first post' in response.content) # Check the second post is not contained in the results self.assertTrue('My second post' not in response.content) # Search for second post response = self.client.get('/search?q=second') self.assertEquals(response.status_code, 200) # Check the first post is not contained in the results self.assertTrue('My first post' not in response.content) # Check the second post is contained in the results self.assertTrue('My second post' in response.content)
Don’t forget to run the tests to make sure they fail:
$ python manage.py test Creating test database for alias 'default'... ..........................F ====================================================================== FAIL: test_search (blogengine.tests.SearchViewTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 819, in test_search self.assertEquals(response.status_code, 200) AssertionError: 404 != 200 ---------------------------------------------------------------------- Ran 27 tests in 6.919s FAILED (failures=1) Destroying test database for alias 'default'...
With that done, we can add the search form to the header:
<!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>{% block title %}My Django Blog{% endblock %}</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="alternate" type="application/rss+xml" title="Blog posts" href="/feeds/posts/" > <!-- Place favicon.ico and apple-touch-icon.png in the root directory --> {% load staticfiles %} <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/normalize.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/main.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap.min.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap-theme.min.css' %}"> <link rel="stylesheet" href="{% static 'css/main.css' %}"> <link rel="stylesheet" href="{% static 'css/code.css' %}"> <script src="{% static 'bower_components/html5-boilerplate/js/vendor/modernizr-2.6.2.min.js' %}"></script> </head> <body> <!--[if lt IE 7]> <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p> <![endif]--> <!-- Add your site or application content here --> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_GB/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <div class="navbar navbar-static-top navbar-inverse"> <div class="container-fluid"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#header-nav"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="/">My Django Blog</a> </div> <div class="collapse navbar-collapse" id="header-nav"> <ul class="nav navbar-nav"> {% load flatpages %} {% get_flatpages as flatpages %} {% for flatpage in flatpages %} <li><a href="{{ flatpage.url }}">{{ flatpage.title }}</a></li> {% endfor %} <li><a href="/feeds/posts/">RSS feed</a></li> <form action="/search" method="GET" class="navbar-form navbar-left"> <div class="form-group"> <input type="text" name="q" placeholder="Search..." class="form-control"></input> </div> <button type="submit" class="btn btn-default">Search</button> </form> </ul> </div> </div> </div> <div class="container"> {% block header %} <div class="page-header"> <h1>My Django Blog</h1> </div> {% endblock %} <div class="row"> {% block content %}{% endblock %} </div> </div> <div class="container footer"> <div class="row"> <div class="span12"> <p>Copyright © {% now "Y" %}</p> </div> </div> </div> <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="{% static 'bower_components/html5-boilerplate/js/vendor/jquery-1.10.2.min.js' %}</script> <script src="{% static 'bower_components/html5-boilerplate/js/plugins.js' %}"></script> <script src="{% static 'bower_components/bootstrap/dist/js/bootstrap.min.js' %}"></script> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. --> <script> (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]= function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date; e=o.createElement(i);r=o.getElementsByTagName(i)[0]; e.src='//www.google-analytics.com/analytics.js'; r.parentNode.insertBefore(e,r)}(window,document,'script','ga')); ga('create','UA-XXXXX-X');ga('send','pageview'); </script> </body> </html>
Now we’ll actually implement our search. Implementing search using Django’s generic views can be fiddly, so we’ll write our search view as a function instead. First, amend the imports at the top of your view file to look like this:
from django.shortcuts import get_object_or_404, render_to_response from django.core.paginator import Paginator, EmptyPage from django.db.models import Q from django.views.generic import ListView from blogengine.models import Category, Post, Tag from django.contrib.syndication.views import Feed from django.utils.encoding import force_unicode from django.utils.safestring import mark_safe import markdown2
Next, add the following code to the end of the file:
def getSearchResults(request): """ Search for a post by title or text """ # Get the query data query = request.GET.get('q', '') page = request.GET.get('page', 1) # Query the database if query: results = Post.objects.filter(Q(text__icontains=query) | Q(title__icontains=query)) else: results = None # Add pagination pages = Paginator(results, 5) # Get specified page try: returned_page = pages.page(page) except EmptyPage: returned_page = pages.page(pages.num_pages) # Display the search results return render_to_response('blogengine/search_post_list.html', {'page_obj': returned_page, 'object_list': returned_page.object_list, 'search': query})
As this is the first time we’ve written a view without using generic views, a little explanation is called for. First we get the values of the q and page parameters passed to the view. q contains the query text and page contains the page number. Note also that our page defaults to 1 if not set.
We then use the Q object to perform a query. The Django ORM will AND together keyword argument queries, but that’s not the behaviour we want here. Instead we want to be able to search for content in the title or text, so we need to use a query with an OR statement, which necessitates using the Q object.
Next, we use the Paginator object to manually paginate the results, and if it raises an EmptyPage exception, to just show the last page instead. Finally we render the template blogengine/search_post_list.html, and pass through the parameters page_obj for the returned page, object_list for the objects, and search for the query.
We also need to add a route for our new view:
# Search posts url(r'^search', 'blogengine.views.getSearchResults'),
Finally, let’s create a new template to show our results:
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post col-md-12"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> {% if post.category %} <div class="col-md-12"> <a href="{{ post.category.get_absolute_url }}"><span class="label label-primary">{{ post.category.name }}</span></a> </div> {% endif %} {% if post.tags %} <div class="col-md-12"> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} <ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="/search?page={{ page_obj.previous_page_number }}&q={{ search }}">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="/search?page={{ page_obj.next_page_number }}&q={{ search }}">Next Page</a></li> {% endif %} </ul> {% endblock %}
Let’s run our tests:
$ python manage.py test Creating test database for alias 'default'... ........................... ---------------------------------------------------------------------- Ran 27 tests in 6.421s OK Destroying test database for alias 'default'...
Don’t forget to do a quick sense check to make sure it’s all working as expected. Then it’s time to commit:
$ git add blogengine/ $ git commit -m 'Implemented search'
And push up your changes:
$ git push origin master $ git push heroku master
And that’s the end of this instalment. Please note this particular search solution is quite basic, and if you want something more powerful, you may want to look at Haystack.
As usual, you can get this lesson with git checkout lesson-7 - if you have any problems, the repository should be the first place you look for answers as this is the working code base for the application, and with judicious use of a tool like diff, it’s generally pretty easy to track down most issues.
In our next, and final instalment, we’ll cover:
Hope to see you then!
25th May 2014 5:23 pm
Welcome back! In this tutorial we’ll cover the following:
Apologies, but I’m holding over implementing the search and additional feeds for a future instalment - in the past I’ve tried to cover too much in one post and that has led to me putting them off for much too long. So this instalment and future ones are going to be shorter so I can get them out the door quicker.
Ready? Let’s get started!
When someone reports a bug, it’s tempting to just dive straight in and start fixing it. But TDD cautions us against this practice. If we have a bug, then our tests should have caught it. If they don’t, then before we fix the bug, we have to make sure we can catch it if it crops up in future by implementing a test for it, and ensuring that it fails. Once that test is in place, we can then go ahead and fix the bug, safe in the knowledge that if it should reappear in future, we will be warned when our tests run.
As it happens, we have a bug in our web app. If you activate your virtualenv in the usual way and run the development server, and then try to create a new post without adding a tag, you’ll see that it fails as the tag is empty. Now, it’s pretty obvious that this is because the tags attribute of the Post model cannot be blank, so we have a good idea of what we need to do to fix this. But to make sure it never occurs again, we need to implement a test first.
Add the following method to AdminTest:
def test_create_post_without_tag(self): # Create the category category = Category() category.name = 'python' category.description = 'The Python programming language' category.save() # Log in self.client.login(username='bobsmith', password="password") # Check response code response = self.client.get('/admin/blogengine/post/add/') self.assertEquals(response.status_code, 200) # Create the new post response = self.client.post('/admin/blogengine/post/add/', { 'title': 'My first post', 'text': 'This is my first post', 'pub_date_0': '2013-12-28', 'pub_date_1': '22:00:04', 'slug': 'my-first-post', 'site': '1', 'category': '1' }, follow=True ) self.assertEquals(response.status_code, 200) # Check added successfully self.assertTrue('added successfully' in response.content) # Check new post now in database all_posts = Post.objects.all() self.assertEquals(len(all_posts), 1)
This is virtually identical to our previous method for adding a post, but doesn’t add a tag to the post. If you run python manage.py jenkins, the test should fail. Let’s commit our changes:
$ git add blogengine/tests.py $ git commit -m 'Added failing test for posts without tags'
Now we’re in a position to fix our bug. Let’s take a look at our models:
class Post(models.Model): title = models.CharField(max_length=200) pub_date = models.DateTimeField() text = models.TextField() slug = models.SlugField(max_length=40, unique=True) author = models.ForeignKey(User) site = models.ForeignKey(Site) category = models.ForeignKey(Category, blank=True, null=True) tags = models.ManyToManyField(Tag)
If you compare category and tags, you’ll immediately see that category has the additional parameters blank and null both set to True. So that’s what we need to do for tags as well. Amend the model to look like this:
class Post(models.Model): title = models.CharField(max_length=200) pub_date = models.DateTimeField() text = models.TextField() slug = models.SlugField(max_length=40, unique=True) author = models.ForeignKey(User) site = models.ForeignKey(Site) category = models.ForeignKey(Category, blank=True, null=True) tags = models.ManyToManyField(Tag, blank=True, null=True)
You shouldn’t have to create a migration for this. Let’s run our tests:
$ python manage.py jenkins --coverage-html-report=htmlcov Creating test database for alias 'default'... ....................... ---------------------------------------------------------------------- Ran 23 tests in 7.634s OK Destroying test database for alias 'default'...
Our tests pass! So we’ve fixed our bug, and we’ve ensured that if it happens again, we’ll catch it. With that done, it’s time to commit:
$ git add blogengine/models.py $ git commit -m 'Fixed a bug with the Post model'
Remember: Always make the effort to create a test to reproduce your bug before fixing it. That way, you know you can catch it in future.
This is one feature that not everyone will want to implement. If you want to be able to show code snippets on your blog, then implementing syntax highlighting is well worth your time. However, if that’s not what you want to use your blog for, feel free to skip over this section.
Now, earlier in the series we implemented Markdown support. In Markdown there are two main ways to denote a code block. One is to indent the code by four spaces, while the other is to use fenced code blocks with syntax highlighting. There are many flavours of Markdown available for Python, and unfortunately the one we’ve been using so far doesn’t support fenced code blocks, so we’ll be switching to one that does.
We also need to be able to generate a suitable stylesheet to highlight the code appropriately. For that, we’ll be using Pygments. So let’s first uninstall our existing implementation of Markdown:
$ pip uninstall MarkdownAnd install the new modules:
$ pip install markdown2 PygmentsDon’t forget to record the changes:
$ pip freeze > requirements.txtNow, we need to amend our Markdown template tags to use the new version of Markdown:
import markdown2 from django import template from django.template.defaultfilters import stringfilter from django.utils.encoding import force_unicode from django.utils.safestring import mark_safe register = template.Library() @register.filter(is_safe=True) @stringfilter def custom_markdown(value): extras = ["fenced-code-blocks"] return mark_safe(markdown2.markdown(force_unicode(value), extras = extras))
All we do here is change the Markdown module that gets imported, and amend how it is called. Note that we pass through the parameter fenced-code-blocks to enable this functionality.
If you now run the development server and create a post with some code in it (just copy the Ruby example from here), then view it on the site, you should be able to see that it’s in a <code> block. However, it’s not highlighted yet. Let’s commit our changes:
$ git add requirements/txt blogengine/templatetags/custom_markdown.py $ git commit -m 'Now use Markdown2 to allow for syntax highlighting'
Now, if you examine the markup for your code blocks using your browser’s developer tools, you’ll notice that the code is wrapped in many different spans with various classes. Pygments can generate a CSS file that uses those classes to highlight your code.
First, let’s create a folder to store our CSS files in:
$ mkdir blogengine/static/cssNext, we’ll add a blank CSS file for any other styling we may want to apply:
$ touch blogengine/static/css/main.cssThen, we generate our CSS file:
$ pygmentize -S default -f html > blogengine/static/css/code.cssWe need to include our CSS files in our HTML template:
<!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>{% block title %}My Django Blog{% endblock %}</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <!-- Place favicon.ico and apple-touch-icon.png in the root directory --> {% load staticfiles %} <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/normalize.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/main.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap.min.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap-theme.min.css' %}"> <link rel="stylesheet" href="{% static 'css/main.css' %}"> <link rel="stylesheet" href="{% static 'css/code.css' %}"> <script src="{% static 'bower_components/html5-boilerplate/js/vendor/modernizr-2.6.2.min.js' %}"></script> </head> <body> <!--[if lt IE 7]> <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p> <![endif]--> <!-- Add your site or application content here --> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_GB/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <div class="navbar navbar-static-top navbar-inverse"> <div class="navbar-inner"> <div class="container"> <a class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </a> <a class="brand" href="/">My Django Blog</a> <div class="nav-collapse collapse"> </div> </div> </div> </div> <div class="container"> {% block header %} <div class="page-header"> <h1>My Django Blog</h1> </div> {% endblock %} <div class="row"> {% block content %}{% endblock %} </div> </div> <div class="container footer"> <div class="row"> <div class="span12"> <p>Copyright © {% now "Y" %}</p> </div> </div> </div> <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="{% static 'bower_components/html5-boilerplate/js/vendor/jquery-1.10.2.min.js' %}</script> <script src="{% static 'bower_components/html5-boilerplate/js/plugins.js' %}"></script> <script src="{% static 'bower_components/bootstrap/dist/js/bootstrap.min.js' %}"></script> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. --> <script> (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]= function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date; e=o.createElement(i);r=o.getElementsByTagName(i)[0]; e.src='//www.google-analytics.com/analytics.js'; r.parentNode.insertBefore(e,r)}(window,document,'script','ga')); ga('create','UA-XXXXX-X');ga('send','pageview'); </script> </body> </html>
Now, if you run the development server and reload the page, your code will be highlighted using the default Pygments style. If you don’t like it, there are plenty to choose from. Run the following command:
$ pygmentize -L stylesThat will list the various styles available. For instance, let’s say we want to try the Tango style:
$ pygmentize -S tango -f html > blogengine/static/css/code.cssIf you like the Monokai theme in Sublime Text, there’s a Pygments version of that:
$ pygmentize -S monokai -f html > blogengine/static/css/code.cssIf you like the Solarized theme, that’s not bundled with Pygments, but can be installed separately:
$ pip install pygments-style-solarizedThen run this for the light version:
$ pygmentize -S solarizedlight -f html > blogengine/static/css/code.cssAnd this for the dark version:
$ pygmentize -S solarizeddark -f html > blogengine/static/css/code.cssPick one that you like - I’m going to go for the dark version of Solarized.
Note that this doesn’t actually change the background colour of the code blocks. You will therefore need to set this manually using the CSS file we created earlier. If you’re using Solarized Dark like I am, then this should set it correctly:
div.codehilite pre { background-color: #002b36; }
If you’re using Solarized Light, then this should be more appropriate:
div.codehilite pre { background-color: #fdf6e3; }
Or, if you’re using Monokai, black will do:
div.codehilite pre { background-color: #000000; }
With that done, let’s record the additional Pygments style:
$ pip freeze > requirements.txtAnd commit our changes:
$ git add requirements.txt blogengine/static/css/ templates/blogengine/includes/base.html $ git commit -m 'Styled code with Solarized Dark'
Let’s run our tests:
$ python manage.py jenkins Creating test database for alias 'default'... E ====================================================================== ERROR: blogengine.tests (unittest.loader.ModuleImportFailure) ---------------------------------------------------------------------- ImportError: Failed to import test module: blogengine.tests Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name __import__(name) File "/Users/matthewdaly/Projects/django_tutorial_blog_ng/blogengine/tests.py", line 7, in <module> import markdown ImportError: No module named markdown ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1) Destroying test database for alias 'default'...
Whoops! We introduced an error here. If we take a look, we can see that the problem is on line 7, where we import the Markdown module. That makes sense, as we now use a different implementation of Markdown. Fortunately, in Python you can import modules with a different name, which makes this a breeze to fix. Change the line to:
import markdown2 as markdownNow, if you run your tests, they should pass. It’s important that if a test breaks, you fix it as soon as possible and don’t put it off. Let’s commit these changes:
$ git add blogengine/tests.py $ git commit -m 'Fixed a broken test'
Our syntax highlighting is now done! If you want to see it in action using the Solarized Dark theme, check out the copy of this blogging engine hosted here.
Now that our blog is up and running on Heroku, it could do with a bit of work on the front end to make it look a bit nicer. If you recall, we’re using Bootstrap for our front end, so you may want to refer to the documentation for that to give you some ideas on how you want to style your blog.
Bootstrap has a nifty pager class for Next and Previous links, so let’s apply that to our post list template:
<ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="/{{ page_obj.previous_page_number }}/">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="/{{ page_obj.next_page_number }}/">Next Page</a></li> {% endif %} </ul>
Let’s also add labels to our categories and tags. We’ll also place our posts and other content inside proper columns:
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post col-md-12"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> {% if post.category %} <div class="col-md-12"> <a href="{{ post.category.get_absolute_url }}"><span class="label label-primary">{{ post.category.name }}</span></a> </div> {% endif %} {% if post.tags %} <div class="col-md-12"> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} <ul class="pager"> {% if page_obj.has_previous %} <li class="previous"><a href="/{{ page_obj.previous_page_number }}/">Previous Page</a></li> {% endif %} {% if page_obj.has_next %} <li class="next"><a href="/{{ page_obj.next_page_number }}/">Next Page</a></li> {% endif %} </ul> {% endblock %}
We’ll also want to tidy up the layout for our individual post pages:
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} <div class="post col-md-12"> <h1>{{ object.title }}</h1> <h3>{{ object.pub_date }}</h3> {{ object.text|custom_markdown }} </div> {% if object.category %} <div class="col-md-12"> <a href="{{ object.category.get_absolute_url }}"><span class="label label-primary">{{ object.category.name }}</span></a> </div> {% endif %} {% if object.tags %} <div class="col-md-12"> {% for tag in object.tags.all %} <a href="{{ tag.get_absolute_url }}"><span class="label label-success">{{ tag.name }}</span></a> {% endfor %} </div> {% endif %} <div class="col-md-12"> <h4>Comments</h4> <div class="fb-comments" data-href="http://{{ object.site }}{{ object.get_absolute_url }}" data-width="470" data-num-posts="10"></div> </div> </div> {% endblock %}
Time to commit our changes:
$ git add templates/blogengine/ $ git commit -m 'Tidied up post templates'
With that done, let’s turn our attention to our base template. We’ll amend the header to collapse down at smaller screen widths. This is easy to do with Bootstrap:
<!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>{% block title %}My Django Blog{% endblock %}</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="alternate" type="application/rss+xml" title="Blog posts" href="/feeds/posts/" > <!-- Place favicon.ico and apple-touch-icon.png in the root directory --> {% load staticfiles %} <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/normalize.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/html5-boilerplate/css/main.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap.min.css' %}"> <link rel="stylesheet" href="{% static 'bower_components/bootstrap/dist/css/bootstrap-theme.min.css' %}"> <link rel="stylesheet" href="{% static 'css/main.css' %}"> <link rel="stylesheet" href="{% static 'css/code.css' %}"> <script src="{% static 'bower_components/html5-boilerplate/js/vendor/modernizr-2.6.2.min.js' %}"></script> </head> <body> <!--[if lt IE 7]> <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p> <![endif]--> <!-- Add your site or application content here --> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_GB/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <div class="navbar navbar-static-top navbar-inverse"> <div class="container-fluid"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#header-nav"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="/">My Django Blog</a> </div> <div class="collapse navbar-collapse" id="header-nav"> <ul class="nav navbar-nav"> {% load flatpages %} {% get_flatpages as flatpages %} {% for flatpage in flatpages %} <li><a href="{{ flatpage.url }}">{{ flatpage.title }}</a></li> {% endfor %} <li><a href="/feeds/posts/">RSS feed</a></li> </ul> </div> </div> </div> <div class="container"> {% block header %} <div class="page-header"> <h1>My Django Blog</h1> </div> {% endblock %} <div class="row"> {% block content %}{% endblock %} </div> </div> <div class="container footer"> <div class="row"> <div class="span12"> <p>Copyright © {% now "Y" %}</p> </div> </div> </div> <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="{% static 'bower_components/html5-boilerplate/js/vendor/jquery-1.10.2.min.js' %}</script> <script src="{% static 'bower_components/html5-boilerplate/js/plugins.js' %}"></script> <script src="{% static 'bower_components/bootstrap/dist/js/bootstrap.min.js' %}"></script> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. --> <script> (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]= function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date; e=o.createElement(i);r=o.getElementsByTagName(i)[0]; e.src='//www.google-analytics.com/analytics.js'; r.parentNode.insertBefore(e,r)}(window,document,'script','ga')); ga('create','UA-XXXXX-X');ga('send','pageview'); </script> </body> </html>
Here we’ve also added a link to our RSS feed in the header, and another in the page head to facilitate working with browsers that support RSS better.
Our blog’s now looking much more presentable. All that remains is to commit it:
$ git add templates/blogengine/includes/base.html $ git commit -m 'Amended base template'
Now we can push it to GitHub and deploy it on Heroku:
$ git push origin master $ git push heroku master
And we’re done! Don’t forget, you can grab this lesson with git checkout lesson-6.
Next time, we’ll cover search (I promise!).
24th May 2014 8:15 pm
Hello again! I was originally planning to cover implementing a search system, adding more feeds, and tidying up the front end in this instalment. However, I felt it was time for a change of pace, so instead, we’re going to look at:
Don’t worry, the original lesson isn’t going anywhere. We’ll still be implementing all of that later on, but today is the day we get your Django blog up and running on the web. That way, you can get a better idea of how Django works in the wild, and you have something concrete to show for your efforts.
If you’re not familiar with continuous integration, it’s basically a process that carries out some tasks automatically when you push a new commit to the repository. These tasks may include running unit tests, linting your code, and checking it to see what percentage of the code base is covered by the tests (after all, if your tests don’t actually cover every scenario, then there’s more of a chance that something might slip through the net. It’s also possible to implement hooks to automatically deploy your application only if the tests pass.
Typically, you will have a continuous integration server running somewhere that regularly polls your Git repository for changes, and when it finds a new commit, will check it out and run the tests (or whatever other task you configure it to). One of the most popular continuous integration servers around is Jenkins - I use it at work and can highly recommend it. However, we aren’t going to cover using Jenkins here because setting it up is quite a big deal and it really is best kept on a server of its own (although feel free to use it if you prefer). Instead, we’re going to use Travis CI, which integrates nicely with GitHub and is free for open source projects. If you don’t mind your code being publicly available on GitHub, then Travis is a really great way to dip your toe into continuous integration.
NB: You don’t have to use continuous integration at all if you don’t want to - this isn’t a big project so you don’t really need it. If you don’t want to put your code on GitHub, then feel free to just follow along and not bother pushing your code to GitHub and configuring Travis.
As mentioned above, code coverage is a measure of the percentage of the code that is covered by tests. While not infallible, it’s a fairly good guide to how comprehensive your tests are. If you have 100% code coverage, you can be fairly confident that your tests are comprehensive enough to catch most errors, so it’s a good rule of thumb to aim for 100% test coverage on a project.
So how do we check our test coverage? The coverage Python module is the most common tool for this. There’s also a handy Django module called django-jenkins, which is designed to work with Jenkins, but can be used with any continuous integration server, that can not only run your tests, but also check code coverage at the same time. So, make sure your virtualenv is up and running:
$ source venv/bin/activateThen, run the following command:
$ pip install coverage django-jenkinsOnce that’s done, add these to our requirements file:
$ pip freeze > requirements.txtWe now need to configure our Django project to use django-jenkins. Add the following to the bottom of the settings file:
INSTALLED_APPS += ('django_jenkins',) JENKINS_TASKS = ( 'django_jenkins.tasks.run_pylint', 'django_jenkins.tasks.with_coverage', ) PROJECT_APPS = ['blogengine']
This adds django-jenkins to our installed apps and tells it to include two additional tasks, besides running the tests. The first task runs Pylint to check our code quality (but we aren’t really concerned about that at this point). The second checks the coverage. Finally, we tell django-jenkins that the blogengine app is the only one to be tested.
You’ll also want to add the following lines to your .gitignore:
reports/ htmlcov/
These are the reports generated by django-jenkins, and should not be kept under version control. With that done, it’s time to commit:
$ git add .gitignore django_tutorial_blog_ng/ requirements.txt $ git commit -m 'Added coverage checking using django-jenkins'
Now, let’s run our tests. From now on, you’ll use the following command to run your tests:
$ python manage.py jenkinsThis ensures we check the coverage at the same time. Now, you’ll notice that the reports folder has been created, and it will contain three files, including one called coverage.xml. However, XML isn’t a very friendly format. Happily, we can easily generate reports in HTML instead:
$ python manage.py jenkins --coverage-html-report=htmlcovRunning this command will create another folder called htmlcov/, and in here you will find your report, nicely formatted as HTML. Open up index.html in your web browser and you should see a file-by-file breakdown of your code coverage. Nice, huh?
Now, if your code so far is largely identical to mine, you’ll notice that the model and view files don’t have 100% coverage yet. If you click on each one, you’ll see a handy line-by-line breakdown of the test coverage for each file. You’ll notice that in the views file, the areas of the code for when tags and categories don’t exist are highlighted in pink - this tells you that these lines of code are never executed during the tests. So let’s fix that.
First, our template needs to be able to handle empty lists.
{% extends "blogengine/includes/base.html" %} {% load custom_markdown %} {% block content %} {% if object_list %} {% for post in object_list %} <div class="post"> <h1><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h1> <h3>{{ post.pub_date }}</h3> {{ post.text|custom_markdown }} </div> <a href="{{ post.category.get_absolute_url }}">{{ post.category.name }}</a> {% for tag in post.tags.all %} <a href="{{ tag.get_absolute_url }}">{{ tag.name }}</a> {% endfor %} {% endfor %} {% else %} <p>No posts found</p> {% endif %} {% if page_obj.has_previous %} <a href="/{{ page_obj.previous_page_number }}/">Previous Page</a> {% endif %} {% if page_obj.has_next %} <a href="/{{ page_obj.next_page_number }}/">Next Page</a> {% endif %} {% endblock %}
Let’s commit the changes:
$ git add templates/blogengine/post_list.html $ git commit -m 'Added a "No posts found" message to post list template'
Next, we need to write tests to check that we get a “No posts found” message when we view a tag or category that does not exist. Add the following methods to the class PostViewTest():
def test_nonexistent_category_page(self): category_url = '/category/blah/' response = self.client.get(category_url) self.assertEquals(response.status_code, 200) self.assertTrue('No posts found' in response.content) def test_nonexistent_tag_page(self): tag_url = '/tag/blah/' response = self.client.get(tag_url) self.assertEquals(response.status_code, 200) self.assertTrue('No posts found' in response.content)
Now, let’s run our tests again:
$ python manage.py jenkins --coverage-html-report=htmlcovAssuming they all pass as expected, then your coverage reports will be regenerated. If you reload the coverage report, you should see that your views now have 100% test coverage. Let’s commit again:
$ git add blogengine/tests.py $ git commit -m 'Views file now has 100% coverage'
Now, our models still don’t have 100% coverage yet. If you look at the breakdown for models.py, you’ll see that the line if not self.slug: has only partial coverage, because we missed out setting a slug for the categories and tags in our test. So, let’s fix that. In PostTest(), amend the test_create_category() and test_create_tag() methods as follows:
def test_create_category(self): # Create the category category = Category() # Add attributes category.name = 'python' category.description = 'The Python programming language' category.slug = 'python' # Save it category.save() # Check we can find it all_categories = Category.objects.all() self.assertEquals(len(all_categories), 1) only_category = all_categories[0] self.assertEquals(only_category, category) # Check attributes self.assertEquals(only_category.name, 'python') self.assertEquals(only_category.description, 'The Python programming language') self.assertEquals(only_category.slug, 'python') def test_create_tag(self): # Create the tag tag = Tag() # Add attributes tag.name = 'python' tag.description = 'The Python programming language' tag.slug = 'python' # Save it tag.save() # Check we can find it all_tags = Tag.objects.all() self.assertEquals(len(all_tags), 1) only_tag = all_tags[0] self.assertEquals(only_tag, tag) # Check attributes self.assertEquals(only_tag.name, 'python') self.assertEquals(only_tag.description, 'The Python programming language') self.assertEquals(only_tag.slug, 'python')
Run the tests again:
$ python manage.py jenkins --coverage-html-report=htmlcovThen refresh your coverage page, and we should have hit 100% coverage. Excellent news! This means that we can be confident that if any problems get introduced in future, we can pick them up easily. To demonstrate this, let’s upgrade our Django install to the latest version and check everything still works as expected:
$ pip install Django --upgradeThis will upgrade the copy of Django in our virtualenv to the latest version. Then we can run our tests again:
$ python manage.py jenkins --coverage-html-report=htmlcovOur tests should still pass, indicating that the upgrade to our Django version does not appear to have broken any functionality. Let’s update our requirements.txt:
$ pip freeze > requirements.txt $ git add requirements.txt $ git commit -m 'Upgraded Django version'
As mentioned previously, I’m going to assume you plan to deploy your site on Heroku. It has good Django support, and you can quite happily host a blog there using their free tariff. If you’d prefer to use another hosting provider, then you should be able to adapt these instructions accordingly.
Now, so far we’ve used SQLite as our database for development purposes. However, SQLite isn’t really suitable for production purposes. Heroku provide a Postgresql database for each web app, so we will use that. To configure it, open up settings.py and amend the database configuration section to look like this:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.', 'NAME': '', } }
Then, add the following at the end of the file:
# Heroku config # Parse database configuration from $DATABASE_URL import dj_database_url DATABASES['default'] = dj_database_url.config(default="sqlite:///db.sqlite3") # Honor the 'X-Forwarded-Proto' header for request.is_secure() SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') # Allow all host headers ALLOWED_HOSTS = ['*'] # Static asset configuration STATIC_ROOT = 'staticfiles' STATICFILES_DIRS = ( os.path.join(BASE_DIR, 'static'), )
A little explanation is called for. Remember when we first started out, we installed the django-toolbelt package, which included dj-database-url? Well, here we use the dj_database_url module to get the database from an environment variable set on Heroku. We set a default value so as to fall back to SQLite when that variable is not set. The other settings are required by Heroku.
You may want to run your tests again to ensure everything is still working before committing:
$ git add django_tutorial_blog_ng/settings.py $ git commit -m 'Amended settings to work on desktop and Heroku'
Now, as mentioned previously, I’ll be demonstrating how to set up Travis CI for our project. Travis CI is only free for open-source projects, so if you don’t want to make your code publicly accessible, you may want to use Jenkins instead. I’ll leave setting that up and running it as an exercise for the reader, but I recommend a plugin called Cobertura which allows you to publish details of your code’s test coverage.
Unfortunately, Travis CI doesn’t have the capability to publish your coverage results. Fortunately, you can use Coveralls.io to pick up the slack. Like Travis, it’s free for open-source projects, and if you host your code on GitHub, it’s pretty easy to use.
You can find instructions on setting up a project on Travis CI here. Once you’ve configured it, you’ll need to set up your .travis.yml file. This is a simple text file that tells Travis CI how to run your tests. Put the following content in the file:
language: python python: - "2.7" # command to install dependencies install: "pip install -r requirements.txt" # command to run tests script: coverage run --include="blogengine/*" --omit="blogengine/migrations/*" manage.py test blogengine after_success: coveralls
Now, this tells Travis that this is a Python application, and we should be using Python 2.7. Please note you can test against multiple versions of Python if you wish - just add another line with the Python version you want to test against.
Then, we see that we use Pip to install our requirements. Finally we run our tests with Coverage, in order to generate coverage data, and afterwards call coveralls to pass the coverage data to Coveralls.io.
We also need to install the coveralls module:
$ pip install coveralls $ pip freeze > requirements.txt
And we need to keep our coverage data out of version control:
env/ *.pyc db.sqlite3 blogengine/static/bower_components/ reports/ htmlcov/ .coverage
You’ll also want to set up your project on Coveralls, as described here - please note that this requires your project be in a public GitHub repository.
With that done, let’s commit our changes:
$ git add .gitignore .travis.yml requirements.txt $ git commit -m 'Added Travis config file and Coveralls support'
With that done, assuming you have Travis CI and Coveralls configured, and your code is already hosted on GitHub, then you should be able to just push your code up to trigger the build:
$ git push origin masterIf you keep an eye on Travis in your browser, you can watch what happens as your tests are run. If for any reason the build fails, then it shouldn’t be too hard to figure out what has gone wrong using the documentation for Travis and Coveralls - both services are pretty easy to use.
Congratulations - you’re now using Continuous Integration! That wasn’t so hard, was it? Now, every time you push to GitHub, Travis will run your tests, and you’ll get an email if they fail, giving you early warning of any problems, and you can check your coverage at the same time. Both Travis and Coveralls offer badges that you can place in your README file on GitHub to show off your coverage and build status - feel free to add these to your repo.
You may want to try making a change that breaks your tests and committing it, then pushing it up, so that you can see what happens when the build breaks.
Our final task today is deploying our blog to Heroku so we can see it in action. First of all, if you don’t already have an account with Heroku, you’ll need to sign up here. You should also be prompted to install the Heroku toolbelt. Once that’s done, run the following command:
$ heroku loginYou’ll be prompted for your credentials - enter these and you should be able to log in successfully.
Now, in order to run our Django app on Heroku, we’ll need to add a Procfile to tell Heroku what command to run in order to start your app. In this case, we will be using Gunicorn as our web server. Assuming our project is called django_tutorial_blog_ng, this is what you need to put in this file:
web: gunicorn django_tutorial_blog_ng.wsgiThat tells Heroku that the file we need to run for this application is django_tutorial_blog_ng/wsgi.py. To test it, run the following command:
$ foreman startThat will start our web server on port 5000, using Gunicorn rather than the Django development server. You should be able to see it in action here, but you’ll notice a very serious issue - namely that the static files aren’t being served. Now, Django has a command called collectstatic that collects all the static files and drops them into one convenient folder. Heroku will run this command automatically, so we need to ensure our static files are available. Amend your .gitignore file so it no longer excludes our static files:
venv/ *.pyc db.sqlite3 reports/ htmlcov/ .coverage
We also need to amend our wsgi.py to serve static files:
import os os.environ.setdefault("DJANGO_SETTINGS_MODULE", "django_tutorial_blog_ng.settings") from django.core.wsgi import get_wsgi_application from dj_static import Cling application = Cling(get_wsgi_application())
This should solve our problem. Let’s commit our changes:
$ git add django_tutorial_blog_ng/wsgi.py Procfile blogengine/static/ .gitignore $ git commit -m 'Configured app for deployment on Heroku'
Now we’re ready to deploy our app!
Every Heroku app needs a unique name. If you don’t specify one, then Heroku will generate one for you. Your app will have the domain name appname.herokuapp.com - however, if you already have a domain name lined up for your blog, you can point that domain name at the Heroku app if you wish. I’m going to deploy mine with the name blog-shellshocked-info.
You also need to consider where you want to deploy it. Heroku has two regions - North America and EU. By default it will deploy to North America, but as I’m in the EU, that’s where I want to deploy my app to - obviously if you’re in the Americas, you may be better off sticking with North America.
So, let’s create our app. Here’s the command you need to run:
$ heroku apps:create blog-shellshocked-info --region euYou will want to change the app name, or remove it entirely if you’re happy for Heroku to generate one for you. If you want to host it in North America, drop the --region eu section.
Once completed, this will have created your new app, and added a Git remote for it, but will not have deployed it. To deploy your app, run this command:
$ git push heroku masterThat will push your code up to Heroku. Please note that building the app may take a little while. Once it’s done, you can run heroku open to open it in your web browser. You should see an error message stating that the relation blogengine_post does not exist. That’s because we need to create our database structure. Heroku allows you to easily run commands on your app with heroku run, so let’s create our database and run our migrations:
$ heroku run python manage.py syncdb $ heroku run python manage.py migrate
These are exactly the same commands you would run locally to create your database, but prefaced with heroku run so that they get run by Heroku. As usual, you will be prompted to create a superuser - you’ll want to do this so you can log into the admin.
That’s all for today! We’ve finally got our site up and running on Heroku, and set up continuous integration so our tests will get run for us. You can see an example of the site working here. As usual, you can check out the latest version of the code with git checkout lesson-5. If you’d like a homework assignment, then take a look at automating deployment to Heroku on successful builds and see if you can get it set up successfully.
Next time around, we’ll get back to implementing our search and tidying up the front end. See you then!
Flow Typed AJAX Responses With React Hooks
Caching the Laravel User Provider With a Decorator
The Trouble With Integrated Static Analysis
F*** Phpstorm Man and the High Horse He Rode in on
I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Zend Framework, Django, Phonegap and React.js.