The NoSQL movement is an exciting one for web developers. While relational databases such as MySQL are applicable to solving a wide range of problems, they aren't the best solution for every problem. Sometimes you may find yourself dealing with a problem where an alternative data store may make more sense.

Redis is one of the data stores that have appeared as part of this movement, and is arguably one of the more generally useful ones. Since it solves different problems to a relational database, it's not generally useful as an alternative to them - instead it is often used alongside them.

What is Redis?

Redis is described as follows on the website:

"Redis is an open source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs".

In other words, its core functionality is that it allows you to store a value by a key, and later retrieve that data using the key. It also allows you to set an optional expiry time for that key-value pair. It's quite similar to Memcached in that respect, and indeed one obvious use case for Redis is as an alternative to Memcached. However, it offers a number of additional benefits - for one thing, it supports more data types, and for another, it allows you to persist your data to disk (unlike Memcached, which only retains the data in memory, meaning it's lost on restart or if Memcached crashes). The latter means that for some very simple web applications, Redis can be used as your sole data store.

In this tutorial, we'll build a simple URL shortener, using Redis as the sole data store. A URL shortener only really requires two fields:

• A string to identify the correct URL
• The URL

That makes Redis a good fit for this use case since all we need to do is generate an ID for each URL, then when a link is followed, look up the URL for that key, and redirect the user to it. As long as this is all our application needs to do, we can quite happily use Redis for this rather than a relational database, and it will be significantly faster than a relational database would be for this use case.

Getting started

We're more interested in the fundamentals of using Redis in our application than a specific language here. As JavaScript is pretty much required to be a web developer, I think it's a fairly safe bet to use Node.js rather than PHP or Python, since that way, even if your only experience of JavaScript is client-side, you shouldn't have too much trouble following along.

You'll need to have Node.js installed, and I'll leave the details of installing this to you. You'll also need the Grunt CLI - install this globally as follows:

$ sudo npm install -g grunt-cli

Finally, you'll want to have Redis itself installed. You might also want to install hiredis, which is a faster Redis client that gets used automatically where available.

Now, let's create our package.json file:

$ npm init

You'll see a number of questions. Your generated package.json file should look something like this:

1{
2  "name": "url-shortener",
3  "version": "1.0.0",
4  "description": "A URL shortener",
5  "main": "index.js",
6  "scripts": {
7    "test": "grunt test"
8  },
9  "keywords": [
10    "URL",
11    "shortener"
12  ],
13  "author": "Matthew Daly <matthew@matthewdaly.co.uk> (http://matthewdaly.co.uk/)",
14  "license": "GPLv2"
15}

Note in particular that we set our test command to grunt test.

Next, we install our required Node.js modules. First, install, the normal dependencies:

$ npm install --save body-parser express redis hiredis jade shortid

Next, install the development dependencies:

$ npm install --save-dev grunt grunt-contrib-jshint grunt-mocha-istanbul istanbul mocha chai request

Now, we're going to use Grunt to run our tests, so that we can easily lint the JavaScript and generate code coverage details. Here's the Gruntfile:

1module.exports = function (grunt) {
2    'use strict';
3
4    grunt.initConfig({
5        jshint: {
6            all: [
7                'test/*.js',
8                'index.js'
9            ]
10        },
11        mocha_istanbul: {
12            coverage: {
13                src: 'test', // the folder, not the files,
14                options: {
15                    mask: '*.js',
16                    reportFormats: ['cobertura', 'html']
17                }
18            }
19        }
20    });
21
22    // Load tasks
23    grunt.loadNpmTasks('grunt-contrib-jshint');
24    grunt.loadNpmTasks('grunt-mocha-istanbul');
25
26    // Register tasks
27    grunt.registerTask('test', ['jshint', 'mocha_istanbul:coverage']);
28};

We'll also create a Procfile in anticipation of deploying the app to Heroku:

web: node index.js

Creating the views

For this application we'll be using the Express framework and the Jade templating system. We need three templates:

• Submission form
• Output form
• 404 page

Create the folder views under the application directory and add the files views/index.jade:

1doctype html
2html(lang="en")
3    head
4        title="Shortbread"
5
6    body
7        div.container
8            div.row
9                h1 Shortbread
10
11            div.row
12                form(action="/", method="POST")
13                    input(type="url", name="url")
14                    input(type="submit", value="Submit")

Also views/output.jade:

1doctype html
2html(lang="en")
3    head
4        title=Shortbread
5
6    body
7        div.container
8            div.row
9                h1 Shortbread
10                p
11                    | Your shortened URL is
12                    a(href=base_url+'/'+id) #{base_url}/#{id}

and views/error.jade:

1doctype html
2html(lang="en")
3    head
4        title="Shortbread"
5
6    body
7        div.container
8            div.row
9                h1 Shortbread
10                p Link not found

Writing our first test

We're going to use Mocha for our tests, together with the Chai assertion library. Create a folder called test, and put the following in test/test.js:

1/*jslint node: true */
2/*global describe: false, before: false, after: false, it: false */
3"use strict";
4
5// Declare the variables used
6var expect = require('chai').expect,
7    request = require('request'),
8    server = require('../index'),
9    redis = require('redis'),
10    client;
11client = redis.createClient();
12
13// Server tasks
14describe('server', function () {
15
16    // Beforehand, start the server
17    before(function (done) {
18        console.log('Starting the server');
19        done();
20    });
21
22    // Afterwards, stop the server and empty the database
23    after(function (done) {
24        console.log('Stopping the server');
25        client.flushdb();
26        done();
27    });
28
29    // Test the index route
30    describe('Test the index route', function () {
31        it('should return a page with the title Shortbread', function (done) {
32            request.get({ url: 'http://localhost:5000' }, function (error, response, body) {
33                expect(body).to.include('Shortbread');
34                expect(response.statusCode).to.equal(200);
35                expect(response.headers['content-type']).to.equal('text/html; charset=utf-8');
36                done();
37            });
38        });
39    });
40});

This code bears a little explanation. First, we import the required modules, as well as our index.js file (which we have yet to add). Then we create a callback to contain our tests.

Inside the callback, we call the before() and after() functions, which let us set up and tear down our tests. As part of the teardown process, we flush the Redis database.

Finally, we fetch our home page and verify that it returns a 200 status code and a content type of text/html, as well as including the name or our application.

We'll need to create our index.js file to avoid a nasty error, but we won't populate it just yet:

$ touch index.js

Let's run our tests:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6
7
8  server
9Starting the server
10    Test the index route
11      1) should return a page with the title Shortbread
12Stopping the server
13
14
15  0 passing (152ms)
16  1 failing
17
18  1) server Test the index route should return a page with the title Shortbread:
19     Uncaught AssertionError: expected undefined to include 'Shortbread'
20      at Request._callback (/Users/matthewdaly/Projects/url-shortener/test/test.js:33:33)
21      at self.callback (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:372:22)
22      at Request.emit (events.js:95:17)
23      at Request.onRequestError (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:963:8)
24      at ClientRequest.emit (events.js:95:17)
25      at Socket.socketErrorListener (http.js:1551:9)
26      at Socket.emit (events.js:95:17)
27      at net.js:440:14
28      at process._tickCallback (node.js:419:13)
29
30
31
32=============================================================================
33Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
34Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
35=============================================================================
36
37=============================== Coverage summary ===============================
38Statements   : 100% ( 0/0 )
39Branches     : 100% ( 0/0 )
40Functions    : 100% ( 0/0 )
41Lines        : 100% ( 0/0 )
42================================================================================
43>>
44Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
45
46Aborted due to warnings.

Now that we have a failing test, we can start work on our app proper. Open up index.js and add the following code:

1/*jslint node: true */
2'use strict';
3
4// Declare variables used
5var app, base_url, bodyParser, client, express, port, rtg, shortid;
6
7// Define values
8express = require('express');
9app = express();
10port = process.env.PORT || 5000;
11shortid = require('shortid');
12bodyParser = require('body-parser');
13base_url = process.env.BASE_URL || 'http://localhost:5000';
14
15// Set up connection to Redis
16if (process.env.REDISTOGO_URL) {
17  rtg  = require("url").parse(process.env.REDISTOGO_URL);
18  client = require("redis").createClient(rtg.port, rtg.hostname);
19  client.auth(rtg.auth.split(":")[1]);
20} else {
21  client = require('redis').createClient();
22}
23
24// Set up templating
25app.set('views', __dirname + '/views');
26app.set('view engine', "jade");
27app.engine('jade', require('jade').__express);
28
29// Set URL
30app.set('base_url', base_url);
31
32// Handle POST data
33app.use(bodyParser.json());
34app.use(bodyParser.urlencoded({
35  extended: true
36}));
37
38// Define index route
39app.get('/', function (req, res) {
40  res.render('index');
41});
42
43// Serve static files
44app.use(express.static(__dirname + '/static'));
45
46// Listen
47app.listen(port);
48console.log('Listening on port ' + port);

Let's go through this code. First we confirm that linting tools should treat this as a Node app, and use strict mode (I recommend always using strict mode in JavaScript).

Then we declare our variables and import the required modules. Note here that we set the port to 5000, but can also set it based on the PORT environment variable, which is used by Heroku. We also define a base URL, which again can be overriden from an environment variable when hosted on Heroku.

We then set up our connection to our Redis instance. When we push the code up to Heroku, we'll use the Redis To Go addon, so we check for an environment variable containing the Redis URL. If it's set, we use that to connect. Otherwise, we just connect as normal.

We then set up templating using Jade, and define the folder containing our views, and store the base URL within the app. Then we set up bodyParser so that Express can handle POST data.

Next, we define our index route to just render the index.jade file. Finally, we set up our static folder and set the app to listen on the correct port.

Let's run our test:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6Listening on port 5000
7
8
9  server
10Starting the server
11    Test the index route
12      ✓ should return a page with the title Shortbread (116ms)
13Stopping the server
14
15
16  1 passing (128ms)
17
18=============================================================================
19Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
20Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
21=============================================================================
22
23=============================== Coverage summary ===============================
24Statements   : 86.96% ( 20/23 )
25Branches     : 83.33% ( 5/6 )
26Functions    : 100% ( 1/1 )
27Lines        : 86.96% ( 20/23 )
28================================================================================
29>> Done. Check coverage folder.
30
31Done, without errors.

Note that Istanbul will have generated a nice HTML coverage report, which will be at coverage/index.html, but this won't show 100% test coverage due to the Heroku-specific Redis section. To fix this, amend that section as follows:

1/* istanbul ignore if */
2if (process.env.REDISTOGO_URL) {
3    rtg  = require("url").parse(process.env.REDISTOGO_URL);
4    client = require("redis").createClient(rtg.port, rtg.hostname);
5    client.auth(rtg.auth.split(":")[1]);
6} else {
7    client = require('redis').createClient();
8}

Telling Istanbul to ignore the if clause resolves that problem nicely.

Submitting a URL

Next, let's add the ability to add a URL. First, add the following test, after the one for the index:

1    // Test submitting a URL
2    describe('Test submitting a URL', function () {
3        it('should return the shortened URL', function (done) {
4            request.post('http://localhost:5000', {form: {url: 'http://www.google.co.uk'}}, function (error, response, body) {
5                expect(body).to.include('Your shortened URL is');
6                expect(response.statusCode).to.equal(200);
7                expect(response.headers['content-type']).to.equal('text/html; charset=utf-8');
8                done();
9            });
10        });
11    });

This test submits a URL via POST, and checks to see that the response view gets returned. Now, let's run our tests again:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6Listening on port 5000
7
8
9  server
10Starting the server
11    Test the index route
12      ✓ should return a page with the title Shortbread (223ms)
13    Test submitting a URL
14      1) should return the shortened URL
15Stopping the server
16
17
18  1 passing (318ms)
19  1 failing
20
21  1) server Test submitting a URL should return the shortened URL:
22     Uncaught AssertionError: expected 'Cannot POST /\n' to include 'Your shortened URL is'
23      at Request._callback (/Users/matthewdaly/Projects/url-shortener/test/test.js:45:33)
24      at Request.self.callback (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:372:22)
25      at Request.emit (events.js:98:17)
26      at Request.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1310:14)
27      at Request.emit (events.js:117:20)
28      at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1258:12)
29      at IncomingMessage.emit (events.js:117:20)
30      at _stream_readable.js:943:16
31      at process._tickCallback (node.js:419:13)
32
33
34
35=============================================================================
36Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
37Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
38=============================================================================
39
40=============================== Coverage summary ===============================
41Statements   : 100% ( 23/23 ), 3 ignored
42Branches     : 100% ( 6/6 ), 1 ignored
43Functions    : 100% ( 1/1 )
44Lines        : 100% ( 23/23 )
45================================================================================
46>>
47Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
48
49Aborted due to warnings.

We have a failing test, so let's make it pass. Add the following route after the index one:

1// Define submit route
2app.post('/', function (req, res) {
3    // Declare variables
4    var url, id;
5
6    // Get URL
7    url = req.body.url;
8
9    // Create a hashed short version
10    id = shortid.generate();
11
12    // Store them in Redis
13    client.set(id, url, function () {
14        // Display the response
15        res.render('output', { id: id, base_url: base_url });
16    });
17});

This route is fairly simple. It handles POST requests to the index route, and first of all it gets the URL from the POST request. Then it randomly generates a hash to use as the key.

The next part is where we see Redis in action. We create a new key-value pair, with the key set to the newly generated ID, and the value set to the URL. Once Redis confirms that has been done, the callback is fired, which renders the output.jade view with the ID and base URL passed through, so that we can see our shortened URL.

With that done, our test should pass:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6Listening on port 5000
7
8
9  server
10Starting the server
11    Test the index route
12      ✓ should return a page with the title Shortbread (89ms)
13    Test submitting a URL
14      ✓ should return the shortened URL (65ms)
15Stopping the server
16
17
18  2 passing (167ms)
19
20=============================================================================
21Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
22Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
23=============================================================================
24
25=============================== Coverage summary ===============================
26Statements   : 100% ( 29/29 ), 3 ignored
27Branches     : 100% ( 6/6 ), 1 ignored
28Functions    : 100% ( 3/3 )
29Lines        : 100% ( 29/29 )
30================================================================================
31>> Done. Check coverage folder.
32
33Done, without errors.

Our final task is to implement the short URL handling. We want to check to see if a short URL exists. If it does, we redirect the user to the destination. If it doesn't, we raise a 404 error. For that we need two more tests. Here they are:

1
2    // Test following a URL
3    describe('Test following a URL', function () {
4        it('should redirect the user to the shortened URL', function (done) {
5            // Create the URL
6            client.set('testurl', 'http://www.google.com', function () {
7                // Follow the link
8                request.get({
9                    url: 'http://localhost:5000/testurl',
10                    followRedirect: false
11                }, function (error, response, body) {
12                    expect(response.headers.location).to.equal('http://www.google.com');
13                    expect(response.statusCode).to.equal(301);
14                    done();
15                });
16            });
17        });
18    });
19
20    // Test non-existent link
21    describe('Test following a non-existent-link', function () {
22        it('should return a 404 error', function (done) {
23            // Follow the link
24            request.get({
25                url: 'http://localhost:5000/nonexistenturl',
26                followRedirect: false
27            }, function (error, response, body) {
28                expect(response.statusCode).to.equal(404);
29                expect(body).to.include('Link not found');
30                done();
31            });
32        });
33    });

The first test creates a URL for testing purposes. It then navigates to that URL. Note that we set followRedirect to true - this is because, by default, request will follow any redirect, so we need to prevent it from doing so to ensure that the headers to redirect the user are set correctly.

Once the response has been received, we then check that the status code is 301 (Moved Permanently), and that the Location header is set to the correct destination. When a real browser visits this page, it will be redirected accordingly.

The second test tries to fetch a non-existent URL, and checks that the status code is 404, and the response contains the words Link not found.

If we run our tests, they should now fail:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6Listening on port 5000
7
8
9  server
10Starting the server
11    Test the index route
12      ✓ should return a page with the title Shortbread (252ms)
13    Test submitting a URL
14      ✓ should return the shortened URL (47ms)
15    Test following a URL
16      1) should redirect the user to the shortened URL
17    Test following a non-existent-link
18      2) should return a 404 error
19Stopping the server
20
21
22  2 passing (322ms)
23  2 failing
24
25  1) server Test following a URL should redirect the user to the shortened URL:
26     Uncaught AssertionError: expected undefined to equal 'http://www.google.com'
27      at Request._callback (/Users/matthewdaly/Projects/url-shortener/test/test.js:63:58)
28      at Request.self.callback (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:372:22)
29      at Request.emit (events.js:98:17)
30      at Request.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1310:14)
31      at Request.emit (events.js:117:20)
32      at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1258:12)
33      at IncomingMessage.emit (events.js:117:20)
34      at _stream_readable.js:943:16
35      at process._tickCallback (node.js:419:13)
36
37  2) server Test following a non-existent-link should return a 404 error:
38     Uncaught AssertionError: expected 'Cannot GET /nonexistenturl\n' to include 'Link not found'
39      at Request._callback (/Users/matthewdaly/Projects/url-shortener/test/test.js:80:33)
40      at Request.self.callback (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:372:22)
41      at Request.emit (events.js:98:17)
42      at Request.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1310:14)
43      at Request.emit (events.js:117:20)
44      at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/url-shortener/node_modules/request/request.js:1258:12)
45      at IncomingMessage.emit (events.js:117:20)
46      at _stream_readable.js:943:16
47      at process._tickCallback (node.js:419:13)
48
49
50
51=============================================================================
52Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
53Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
54=============================================================================
55
56=============================== Coverage summary ===============================
57Statements   : 100% ( 29/29 ), 3 ignored
58Branches     : 100% ( 6/6 ), 1 ignored
59Functions    : 100% ( 3/3 )
60Lines        : 100% ( 29/29 )
61================================================================================
62>>
63Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
64
65Aborted due to warnings.

Now, let's add our final route:

1// Define link route
2app.route('/:id').all(function (req, res) {
3    // Get ID
4    var id = req.params.id.trim();
5
6    // Look up the URL
7    client.get(id, function (err, reply) {
8        if (!err && reply) {
9            // Redirect user to it
10            res.status(301);
11            res.set('Location', reply);
12            res.send();
13        } else {
14            // Confirm no such link in database
15            res.status(404);
16            res.render('error');
17        }
18    });
19});

We accept the ID as a parameter in the URL. We trim off any whitespace around it, and then we query Redis for a URL with that ID. If we find one, we set the status code to 301, and the location to the URL, and send the response. Otherwise, we set the status to 404 and render the error view.

Now, let's check it passes:

1$ grunt test
2Running "jshint:all" (jshint) task
3>> 2 files lint free.
4
5Running "mocha_istanbul:coverage" (mocha_istanbul) task
6Listening on port 5000
7
8
9  server
10Starting the server
11    Test the index route
12      ✓ should return a page with the title Shortbread (90ms)
13    Test submitting a URL
14      ✓ should return the shortened URL (47ms)
15    Test following a URL
16      ✓ should redirect the user to the shortened URL
17    Test following a non-existent-link
18      ✓ should return a 404 error
19Stopping the server
20
21
22  4 passing (191ms)
23
24=============================================================================
25Writing coverage object [/Users/matthewdaly/Projects/url-shortener/coverage/coverage.json]
26Writing coverage reports at [/Users/matthewdaly/Projects/url-shortener/coverage]
27=============================================================================
28
29=============================== Coverage summary ===============================
30Statements   : 100% ( 38/38 ), 3 ignored
31Branches     : 100% ( 10/10 ), 1 ignored
32Functions    : 100% ( 5/5 )
33Lines        : 100% ( 38/38 )
34================================================================================
35>> Done. Check coverage folder.
36
37Done, without errors.

Excellent! Our URL shortener is now complete. From here, deploying it to Heroku is straightforward - you'll need to install the Redis to Go addon, and refer to Heroku's documentation on deploying Node.js applications for more details.

Wrapping up

You'll find the source for this application here and a demo here.

I hope you've enjoyed this brief introduction to Redis, and that it's opened your eyes to at least one of the alternatives out there to a relational database. I'll hopefully be able to follow this up with examples of some other problems Redis is ideal for solving.