Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

2nd March 2015 11:25 pm

Syntax Highlighting in Fenced Code Blocks in Vim

Just thought I’d share a little trick I picked up recently. As you may know, GitHub flavoured Markdown (which I use for this blog) supports fenced code blocks, allowing you to specify a language for a block of code in a Markdown file.

If you put the following code in your .vimrc, you can get syntax highlighting in those code blocks when you open up a Markdown file in Vim:

"Syntax highlighting in Markdown
au BufNewFile,BufReadPost *.md set filetype=markdown
let g:markdown_fenced_languages = ['bash=sh', 'css', 'django', 'handlebars', 'javascript', 'js=javascript', 'json=javascript', 'perl', 'php', 'python', 'ruby', 'sass', 'xml', 'html']

This does depend on having the appropriate syntax files installed. However, you can easily add in syntax files for many other languages that Vim supports, and there are third-party ones available to install - in my case, I’ve got the handlebars one installed, which doesn’t come with Vim.

2nd March 2015 11:03 pm

Extending Our Node.js and Redis Chat Server

In this tutorial, we’re going to extend the chat system we built in the first tutorial to include the following functionality:

  • Persisting the data
  • Prompting users to sign in and storing their details in a Redis-backed session

In the process, we’ll pick up a bit more about using Redis.

Persistence

Our first task is to make our messages persist when the session ends. Now, in order to do this, we’re going to use a list. A list in Redis can be thought of as equivalent to an array or list in most programming languages, and can be retrieved by passing the key in a similar fashion to how you would retrieve a string.

As usual, we will write our test first. Open up test/test.js and replace the test for sending a message with this:

// Test sending a message
describe('Test sending a message', function () {
it("should return 'Message received'", function (done) {
// Connect to server
var socket = io.connect('http://localhost:5000', {
'reconnection delay' : 0,
'reopen delay' : 0,
'force new connection' : true
});
// Handle the message being received
socket.on('message', function (data) {
expect(data).to.include('Message received');
client.lrange('chat:messages', 0, -1, function (err, messages) {
// Check message has been persisted
var message_list = [];
messages.forEach(function (message, i) {
message_list.push(message);
});
expect(message_list[0]).to.include('Message received');
// Finish up
socket.disconnect();
done();
});
});
// Send the message
socket.emit('send', { message: 'Message received' });
});
});

The main difference here is that we use our Redis client to get the list chat:messages, and check to see if our message appears in it. Now, let’s run our test to ensure it fails:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (484ms)
Test sending a message
1) should return 'Message received'
Stopping the server
1 passing (552ms)
1 failing
1) server Test sending a message should return 'Message received':
Uncaught AssertionError: expected undefined to include 'Message received'
at /Users/matthewdaly/Projects/babblr/test/test.js:62:48
at try_callback (/Users/matthewdaly/Projects/babblr/node_modules/redis/index.js:592:9)
at RedisClient.return_reply (/Users/matthewdaly/Projects/babblr/node_modules/redis/index.js:685:13)
at HiredisReplyParser.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/redis/index.js:321:14)
at HiredisReplyParser.emit (events.js:95:17)
at HiredisReplyParser.execute (/Users/matthewdaly/Projects/babblr/node_modules/redis/lib/parser/hiredis.js:43:18)
at RedisClient.on_data (/Users/matthewdaly/Projects/babblr/node_modules/redis/index.js:547:27)
at Socket.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/redis/index.js:102:14)
at Socket.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:765:14)
at Socket.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:427:10)
at emitReadable (_stream_readable.js:423:5)
at readableAddChunk (_stream_readable.js:166:9)
at Socket.Readable.push (_stream_readable.js:128:10)
at TCP.onread (net.js:529:21)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 96.97% ( 32/33 ), 5 ignored
Branches : 100% ( 6/6 ), 1 ignored
Functions : 80% ( 4/5 )
Lines : 96.97% ( 32/33 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

Our test fails, so now we can start work on implementing the functionality we need. First of all, when a new message is sent, we need to push it to the list. Amend the new message handler in index.js to look like this:

// Handle new messages
io.sockets.on('connection', function (socket) {
// Subscribe to the Redis channel
subscribe.subscribe('ChatChannel');
// Handle incoming messages
socket.on('send', function (data) {
// Publish it
client.publish('ChatChannel', data.message);
// Persist it to a Redis list
client.rpush('chat:messages', 'Anonymous Coward : ' + data.message);
});
// Handle receiving messages
var callback = function (channel, data) {
socket.emit('message', data);
};
subscribe.on('message', callback);
// Handle disconnect
socket.on('disconnect', function () {
subscribe.removeListener('message', callback);
});
});

The only significant change is the Persist it to a Redis list section. Here we call the RPUSH command to push the current message to chat:messages. RPUSH pushes a message to the end of the list. There’s a similar command, LPUSH, which pushes an item to the beginning of the list, as well as LPOP and RPOP, which remove and return an item from the beginning and end of the list respectively.

Next we need to handle displaying the list when the main route loads. Replace the index route in index.js with this:

// Define index route
app.get('/', function (req, res) {
// Get messages
client.lrange('chat:messages', 0, -1, function (err, messages) {
/* istanbul ignore if */
if (err) {
console.log(err);
} else {
// Get messages
var message_list = [];
messages.forEach(function (message, i) {
/* istanbul ignore next */
message_list.push(message);
});
// Render page
res.render('index', { messages: message_list});
}
});
});

Here we use the client to return all messages in the list by using the LRANGE command and defining the slice as being from the start to the end of the list. We then loop through the mesages and push each to a list, before passing that list to the view.

Speaking of which, we also need to update views/index.hbs:

{{> header }}
<div class="container">
<div class="row">
<div class="col-md-8">
<div class="conversation">
{{#each messages}}
<p>{{this}}</p>
{{/each}}
</div>
</div>
<div class="col-md-4">
<form>
<div class="form-group">
<label for="message">Message</label>
<textarea class="form-control" id="message" rows="20"></textarea>
<a id="submitbutton" class="btn btn-primary form-control">Submit</a>
<div>
</form>
</div>
</div>
</div>
{{> footer }}

This just loops through the messages and prints each one in turn. Now let’s run our tests and make sure they pass:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (1262ms)
Test sending a message
✓ should return 'Message received' (48ms)
Stopping the server
2 passing (2s)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 40/40 ), 7 ignored
Branches : 100% ( 8/8 ), 2 ignored
Functions : 85.71% ( 6/7 )
Lines : 100% ( 40/40 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls" task
Running "coveralls:app" (coveralls) task
Verifying property coveralls.app exists in config...OK
Files: coverage/lcov.info
Options: src="coverage/lcov.info", force=false
Submitting file to coveralls.io: coverage/lcov.info
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

As before, don’t worry about Coveralls not working - it’s only an issue when it runs on Travis CI. If everything else is fine, our chat server should now persist our changes.

Sessions and user login

At present, it’s hard to carry on a conversation with someone using this site because you can’t see who is responding to you. We need to implement a mechanism to obtain a username for each user, store it in a session, and then use it to identify all of a user’s messages. In this case, we’re going to just prompt the user to enter a username of their choice, but if you wish, you can use something like Passport.js to allow authentication using third-party services - I’ll leave that as an exercise for the reader.

Now, Express doesn’t include any support for sessions out of the box, so we have to install some additional libraries:

$ npm install connect-redis express-session body-parser --save

The express-session library is middleware for Express that allows for storing and retrieving session variables, while connect-redis allows it to use Redis to store this data. We used body-parser for the URL shortener to process POST data, so we will use it again here. Now, we need to set it up. Replace the part of index.js before we set up the templating with this:

/*jslint node: true */
'use strict';
// Declare variables used
var app, base_url, client, express, hbs, io, port, RedisStore, rtg, session, subscribe;
// Define values
express = require('express');
app = express();
port = process.env.PORT || 5000;
base_url = process.env.BASE_URL || 'http://localhost:5000';
hbs = require('hbs');
session = require('express-session');
RedisStore = require('connect-redis')(session);
// Set up connection to Redis
/* istanbul ignore if */
if (process.env.REDISTOGO_URL) {
rtg = require('url').parse(process.env.REDISTOGO_URL);
client = require('redis').createClient(rtg.port, rtg.hostname);
subscribe = require('redis').createClient(rtg.port, rtg.hostname);
client.auth(rtg.auth.split(':')[1]);
subscribe.auth(rtg.auth.split(':')[1]);
} else {
client = require('redis').createClient();
subscribe = require('redis').createClient();
}
// Set up session
app.use(session({
store: new RedisStore({
client: client
}),
secret: 'blibble'
}));

This just sets up the session and configures it to use Redis as the back end. Don’t forget to change the value of secret.

Now, let’s plan out how our username system is going to work. If a user visits the site and there is no session set, then they should be redirected to a new route, /login. Here they will be prompted to enter a username. Once a satisfactory username (eg one or more characters) has been submitted via the form, it should be stored in the session and the user redirected to the index. There should also be a /logout route to destroy the session and redirect the user back to the login form.

First, let’s implement a test for fetching the login form in test/test.js:

// Test submitting to the login route
describe('Test submitting to the login route', function () {
it('should store the username in the session and redirect the user to the index', function (done) {
request.post({ url: 'http://localhost:5000/login',
form:{username: 'bobsmith'},
followRedirect: false},
function (error, response, body) {
expect(response.headers.location).to.equal('/');
expect(response.statusCode).to.equal(302);
done();
});
});
});

This test sends a POST request containing the field username with the value bobsmith. We expect to be redirected to the index route.

Let’s run the test to make sure it fails:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
express-session deprecated undefined resave option; provide resave option index.js:9:1585
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1585
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (45ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test submitting to the login route
1) should store the username in the session and redirect the user to the index
Test sending a message
✓ should return 'Message received' (42ms)
Stopping the server
3 passing (122ms)
1 failing
1) server Test submitting to the login route should store the username in the session and redirect the user to the index:
Uncaught AssertionError: expected undefined to equal '/'
at Request._callback (/Users/matthewdaly/Projects/babblr/test/test.js:61:58)
at Request.self.callback (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:373:22)
at Request.emit (events.js:98:17)
at Request.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1318:14)
at Request.emit (events.js:117:20)
at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1266:12)
at IncomingMessage.emit (events.js:117:20)
at _stream_readable.js:944:16
at process._tickCallback (node.js:442:13)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 45/45 ), 7 ignored
Branches : 100% ( 8/8 ), 2 ignored
Functions : 87.5% ( 7/8 )
Lines : 100% ( 45/45 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

Now, all we need to do to make this test pass is create a view containing the form and define a route to display it. First, we’ll define our new route in index.js:

// Define login route
app.get('/login', function (req, res) {
// Render view
res.render('login');
});

Next, we’ll create our new template at views/login.hbs:

{{> header }}
<div class="container">
<div class="row">
<div class="col-md-12">
<form action="/login" method="POST">
<div class="form-group">
<label for="Username">Please enter a handle</label>
<input type="text" class="form-control" size="20" required id="username" name="username"></input>
<input type="submit" class="btn btn-primary form-control"></input>
<div>
</form>
</div>
</div>
</div>
{{> footer }}

Let’s run our tests and make sure they pass:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
express-session deprecated undefined resave option; provide resave option index.js:9:1585
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1585
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (64ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test sending a message
✓ should return 'Message received' (78ms)
Stopping the server
3 passing (179ms)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 45/45 ), 7 ignored
Branches : 100% ( 8/8 ), 2 ignored
Functions : 87.5% ( 7/8 )
Lines : 100% ( 45/45 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls" task
Running "coveralls:app" (coveralls) task
Verifying property coveralls.app exists in config...OK
Files: coverage/lcov.info
Options: src="coverage/lcov.info", force=false
Submitting file to coveralls.io: coverage/lcov.info
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

Next, we need to process the submitted form, set the session, and redirect the user back to the index. First, let’s add another test:

// Test submitting to the login route
describe('Test submitting to the login route', function () {
it('should store the username in the session and redirect the user to the index', function (done) {
request.post({ url: 'http://localhost:5000/login',
form:{username: 'bobsmith'},
followRedirect: false},
function (error, response, body) {
expect(response.headers.location).to.equal('http://localhost:5000');
expect(response.statusCode).to.equal(301);
done();
});
});
});

This test submits the username, and makes sure that the response received is a 301 redirect to the index route. Let’s check to make sure it fails:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
express-session deprecated undefined resave option; provide resave option index.js:9:1585
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1585
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (476ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test submitting to the login route
1) should store the username in the session and redirect the user to the index
Test sending a message
✓ should return 'Message received' (42ms)
Stopping the server
3 passing (557ms)
1 failing
1) server Test submitting to the login route should store the username in the session and redirect the user to the index:
Uncaught AssertionError: expected undefined to equal 'http://localhost:5000'
at Request._callback (/Users/matthewdaly/Projects/babblr/test/test.js:61:58)
at Request.self.callback (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:373:22)
at Request.emit (events.js:98:17)
at Request.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1318:14)
at Request.emit (events.js:117:20)
at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1266:12)
at IncomingMessage.emit (events.js:117:20)
at _stream_readable.js:944:16
at process._tickCallback (node.js:442:13)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 45/45 ), 7 ignored
Branches : 100% ( 8/8 ), 2 ignored
Functions : 87.5% ( 7/8 )
Lines : 100% ( 45/45 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

Now, in order to process POST data we’ll need to use body-parser. Amend the top of index.js to look like this::

/*jslint node: true */
'use strict';
// Declare variables used
var app, base_url, bodyParser, client, express, hbs, io, port, RedisStore, rtg, session, subscribe;
// Define values
express = require('express');
app = express();
bodyParser = require('body-parser');
port = process.env.PORT || 5000;
base_url = process.env.BASE_URL || 'http://localhost:5000';
hbs = require('hbs');
session = require('express-session');
RedisStore = require('connect-redis')(session);
// Set up connection to Redis
/* istanbul ignore if */
if (process.env.REDISTOGO_URL) {
rtg = require('url').parse(process.env.REDISTOGO_URL);
client = require('redis').createClient(rtg.port, rtg.hostname);
subscribe = require('redis').createClient(rtg.port, rtg.hostname);
client.auth(rtg.auth.split(':')[1]);
subscribe.auth(rtg.auth.split(':')[1]);
} else {
client = require('redis').createClient();
subscribe = require('redis').createClient();
}
// Set up session
app.use(session({
store: new RedisStore({
client: client
}),
secret: 'blibble'
}));
// Set up templating
app.set('views', __dirname + '/views');
app.set('view engine', "hbs");
app.engine('hbs', require('hbs').__express);
// Register partials
hbs.registerPartials(__dirname + '/views/partials');
// Set URL
app.set('base_url', base_url);
// Handle POST data
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
extended: true
}));

Next, we define a POST route to handle the username input:

// Process login
app.post('/login', function (req, res) {
// Get username
var username = req.body.username;
// If username length is zero, reload the page
if (username.length === 0) {
res.render('login');
} else {
// Store username in session and redirect to index
req.session.username = username;
res.redirect('/');
}
});

This should be fairly straightforward. This route accepts a username parameter. If this parameter is not present, the user will see the login form again. Otherwise, they are redirected back to the index.

Now, if you check coverage/index.html after running the tests again, you’ll notice that there’s a gap in our coverage for the scenario when a user submits an empty username. Let’s fix that - add the following test to test/test.js:

// Test empty login
describe('Test empty login', function () {
it('should show the login form', function (done) {
request.post({ url: 'http://localhost:5000/login',
form:{username: ''},
followRedirect: false},
function (error, response, body) {
expect(response.statusCode).to.equal(200);
expect(body).to.include('Please enter a handle');
done();
});
});
});

Let’s run our tests again:

$ npm test
> babblr@1.0.0 test /Users/matthewdaly/Projects/babblr
> grunt test --verbose
Initializing
Command-line options: --verbose
Reading "Gruntfile.js" Gruntfile...OK
Registering Gruntfile tasks.
Initializing config...OK
Registering "grunt-contrib-jshint" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-contrib-jshint/package.json...OK
Loading "jshint.js" tasks...OK
+ jshint
Registering "grunt-coveralls" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-coveralls/package.json...OK
Loading "coverallsTask.js" tasks...OK
+ coveralls
Registering "grunt-mocha-istanbul" local Npm module tasks.
Reading /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Parsing /Users/matthewdaly/Projects/babblr/node_modules/grunt-mocha-istanbul/package.json...OK
Loading "index.js" tasks...OK
+ istanbul_check_coverage, mocha_istanbul
Loading "Gruntfile.js" tasks...OK
+ test
Running tasks: test
Running "test" task
Running "jshint" task
Running "jshint:all" (jshint) task
Verifying property jshint.all exists in config...OK
Files: test/test.js, index.js -> all
Options: force=false, reporterOutput=null
OK
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Verifying property mocha_istanbul.coverage exists in config...OK
Files: test
Options: require=[], ui=false, globals=[], reporter=false, timeout=false, coverage=false, slow=false, grep=false, dryRun=false, quiet=false, recursive=false, mask="*.js", root=false, print=false, noColors=false, harmony=false, coverageFolder="coverage", reportFormats=["cobertura","html","lcovonly"], check={"statements":false,"lines":false,"functions":false,"branches":false}, excludes=false, mochaOptions=false, istanbulOptions=false
>> Will execute: node /Users/matthewdaly/Projects/babblr/node_modules/istanbul/lib/cli.js cover --dir=/Users/matthewdaly/Projects/babblr/coverage --report=cobertura --report=html --report=lcovonly /Users/matthewdaly/Projects/babblr/node_modules/mocha/bin/_mocha -- test/*.js
express-session deprecated undefined resave option; provide resave option index.js:9:1669
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1669
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (44ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test submitting to the login route
✓ should store the username in the session and redirect the user to the index
Test empty login
✓ should show the login form
Test sending a message
✓ should return 'Message received' (41ms)
Stopping the server
5 passing (145ms)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 54/54 ), 7 ignored
Branches : 100% ( 10/10 ), 2 ignored
Functions : 88.89% ( 8/9 )
Lines : 100% ( 54/54 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls" task
Running "coveralls:app" (coveralls) task
Verifying property coveralls.app exists in config...OK
Files: coverage/lcov.info
Options: src="coverage/lcov.info", force=false
Submitting file to coveralls.io: coverage/lcov.info
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

Our test now passes (bar, of course, Coveralls failing). Our next step is to actually do something with the session. Now, the request module we use in our test requires a third-party module called tough-cookie to work with cookies, so we need to install that:

$ npm install tough-cookie --save-dev

Next, amend the login test as follows:

// Test submitting to the login route
describe('Test submitting to the login route', function () {
it('should store the username in the session and redirect the user to the index', function (done) {
request.post({ url: 'http://localhost:5000/login',
form:{username: 'bobsmith'},
jar: true,
followRedirect: false},
function (error, response, body) {
expect(response.headers.location).to.equal('/');
expect(response.statusCode).to.equal(302);
// Check the username
request.get({ url: 'http://localhost:5000/', jar: true }, function (error, response, body) {
expect(body).to.include('bobsmith');
done();
});
});
});
});

Here we’re using a new parameter, namely jar - this tells request to store the cookies. We POST the username to the login form, and then we get the index route and verify that the username is shown in the request. Check the test fails, then amend the index route in index.js as follows:

// Define index route
app.get('/', function (req, res) {
// Get messages
client.lrange('chat:messages', 0, -1, function (err, messages) {
/* istanbul ignore if */
if (err) {
console.log(err);
} else {
// Get username
var username = req.session.username;
// Get messages
var message_list = [];
messages.forEach(function (message, i) {
/* istanbul ignore next */
message_list.push(message);
});
// Render page
res.render('index', { messages: message_list, username: username });
}
});
});

Note we get the username and pass it through to the view. We need to adapt the header view to display the username. Amend views/partials/header.hbs to look like this:

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>Babblr</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Place favicon.ico and apple-touch-icon.png in the root directory -->
<link rel="stylesheet" href="/bower_components/bootstrap/dist/css/bootstrap.min.css">
<link rel="stylesheet" href="/bower_components/bootstrap/dist/css/bootstrap-theme.min.css">
<link rel="stylesheet" href="/css/style.css">
</head>
<body>
<!--[if lt IE 7]>
<p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
<![endif]-->
<nav class="navbar navbar-inverse navbar-static-top" role="navigation">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#header-nav">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/">Babblr</a>
<div class="collapse navbar-collapse navbar-right" id="header-nav">
<ul class="nav navbar-nav">
{{#if username}}
<li><a href="/logout">Logged in as {{ username }}</a></li>
{{else}}
<li><a href="/login">Log in</a></li>
{{/if}}
</ul>
</div>
</div>
</div>
</nav>

Note the addition of a logout link, which we will implement later. Let’s check our tests pass:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
express-session deprecated undefined resave option; provide resave option index.js:9:1669
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1669
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (44ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test submitting to the login route
✓ should store the username in the session and redirect the user to the index
Test empty login
✓ should show the login form
Test sending a message
✓ should return 'Message received' (45ms)
Stopping the server
5 passing (156ms)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 55/55 ), 7 ignored
Branches : 100% ( 10/10 ), 2 ignored
Functions : 88.89% ( 8/9 )
Lines : 100% ( 55/55 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls:app" (coveralls) task
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.

Excellent! Next, let’s implement the test for our logout route:

// Test logout
describe('Test logout', function () {
it('should log the user out', function (done) {
request.post({ url: 'http://localhost:5000/login',
form:{username: 'bobsmith'},
jar: true,
followRedirect: false},
function (error, response, body) {
expect(response.headers.location).to.equal('/');
expect(response.statusCode).to.equal(302);
// Check the username
request.get({ url: 'http://localhost:5000/', jar: true }, function (error, response, body) {
expect(body).to.include('bobsmith');
// Log the user out
request.get({ url: 'http://localhost:5000/logout', jar: true }, function (error, response, body) {
expect(body).to.include('Log in');
done();
});
});
});
});
});

This is largely the same as the previous test, but adds some additional content at the end to test logging out afterwards. Let’s run the test:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
express-session deprecated undefined resave option; provide resave option index.js:9:1669
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option index.js:9:1669
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (536ms)
Test the login route
✓ should return a page with the text Please enter a handle
Test submitting to the login route
✓ should store the username in the session and redirect the user to the index
Test empty login
✓ should show the login form
Test logout
1) should log the user out
Test sending a message
✓ should return 'Message received' (49ms)
Stopping the server
5 passing (682ms)
1 failing
1) server Test logout should log the user out:
Uncaught AssertionError: expected 'Cannot GET /logout\n' to include 'Log in'
at Request._callback (/Users/matthewdaly/Projects/babblr/test/test.js:105:45)
at Request.self.callback (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:373:22)
at Request.emit (events.js:98:17)
at Request.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1318:14)
at Request.emit (events.js:117:20)
at IncomingMessage.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:1266:12)
at IncomingMessage.emit (events.js:117:20)
at _stream_readable.js:944:16
at process._tickCallback (node.js:442:13)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 55/55 ), 7 ignored
Branches : 100% ( 10/10 ), 2 ignored
Functions : 88.89% ( 8/9 )
Lines : 100% ( 55/55 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.

Now we have a failing test, let’s implement our logout route. Add the following route to index.js:

// Process logout
app.get('/logout', function (req, res) {
// Delete username from session
req.session.username = null;
// Redirect user
res.redirect('/');
});

If you run your tests again, they should now pass.

Now that we have the user’s name stored in the session, we can make use of it. First, let’s amend static/js/main.js so that it no longer adds a default username:

$(document).ready(function () {
'use strict';
// Set up the connection
var field, socket, output;
socket = io.connect(window.location.href);
// Get a reference to the input
field = $('textarea#message');
// Get a reference to the output
output = $('div.conversation');
// Handle message submit
$('a#submitbutton').on('click', function () {
// Create the message
var msg;
msg = field.val();
socket.emit('send', { message: msg });
field.val('');
});
// Handle incoming messages
socket.on('message', function (data) {
// Insert the message
output.append('<p>' + data + '</p>');
});
});

Then, in index.js, we need to declare a variable for our session middleware, which will be shared between Socket.IO and Express:

// Declare variables used
var app, base_url, bodyParser, client, express, hbs, io, port, RedisStore, rtg, session, sessionMiddleware, subscribe;

Then we amend the session setup to make it easier to reuse for Socket.IO:

// Set up session
sessionMiddleware = session({
store: new RedisStore({
client: client
}),
secret: 'blibble'
});
app.use(sessionMiddleware);

Towards the end of the file, before we set up our handlers for Socket.IO, we integrate our sessions:

// Integrate sessions
io.use(function(socket, next) {
sessionMiddleware(socket.request, socket.request.res, next);
});

Finally, we rewrite our session handlers to use the username from the session:

// Handle new messages
io.sockets.on('connection', function (socket) {
// Subscribe to the Redis channel
subscribe.subscribe('ChatChannel');
// Handle incoming messages
socket.on('send', function (data) {
// Define variables
var username, message;
// Get username
username = socket.request.session.username;
if (!username) {
username = 'Anonymous Coward';
}
message = username + ': ' + data.message;
// Publish it
client.publish('ChatChannel', message);
// Persist it to a Redis list
client.rpush('chat:messages', message);
});
// Handle receiving messages
var callback = function (channel, data) {
socket.emit('message', data);
};
subscribe.on('message', callback);
// Handle disconnect
socket.on('disconnect', function () {
subscribe.removeListener('message', callback);
});
});

Note here that when a message is sent, we get the username from the session, and if it’s empty, set it to Anonymous Coward. We then prepend it to the message, publish it, and persist it.

One final thing…

One last job remains. At present, users can pass JavaScript through in messages, which is not terribly secure! We need to fix it. Amend the send handler as follows:

// Handle incoming messages
socket.on('send', function (data) {
// Define variables
var username, message;
// Strip tags from message
message = data.message.replace(/<[^>]*>/g, '');
// Get username
username = socket.request.session.username;
if (!username) {
username = 'Anonymous Coward';
}
message = username + ': ' + message;
// Publish it
client.publish('ChatChannel', message);
// Persist it to a Redis list
client.rpush('chat:messages', message);
});

Here we use a regex to strip out any HTML tags from the message - this will prevent anyone injecting JavaScript into our chat client.

And that’s all, folks! If you want to check out the source for this lesson it’s in the repository on GitHub, tagged lesson-2. If you want to carry on working on this on your own, there’s still plenty you can do, such as:

  • Adding support for multiple rooms
  • Using Passport.js to allow logging in using third-party services such as Twitter or Facebook
  • Adding formatting for messages, either by using something like Markdown, or a client-side rich text editor

As you can see, it’s surprising how much you can accomplish using only Redis, and under certain circumstances it offers a lot of advantages over a relational database. It’s always worth thinking about whether Redis can be used for your project.

15th February 2015 6:11 pm

Switching to My Own Static Site Generator

As you may have seen if you’re visiting the site, I’ve finally switched over from Octopress to the static site generator I’ve been working on for the last few months. Apologies if you’re seeing lots of old posts in your RSS reader - there must have been an inconsistency between the RSS feed for this and that for Octopress.

I actually still really like Octopress, however I’m not and have never been a big fan of Ruby. Python and JavaScript are my two main go-to languages (although I do a lot of work professionally with PHP as well), so I wanted a solution in one of those languages, but I wanted something that was very similar to Octopress in every other way. I also wanted the facility to easily concatenate and minify static files as part of my deployment process to make the whole thing as lean as possible, so it made sense to build it as a Grunt plugin and create a Yeoman generator for building the boilerplate for the blog. Also, it’s always easier to work with your own code, and so using templates I wrote myself should make it quicker and easier for me to customise the blog how I want.

While deploying it did throw up a few errors that I’ve had to fix, it’s gone fairly smoothly and I’m pretty happy with it, although I will no doubt spend some time tweaking it over the next few weeks. It’s built with GitHub Pages in mind, but the fact that it’s built using Grunt should make it straightforward to switch to a different deployment method - during development I’ve actually used grunt-rsync to deploy to my Raspberry Pi and grunt-bitbucket-pages to deploy to Bitbucket in order to test it and both work absolutely fine. There are also Grunt plugins for deploying via FTP around, so if you want to check it out, then as long as you have at least some familiarity with Grunt you should be able to deploy it however you wish. The generator is meant to be only a starting point for your own site, so by all means check it out, tinker with the styling and templates, and make it your own. I will be very happy indeed if I see someone else using it in the wild.

Static site generators are generally somewhat harder to use than a CMS like WordPress, but they have many advantages:

  • Lighter - you can quite easily host a static site with just Nginx on a Raspberry Pi
  • Faster - with no database or actual dynamic content on the server, just flat HTML, your site will be far quicker to load than a WordPress blog
  • Cheaper to host
  • Easy to deploy - if your workflow is very command-line based like mine is, it’s very quick and easy to get blogging

If you can get away with using a static site generator rather than a database-driven blogging system, then it’s well worth doing so.

31st December 2014 2:10 pm

Building a Chat Server With Node.js and Redis

One of the more interesting capabilities Redis offers is its support for Pub/Sub. This allows you to subscribe to a specific channel, and then react when some content is published to that channel. In this tutorial, we’ll build a very simple web-based chat system that demonstrates Redis’s Pub/Sub support in action. Chat systems are pretty much synonymous with Node.js - it’s widely considered the “Hello, World!” of Node.js. Since we already used Node with the prior Redis tutorial, then it also makes sense to stick with it for this project too.

Installing Node.js

Since the last tutorial, I’ve discovered NVM, and if you’re using any flavour of Unix, I highly recommend using it. It’s not an option if you’re using Windows, however Redis doesn’t officially support Windows anyway, so if you want to follow along on a Windows machine I’d recommend using a VM.

If you followed the URL shortener tutorial, you should already have everything you need, though I’d still recommend switching to NVM as it’s very convenient. We’ll be using Grunt again, so you’ll need to make sure you have grunt-cli installed with the following command:

$ npm install -g grunt-cli

This assumes you used NVM to install Node - if it’s installed globally, you may need to use sudo.

Installing dependencies

As usual with a Node.js project, our first step is to create our package.json file:

$ npm init

Answer the questions so you end up with something like this (or just paste this into package.json and amend it as you see fit):

{
"name": "babblr",
"version": "1.0.0",
"description": "Chat client",
"main": "index.js",
"scripts": {
"test": "grunt test --verbose"
},
"keywords": [
"chat"
],
"author": "Matthew Daly <matthew@matthewdaly.co.uk> (http://matthewdaly.co.uk/)",
"license": "GPLv2"
}

Now let’s install our dependencies:

$ npm install express hbs redis hiredis socket.io socket.io-client --save
$ npm install chai grunt grunt-contrib-jshint grunt-coveralls grunt-mocha-istanbul istanbul mocha request --save-dev

These two commands will install our dependencies.

Now, if you followed on with the URL shortener tutorial, you’ll notice that we aren’t using Jade - instead we’re going to use Handlebars. Jade is quite a nice templating system, but I find it gets in the way for larger projects - you spend too much time looking up the syntax for things you already know in HTML. Handlebars is closer to HTML so we will use that. We’ll also use Socket.IO extensively on this project.

Support files

As before, we’ll also use Mocha for our unit tests and Istanbul to generate coverage stats. We’ll need a Grunt configuration for that, so here it is:

module.exports = function (grunt) {
'use strict';
grunt.initConfig({
jshint: {
all: [
'test/*.js',
'index.js'
]
},
mocha_istanbul: {
coverage: {
src: 'test', // the folder, not the files,
options: {
mask: '*.js',
reportFormats: ['cobertura', 'html', 'lcovonly']
}
}
},
coveralls: {
options: {
src: 'coverage/lcov.info',
force: false
},
app: {
src: 'coverage/lcov.info'
}
}
});
// Load tasks
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-coveralls');
grunt.loadNpmTasks('grunt-mocha-istanbul');
// Register tasks
grunt.registerTask('test', ['jshint', 'mocha_istanbul:coverage', 'coveralls']);
};

We also need a .bowerrc:

{
"directory": "static/bower_components"
}

And a bower.json:

{
"name": "babblr",
"main": "index.js",
"version": "1.0.0",
"authors": [
"Matthew Daly <matthewbdaly@gmail.com>"
],
"description": "A simple chat server",
"moduleType": [
"node"
],
"keywords": [
"chat"
],
"license": "GPLv2",
"homepage": "http://matthewdaly.co.uk",
"private": true,
"ignore": [
"**/.*",
"node_modules",
"bower_components",
"test",
"tests"
],
"dependencies": {
"html5-boilerplate": "~4.3.0",
"jquery": "~2.1.1",
"bootstrap": "~3.3.1"
}
}

Then install the Bower dependencies:

$ bower install

We also need a Procfile so we can run it on Heroku:

web: node index.js

Now, let’s create the main file:

$ touch index.js

And our test file:

$ mkdir test
$ touch test/test.js

Implementing the chat server

Next, let’s implement our first test. First of all, we’ll verify that the index route works:

/*jslint node: true */
/*global describe: false, before: false, after: false, it: false */
"use strict";
// Declare the variables used
var expect = require('chai').expect,
request = require('request'),
server = require('../index'),
redis = require('redis'),
io = require('socket.io-client'),
client;
client = redis.createClient();
// Server tasks
describe('server', function () {
// Beforehand, start the server
before(function (done) {
console.log('Starting the server');
done();
});
// Afterwards, stop the server and empty the database
after(function (done) {
console.log('Stopping the server');
client.flushdb();
done();
});
// Test the index route
describe('Test the index route', function () {
it('should return a page with the title Babblr', function (done) {
request.get({ url: 'http://localhost:5000/' }, function (error, response, body) {
expect(body).to.include('Babblr');
expect(response.statusCode).to.equal(200);
expect(response.headers['content-type']).to.equal('text/html; charset=utf-8');
done();
});
});
});
});

Note that this is very similar to the first test for the URL shortener, because it’s doing basically the same thing.

Now, run the test and make sure it fails:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
server
Starting the server
Test the index route
1) should return a page with the title Babblr
Stopping the server
0 passing (873ms)
1 failing
1) server Test the index route should return a page with the title Babblr:
Uncaught AssertionError: expected undefined to include 'Babblr'
at Request._callback (/Users/matthewdaly/Projects/babblr/test/test.js:34:33)
at self.callback (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:373:22)
at Request.emit (events.js:95:17)
at Request.onRequestError (/Users/matthewdaly/Projects/babblr/node_modules/request/request.js:971:8)
at ClientRequest.emit (events.js:95:17)
at Socket.socketErrorListener (http.js:1552:9)
at Socket.emit (events.js:95:17)
at net.js:441:14
at process._tickCallback (node.js:442:13)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 0/0 )
Branches : 100% ( 0/0 )
Functions : 100% ( 0/0 )
Lines : 100% ( 0/0 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.

With that confirmed, we can start writing code to make the test pass:

/*jslint node: true */
'use strict';
// Declare variables used
var app, base_url, client, express, hbs, io, port, rtg, subscribe;
// Define values
express = require('express');
app = express();
port = process.env.PORT || 5000;
base_url = process.env.BASE_URL || 'http://localhost:5000';
hbs = require('hbs');
// Set up connection to Redis
/* istanbul ignore if */
if (process.env.REDISTOGO_URL) {
rtg = require("url").parse(process.env.REDISTOGO_URL);
client = require("redis").createClient(rtg.port, rtg.hostname);
subscribe = require("redis").createClient(rtg.port, rtg.hostname);
client.auth(rtg.auth.split(":")[1]);
subscribe.auth(rtg.auth.split(":")[1]);
} else {
client = require('redis').createClient();
subscribe = require('redis').createClient();
}
// Set up templating
app.set('views', __dirname + '/views');
app.set('view engine', "hbs");
app.engine('hbs', require('hbs').__express);
// Register partials
hbs.registerPartials(__dirname + '/views/partials');
// Set URL
app.set('base_url', base_url);
// Define index route
app.get('/', function (req, res) {
res.render('index');
});
// Serve static files
app.use(express.static(__dirname + '/static'));
// Listen
io = require('socket.io')({
}).listen(app.listen(port));
console.log("Listening on port " + port);

If you compare this to the code for the URL shortener, you’ll notice a few fairly substantial differences. For one thing, we set up two Redis connections, not one - that’s because we need to do so when using Pub/Sub with Redis. You’ll also notice that we register Handlebars (hbs) rather than Jade, and define not just a directory for views, but another directory inside it for partials. Finally, setting it up to listen at the end is a bit more involved because we’ll be using Socket.IO.

Now, you can run your tests again at this point, but they won’t pass because we haven’t created our views. So let’s do that. Create the directory views and the subdirectory partials inside it. Then add the following content to views/index.hbs:

{{> header }}
<div class="container">
<div class="row">
<div class="col-md-8">
<div class="conversation">
</div>
</div>
<div class="col-md-4">
<form>
<div class="form-group">
<label for="message">Message</label>
<textarea class="form-control" id="message" rows="20"></textarea>
<a id="submitbutton" class="btn btn-primary form-control">Submit</a>
<div>
</form>
</div>
</div>
</div>
{{> footer }}

Add this to views/partials/header.hbs:

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>Babblr</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Place favicon.ico and apple-touch-icon.png in the root directory -->
<link rel="stylesheet" href="/bower_components/bootstrap/dist/css/bootstrap.min.css">
<link rel="stylesheet" href="/bower_components/bootstrap/dist/css/bootstrap-theme.min.css">
<link rel="stylesheet" href="/css/style.css">
</head>
<body>
<!--[if lt IE 7]>
<p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
<![endif]-->
<nav class="navbar navbar-inverse navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<a class="navbar-brand" href="#">Babblr</a>
</div>
</div>
</nav>

And add this to views/partials/footer.hbs:

<script src="/bower_components/jquery/dist/jquery.min.js"></script>
<script src="/bower_components/bootstrap/dist/js/bootstrap.min.js"></script>
<script src="/socket.io/socket.io.js"></script>
<script src="/js/main.js"></script>
</body>
</html>

You’ll also want to create placeholder CSS and JavaScript files:

$ mkdir static/js
$ mkdir static/css
$ touch static/js/main.js
$ touch static/css/style.css

The test should now pass:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (41ms)
Stopping the server
1 passing (54ms)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 24/24 ), 5 ignored
Branches : 100% ( 6/6 ), 1 ignored
Functions : 100% ( 1/1 )
Lines : 100% ( 24/24 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls:app" (coveralls) task
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.

Don’t worry about the coveralls task failing, as that only needs to pass when it runs on Travis CI.

So we now have our main route in place. The next step is to actually implement the chat functionality. Add this code to the test file:

// Test sending a message
describe('Test sending a message', function () {
it("should return 'Message received'", function (done) {
// Connect to server
var socket = io.connect('http://localhost:5000', {
'reconnection delay' : 0,
'reopen delay' : 0,
'force new connection' : true
});
// Handle the message being received
socket.on('message', function (data) {
expect(data).to.include('Message received');
socket.disconnect();
done();
});
// Send the message
socket.emit('send', { message: 'Message received' });
});
});

This code should be fairly straightforward to understand. First, we connect to the server. Then, we set up a handler to verify the content of the message when it gets sent. Finally, we send the message. Let’s run the tests to make sure we get the expected result:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (337ms)
Test sending a message
1) should return 'Message received'
Stopping the server
1 passing (2s)
1 failing
1) server Test sending a message should return 'Message received':
Error: timeout of 2000ms exceeded
at null.<anonymous> (/Users/matthewdaly/Projects/babblr/node_modules/mocha/lib/runnable.js:159:19)
at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 24/24 ), 5 ignored
Branches : 100% ( 6/6 ), 1 ignored
Functions : 100% ( 1/1 )
Lines : 100% ( 24/24 )
================================================================================
>>
Warning: Task "mocha_istanbul:coverage" failed. Use --force to continue.
Aborted due to warnings.

Now, let’s implement this functionality. Add this at the end of index.js:

// Handle new messages
io.sockets.on('connection', function (socket) {
// Subscribe to the Redis channel
subscribe.subscribe('ChatChannel');
// Handle incoming messages
socket.on('send', function (data) {
// Publish it
client.publish('ChatChannel', data.message);
});
// Handle receiving messages
var callback = function (channel, data) {
socket.emit('message', data);
};
subscribe.on('message', callback);
// Handle disconnect
socket.on('disconnect', function () {
subscribe.removeListener('message', callback);
});
});

We’ll go through this. First, we create a callback for when a new connection is received. Inside the callback, we then subscribe to a Pub/Sub channel in Redis called ChatChannel.

Then, we define another callback so that on a send event from Socket.IO, we get the message and publish it to ChatChannel. After that, we define another callback to handle receiving messages, and set it to run when a new message is published to ChatChannel. Finally, we set up a callback to handle removing the listener when a user disconnects.

Note the two different connections to Redis - client and subscribe. As mentioned earlier, you need to use two connections to Redis when using Pub/Sub. This is because a client subscribed to one or more channels should not issue commands, so we use subscribe as a dedicated connection to handle subscriptions, and use client to publish new messages.

We’ll also need a bit of client-side JavaScript to handle sending and receiving messages. Amend main.js as follows:

$(document).ready(function () {
'use strict';
// Set up the connection
var field, socket, output;
socket = io.connect(window.location.href);
// Get a reference to the input
field = $('textarea#message');
// Get a reference to the output
output = $('div.conversation');
// Handle message submit
$('a#submitbutton').on('click', function () {
// Create the message
var msg;
msg = field.val();
socket.emit('send', { message: msg });
field.val('');
});
// Handle incoming messages
socket.on('message', function (data) {
// Insert the message
output.append('<p>Anonymous Coward : ' + data + '</p>');
});
});

Here we have one callback that handles sending messages, and another that handles receiving messages. Note that every message will be preceded with Anonymous Coward - we won’t implement user names at this point (though I plan it for a future instalment).

We’ll also add a little bit of additional styling:

div.conversation {
height: 500px;
overflow-y: scroll;
border: 1px solid #000;
padding: 10px;
}

Now, if you run your tests, they should pass:

$ grunt test
Running "jshint:all" (jshint) task
>> 2 files lint free.
Running "mocha_istanbul:coverage" (mocha_istanbul) task
Listening on port 5000
server
Starting the server
Test the index route
✓ should return a page with the title Babblr (40ms)
Test sending a message
✓ should return 'Message received' (45ms)
Stopping the server
2 passing (101ms)
=============================================================================
Writing coverage object [/Users/matthewdaly/Projects/babblr/coverage/coverage.json]
Writing coverage reports at [/Users/matthewdaly/Projects/babblr/coverage]
=============================================================================
=============================== Coverage summary ===============================
Statements : 100% ( 33/33 ), 5 ignored
Branches : 100% ( 6/6 ), 1 ignored
Functions : 100% ( 5/5 )
Lines : 100% ( 33/33 )
================================================================================
>> Done. Check coverage folder.
Running "coveralls:app" (coveralls) task
>> Failed to submit 'coverage/lcov.info' to coveralls: Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}
>> Failed to submit coverage results to coveralls
Warning: Task "coveralls:app" failed. Use --force to continue.
Aborted due to warnings.

If you now run the following command:

$ node index.js

Then visit http://localhost:5000, you should be able to create new messages. If you then open it up in a second tab, you can see messages added in one tab appear in another. Deploying to Heroku using Redis To Go will be straightforward, and you can then access it from multiple devices and see new chat messages appear in real time.

Wrapping up

This illustrates just how straightforward it is to use Redis’s Pub/Sub capability. The chat system is still quite limited, so in a future instalment we’ll develop it further. You can get the source code from the Github repository - just switch to the lesson-1 tag.

28th December 2014 5:04 pm

My First Grunt Plugin

A while back, I mentioned that I’d written a Yeoman generator for creating a flat HTML blog, called generator-simple-static-blog. For this, I’d used the first Grunt plugin I could find for the purpose, which was grunt-markdown-blog. This worked, but I wasn’t really very happy with it.

The ideal Grunt plugin I had in mind was as follows:

  • Used Handlebars for templating
  • Generated posts from Markdown files
  • Saved files in named folders with a single index.html file in each one (like Octopress does) so that no file extension is visible on a page
  • Generated index pages, rather than just showing the latest post as the first page

Unfortunately, grunt-markdown-blog only fulfilled the second criteria, so it was never going to be something I stuck with long-term. However, I couldn’t find anything else that would do the trick, so it looked like my only option was to write a suitable plugin myself.

I started a new Git repository a while back, but didn’t make much progress. Then, on Christmas Eve, I suddenly got the urge to start working on this again, and in a matter of a few hours I’d gotten a working Grunt plugin that ticked all of these boxes. I had to delay getting it integrated into the generator due to Christmas day, and then an unfortunate bout of flu, but I’ve now published it as grunt-blogbuilder and amended the Yeoman generator to use it instead.

I’m really pleased with the outcome, and while I’m still not yet ready to migrate over to it from Octopress, it’s a massive step forward, and building a Grunt plugin has been an interesting experience.

Recent Posts

Mutation Testing With Infection

Switching from Vim to Neovim

Better Strings in PHP

Forcing SSL in Codeigniter

Logging to the ELK Stack With Laravel

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Django, Phonegap and Angular.js.