Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

11th September 2016 7:33 pm

Building a Phonegap App With Laravel and Angular - Part 1

A lot of my work over the last few years has been on Phonegap apps. Phonegap isn’t terribly hard to use, but the difference in context between that and a more conventional web app means that you have to move a lot of functionality to the client side, and unless you’ve used client-side Javascript frameworks before it can be a struggle.

In this series of tutorials I’ll show you how I might build a Phonegap app. The work involved will include:

  • Building a REST API using Laravel to expose the data
  • Building an admin interface to manage the data
  • Building a Phonegap app using Angular.js
  • Testing and deploying it

In the process we’ll cover issues like authentication, authorization, real-time notifications and working with REST APIs. Note that we won’t cover the app submission process - you can find plenty of resources on that. We will, however, be using Phonegap Build to build the app.

The brief

Let’s say our new client is an animal shelter. The brief for the app is as follows:

My New Animal Friend will be an app for finding a new pet. Once a user signs in, they’ll be able to choose what type of pet they’re looking for, then look through a list of pets available to adopt. They can reject them by swiping left or save them by swiping right. They can see more about the ones they swipe right on, and arrange to meet them, from within the app. Users can also message the staff to ask questions about a pet.

Nice idea, but there’s a lot of work involved! Our very first task is to build the REST API, since everything else relies on that. Before starting, make sure you have the following installed:

  • PHP (I’m using PHP 7, but 5.6 should be fine)
  • Composer
  • Git
  • A compatible relational database (I use PostgreSQL)
  • Redis
  • Your usual text editor
  • Node.js

As long as you have this, you should be ready to go. Using Homestead is the simplest way to get started if you don’t have all this stuff already.

Starting the API

To start building our REST API, run the following command from the shell:

$ composer create-project --prefer-dist laravel/laravel mynewanimalfriend-backend

We also have some other dependencies we need to install, so switch into the new directory and run the following command:

$ composer require barryvdh/laravel-cors tymon/jwt-auth predis/predis

Next, we need to add the new packages to the Laravel config. Open up config/app.php and add the following to the providers array:

Tymon\JWTAuth\Providers\JWTAuthServiceProvider::class,
Barryvdh\Cors\ServiceProvider::class,

And the following to the aliases array:

'JWTAuth' => Tymon\JWTAuth\Facades\JWTAuth::class,

We also need to ensure that the CORS middleware is applied to all API routes. Open up app/Http/Kernel.php and under the api array in protected $middlewareGroups paste the following:

   \Barryvdh\Cors\HandleCors::class,

Now that the packages are included, we can publish the files for them:

$ php artisan vendor:publish

Next, we need to set a key for our API authentication:

$ php artisan jwt:generate

And set a custom namespace:

$ php artisan app:name AnimalFriend

You’ll also want to set up the .env file with the configuration settings for your application. There’s one at .env.example by default that you can copy and customise. Then run the following command to generate the application key:

$ php artisan key:generate

I had to change the namespace for the user model in config/jwt.php as well:

'user' => 'AnimalFriend\User',

I also tend to amend the settings in phpunit.xml as follows so that it uses an in-memory SQLite database for tests:

<env name="APP_ENV" value="testing"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="QUEUE_DRIVER" value="sync"/>
<env name="CACHE_DRIVER" value="redis"/>
<env name="DB_CONNECTION" value="sqlite"/>
<env name="DB_DATABASE" value=":memory:"/>

Also, delete tests/ExampleTest.php and amend tests/TestCase.php as follows in order to use database migrations in tests:

<?php
use Illuminate\Foundation\Testing\DatabaseMigrations;
abstract class TestCase extends Illuminate\Foundation\Testing\TestCase
{
use DatabaseMigrations;
/**
* The base URL to use while testing the application.
*
* @var string
*/
protected $baseUrl = 'http://localhost';
/**
* Creates the application.
*
* @return \Illuminate\Foundation\Application
*/
public function createApplication()
{
$app = require __DIR__.'/../bootstrap/app.php';
$app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();
return $app;
}
}

With that in place, we can start work on our API proper.

Authenticating our API

We’re going to start out with a very limited subset of our API. First, we’ll implement the authentication for our app, then we’ll add the facility to view a list of pets or an individual pet. Other functionality will come later. This will be sufficient to get the app working.

First, we need to create our user model. As we’ll be practicing TDD throughout, we write a test for the user model first. Save the following as tests/UserModelTest.php:

<?php
use AnimalFriend\User;
class UserModelTest extends TestCase
{
/**
* Test creating a user
*
* @return void
*/
public function testCreatingAUser()
{
// Create a User
$user = factory(AnimalFriend\User::class)->create([
'name' => 'bobsmith',
'email' => 'bob@example.com',
]);
$this->seeInDatabase('users', ['email' => 'bob@example.com']);
// Verify it works
$saved = User::where('email', 'bob@example.com')->first();
$this->assertEquals($saved->id, 1);
$this->assertEquals($saved->name, 'bobsmith');
}
}

If we run the tests:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
. 1 / 1 (100%)
Time: 169 ms, Memory: 12.00MB
OK (1 test, 3 assertions)

We already have a perfectly good User model and the appropriate migrations, so our test already passes.

Next, we need to implement the authentication system. Save this as tests/AuthTest.php:

<?php
use Illuminate\Foundation\Testing\DatabaseMigrations;
class AuthTest extends TestCase
{
use DatabaseMigrations;
/**
* Test the auth
*
* @return void
*/
public function testAuth()
{
// Create a User
$user = factory(AnimalFriend\User::class)->create([
'name' => 'bobsmith',
'email' => 'bob@example.com',
'password' => bcrypt('password')
]);
// Create request
$data = array(
'email' => $user->email,
'password' => 'password',
);
$response = $this->call('POST', 'api/authenticate', $data);
$this->assertResponseStatus(200);
$content = json_decode($response->getContent());
$this->assertTrue(array_key_exists('token', $content));
}
/**
* Test the auth when user does not exist
*
* @return void
*/
public function testAuthFailure()
{
// Create data for request
$data = array(
'email' => 'user@example.com',
'password' => 'password',
);
$response = $this->call('POST', 'api/authenticate', $data);
// Check the status code
$this->assertResponseStatus(401);
}
}

The first test creates a user and sends an authentication request, then confirms that it returns the JSON Web Token. The second checks that a user that doesn’t exist cannot log in.

Let’s run the tests:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
FF. 3 / 3 (100%)
Time: 328 ms, Memory: 14.00MB
There were 2 failures:
1) AuthTest::testAuth
Expected status code 200, got 404.
Failed asserting that 404 matches expected 200.
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:648
/home/matthew/Projects/mynewanimalfriend-backend/tests/AuthTest.php:29
2) AuthTest::testAuthFailure
Expected status code 401, got 404.
Failed asserting that 404 matches expected 401.
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:648
/home/matthew/Projects/mynewanimalfriend-backend/tests/AuthTest.php:49
FAILURES!
Tests: 3, Assertions: 5, Failures: 2.

With a failing test in place, we can implement login. First let’s create our controller at app/Http/Controllers/AuthenticateController.php:

<?php
namespace AnimalFriend\Http\Controllers;
use Illuminate\Http\Request;
use AnimalFriend\Http\Requests;
use AnimalFriend\Http\Controllers\Controller;
use JWTAuth;
use Tymon\JWTAuth\Exceptions\JWTException;
use AnimalFriend\User;
use Hash;
class AuthenticateController extends Controller
{
private $user;
public function __construct(User $user) {
$this->user = $user;
}
public function authenticate(Request $request)
{
// Get credentials
$credentials = $request->only('email', 'password');
// Get user
$user = $this->user->where('email', $credentials['email'])->first();
try {
// attempt to verify the credentials and create a token for the user
if (! $token = JWTAuth::attempt($credentials)) {
return response()->json(['error' => 'invalid_credentials'], 401);
}
} catch (JWTException $e) {
// something went wrong whilst attempting to encode the token
return response()->json(['error' => 'could_not_create_token'], 500);
}
// all good so return the token
return response()->json(compact('token'));
}
}

And we need to set up the route in routes/api.php:

<?php
use Illuminate\Http\Request;
/*
|--------------------------------------------------------------------------
| API Routes
|--------------------------------------------------------------------------
|
| Here is where you can register API routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| is assigned the "api" middleware group. Enjoy building your API!
|
*/
Route::post('authenticate', 'AuthenticateController@authenticate');

Note that because it’s an API route, it’s automatically prefixed with api/ without us having to do anything.

Now if we run our tests, they should pass:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
... 3 / 3 (100%)
Time: 402 ms, Memory: 14.00MB
OK (3 tests, 6 assertions)

Now we can obtain a JSON Web Token to authenticate users with. To start with we’ll only support existing users, but later we’ll add a method to sign up. However, we need at least one user to test with, so we’ll create a seeder for that at database/seeds/UserTableSeeder.php:

<?php
use Illuminate\Database\Seeder;
use Carbon\Carbon;
class UserTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
// Add user
DB::table('users')->insert([
'name' => 'bobsmith',
'email' => 'bob@example.com',
'created_at' => Carbon::now(),
'updated_at' => Carbon::now(),
'password' => Hash::make("password")
]);
}
}

You can run php artisan make:seeder UserTableSeeder to generate the file, or just paste it in. You also need to amend database/seeds/DatabaseSeeder.php as follows:

<?php
use Illuminate\Database\Seeder;
class DatabaseSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
$this->call(UserTableSeeder::class);
}
}

This ensures the seeder will actually be called. Then, run the following commands:

$ php artisan migrate
$ php artisan db:seed

That sets up our user in the database.

Adding the Pets endpoint

Our next step is to add the pets model and endpoint. Our Pet model should have the following fields:

  • ID
  • Timestamps (created_at and updated_at)
  • Name
  • Path to photo
  • Availability
  • Type (eg cat, dog)

Let’s create a test for that model:

<?php
use AnimalFriend\Pet;
class PetModelTest extends TestCase
{
/**
* Test creating a pet
*
* @return void
*/
public function testCreatingAPet()
{
// Create a Pet
$pet = factory(AnimalFriend\Pet::class)->create([
'name' => 'Freddie',
'type' => 'Cat',
]);
$this->seeInDatabase('pets', ['type' => 'Cat']);
// Verify it works
$saved = Pet::where('name', 'Freddie')->first();
$this->assertEquals($saved->id, 1);
$this->assertEquals($saved->name, 'Freddie');
$this->assertEquals($saved->type, 'Cat');
$this->assertEquals($saved->available, 1);
$this->assertEquals($saved->picture, '1.jpg');
}
}

Save this as tests/PetModelTest.php. Then run the tests:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
..E. 4 / 4 (100%)
Time: 414 ms, Memory: 16.00MB
There was 1 error:
1) PetModelTest::testCreatingAUser
InvalidArgumentException: Unable to locate factory with name [default] [AnimalFriend\Pet].
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Database/Eloquent/FactoryBuilder.php:126
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php:2280
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Database/Eloquent/FactoryBuilder.php:139
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Database/Eloquent/FactoryBuilder.php:106
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Database/Eloquent/FactoryBuilder.php:84
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetModelTest.php:16
ERRORS!
Tests: 4, Assertions: 6, Errors: 1.

First we need to create a factory for creating a pet in database/factories/ModelFactory.php:

$factory->define(AnimalFriend\Pet::class, function (Faker\Generator $faker) {
return [
'name' => $faker->firstNameMale,
'type' => 'Cat',
'available' => 1,
'picture' => '1.jpg'
];
});

Then, we create the model:

$ php artisan make:model Pet

Next, we create a migration for the Pet model:

$ php artisan make:migration create_pets_table
Created Migration: 2016_09_11_145010_create_pets_table

And paste in the following code:

<?php
use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;
class CreatePetsTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('pets', function (Blueprint $table) {
$table->increments('id');
$table->string('name');
$table->string('type');
$table->string('available');
$table->string('picture')->nullable();
$table->timestamps();
});
}
/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
Schema::drop('pets');
}
}

Time to run the tests again:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
.... 4 / 4 (100%)
Time: 412 ms, Memory: 16.00MB
OK (4 tests, 12 assertions)

With that done, we can start work on implementing the endpoint. We need to check that unauthorised users cannot retrieve the data, and that authorised users can. First, let’s create tests/PetControllerTest.php:

<?php
use Illuminate\Foundation\Testing\DatabaseMigrations;
class PetControllerTest extends TestCase
{
use DatabaseMigrations;
/**
* Test fetching pets when unauthorised
*
* @return void
*/
public function testFetchingPetsWhenUnauthorised()
{
// Create a Pet
$pet = factory(AnimalFriend\Pet::class)->create([
'name' => 'Freddie',
'type' => 'Cat',
]);
$this->seeInDatabase('pets', ['type' => 'Cat']);
// Create request
$response = $this->call('GET', '/api/pets');
$this->assertResponseStatus(400);
}
/**
* Test fetching pets when authorised
*
* @return void
*/
public function testFetchingPets()
{
// Create a Pet
$pet = factory(AnimalFriend\Pet::class)->create([
'name' => 'Freddie',
'type' => 'Cat',
]);
$this->seeInDatabase('pets', ['type' => 'Cat']);
// Create a User
$user = factory(AnimalFriend\User::class)->create([
'name' => 'bobsmith',
'email' => 'bob@example.com',
]);
$this->seeInDatabase('users', ['email' => 'bob@example.com']);
// Create request
$token = JWTAuth::fromUser($user);
$headers = array(
'Authorization' => 'Bearer '.$token
);
// Send it
$this->json('GET', '/api/pets', [], $headers)
->seeJsonStructure([
'*' => [
'id',
'name',
'type',
'available',
'picture',
'created_at',
'updated_at'
]
]);
$this->assertResponseStatus(200);
}
}

First, we create a pet, make an HTTP request to /api/pets, and check we are not authorised. Next, we do the same, but also create a user and a JSON Web Token, and pass the token through in the request. Then we verify the response data and that it was successful.

Let’s run the tests:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
..FF.. 6 / 6 (100%)
Time: 509 ms, Memory: 16.00MB
There were 2 failures:
1) PetControllerTest::testFetchingPetsWhenUnauthorised
Expected status code 400, got 404.
Failed asserting that 404 matches expected 400.
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:648
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:25
2) PetControllerTest::testFetchingPets
Failed asserting that null is of type "array".
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:295
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:67
FAILURES!
Tests: 6, Assertions: 17, Failures: 2.

That looks correct, so we can start building our endpoint. We can generate a boilerplate for it as follows:

$ $ php artisan make:controller PetController --resource

Note the --resource flag - this tells Laravel to set it up to be a RESTful controller with certain predefined functions. Next, let’s amend the new file at app\Http\Controllers/PetController.php as follows:

<?php
namespace AnimalFriend\Http\Controllers;
use Illuminate\Http\Request;
use AnimalFriend\Http\Requests;
use AnimalFriend\Pet;
class PetController extends Controller
{
private $pet;
public function __construct(Pet $pet) {
$this->pet = $pet;
}
/**
* Display a listing of the resource.
*
* @return \Illuminate\Http\Response
*/
public function index()
{
// Get all pets
$pets = $this->pet->get();
// Send response
return response()->json($pets, 200);
}
}

This implements an index route that shows all pets. Next, we hook up the route in routes/api.php:

// Auth routes
Route::group(['middleware' => ['jwt.auth']], function () {
Route::resource('pets', 'PetController');
});

Note that we wrap this resource in the jwt.auth middleware to prevent access by unauthorised users. Implementing this as middleware makes it very easy to reuse. Also note that we can specify it as a resource, meaning we don’t have to explicitly hook up each route to a controller method.

Let’s run the tests again:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
..EE.. 6 / 6 (100%)
Time: 511 ms, Memory: 16.00MB
There were 2 errors:
1) PetControllerTest::testFetchingPetsWhenUnauthorised
ReflectionException: Class jwt.auth does not exist
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Container/Container.php:734
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Container/Container.php:629
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:709
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php:173
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:517
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:24
2) PetControllerTest::testFetchingPets
ReflectionException: Class jwt.auth does not exist
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Container/Container.php:734
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Container/Container.php:629
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:709
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php:173
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:517
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:72
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:56
ERRORS!
Tests: 6, Assertions: 15, Errors: 2.

Looks like JWT isn’t configured correctly. We can fix that in app/Http/Kernel.php by adding it to $routeMiddleware:

'jwt.auth' => 'Tymon\JWTAuth\Middleware\GetUserFromToken',
'jwt.refresh' => 'Tymon\JWTAuth\Middleware\RefreshToken',

And run the tests again:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
...... 6 / 6 (100%)
Time: 514 ms, Memory: 16.00MB
OK (6 tests, 25 assertions)

Our final task for today on the API is building a route for fetching a single pet. Our tests need to handle three situations:

  • An unauthorised request
  • A request for a pet that does not exist
  • A request for a pet that does exist

Add these methods to tests/PetControllerTest.php:

/**
* Test fetching pet when unauthorised
*
* @return void
*/
public function testFetchingPetWhenUnauthorised()
{
// Create a Pet
$pet = factory(AnimalFriend\Pet::class)->create([
'name' => 'Freddie',
'type' => 'Cat',
]);
$this->seeInDatabase('pets', ['type' => 'Cat']);
// Send request
$response = $this->call('GET', '/api/pets/'.$pet->id);
$this->assertResponseStatus(400);
}
/**
* Test fetching pet which does not exist
*
* @return void
*/
public function testFetchingPetDoesNotExist()
{
// Create a User
$user = factory(AnimalFriend\User::class)->create([
'name' => 'bobsmith',
'email' => 'bob@example.com',
]);
$this->seeInDatabase('users', ['email' => 'bob@example.com']);
// Create request
$token = JWTAuth::fromUser($user);
$headers = array(
'Authorization' => 'Bearer '.$token
);
// Send it
$this->json('GET', '/api/pets/1', [], $headers);
$this->assertResponseStatus(404);
}
/**
* Test fetching pet when authorised
*
* @return void
*/
public function testFetchingPet()
{
// Create a Pet
$pet = factory(AnimalFriend\Pet::class)->create([
'name' => 'Freddie',
'type' => 'Cat',
]);
$this->seeInDatabase('pets', ['type' => 'Cat']);
// Create a User
$user = factory(AnimalFriend\User::class)->create([
'name' => 'bobsmith',
'email' => 'bob@example.com',
]);
$this->seeInDatabase('users', ['email' => 'bob@example.com']);
// Create request
$token = JWTAuth::fromUser($user);
$headers = array(
'Authorization' => 'Bearer '.$token
);
// Send it
$this->json('GET', '/api/pets/'.$pet->id, [], $headers)
->seeJsonStructure([
'id',
'name',
'type',
'available',
'picture',
'created_at',
'updated_at'
]);
$this->assertResponseStatus(200);
}

Let’s check our tests fail:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
.....FE.. 9 / 9 (100%)
Time: 974 ms, Memory: 16.00MB
There was 1 error:
1) PetControllerTest::testFetchingPet
PHPUnit_Framework_Exception: Argument #2 (No Value) of PHPUnit_Framework_Assert::assertArrayHasKey() must be a array or ArrayAccess
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:304
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:145
--
There was 1 failure:
1) PetControllerTest::testFetchingPetDoesNotExist
Expected status code 404, got 400.
Failed asserting that 400 matches expected 404.
/home/matthew/Projects/mynewanimalfriend-backend/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:648
/home/matthew/Projects/mynewanimalfriend-backend/tests/PetControllerTest.php:112
ERRORS!
Tests: 9, Assertions: 31, Errors: 1, Failures: 1.

Now, we already have the show() method hooked up by default, so we just have to implement it in app/Http/Controllers/PetController.php:

/**
* Display the specified resource.
*
* @param int $id
* @return \Illuminate\Http\Response
*/
public function show($id)
{
// Get pet
$pet = $this->pet->findOrFail($id);
// Send response
return response()->json($pet, 200);
}

And let’s run our tests again:

$ vendor/bin/phpunit
PHPUnit 5.5.4 by Sebastian Bergmann and contributors.
......... 9 / 9 (100%)
Time: 693 ms, Memory: 16.00MB
OK (9 tests, 39 assertions)

Now we have all the endpoints we need to get started with the app. You can find the source code for this backend on Github - check out the lesson-1 tag.

That seems like a good place to stop for now. We have our first pass at the back end. It’s not complete by any means, but it’s a good start, and is sufficient for us to get some basic functionality up and running in the app. In the next instalment we’ll start working with Phonegap to build the first pass at the app itself. Later instalments will see us working with both the app and backend to build it into a more useful whole.

5th September 2016 10:22 pm

Deploying New Versions of a Laravel App With Fabric

Envoy is the official way to run tasks on a remote server for Laravel apps. A typical Envoy task for deploying a new version might look like this:

@servers(['web' => 'matthew@server1.example.com'])
@task('deploy', ['on' => 'web'])
cd /var/www
sudo chown -R matthew:matthew .
git pull origin master
php artisan migrate
php artisan view:clear
composer dump-autoload
sudo chown -R www-data:www-data .
sudo supervisorctl restart mail-queue
@endtask

This would be defined in Envoy.blade.php. With this in place, and Envoy set up globally, you can then run envoy run deploy to run the deploy command.

However, Envoy requires the PHP SSH library, which I haven’t been able to get working with PHP 7. Fortunately I was already familiar with Fabric, which makes an excellent alternative as long as you don’t mind writing the task in Python.

The same kind of task might look like this in a Fabric script, saved as fabfile.py:

#!/usr/bin/env python
from fabric.api import local, env, run, sudo
from fabric.context_managers import cd, prefix
env.hosts = ['server1.example.com']
env.path = "/var/www"
env.user = "matthew"
env.password = "password"
# Or...
env.key_filename = '/path/to/ssh/key'
def deploy():
"""
Deploy the latest version
"""
# Push changes to Bitbucket
local("git push origin master")
# Switch to project directory
with cd(env.path):
# Change owner
sudo('chown -R matthew:matthew .')
# Pull changes to server
run('git pull origin master')
# Run migrations
run('php artisan migrate')
# Clear cached files
run('php artisan view:clear')
run('composer dump-autoload')
# Change owner back
sudo('chown -R www-data:www-data .')
# restart mail queue
sudo('supervisorctl restart mail-queue')

Then, assuming Fabric is already installed locally, you can run fab deploy to push up the latest revision.

Either of these solutions will do a fine job of deploying your app. If you do need to store user-specific data in your Fabric script, it’s probably prudent to keep it out of version control.

Whichever way you choose, it’s a really good idea to do what you can to automate deployment. It can be a boring, repetitive job, and both of these solutions make it much easier.

29th August 2016 4:40 pm

Maintaining Your CV With Markdown and Emacs

I’ve recently been jobhunting, so that has meant having to update my CV. Fortunately, I’ve got into the habit of keeping it up to date easily by writing it in Markdown and generating it in the required format on demand. That way I can easily convert it to HTML, PDF or Microsoft DocX format as and when I need it. I thought I’d share this method as it works very well for me.

Maintaining your CV in Emacs?

Yes, you read that right! Although I’m a die-hard Vim user, I do use Emacs for a few things. One of them is time-tracking using org-mode, and another is maintaining my CV.

First of all you’ll need to install pandoc, texlive and markdown. On Ubuntu this is easily done using apt-get:

$ sudo apt-get install pandoc markdown texlive

You’ll also need to install Emacs and the appropriate packages, namely markdown-mode and markdown-mode+. To do so, first ensure this is in your .emacs.d/init.el:

(require 'package)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(package-initialize)
;; Markdown support
(require 'markdown-mode)
(require 'markdown-mode+)
(setq markdown-command "/usr/bin/markdown")
(add-to-list 'auto-mode-alist '("\\.markdown$" . markdown-mode))
(add-to-list 'auto-mode-alist '("\\.md$" . markdown-mode))
(setq markdown-css-paths `(,(expand-file-name "Documents/markdown.css")))

Then fire up Emacs, ignoring the warnings you get, and run M-x package-list-packages to load the list of available packages. I’ll leave navigating and installing this list of packages to you, but once they’re done you should have everything you need.

This assumes the stylesheet you wish to use is at ~/Documents/markdown.css - adjust the path if necessary. You may also need to amend the path to your Markdown install if the location differs. You can put what you like in the stylesheet, but my advice is to keep it as simple as you can - it’s your CV, not a web page. Here’s what I use:

body {
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
width: 80%;
margin: auto;
background: #ffffff;
padding: 10px;
}
h2 {
font-size: 30px;
color: #757575;
text-align: center;
margin-bottom: 15px;
}
h1 {
font-size: 55px;
color: #757575;
text-align: center;
margin-bottom: 15px;
}
hr {
color: #000000;
}
ul li {
list-style-type: disc;
}
blockquote {
text-align: center;
}
a, a:visited, a:hover {
text-decoration: none;
color: #000000;
}
code {
white-space: pre-wrap;
word-wrap: break-word;
}

Next, we write our CV in Markdown. Here’s a sample one based on mine:

James Smith
============
About me
--------
I'm a full-stack web developer. I have built a wide variety of web applications (including single-page web apps), content based sites and REST APIs.
---
Skills
----------
* HTML5
* CSS, Sass and Compass
* Javascript, including Angular.js
* PHP, including Laravel and Lumen
---
Employment
----------
**Agency ltd**
June 2014 - present
I worked for a busy digital agency, building custom web apps using Laravel and Angular.js
---
Education
----------
* **2009-2014 My Secondary School, London** - 7 GCSEs:
---
Hobbies and Interests
---------------------
Real ale, learning more about webdev, reading, socialising.
---
Contact
-------
> **Mobile:** 01234 567890
> **[Email](mailto:user@example.com)** - **[Website](http://www.example.com)** - **[GitHub](https://github.com/username)**

Now, if you save this file as something like cv.md and then open it up in Emacs, you should be able to preview it in your browser with C-c C-c p. Nice, huh? To export it to HTML, run C-c C-c v instead.

What if you want to view it in other formats? Say a potential employer is asking for your CV in Microsoft DocX format (ugh…)? Just run this command in the shell:

$ pandoc -s -S cv.md -o cv.docx

Or how about PDF?

$ pandoc -s -S cv.md -o cv.pdf

Using this method it’s straightforward to maintain a single master copy of your CV which you can then convert to other formats on demand.

Keeping your CV backed up

If you want to keep your CV safe, there’s a couple of ways to do it. One is to keep it in a Git or Mercurial repository, and another is to use Dropbox to keep it in sync. I tend to use the latter approach, although I’m considering switching to the former. If you wanted to generate the various versions automatically, you could set up a hook to generate the various versions using Pandoc during the commit process.

I used to hate updating my CV, but that was largely because I left it too long, and often had nothing much to put on it. Nowadays I’m often learning something new so I quite often have reason to update it to reflect that, and adopting this workflow has made things a lot easier.

15th August 2016 11:18 pm

Creating a Personal Dashboard With React and Webpack

The Raspberry Pi is a great device for running simple web apps at home on a permanent basis, and you can pick up a small touchscreen for it quite cheaply. This makes it easy to build and host a small personal dashboard that pulls important data from various APIs or RSS feeds and displays it. You’ll often see dashboards like this on Raspberry Pi forums and subreddits. As I’m currently between jobs, and have some time to spare before my new job starts, I decided to start creating my own version of it. It was obvious that React.js is a good fit for this as it allows you to break up your user interface into multiple independent components and keep the functionality close to the UI. It also makes it easy to reuse widgets by passing different parameters through each time.

In this tutorial I’ll show you how to start building a simple personal dashboard using React and Webpack. You can then install Nginx on your Raspberry Pi and host it from there. In the process, you’ll be able to pick up a bit of knowledge about Webpack and ECMAScript 2015 (using Babel). Our initial implementation will have only two widgets, a clock and a feed, but those should show you enough of the basics that you should then be able to build other widgets you may have in mind.

Installing our dependencies

First, let’s create our package.json:

$ npm init -y

Then install the dependencies:

$ npm install --save-dev babel-cli babel-register babel-core babel-eslint babel-loader babel-preset-es2015 babel-preset-react chai css-loader eslint eslint-loader eslint-plugin-react file-loader istanbul@^1.0.0-alpha.2 jquery jsdom mocha moment node-sass react react-addons-pure-render-mixin react-addons-test-utils react-dom react-hot-loader request sass-loader style-loader url-loader webpack webpack-dev-server

Note that we need to install a specific version of Istanbul to get code coverage.

Next, we create our Webpack config. Save this as webpack.config.js:

var webpack = require('webpack');
module.exports = {
entry: [
'webpack/hot/only-dev-server',
"./js/app.js"
],
debug: true,
devtool: 'source-map',
output: {
path: __dirname + '/static',
filename: "bundle.js"
},
module: {
preLoaders: [
{
test: /(\.js$|\.jsx$)/,
exclude: /node_modules/,
loader: "eslint-loader"
}
],
loaders: [
{ test: /\.jsx?$/, loaders: ['react-hot', 'babel'], exclude: /node_modules/ },
{ test: /\.js$/, exclude: /node_modules/, loader: 'babel-loader'},
{ test: /\.woff2?$/, loader: "url-loader?limit=25000" },
{ test: /\.(eot|svg|ttf)?$/, loader: "file-loader" },
{ test: /\.scss$/, loader: "style!css!sass" }
]
},
eslint: {
configFile: '.eslintrc.yml'
},
plugins: [
new webpack.HotModuleReplacementPlugin(),
new webpack.NoErrorsPlugin()
]
};

Note the various loaders we’re using. We use ESLint to lint our Javascript files for code quality, and the build will fail if they do not match the required standards. We’re also using loaders for CSS, Sass, Babel (so we can use ES2015 for our Javascript) and fonts. Also, note the hot module replacement plugin - this allows us to reload the application automatically. If you haven’t used Webpack before, this config should be sufficient to get you started, but I recommend reading the documentation.

We also need to configure ESLint how we want. Here is the configuration we will be using, which should be saved as .eslintrc.yml:

rules:
no-debugger:
- 0
no-console:
- 0
no-unused-vars:
- 0
indent:
- 2
- 2
quotes:
- 2
- single
linebreak-style:
- 2
- unix
semi:
- 2
- always
env:
es6: true
browser: true
node: true
extends: 'eslint:recommended'
parserOptions:
sourceType: module
ecmaFeatures:
jsx: true
experimentalObjectRestSpread: true
modules: true
plugins:
- react

We also need a base HTML file. Save this as index.html:

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Personal Dashboard</title>
</head>
<body>
<div id="view"></section>
<script src="bundle.js"></script>
</body>
</html>

We also need to set the commands for building and testing our app in package.json:

"scripts": {
"test": "istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'",
"test:watch": "npm run test -- --watch",
"start": "webpack-dev-server --progress --colors",
"build": "webpack --progress --colors"
},
"babel": {
"presets": [
"es2015",
"react"
]
},

The npm test command will call Mocha to run the tests, but will also use Istanbul to generate test coverage. For the sake of brevity, our tests won’t be terribly comprehensive. The npm start command will run a development server, while npm run build will build our application.

We also need to create the test/ folder and the test/setup.js file:

import jsdom from 'jsdom';
import chai from 'chai';
const doc = jsdom.jsdom('<!doctype html><html><body></body></html>');
const win = doc.defaultView;
global.document = doc;
global.window = win;
Object.keys(window).forEach((key) => {
if (!(key in global)) {
global[key] = window[key];
}
});

This sets up Chai and creates a dummy DOM for our tests. We also need to create the folder js/ and the file js/app.js. You can leave that file empty for now.

If you now run npm start and navigate to http://localhost:8080/webpack-dev-server/, you can see the current state of the application.

Our dashboard component

Our first React component will be a wrapper for all the other ones. Each of the rest of the components will be a self-contained widget that will populate itself without the need for a centralized data store like Redux. I will mention that Redux is a very useful library, and for larger React applications it makes a lot of sense to use it, but here we’re better off having each widget manage its own data internally, rather than have it be passed down from a single data store.

Save the following as test/components/dashboard.js:

import TestUtils from 'react-addons-test-utils';
import React from 'react';
import {findDOMNode} from 'react-dom';
import Dashboard from '../../js/components/dashboard';
import {expect} from 'chai';
const {renderIntoDocument, scryRenderedDOMComponentsWithClass, Simulate} = TestUtils;
describe('Dashboard', () => {
it('renders the dashboard', () => {
const component = renderIntoDocument(
<Dashboard title="My Dashboard" />
);
const title = findDOMNode(component.refs.title);
expect(title).to.be.ok;
expect(title.textContent).to.contain('My Dashboard');
});
}

This tests that we can set the title of our dashboard component. Let’s run our tests:

$ npm test
> personal-dashboard@1.0.0 test /home/matthew/Projects/personal-dashboard
> istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'
No coverage information was collected, exit without writing coverage information
module.js:327
throw err;
^
Error: Cannot find module '../../js/components/dashboard'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (dashboard.js:4:1)
at Module._compile (module.js:409:26)
at loader (/home/matthew/Projects/personal-dashboard/node_modules/babel-register/lib/node.js:148:5)
at Object.require.extensions.(anonymous function) [as .js] (/home/matthew/Projects/personal-dashboard/node_modules/babel-register/lib/node.js:158:7)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at /home/matthew/Projects/personal-dashboard/node_modules/mocha/lib/mocha.js:220:27
at Array.forEach (native)
at Mocha.loadFiles (/home/matthew/Projects/personal-dashboard/node_modules/mocha/lib/mocha.js:217:14)
at Mocha.run (/home/matthew/Projects/personal-dashboard/node_modules/mocha/lib/mocha.js:485:10)
at Object.<anonymous> (/home/matthew/Projects/personal-dashboard/node_modules/mocha/bin/_mocha:403:18)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Object.Module._extensions.(anonymous function) (/home/matthew/Projects/personal-dashboard/node_modules/istanbul/lib/hook.js:109:37)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
at runFn (/home/matthew/Projects/personal-dashboard/node_modules/istanbul/lib/command/common/run-with-cover.js:122:16)
at /home/matthew/Projects/personal-dashboard/node_modules/istanbul/lib/command/common/run-with-cover.js:251:17
at /home/matthew/Projects/personal-dashboard/node_modules/istanbul/lib/util/file-matcher.js:68:16
at /home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:52:16
at /home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:361:13
at /home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:52:16
at done (/home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:246:17)
at /home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:44:16
at /home/matthew/Projects/personal-dashboard/node_modules/async/lib/async.js:358:17
at LOOP (fs.js:1530:14)
at nextTickCallbackWith0Args (node.js:420:9)
at process._tickCallback (node.js:349:13)
npm ERR! Test failed. See above for more details.

Our dashboard file doesn’t exist. So let’s create it:

$ mkdir js/components
$ touch js/components/dashboard.js

And run our test again:

$ npm test
> personal-dashboard@1.0.0 test /home/matthew/Projects/personal-dashboard
> istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'
Dashboard
Warning: React.createElement: type should not be null, undefined, boolean, or number. It should be a string (for DOM elements) or a ReactClass (for composite components).
1) renders the dashboard
0 passing (31ms)
1 failing
1) Dashboard renders the dashboard:
Invariant Violation: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object.
at invariant (node_modules/fbjs/lib/invariant.js:38:15)
at [object Object].instantiateReactComponent [as _instantiateReactComponent] (node_modules/react/lib/instantiateReactComponent.js:86:134)
at [object Object].ReactCompositeComponentMixin.performInitialMount (node_modules/react/lib/ReactCompositeComponent.js:388:22)
at [object Object].ReactCompositeComponentMixin.mountComponent (node_modules/react/lib/ReactCompositeComponent.js:262:21)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at mountComponentIntoNode (node_modules/react/lib/ReactMount.js:105:32)
at ReactReconcileTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at batchedMountComponentIntoNode (node_modules/react/lib/ReactMount.js:126:15)
at ReactDefaultBatchingStrategyTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at Object.ReactDefaultBatchingStrategy.batchedUpdates (node_modules/react/lib/ReactDefaultBatchingStrategy.js:63:19)
at Object.batchedUpdates (node_modules/react/lib/ReactUpdates.js:98:20)
at Object.ReactMount._renderNewRootComponent (node_modules/react/lib/ReactMount.js:285:18)
at Object.ReactMount._renderSubtreeIntoContainer (node_modules/react/lib/ReactMount.js:371:32)
at Object.ReactMount.render (node_modules/react/lib/ReactMount.js:392:23)
at ReactTestUtils.renderIntoDocument (node_modules/react/lib/ReactTestUtils.js:85:21)
at Context.<anonymous> (dashboard.js:11:23)
No coverage information was collected, exit without writing coverage information
npm ERR! Test failed. See above for more details.

Now we have a failing test, we can create our component. Save this as js/components/dashboard.js:

import React from 'react';
export default React.createClass({
render() {
return (
<div className="dashboard">
<h1 ref="title">{this.props.title}</h1>
<div className="wrapper">
</div>
</div>
);
}
});

And let’s run our tests again:

$ npm test
> personal-dashboard@1.0.0 test /home/matthew/Projects/personal-dashboard
> istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'
Dashboard
✓ renders the dashboard
1 passing (50ms)
No coverage information was collected, exit without writing coverage information

Our first component is in place. However, it isn’t getting loaded. We also need to start thinking about styling. Create the file scss/style.scss, but leave it blank for now. Then save this in js/app.js:

import React from 'react';
import ReactDOM from 'react-dom';
import Dashboard from './components/dashboard';
import styles from '../scss/style.scss';
ReactDOM.render(
<Dashboard title="My Dashboard" />,
document.getElementById('view')
);

Note that we’re importing CSS or Sass files in the same way as Javascript files. This is unique to Webpack, and while it takes a bit of getting used to, it has its advantages - if you import only the styles relating to each component, you can be sure there’s no orphaned CSS files. Here, we only have one CSS file anyway, so it’s a non-issue.

If you now run npm start, our dashboard gets loaded and the title is displayed. With our dashboard in place, we can now implement our first widget.

Creating the clock widget

Our first widget will be a simple clock. This demonstrates changing the state of the widget on an interval. First let’s write a test - save this as test/components/clockwidget.js:

import TestUtils from 'react-addons-test-utils';
import React from 'react';
import {findDOMNode} from 'react-dom';
import ClockWidget from '../../js/components/clockwidget';
import {expect} from 'chai';
const {renderIntoDocument, scryRenderedDOMComponentsWithClass, Simulate} = TestUtils;
describe('Clock Widget', () => {
it('renders the clock widget', () => {
const currentTime = 1465160300530;
const component = renderIntoDocument(
<ClockWidget time={currentTime} />
);
const time = findDOMNode(component.refs.time);
expect(time).to.be.ok;
expect(time.textContent).to.contain('Sunday');
});
});

And create an empty file at js/components/clockwidget.js. Then we run our tests again:

$ npm test
> personal-dashboard@1.0.0 test /home/matthew/Projects/personal-dashboard
> istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'
Clock Widget
Warning: React.createElement: type should not be null, undefined, boolean, or number. It should be a string (for DOM elements) or a ReactClass (for composite components).
1) renders the clock widget
Dashboard
✓ renders the dashboard
1 passing (46ms)
1 failing
1) Clock Widget renders the clock widget:
Invariant Violation: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object.
at invariant (node_modules/fbjs/lib/invariant.js:38:15)
at [object Object].instantiateReactComponent [as _instantiateReactComponent] (node_modules/react/lib/instantiateReactComponent.js:86:134)
at [object Object].ReactCompositeComponentMixin.performInitialMount (node_modules/react/lib/ReactCompositeComponent.js:388:22)
at [object Object].ReactCompositeComponentMixin.mountComponent (node_modules/react/lib/ReactCompositeComponent.js:262:21)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at mountComponentIntoNode (node_modules/react/lib/ReactMount.js:105:32)
at ReactReconcileTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at batchedMountComponentIntoNode (node_modules/react/lib/ReactMount.js:126:15)
at ReactDefaultBatchingStrategyTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at Object.ReactDefaultBatchingStrategy.batchedUpdates (node_modules/react/lib/ReactDefaultBatchingStrategy.js:63:19)
at Object.batchedUpdates (node_modules/react/lib/ReactUpdates.js:98:20)
at Object.ReactMount._renderNewRootComponent (node_modules/react/lib/ReactMount.js:285:18)
at Object.ReactMount._renderSubtreeIntoContainer (node_modules/react/lib/ReactMount.js:371:32)
at Object.ReactMount.render (node_modules/react/lib/ReactMount.js:392:23)
at ReactTestUtils.renderIntoDocument (node_modules/react/lib/ReactTestUtils.js:85:21)
at Context.<anonymous> (clockwidget.js:12:23)
No coverage information was collected, exit without writing coverage information
npm ERR! Test failed. See above for more details.

With a failing test in place, we can create our component:

import React from 'react';
import moment from 'moment';
export default React.createClass({
getInitialState() {
return {
time: this.props.time || moment()
};
},
render() {
const time = moment(this.state.time).format('dddd, Do MMMM YYYY, h:mm:ss a');
return (
<div className="clockwidget widget">
<div className="widget-content">
<h2 ref="time">{time}</h2>
</div>
</div>
);
}
});

Note that the component accepts a property of time. The getInitialState() method then converts this.props.time into this.state.time so that it can be displayed on render. Note we also set a default of the current time using Moment.js.

We also need to update the dashboard component to load this new component:

import React from 'react';
import ClockWidget from './clockwidget';
export default React.createClass({
render() {
return (
<div className="dashboard">
<h1 ref="title">{this.props.title}</h1>
<div className="wrapper">
<ClockWidget />
</div>
</div>
);
}
});

Now, if you try running npm start and viewing the dashboard in the browser, you will see that it displays the current time and date, but it’s not being updated. You can force the page to reload every now and then, but we can do better than that. We can set an interval in which the time will refresh. As the smallest unit we show is seconds, this interval should be 1 second.

Amend the clock component as follows:

import React from 'react';
import moment from 'moment';
export default React.createClass({
getInitialState() {
return {
time: this.props.time || moment()
};
},
tick() {
this.setState({
time: moment()
});
},
componentDidMount() {
this.interval = setInterval(this.tick, 1000);
},
componentWillUnmount() {
clearInterval(this.interval);
},
render() {
const time = moment(this.state.time).format('dddd, Do MMMM YYYY, h:mm:ss a');
return (
<div className="clockwidget widget">
<div className="widget-content">
<h2 ref="time">{time}</h2>
</div>
</div>
);
}
});

When our component has mounted, we set an interval of 1,000 milliseconds, and each time it elapses we call the tick() method. This method sets the state to the current time, and as a result the user interface is automatically re-rendered. On unmount, we clear the interval.

In this case we’re just calling a single function on a set interval. In principle, the same approach can be used to populate components in other ways, such as by making an AJAX request.

Creating an RSS widget

Our next widget will be a simple RSS feed reader. We’ll fetch the content with jQuery and render it using React. We’ll also reload it regularly. First, let’s create our test:

import TestUtils from 'react-addons-test-utils';
import React from 'react';
import {findDOMNode} from 'react-dom';
import FeedWidget from '../../js/components/feedwidget';
import {expect} from 'chai';
const {renderIntoDocument, scryRenderedDOMComponentsWithClass, Simulate} = TestUtils;
describe('Feed Widget', () => {
it('renders the Feed widget', () => {
const url = "http://feeds.bbci.co.uk/news/rss.xml?edition=uk"
const component = renderIntoDocument(
<FeedWidget feed={url} size={5} delay={60} />
);
const feed = findDOMNode(component.refs.feed);
expect(feed).to.be.ok;
expect(feed.textContent).to.contain(url);
});
});

Our feed widget will accept an external URL as an argument, and will then poll this URL regularly to populate the feed. It also allows us to specify the size attribute, which denotes the number of feed items, and the delay attribute, which denotes the number of seconds it should wait before fetching the data again.

We also need to amend the dashboard component to include this widget:

import React from 'react';
import ClockWidget from './clockwidget';
import FeedWidget from './feedwidget';
export default React.createClass({
render() {
return (
<div className="dashboard">
<h1 ref="title">{this.props.title}</h1>
<div className="wrapper">
<ClockWidget />
<FeedWidget feed="http://feeds.bbci.co.uk/news/rss.xml?edition=uk" size="5" delay="60" />
</div>
</div>
);
}
});

If we then create js/components/feedwidget.js and run npm test:

$ npm test
> personal-dashboard@1.0.0 test /home/matthew/Projects/personal-dashboard
> istanbul cover _mocha -- --compilers js:babel-core/register --require ./test/setup.js 'test/**/*.@(js|jsx)'
Clock Widget
✓ renders the clock widget (92ms)
Dashboard
Warning: React.createElement: type should not be null, undefined, boolean, or number. It should be a string (for DOM elements) or a ReactClass (for composite components). Check the render method of `dashboard`.
1) renders the dashboard
Feed Widget
Warning: React.createElement: type should not be null, undefined, boolean, or number. It should be a string (for DOM elements) or a ReactClass (for composite components).
2) renders the Feed widget
1 passing (286ms)
2 failing
1) Dashboard renders the dashboard:
Invariant Violation: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object. Check the render method of `dashboard`.
at invariant (node_modules/fbjs/lib/invariant.js:38:15)
at instantiateReactComponent (node_modules/react/lib/instantiateReactComponent.js:86:134)
at instantiateChild (node_modules/react/lib/ReactChildReconciler.js:43:28)
at node_modules/react/lib/ReactChildReconciler.js:70:16
at traverseAllChildrenImpl (node_modules/react/lib/traverseAllChildren.js:69:5)
at traverseAllChildrenImpl (node_modules/react/lib/traverseAllChildren.js:85:23)
at traverseAllChildren (node_modules/react/lib/traverseAllChildren.js:164:10)
at Object.ReactChildReconciler.instantiateChildren (node_modules/react/lib/ReactChildReconciler.js:69:7)
at ReactDOMComponent.ReactMultiChild.Mixin._reconcilerInstantiateChildren (node_modules/react/lib/ReactMultiChild.js:194:41)
at ReactDOMComponent.ReactMultiChild.Mixin.mountChildren (node_modules/react/lib/ReactMultiChild.js:231:27)
at ReactDOMComponent.Mixin._createInitialChildren (node_modules/react/lib/ReactDOMComponent.js:715:32)
at ReactDOMComponent.Mixin.mountComponent (node_modules/react/lib/ReactDOMComponent.js:531:12)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at ReactDOMComponent.ReactMultiChild.Mixin.mountChildren (node_modules/react/lib/ReactMultiChild.js:242:44)
at ReactDOMComponent.Mixin._createInitialChildren (node_modules/react/lib/ReactDOMComponent.js:715:32)
at ReactDOMComponent.Mixin.mountComponent (node_modules/react/lib/ReactDOMComponent.js:531:12)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at [object Object].ReactCompositeComponentMixin.performInitialMount (node_modules/react/lib/ReactCompositeComponent.js:397:34)
at [object Object].ReactCompositeComponentMixin.mountComponent (node_modules/react/lib/ReactCompositeComponent.js:262:21)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at [object Object].ReactCompositeComponentMixin.performInitialMount (node_modules/react/lib/ReactCompositeComponent.js:397:34)
at [object Object].ReactCompositeComponentMixin.mountComponent (node_modules/react/lib/ReactCompositeComponent.js:262:21)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at mountComponentIntoNode (node_modules/react/lib/ReactMount.js:105:32)
at ReactReconcileTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at batchedMountComponentIntoNode (node_modules/react/lib/ReactMount.js:126:15)
at ReactDefaultBatchingStrategyTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at Object.ReactDefaultBatchingStrategy.batchedUpdates (node_modules/react/lib/ReactDefaultBatchingStrategy.js:63:19)
at Object.batchedUpdates (node_modules/react/lib/ReactUpdates.js:98:20)
at Object.ReactMount._renderNewRootComponent (node_modules/react/lib/ReactMount.js:285:18)
at Object.ReactMount._renderSubtreeIntoContainer (node_modules/react/lib/ReactMount.js:371:32)
at Object.ReactMount.render (node_modules/react/lib/ReactMount.js:392:23)
at ReactTestUtils.renderIntoDocument (node_modules/react/lib/ReactTestUtils.js:85:21)
at Context.<anonymous> (dashboard.js:11:23)
2) Feed Widget renders the Feed widget:
Invariant Violation: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object.
at invariant (node_modules/fbjs/lib/invariant.js:38:15)
at [object Object].instantiateReactComponent [as _instantiateReactComponent] (node_modules/react/lib/instantiateReactComponent.js:86:134)
at [object Object].ReactCompositeComponentMixin.performInitialMount (node_modules/react/lib/ReactCompositeComponent.js:388:22)
at [object Object].ReactCompositeComponentMixin.mountComponent (node_modules/react/lib/ReactCompositeComponent.js:262:21)
at Object.ReactReconciler.mountComponent (node_modules/react/lib/ReactReconciler.js:47:35)
at mountComponentIntoNode (node_modules/react/lib/ReactMount.js:105:32)
at ReactReconcileTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at batchedMountComponentIntoNode (node_modules/react/lib/ReactMount.js:126:15)
at ReactDefaultBatchingStrategyTransaction.Mixin.perform (node_modules/react/lib/Transaction.js:138:20)
at Object.ReactDefaultBatchingStrategy.batchedUpdates (node_modules/react/lib/ReactDefaultBatchingStrategy.js:63:19)
at Object.batchedUpdates (node_modules/react/lib/ReactUpdates.js:98:20)
at Object.ReactMount._renderNewRootComponent (node_modules/react/lib/ReactMount.js:285:18)
at Object.ReactMount._renderSubtreeIntoContainer (node_modules/react/lib/ReactMount.js:371:32)
at Object.ReactMount.render (node_modules/react/lib/ReactMount.js:392:23)
at ReactTestUtils.renderIntoDocument (node_modules/react/lib/ReactTestUtils.js:85:21)
at Context.<anonymous> (feedwidget.js:12:23)
=============================== Coverage summary ===============================
Statements : 83.33% ( 10/12 )
Branches : 50% ( 1/2 )
Functions : 66.67% ( 4/6 )
Lines : 83.33% ( 10/12 )
================================================================================
npm ERR! Test failed. See above for more details.

Our test fails, so we can start work on the widget proper. Here it is:

import React from 'react';
import jQuery from 'jquery';
window.jQuery = jQuery;
const FeedItem = React.createClass({
render() {
return (
<a href={this.props.link} target="_blank">
<li className="feeditem">{this.props.title}</li>
</a>
);
}
});
export default React.createClass({
getInitialState() {
return {
feed: [],
size: this.props.size || 5
};
},
componentDidMount() {
this.getFeed();
this.interval = setInterval(this.getFeed, (this.props.delay * 1000));
},
componentWillUnmount() {
clearInterval(this.interval);
},
getFeed() {
let that = this;
jQuery.ajax({
url: this.props.feed,
success: function (response) {
let xml = jQuery(response);
let feed = [];
xml.find('item').each(function () {
let item = {};
item.title = jQuery(this).find('title').text();
item.link = jQuery(this).find('guid').text();
feed.push(item);
});
that.setState({
feed: feed.slice(0,that.state.size)
});
}
});
},
render() {
let feedItems = this.state.feed.map(function (item, index) {
return (
<FeedItem title={item.title} link={item.link} key={item.link}></FeedItem>
);
});
return (
<div className="feedwidget widget">
<div className="widget-content">
<h2 ref="feed"> Fetched from {this.props.feed}</h2>
<ul>
{feedItems}
</ul>
</div>
</div>
);
}
});

This is by far the most complex component, so a little explanation is called for. We include jQuery as a dependency at the top of the file. Then we create a component for rendering an individual feed item, called FeedItem. This is very simple, consisting of an anchor tag wrapped around a list item. Note the use of the const keyword - in ES6 this denotes a constant.

Next, we move onto the feed widget proper. We set the initial state of the feed to be an empty array. Then, we define a componentDidMount() method that calls getFeed() and sets up an interval to call it again, based on the delay property. The getFeed() method fetches the URL in question and sets this.state.feed to an array of the most recent entries in the feed, with the size denoted by the size property passed through. We also clear that interval when the component is about to be umounted.

Note that you may have problems with the Access-Control-Allow-Origin HTTP header. It’s possible to disable this in your web browser, so if you want to run this as a dashboard you’ll probably need to do so. On Chrome there’s a useful plugin that allows you to disable this when needed.

Because our FeedWidget has been created in a generic manner, we can then include multiple feed widgets easily, as in this example:

import React from 'react';
import ClockWidget from './clockwidget';
import FeedWidget from './feedwidget';
export default React.createClass({
render() {
return (
<div className="dashboard">
<h1 ref="title">{this.props.title}</h1>
<div className="wrapper">
<ClockWidget />
<FeedWidget feed="http://feeds.bbci.co.uk/news/rss.xml?edition=uk" size="5" delay="60" />
<FeedWidget feed="https://www.sitepoint.com/feed/" size="10" delay="120" />
</div>
</div>
);
}
});

We also need to style our widgets. Save this as scss/_colours.scss:

$bgColour: #151515;
$txtColour: #cfcfcf;
$clockBg: #fa8c00;
$clockHoverBg: #0099ff;
$clockTxt: #fff;
$feedBg: #0099ff;
$feedTxt: #fff;
$feedHoverBg: #fa8c00;

And this as scss/style.scss:

@import 'colours';
html, body {
background-color: $bgColour;
color: $txtColour;
font-family: Arial, Helvetica, sans-serif;
}
div.dashboard {
padding: 10px;
}
div.wrapper {
-moz-column-count: 4;
-webkit-column-count: 4;
column-count: 4;
-moz-column-gap: 1em;
-webkit-column-gap: 1em;
column-gap: 1em;
}
div.widget {
display: inline-block;
margin: 0 0 1em;
width: 100%;
min-height: 100px;
margin: 5px;
opacity: 0.8;
transition: opacity 1s;
&:hover {
opacity: 1;
}
h2, h4 {
padding: 20px;
}
div.widget-content {
width: 100%;
}
}
div.clockwidget {
background-color: $clockBg;
color: $clockTxt;
}
div.feedwidget {
background-color: $feedBg;
color: $feedTxt;
h2 {
word-wrap: break-word;
}
ul {
margin-left: 0;
padding-left: 20px;
a {
text-decoration: none;
padding: 5px;
li {
list-style-type: none;
font-weight: bold;
color: $feedTxt;
}
}
}
}

The end result should look something like this:

The personal dashboard in action

With that done, feel free to add whatever other feeds you want to include.

Deploying our dashboard

The final step is deploying our dashboard to our Raspberry Pi or other device. Run the following command to generate the Javascript:

$ npm run build

This will create static/bundle.js. You can then copy that file over to your web server with index.html and place both files in the web root. I recommend using Nginx if you’re using a Raspberry Pi as it’s faster and simpler for static content. If you’re likely to make a lot of changes you might want to create a command in the scripts section of your package.json to deploy the files more easily.

These basic widgets should be enough to get you started. You should be able to use the feed widget with virtually any RSS feed, and you should be able to use a similar approach to poll third-party APIs, although you might need to authenticate in some way (if you do, you won’t want to expose your authentication details, so ensure that nobody from outside the network can view your application). I’ll leave it to you to see what kind of interesting widgets you come up with for your own dashboard, but some ideas to get you started include:

  • Public transport schedules/Traffic issues
  • Weather reports
  • Shopping lists/Todo lists, with HTML5 local storage used to persist them
  • Galleries of recent photos on social networks
  • Status of servers on cloud hosting providers

With a little thought, you can probably come up with a few more than that! I’ve created a Github repository with the source code so you can check your own implementation against it.

10th August 2016 8:45 pm

An Introduction to Managing Your Servers With Ansible

If, like me, you’re a web developer who sometimes also has to wear a sysadmin’s hat, then you’ll probably be coming across the same set of tasks each time you set up a new server. These may include:

  • Provisioning new servers on cloud hosting providers such as Digital Ocean
  • Setting up Cloudflare
  • Installing a web server, database and other required packages
  • Installing an existing web application, such as Wordpress
  • Configuring the firewall and Fail2ban
  • Keeping existing servers up to date

These can get tedious and repetitive fairly quickly - who genuinely wants to SSH into each server individually and run the updates regularly? Also, if done manually, there’s a danger of the setup for each server being inconsistent. Shell scripts will do this, but aren’t easy to read and not necessarily easy to adapt to different operating systems. You need a way to be able to manage multiple servers easily, maintain a series of reusable “recipes” and do it all in a way that’s straightforward to read - in other words, a configuration management system.

There are others around, such as Chef, Puppet, and Salt, but my own choice is Ansible. Here’s why I went for Ansible:

  • Playbooks and roles are defined as YAML, making them fairly straightforward to read and understand
  • It’s written in Python, making it easy to create your own modules that leverage existing Python modules to get things done
  • It’s distributed via pip, making it easy to install
  • It doesn’t require you to install anything new on the servers, so you can get started straight away as soon as you can access a new server
  • It has modules for interfacing with cloud services such as Digital Ocean and Amazon Web Services

Ansible is very easy to use, but you do still need to know what is actually going on to get the best out of it. It’s intended as a convenient abstraction on top of the underlying commands, not a replacement, and you should know how to do what you want to do manually before you write an Ansible playbook to do it.

Setting up

You need to have Python 2 available. Ansible doesn’t yet support Python 3 (Grr…) so if you’re using an operating system that has switched to Python 3, such as Arch Linux, you’ll need to have Python 2 installed as well. Assuming you have pip installed, then run this command to install it:

$ sudo pip install ansible

Or for users on systems with Python 3 as the main Python:

$ sudo pip2 install ansible

For Windows users, you’ll want to drop sudo. On Unix-like OS’s that don’t have sudo installed, drop it and run the command as root.

Our first Ansible command

We’ll demonstrate Ansible in action with a Vagrant VM. Drop the following Vagrantfile into your working directory:

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "debian/jessie64"
config.vm.network "forwarded_port", guest: 80, host: 8080
end

Then fire up the VM:

$ vagrant up

This VM will be our test bed for running Ansible. If you prefer, you can use a remote server instead.

Next, we’ll configure Ansible. Save this as ansible.cfg:

[defaults]
hostfile = inventory
remote_user = vagrant
private_key_file = .vagrant/machines/default/virtualbox/private_key

In this case the remote user is vagrant because we’re using Vagrant, but to manage remote machines you would need to change this to the name of the account that you use on the server. The value of private_key_file will also normally be something like /home/matthew/.ssh/id_rsa.pub, but here we’re using the Vagrant-specific key.

Note the hostfile entry - this points to the list of hosts you want to manage with Ansible. Let’s create this next. Save the following as inventory:

testserver ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222

Note that we explicitly need to set the port here because we’re using Vagrant. Normally it will default to port 22. A typical entry for a remote server might look like this:

example.com ansible_ssh_host=192.168.56.101

Note also that we can refer to hosts by the name we give it, which can be as meaningful (or not) as you want.

Let’s run our first command:

$ ansible all -m ping
testserver | SUCCESS => {
"changed": false,
"ping": "pong"
}

We called Ansible with the hosts set to all, therefore every host in the inventory was contacted. We used the -m flag to say we were calling a module, and then specified the ping module. Ansible therefore pinged each server in turn.

We can call ad-hoc commands using the -a flag, as in this example:

$ ansible all -a "uptime"
testserver | SUCCESS | rc=0 >>
17:26:57 up 19 min, 1 user, load average: 0.00, 0.04, 0.13

This command gets the uptime for the server. If you only want to run the command on a single server, you can specify it by name:

$ ansible testserver -a "uptime"
testserver | SUCCESS | rc=0 >>
17:28:21 up 20 min, 1 user, load average: 0.02, 0.04, 0.13

Here we specified the server as testserver. What about if you want to specify more than one server, but not all of them? You can create groups of servers in inventory, as in this example:

[webservers]
testserver ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222
example.com ansible_ssh_host=192.168.56.101

You could then call the following to run the uptime command on all the servers in the webservers group:

$ ansible webservers -a 'uptime'

If you want to run the command as a different user, you can do so:

$ ansible webservers -a 'uptime' -u bob

Note that for running uptime we haven’t specified the -m flag. This is because the command module is the default, but it’s very basic and doesn’t support shell variables. For more complex interactions you might need to use the shell module, as in this example:

$ ansible testserver -m shell -a 'echo $PATH'
testserver | SUCCESS | rc=0 >>
/usr/local/bin:/usr/bin:/bin:/usr/games

For installing a package on Debian or Ubuntu, you might use the apt module:

$ ansible testserver -m apt -a "name=git state=present" --become
testserver | SUCCESS => {
"cache_update_time": 0,
"cache_updated": false,
"changed": true,
"stderr": "",
"stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following extra packages will be installed:\n git-man liberror-perl\nSuggested packages:\n git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-arch git-cvs git-mediawiki git-svn\nThe following NEW packages will be installed:\n git git-man liberror-perl\n0 upgraded, 3 newly installed, 0 to remove and 83 not upgraded.\nNeed to get 4552 kB of archives.\nAfter this operation, 23.5 MB of additional disk space will be used.\nGet:1 http://httpredir.debian.org/debian/ jessie/main liberror-perl all 0.17-1.1 [22.4 kB]\nGet:2 http://httpredir.debian.org/debian/ jessie/main git-man all 1:2.1.4-2.1+deb8u2 [1267 kB]\nGet:3 http://httpredir.debian.org/debian/ jessie/main git amd64 1:2.1.4-2.1+deb8u2 [3262 kB]\nFetched 4552 kB in 1s (3004 kB/s)\nSelecting previously unselected package liberror-perl.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 32784 files and directories currently installed.)\r\nPreparing to unpack .../liberror-perl_0.17-1.1_all.deb ...\r\nUnpacking liberror-perl (0.17-1.1) ...\r\nSelecting previously unselected package git-man.\r\nPreparing to unpack .../git-man_1%3a2.1.4-2.1+deb8u2_all.deb ...\r\nUnpacking git-man (1:2.1.4-2.1+deb8u2) ...\r\nSelecting previously unselected package git.\r\nPreparing to unpack .../git_1%3a2.1.4-2.1+deb8u2_amd64.deb ...\r\nUnpacking git (1:2.1.4-2.1+deb8u2) ...\r\nProcessing triggers for man-db (2.7.0.2-5) ...\r\nSetting up liberror-perl (0.17-1.1) ...\r\nSetting up git-man (1:2.1.4-2.1+deb8u2) ...\r\nSetting up git (1:2.1.4-2.1+deb8u2) ...\r\n",
"stdout_lines": [
"Reading package lists...",
"Building dependency tree...",
"Reading state information...",
"The following extra packages will be installed:",
" git-man liberror-perl",
"Suggested packages:",
" git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk",
" gitweb git-arch git-cvs git-mediawiki git-svn",
"The following NEW packages will be installed:",
" git git-man liberror-perl",
"0 upgraded, 3 newly installed, 0 to remove and 83 not upgraded.",
"Need to get 4552 kB of archives.",
"After this operation, 23.5 MB of additional disk space will be used.",
"Get:1 http://httpredir.debian.org/debian/ jessie/main liberror-perl all 0.17-1.1 [22.4 kB]",
"Get:2 http://httpredir.debian.org/debian/ jessie/main git-man all 1:2.1.4-2.1+deb8u2 [1267 kB]",
"Get:3 http://httpredir.debian.org/debian/ jessie/main git amd64 1:2.1.4-2.1+deb8u2 [3262 kB]",
"Fetched 4552 kB in 1s (3004 kB/s)",
"Selecting previously unselected package liberror-perl.",
"(Reading database ... ",
"(Reading database ... 5%",
"(Reading database ... 10%",
"(Reading database ... 15%",
"(Reading database ... 20%",
"(Reading database ... 25%",
"(Reading database ... 30%",
"(Reading database ... 35%",
"(Reading database ... 40%",
"(Reading database ... 45%",
"(Reading database ... 50%",
"(Reading database ... 55%",
"(Reading database ... 60%",
"(Reading database ... 65%",
"(Reading database ... 70%",
"(Reading database ... 75%",
"(Reading database ... 80%",
"(Reading database ... 85%",
"(Reading database ... 90%",
"(Reading database ... 95%",
"(Reading database ... 100%",
"(Reading database ... 32784 files and directories currently installed.)",
"Preparing to unpack .../liberror-perl_0.17-1.1_all.deb ...",
"Unpacking liberror-perl (0.17-1.1) ...",
"Selecting previously unselected package git-man.",
"Preparing to unpack .../git-man_1%3a2.1.4-2.1+deb8u2_all.deb ...",
"Unpacking git-man (1:2.1.4-2.1+deb8u2) ...",
"Selecting previously unselected package git.",
"Preparing to unpack .../git_1%3a2.1.4-2.1+deb8u2_amd64.deb ...",
"Unpacking git (1:2.1.4-2.1+deb8u2) ...",
"Processing triggers for man-db (2.7.0.2-5) ...",
"Setting up liberror-perl (0.17-1.1) ...",
"Setting up git-man (1:2.1.4-2.1+deb8u2) ...",
"Setting up git (1:2.1.4-2.1+deb8u2) ..."
]
}

Here we specify that a particular package should be state=present or state=absent. Also, note the --become flag, which allows us to become root. If you’re using an RPM-based Linux distro, you can use the yum module in the same way.

Finally, let’s use the git module to check out a project on the server:

$ ansible testserver -m git -a "repo=https://github.com/matthewbdaly/django_tutorial_blog_ng.git dest=/home/vagrant/example version=HEAD"
testserver | SUCCESS => {
"after": "3542098e3b01103db4d9cfc724ba3c71c45cb314",
"before": null,
"changed": true,
"warnings": []
}

Here we check out a Git repository. We specify the repo, destination and version.

You can call any installed Ansible module in an ad-hoc fashion in the same way. Refer to the documentation for a list of modules.

Playbooks

Ad-hoc commands are useful, but they don’t offer much extra over using SSH. Playbooks allow you to define a repeatable set of commands for a particular use case. In this example, I’ll show you how to write a playbook that does the following:

  • Installs and configures Nginx
  • Clones the repository for my site into the web root

This is sufficiently complex to demonstrate some more of the functionality of Ansible, while also demonstrating playbooks in action.

Create a new folder called playbooks, and inside it save the following as sitecopy.yml:

---
- name: Copy personal website
hosts: testserver
become: True
tasks:
- name: Install Nginx
apt: name=nginx update_cache=yes
- name: Copy config
copy: >
src=files/nginx.conf
dest=/etc/nginx/sites-available/default
- name: Activate config
file: >
dest=/etc/nginx/sites-enabled/default
src=/etc/nginx/sites-available/default
state=link
- name: Delete /var/www directory
file: >
path=/var/www
state=absent
- name: Clone repository
git: >
repo=https://github.com/matthewbdaly/matthewbdaly.github.io.git
dest=/var/www
version=HEAD
- name: Restart Nginx
service: name=nginx state=restarted

Note the name fields - these are comments that will show up in the output when each step is run. First we use the apt module to install Nginx, then we copy over the config file and activate it, then we empty the existing /var/www and clone the repository, and finally we restart Nginx.

Also, note the following fields:

  • hosts defines the hosts affected
  • become specifies that the commands are run using sudo

We also need to create the config for Nginx. Create the files directory under playbooks and save this file as playbooks/files/nginx.conf:

server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /var/www;
index index.html index.htm;
server_name localhost;
location / {
try_files $uri $uri/ =404;
}
}

Obviously if your Nginx config will be different, feel free to amend it as necessary. Finally, we run the playbook using the ansible-playbook command:

$ ansible-playbook playbooks/sitecopy.yml
PLAY [Copy personal website] ***************************************************
TASK [setup] *******************************************************************
ok: [testserver]
TASK [Install Nginx] ***********************************************************
changed: [testserver]
TASK [Copy config] *************************************************************
changed: [testserver]
TASK [Activate config] *********************************************************
changed: [testserver]
TASK [Delete /var/www directory] ***********************************************
changed: [testserver]
TASK [Clone repository] ********************************************************
changed: [testserver]
TASK [Restart Nginx] ***********************************************************
changed: [testserver]
PLAY RECAP *********************************************************************
testserver : ok=7 changed=6 unreachable=0 failed=0

If we had a playbook that we wanted to run on only a subset of the hosts it applied to, we could use the -l flag, as in this example:

$ ansible-playbook playbooks/sitecopy.yml -l testserver

Using these same basic concepts, you can invoke many different Ansible modules to achieve many different tasks. You can spin up new servers on supported cloud hosting companies, you can set up a known good fail2ban config, you can configure your firewall, and many more tasks. As your playbooks get bigger, it’s worth moving sections into separate roles that get invoked within multiple playbooks, in order to reduce repetition.

Finally, I mentioned earlier that you can use Ansible to update all of your servers regularly. Here’s the playbook I use for that:

---
- name: Update system
hosts: all
become: True
tasks:
- name: update system
apt: upgrade=full update_cache=yes

This connects to all hosts using the all shortcut we saw earlier, and upgrades all existing packages. Using this method is a lot easier than connecting to each one in turn via SSH and updating it manually.

Summary

Ansible is an extremely useful tool for managing servers, but to get the most out of it you have to put in a fair bit of work reading the documentation and writing your own playbooks for your own use cases. It’s simple to get started with, and if you’re willing to put in the time writing your own playbooks then in the long run you’ll save yourself a lot of time and grief by making it easy to set up new servers and administer existing ones. Hopefully this has given you a taster of what you can do with Ansible - from here on the documentation is worth a look as it lists all of the modules that ship with Ansible. If there’s a particular task you dread, such as setting up a mail server, then Ansible is a very good way to automate that away so it’s easier next time.

My experience is that it’s best to make an effort to try to standardise on two or three different stacks for different purposes, and create Ansible playbooks for those stacks. For instance, I’ve tended to use PHP 5, Apache, MySQL, Memcached and Varnish for Wordpress sites, and PHP 7, Nginx, Redis and PostgreSQL for Laravel sites. That way I know that any sites I build with Laravel will be using that stack. Knowing my servers are more consistent makes it easier to work with them and identify problems.

Recent Posts

Enforcing a Coding Standard With PHP Codesniffer

Decorating Laravel Repositories

My First Laravel Package

Integrating Behat With Laravel

Testing Laravel Middleware

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Django, Phonegap and Angular.js.