Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

22nd January 2018 12:00 pm

Deploying Your Laravel Application With Deployer

Deployment processes have a nasty tendency to be a mish-mash of cobbled-together scripts or utilities in many web shops, with little or no consistency in practice between them. As a result, it’s all too easy for even the most experienced developer to mess up a deployment.

I personally have used all kinds of bodged-together solutions. For a while I used Envoy scripts to deploy my Laravel apps, but then there was an issue with the SSH library in PHP 7 that made it impractical to use it. Then I adopted Fabric, which I’d used before for deploying Django apps and will do fine for deploying PHP apps too, but it wasn’t much more sophisticated than using shell scripts for deployment purposes. There are third-party services like Deploybot, but these are normally quite expensive for what they are.

A while back I heard of Deployer, but I didn’t have the opportunity to try it until recently on a personal project as I was working somewhere that had its own in-house deployment process. It’s a PHP-specific deployment tool with recipes for deploying applications built with various frameworks and CMS’s, including Laravel, Symfony, CodeIgniter and Drupal.

Installing Deployer

Deployer is installed as a .phar file, much like you would with Composer:

$ curl -LO https://deployer.org/deployer.phar
$ mv deployer.phar /usr/local/bin/dep
$ chmod +x /usr/local/bin/dep

With that done, you should be able to run the following command in your project’s directory to create a Deployer script:

$ dep init

In response, you should see a list of project types:

Welcome to the Deployer config generator
This utility will walk you through creating a deploy.php file.
It only covers the most common items, and tries to guess sensible defaults.
Press ^C at any time to quit.
Please select your project type [Common]:
[0] Common
[1] Laravel
[2] Symfony
[3] Yii
[4] Yii2 Basic App
[5] Yii2 Advanced App
[6] Zend Framework
[7] CakePHP
[8] CodeIgniter
[9] Drupal
>

Here I chose Laravel as I was deploying a Laravel project. I was then prompted for the repository URL - this will be filled in with the origin remote if the current folder is already a Git repository:

Repository [git@gitlab.com:Group/Project.git]:
>

You’ll also see a message about contributing anonymous usage data. After answering this, the file deploy.php will be generated:

<?php
namespace Deployer;
require 'recipe/laravel.php';
// Configuration
set('repository', 'git@gitlab.com:Group/Project.git');
set('git_tty', true); // [Optional] Allocate tty for git on first deployment
add('shared_files', []);
add('shared_dirs', []);
add('writable_dirs', []);
// Hosts
host('project.com')
->stage('production')
->set('deploy_path', '/var/www/project.com');
host('beta.project.com')
->stage('beta')
->set('deploy_path', '/var/www/project.com');
// Tasks
desc('Restart PHP-FPM service');
task('php-fpm:restart', function () {
// The user must have rights for restart service
// /etc/sudoers: username ALL=NOPASSWD:/bin/systemctl restart php-fpm.service
run('sudo systemctl restart php-fpm.service');
});
after('deploy:symlink', 'php-fpm:restart');
// [Optional] if deploy fails automatically unlock.
after('deploy:failed', 'deploy:unlock');
// Migrate database before symlink new release.
before('deploy:symlink', 'artisan:migrate');

By default it has two hosts, beta and production, and you can refer to them by these names. You can also add or remove hosts, and amend the existing ones. Note the deploy path as well - this sets the place where the application gets deployed to.

Note that it’s set up to expect the server to be using PHP-FPM and Nginx by default, so if you’re using Apache you may need to amend the command to restart the server. Also, note that if like me you’re using PHP 7 on a distro like Debian that also has PHP 5 around, you’ll probably need to change the references to php-fpm as follows:

desc('Restart PHP-FPM service');
task('php-fpm:restart', function () {
// The user must have rights for restart service
// /etc/sudoers: username ALL=NOPASSWD:/bin/systemctl restart php-fpm.service
run('sudo systemctl restart php7.0-fpm.service');
});
after('deploy:symlink', 'php-fpm:restart');

You will also need to make sure the acl package is installed - on Debian and Ubuntu you can install it as follows:

$ sudo apt-get install acl

Now, the recipe for deploying a Laravel app will include the following:

  • Pulling from the Git remote
  • Updating any Composer dependencies to match composer.json
  • Running the migrations
  • Optimizing the application

In addition, one really great feature Deployer offers is rollbacks. Rather than checking out your application directly into the project root you specify, it numbers each release and deploys it in a separate folder, before symlinking that folder to the project root as current. That way, if a release cannot be deployed successfully, rather than leaving your application in an unfinished state, Deployer will symlink the previous version so that you still have a working version of your application.

If you have configured Deployer for that project, you can deploy using the following command where production is the name of the host you’re deploying to:

$ dep deploy production

The output will look something like this:

✔ Executing task deploy:prepare
✔ Executing task deploy:lock
✔ Executing task deploy:release
➤ Executing task deploy:update_code
Counting objects: 761, done.
Compressing objects: 100% (313/313), done.
Writing objects: 100% (761/761), done.
Total 761 (delta 384), reused 757 (delta 380)
Connection to linklater.shellshocked.info closed.
✔ Ok
✔ Executing task deploy:shared
✔ Executing task deploy:vendors
✔ Executing task deploy:writable
✔ Executing task artisan:storage:link
✔ Executing task artisan:view:clear
✔ Executing task artisan:cache:clear
✔ Executing task artisan:config:cache
✔ Executing task artisan:optimize
✔ Executing task artisan:migrate
✔ Executing task deploy:symlink
✔ Executing task php-fpm:restart
✔ Executing task deploy:unlock
✔ Executing task cleanup
✔ Executing task success
Successfully deployed!

As you can see, we first of all lock the application and pull the latest version from the Git remote. Next we copy the files shared between releases (eg the .env file, the storage/ directory etc), update the dependencies, and make sure the permissions are correct. Next we link the storage, clear all the cached content, optimize our app, and migrate the database, before we set up the symlink. Finally we restart the web server and unlock the application.

In the event you discover a problem after deploy and need to rollback manually, you can do so with the following command:

$ dep rollback production

That makes it easy to ensure that in the event of something going wrong, you can quickly switch back to an earlier version with zero downtime.

Deployer has made deployments a lot less painful for me than any other solution I’ve tried. The support for rollbacks means that if something goes wrong it’s trivial to switch back to an earlier revision.

12th January 2018 1:16 pm

Creating a Caching User Provider for Laravel

If you have a Laravel application that requires users to log in and you use Clockwork or Laravel DebugBar to examine the queries that take place, you’ll probably notice a query that fetches the user model occurs quite a lot. This is because the user’s ID gets stored in the session, and is then used to retrieve the model.

This query is a good candidate for caching because not only is that query being made often, but it’s also not something that changes all that often. If you’re careful, it’s quite easy to set your application up to cache the user without having to worry about invalidating the cache.

Laravel allows you to define your own user providers in order to fetch the user’s details. These must implement Illuminate\Contracts\Auth\UserProvider and must return a user model from the identifier provided. Out of the box it comes with two implementations, Illuminate\Auth\EloquentUserProvider and Illuminate\Auth\DatabaseUserProvider, with the former being the default. Our caching user provider can extend the Eloquent one as follows:

<?php
namespace App\Auth;
use Illuminate\Auth\EloquentUserProvider;
use Illuminate\Contracts\Cache\Repository;
use Illuminate\Contracts\Hashing\Hasher as HasherContract;
class CachingUserProvider extends EloquentUserProvider
{
/**
* The cache instance.
*
* @var Repository
*/
protected $cache;
/**
* Create a new database user provider.
*
* @param \Illuminate\Contracts\Hashing\Hasher $hasher
* @param string $model
* @param Repository $cache
* @return void
*/
public function __construct(HasherContract $hasher, $model, Repository $cache)
{
$this->model = $model;
$this->hasher = $hasher;
$this->cache = $cache;
}
/**
* Retrieve a user by their unique identifier.
*
* @param mixed $identifier
* @return \Illuminate\Contracts\Auth\Authenticatable|null
*/
public function retrieveById($identifier)
{
return $this->cache->tags($this->getModel())->remember('user_by_id_'.$identifier, 60, function () use ($identifier) {
return parent::retrieveById($identifier);
});
}
}

Note that we override the constructor to accept a cache instance as well as the other arguments. We also override the retrieveById() method to wrap a call to the parent’s implementation inside a callback that caches the response. I usually tag anything I cache with the model name, but if you need to use a cache backend that doesn’t support tagging this may not be an option. Our cache key also includes the identifier so that it’s unique to that user.

We then need to add our user provider to the auth service provider:

<?php
namespace App\Providers;
use Illuminate\Support\Facades\Gate;
use Illuminate\Foundation\Support\Providers\AuthServiceProvider as ServiceProvider;
use App\Auth\CachingUserProvider;
use Illuminate\Support\Facades\Auth;
class AuthServiceProvider extends ServiceProvider
{
/**
* Register any authentication / authorization services.
*
* @return void
*/
public function boot()
{
$this->registerPolicies();
Auth::provider('caching', function ($app, array $config) {
return new CachingUserProvider(
$app->make('Illuminate\Contracts\Hashing\Hasher'),
$config['model'],
$app->make('Illuminate\Contracts\Cache\Repository')
);
});
}
}

Note here that we call this provider caching, and we pass it the hasher, the model name, and an instance of the cache. Then, we need to update config/auth.php to use this provider:

'providers' => [
'users' => [
'driver' => 'caching',
'model' => App\Eloquent\Models\User::class,
],
],

The only issue now is that our user models will continue to be cached, even when they are updated. To be able to flush the cache, we can create a model event that fires whenever the user model is updated:

<?php
namespace App\Eloquent\Models;
use Illuminate\Notifications\Notifiable;
use Illuminate\Foundation\Auth\User as Authenticatable;
use App\Events\UserAmended;
class User extends Authenticatable
{
use Notifiable;
protected $dispatchesEvents = [
'saved' => UserAmended::class,
'deleted' => UserAmended::class,
'restored' => UserAmended::class,
];
}

This will call the UserAmended event when a user model is created, updated, deleted or restored. Then we can define that event:

<?php
namespace App\Events;
use Illuminate\Broadcasting\Channel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Broadcasting\PresenceChannel;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use App\Eloquent\Models\User;
class UserAmended
{
use Dispatchable, InteractsWithSockets, SerializesModels;
/**
* Create a new event instance.
*
* @return void
*/
public function __construct(User $model)
{
$this->model = $model;
}
}

Note our event contains an instance of the user model. Then we set up a listener to do the work of clearing the cache:

<?php
namespace App\Listeners;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use App\Events\UserAmended;
use Illuminate\Contracts\Cache\Repository;
class ClearUserId
{
/**
* Create the event listener.
*
* @return void
*/
public function __construct(Repository $cache)
{
$this->cache = $cache;
}
/**
* Handle the event.
*
* @param object $event
* @return void
*/
public function handle(UserAmended $event)
{
$this->cache->tags(get_class($event->model))->forget('user_by_id_'.$event->model->id);
}
}

Here, we get the user model’s class again, and clear the cache entry for that user model.

Finally, we hook up the event and listener in the event service provider:

<?php
namespace App\Providers;
use Illuminate\Support\Facades\Event;
use Illuminate\Foundation\Support\Providers\EventServiceProvider as ServiceProvider;
class EventServiceProvider extends ServiceProvider
{
/**
* The event listener mappings for the application.
*
* @var array
*/
protected $listen = [
'App\Events\UserAmended' => [
'App\Listeners\ClearUserId',
],
];
/**
* Register any events for your application.
*
* @return void
*/
public function boot()
{
parent::boot();
//
}
}

With that done, our user should be cached after the first load, and flushed when the model is amended.

Handling eager-loaded data

It may be that you’re pulling in additional data from the user model in your application, such as roles, permissions, or a separate profile model. Under those circumstances it makes sense to treat that data in the same way by eager-loading it along with your user model.

<?php
namespace App\Auth;
use Illuminate\Auth\EloquentUserProvider;
use Illuminate\Contracts\Cache\Repository;
use Illuminate\Contracts\Hashing\Hasher as HasherContract;
class CachingUserProvider extends EloquentUserProvider
{
/**
* The cache instance.
*
* @var Repository
*/
protected $cache;
/**
* Create a new database user provider.
*
* @param \Illuminate\Contracts\Hashing\Hasher $hasher
* @param string $model
* @param Repository $cache
* @return void
*/
public function __construct(HasherContract $hasher, $model, Repository $cache)
{
$this->model = $model;
$this->hasher = $hasher;
$this->cache = $cache;
}
/**
* Retrieve a user by their unique identifier.
*
* @param mixed $identifier
* @return \Illuminate\Contracts\Auth\Authenticatable|null
*/
public function retrieveById($identifier)
{
return $this->cache->tags($this->getModel())->remember('user_by_id_'.$identifier, 60, function () use ($identifier) {
$model = $this->createModel();
return $model->newQuery()
->with('roles', 'permissions', 'profile')
->where($model->getAuthIdentifierName(), $identifier)
->first();
});
}
}

Because we need to amend the query itself, we can’t just defer to the parent implementation like we did above and must instead copy it over and amend it to eager-load the data.

You’ll also need to set up model events to clear the cache whenever one of the related fields is updated, but it should be fairly straightforward to do so.

Summary

Fetching a user model (and possibly some relations) on every page load while logged in can be a bit much, and it makes sense to cache as much as you can without risking serving stale data. Using this technique you can potentially cache a lot of repetitive, unnecessary queries and make your application faster.

This technique will also work in cases where you’re using other methods of maintaining user state, such as JWT, as long as you’re making use of a guard for authentication purposes, since all of these guards will still be using the same user provider. In fact, I first used this technique on a REST API that used JWT for authentication, and it’s worked well in that case.

10th January 2018 10:07 pm

Adding Opensearch Support to Your Site

For the uninitiated, OpenSearch is the technology that lets you enter a site’s URL, and then press Tab to start searching on that site - you can see it in action on this site. It’s really useful, and quite easy to implement if you know how.

OpenSearch relies on having a particular XML file available. Here’s the opensearch.xml file for this site:

<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns:moz="http://www.mozilla.org/2006/browser/search/"
xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>matthewdaly.co.uk</ShortName>
<Description>Search matthewdaly.co.uk</Description>
<InputEncoding>UTF-8</InputEncoding>
<Url method="get" type="text/html"
template="http://www.google.com/search?q={searchTerms}&amp;sitesearch=matthewdaly.co.uk"/>
</OpenSearchDescription>

In this case, as this site uses a static site generator I can’t really do the search on the site, so it’s handed off to a Google site-specific search, but the principle is the same. The three relevant fields are as follows:

  • ShortName - The short name of the site (this should usually just be the domain name)
  • Description - A human-readable description such as Search mysite.com
  • Url - Specifies the HTTP method that should be used to search (GET or POST), and a template for the URL. The search is automatically inserted where {searchTerms} appears

A more typical example of the Url field might be as follows:

<Url method="get" type="text/html"
template="http://www.example.com/search?q={searchTerms}"/>

Normally you will be pointing the template to your site’s own search page. Note that OpenSearch doesn’t actually do any searching itself - it just tells your browser where to send your search request.

With that file saved as opensearch.xml, all you have to do is add it to the <head> in your HTML:

<link href="/opensearch.xml" rel="search" title="Search title" type="application/opensearchdescription+xml">

And that should be all you need to do to get OpenSearch working.

For Laravel sites, I’ve recently created a package for implementing Opensearch that should help as well. With that you need only install the package, and set the fields in the config to point at your existing search page, in order to get OpenSearch working.

10th January 2018 12:22 pm

Easy Repositories and Decorators With Laravel Repositories

Creating repositories for your Laravel models, as well as creating caching decorators for them, is a useful way of not only implementing caching in your web app, but decoupling the application from a specific ORM. Unfortunately, it can involve writing a fair amount of boilerplate code.

Laravel Repositories is a set of base classes and interfaces for creating repositories and decorators in your application. It consists of:

  • A generic interface for repositories
  • A base repository that implements the interface and can be extended for your own repositories
  • A base decorator that also implements the interface and can similarly be extended

By using these, not only are you able to implement caching quickly and easily for most use cases, but you can easily extend the base classes to add additional methods for your use case. By creating new interfaces that extend the base interface, then having your repositories extend the repository and decorator, you can minimise the amount of work required to set up new repositories.

The main interface used is Matthewbdaly\LaravelRepositories\Repositories\Interfaces\AbstractRepositoryInterface, and your interfaces should extend this. Your decorators should extend Matthewbdaly\LaravelRepositories\Repositories\Decorators\BaseDecorator, and your repositories should extend Matthewbdaly\LaravelRepositories\Repositories\Base. Then, if you add any additional methods to your interface and ensure your repository and decorator implement that interface, it should be straightforward to type-hint the interface and get back the decorated repository, which will handle caching for you.

To be able to type-hint the repositories, you need to set them up in a service provider:

<?php
namespace App\Providers;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap any application services.
*
* @return void
*/
public function boot()
{
//
}
/**
* Register any application services.
*
* @return void
*/
public function register()
{
$this->app->singleton('App\Repositories\Interfaces\ExampleRepositoryInterface', function () {
$baseRepo = new \App\Repositories\EloquentExampleRepository(new \App\Example);
$cachingRepo = new \App\Repositories\Decorators\ExampleDecorator($baseRepo, $this->app['cache.store']);
return $cachingRepo;
});
}
}

Also, note that the cache backend used must be one that supports tags, such as Redis or Memcached. Data is cached using a tag derived from the model name. This also means you have to be careful when eager-loading relations, as the data will be cached under the main model’s name, not that of the relation. You may want to set up separate model events to flush those tags when the related field is updated.

9th January 2018 5:26 pm

Creating Laravel Helpers

Although helpers are an important part of Laravel, the documentation doesn’t really touch on creating them. Fortunately, doing so it fairly easy.

Here I’m building a helper for formatting dates for the HTML5 datetime-local form input. First we define the helper function in app\Helpers.php:

<?php
use Carbon\Carbon;
if (!function_exists('format_date')) {
function format_date(string $date)
{
return Carbon::parse($date, config('app.timezone'))->format('Y-m-d\TH:i:s');
}
}

Then we create a service provider to load them:

<?php
namespace App\Providers;
use Illuminate\Support\ServiceProvider;
class HelperServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
//
}
/**
* Register the application services.
*
* @return void
*/
public function register()
{
//
require_once app_path() . '/Helpers.php';
}
}

Finally,we register the service provider in config/app.php:

'providers' => [
...
App\Providers\HelperServiceProvider::class,
],

Of course, once you have this all set up for one helper, it’s easy to add more because they can all go in app/Helpers.php.

Creating your own helpers is a good way of refactoring unwanted logic out of your Blade templates or controllers and making it more reusable and maintainable, particularly for things like formatting dates or strings.

Recent Posts

How Much Difference Does Adding An Index to a Database Table Make?

Searching Content With Fuse.js

Higher-order Components in React

Creating Your Own Dependency Injection Container in PHP

Understanding Query Objects

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Zend Framework, Django, Phonegap and React.js.