Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

2nd January 2018 12:01 pm

Using Artisan from Standalone Laravel Packages

Recently I’ve been building and publishing a significant number of Laravel packages, and I thought I’d share details of some of them over the next few days.

Artisan Standalone is a package that, when installed in a standalone Laravel package (eg, not in an actual Laravel install, but in a package that you’re building that is intended for use with Laravel), allows you to use Artisan. It’s intended largely to make it quicker and easier to build functionality as separate packages by giving you access to the same generator commands as you have when working with a Laravel application. It came about largely from a need to scratch my own itch, as when building packages I was having to either run Artisan commands in a Laravel app and move them over, or copy them from existing files, which was obviously a pain in the proverbial.

You can install it with the following command:

$ composer require --dev matthewbdaly/artisan-standalone

Once it’s installed, you can access Artisan as follows:

$ vendor/bin/artisan

Note that it doesn’t explicitly include Laravel as a dependency - you’ll need to add that in the parent package to pull in the libraries it needs (which you should be doing anyway). It’s possible that there are some commands that won’t work in this context, but they’re almost certainly ones you won’t need here, such as the migrate command. As far as I can tell the generator commands, which are the only ones we’re really interested in here, all work OK.

1st January 2018 4:06 pm

Creating Artisan Tasks That Generate Files

While the documentation for creating Artisan tasks is generally pretty good, it doesn’t really touch on creating tasks that generate new files. The only way to figure it out was to go digging through the source code. In this case, I was building an Artisan command to create Fractal transformers as part of a package I’m working on.

There’s a specialised class for generating files at Illuminate\Console\GeneratorCommand, which your command class should extend instead of Illuminate\Console\Command. In addition to the usual properties such as the signature and description, you also need to specify $type to give the type of class being generated. Also, note that the constructor is different, so if you use php artisan make:console to create the boilerplate for this command, you’ll need to delete the constructor.

<?php
namespace Matthewbdaly\MyPackage\Console\Commands;
use Illuminate\Console\GeneratorCommand;
use Symfony\Component\Console\Input\InputArgument;
class TransformerMakeCommand extends GeneratorCommand
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'make:transformer {name : The required name of the transformer class}';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Create a Fractal transformer';
/**
* The type of class being generated.
*
* @var string
*/
protected $type = 'Fractal transformer';
/**
* Get the stub file for the generator.
*
* @return string
*/
protected function getStub()
{
return __DIR__.'/stubs/transformer.stub';
}
/**
* Get the console command arguments.
*
* @return array
*/
protected function getArguments()
{
return [
['name', InputArgument::REQUIRED, 'The name of the command.'],
];
}
/**
* Get the default namespace for the class.
*
* @param string $rootNamespace
* @return string
*/
protected function getDefaultNamespace($rootNamespace)
{
return $rootNamespace.'\Transformers';
}
}

Note the getDefaultNamespace() method. If your class will live directly under the app folder this is not necessary. Otherwise, it needs to return the root namespace, with the folder structure you want after it. Here my class will live under app\Transformers, so I’ve set it to reflect that.

Also, note the getStub() method. This tells Artisan that it should use the specified stub file as the basis for our class. Below you’ll find the stub file I used for my transformer:

<?php
namespace DummyNamespace;
use Matthewbdaly\MyPackage\Transformers\BaseTransformer;
use Illuminate\Database\Eloquent\Model;
class DummyClass extends BaseTransformer
{
public function transform(Model $model)
{
return [
'id' => (int) $model->id,
];
}
}

Note that the DummyNamespace and DummyClass fields will be overwritten with the correct values.

Once this Artisan command is registered in the usual way, you can then run it as follows:

$ php artisan make:transformer Example

And it will generate a boilerplate class something like this:

<?php
namespace App\Transformers;
use Matthewbdaly\MyPackage\Transformers\BaseTransformer;
use Illuminate\Database\Eloquent\Model;
class Example extends BaseTransformer
{
public function transform(Model $model)
{
return [
'id' => (int) $model->id,
];
}
}

You can then replace the model with your own one as necessary, and add any further content to this class.

29th December 2017 6:01 pm

Using Uuids As Primary Keys With Laravel and Postgresql

For many applications, using UUID’s as the primary keys on a database table can make a lot of sense. For mobile or offline apps, in particular, they mean you can create new objects locally and assign them a primary key without having to worry about it colliding with another object that was created in the meantime once it gets synchronised to the server. Also, they are less informative to nefarious users - an autoincrementing value in a URL tells a user that that value is the primary key, and means the app may potentially allow gathering of information via user enumeration (eg calling /api/v1/users/1, /api/v1/users/2 etc).

It’s fairly straightforward to use UUID’s as primary keys on your models when using PostgreSQL. First, you need to set up your migrations to use the uuid-ossp extension and set up the id field as both a UUID and the primary key. You also need to set a default value manually so that if it’s left empty it will generate a UUID for it.

DB::statement('CREATE EXTENSION IF NOT EXISTS "uuid-ossp";');
Schema::create('items', function (Blueprint $table) {
$table->uuid('id')->primary();
$table->text('text')->nullable();
$table->timestamps();
});
DB::statement('ALTER TABLE items ALTER COLUMN id SET DEFAULT uuid_generate_v4();');

Then, in the model definition, you need to tell Laravel to cast the id field to a string, and explicitly set the primary key to id:

class Item extends Model
{
protected $casts = [
'id' => 'string',
];
protected $primaryKey = "id";
}

Once this is done, the model should generate the primary keys for you as usual, except as UUID’s. If your application needs to accept UUID primary keys that were created offline, such as in a mobile app, you will probably want to add the id field to the $fillable array on the model to allow this.

2nd December 2017 11:30 pm

Full Text Search With Laravel and Postgresql

I’ve touched on using PostgreSQL to implement fuzzy search with Laravel before, but another type of search that PostgreSQL can handle fairly easily is full-text search. Here I’ll show you how to use it in a Laravel application.

An obvious use case for this kind of search is a personal blogging engine. It’s unlikely something like this is going to have enough content for it to be worth using a heavier solution like Elasticsearch, but a LIKE or ILIKE statement doesn’t really cut it either, so Postgres’s full text search is a good fit. Below you’ll see a Laravel migration for the blog posts table:

<?php
use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;
class CreatePostsTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('posts', function (Blueprint $table) {
$table->increments('id');
$table->string('title');
$table->datetime('pub_date');
$table->text('text');
$table->string('slug');
$table->integer('author_id');
$table->timestamps();
});
DB::statement("ALTER TABLE posts ADD COLUMN searchtext TSVECTOR");
DB::statement("UPDATE posts SET searchtext = to_tsvector('english', title || '' || text)");
DB::statement("CREATE INDEX searchtext_gin ON posts USING GIN(searchtext)");
DB::statement("CREATE TRIGGER ts_searchtext BEFORE INSERT OR UPDATE ON posts FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger('searchtext', 'pg_catalog.english', 'title', 'text')");
}
/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
DB::statement("DROP TRIGGER IF EXISTS tsvector_update_trigger ON posts");
DB::statement("DROP INDEX IF EXISTS searchtext_gin");
DB::statement("ALTER TABLE posts DROP COLUMN searchtext");
Schema::dropIfExists('posts');
}
}

Note that after we create the basic layout of our posts table, we then have to drop down to raw DB statements to achieve the next steps:

  • We add a column called searchtext with a type of TSVECTOR (unfortunately Laravel doesn’t have a convenient method to create this column type, so we need to do it with a raw statement). This column will hold our searchable document.
  • We use the to_tsvector() method to generate a document on each row that combines the title and text fields and store it in the searchtext column. Note also that we specify the language as the first argument. This is because Postgres’s full text search understands so-called “stopwords”, which are words that are so common as to not be worth bothering with at all, such as “the” - these will obviously differ between languages, so it’s prudent to explicitly state this so Postgres knows what stopwords to expect.
  • We create a GIN index on the posts table using our new searchtext column.
  • Finally we create a trigger which, when the table is amended, regenerates the search text.

With that done, we can now look at actually performing a full-text search. To facilitate easy re-use, we’ll create a local scope on our Post model. If you haven’t used scopes in Laravel before, they essentially allow you to break queries into reusable chunks. In this case, we expect our scope to receive two arguments, the query instance (which is passed through automatically), and the search text:

<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class Post extends Model
{
protected $fillable = [
'title',
'pub_date',
'text',
'slug',
'author_id'
];
public function scopeSearch($query, $search)
{
if (!$search) {
return $query;
}
return $query->whereRaw('searchtext @@ to_tsquery(\'english\', ?)', [$search])
->orderByRaw('ts_rank(searchtext, to_tsquery(\'english\', ?)) DESC', [$search]);
}
}

If $search is empty, we just return the query object as is. Otherwise, we first of all construct a WHERE clause that matches our search text against the searchtext column. Note the syntax used here:

searchtext @@ to_tsquery('english', 'foo')

We use the to_tsquery() method to match our text against our search document. As before, note that we specify the language.

Finally, we specify an order - we want the highest ranked matches to appear first, and this section of the query does that:

ts_rank(searchtext, to_tsquery('english', 'foo')) DESC

Here we use ts_rank() to ensure we get our results in the appropriate order. Note that for both queries, we passed the arguments through as parameterized queries, rather than constructing a raw string - we have to watch out for SQL injection when we’re writing raw queries, but we can use PDO’s parameterized queries from Eloquent in a raw statement, which makes things a bit easier.

Now we can call our new search scope as follows:

$posts = Post::search($search)->get();

Because the scope receives and returns a query builder instance, you can continue to add the rest of your query, or paginate it, as necessary:

$posts = Post::search($search)->where('draft', false)->simplePaginate(5);

If you’re working in a language that makes heavy use of accents, such as French, you might also want to install the unaccent extension (you can do this in the migration with CREATE EXTENSION unaccent). Then, any time we call to_tsvector(), you should pass any strings through the unaccent() method to strip out the accents.

Do we need the migrations?

Technically, we could do without the additional changes to the database structure - we could create a document on the fly inside a subquery and use that to query against, which would look something like this in SQL:

SELECT *
FROM
(SELECT *,
to_tsvector('english', posts.title) || to_tsvector('english', posts.text) AS document
FROM "posts") search
WHERE search.document @@ to_tsquery('Redis')
ORDER BY ts_rank(search.document, to_tsquery('english', 'Redis')) DESC;

However, the performance is likely to be significantly worse using this approach as it has to recreate the document, and doesn’t have an existing index to query against. It’s also a pig to write something like this with an ORM.

I’m currently working on a more generic solution for implementing full text search with Postgres and Laravel, however so far it looks like that solution will not only be considerably more complex than this (consistently producing a suitable query for unknown data is rather fiddly), but you can’t create a column for the vector ahead of time, meaning the query will be slower. This approach, while it requires more work than simply installing a package, is not terribly hard to implement on a per-model basis and is easy to customise for your use case.

28th November 2017 11:40 am

Building a Postcode Lookup Client With Httplug and Phpspec

While PHPUnit is my normal go-to PHP testing framework, for some applications I find PHPSpec superior, in particular REST API clients. I’ve found that it makes for a better flow when doing test-driven development, because it makes it very natural to write a test first, then run it, then make the test pass.

In this tutorial I’ll show you how to build a lookup API client for UK postcodes. In the process of doing so, we’ll use PHPSpec to drive our development process. We’ll also use HTTPlug as our underlying HTTP library. The advantage of this over using something like Guzzle is that we give library users the freedom to choose the HTTP library they feel is most appropriate to their situation.

Background

If you’re unfamiliar with it, the UK postcode system is our equivalent of a zip code in the US, but with two major differences:

  • The codes themselves are alphanumeric instead of numeric, with the first part including one or two letters usually (but not always) derived from the nearest large town or city (eg L for Liverpool, B for Birmingham, OX for Oxford), or for London, based on the part of the city (eg NW for the north-west of London)
  • A full postcode is in two parts (eg NW1 8TQ), and the first part narrows the location down to a similar area to a US-style zip code, while the second part usually narrows it down to a street (although sometimes large organisations that receive a lot of mail will have a postcode to themselves).

This means that if you have someone’s postcode and house name or address, you can use those details to look up the rest of their address details. This obviously makes it easier for users to fill out a form, such as when placing an order on an e-commerce site - you can just request those two details and then autofill the rest from them.

Unfortunately, it’s not quite that simple. The data is owned by Royal Mail, and they charge through the nose for access to the raw data, which places this data well outside the budgets of many web app developers. Fortunately, Ideal Postcodes offer a REST API for querying this data. It’s not free, but at 2.5p per request it’s not going to break the bank unless used excessively, and they offer some dummy postcodes that are free to query, which is perfectly fine for testing.

For those of you outside the UK, this may not be of much immediate use, but the underlying principles will still be useful, and you can probably build a similar client for your own nation’s postal code system. For instance, there’s a Zipcode API that those of you in the US can use, and if you understand what’s going on here it shouldn’t be hard to adapt it to work with that. If you do produce a similar client for your country’s postal code system, submit a pull request to update the README with a link to it and I’ll include it.

Setting up

First we’ll create a composer.json to specify our dependencies:

{
"name": "matthewbdaly/postcode-client",
"description": "A postcode lookup client.",
"type": "library",
"keywords": ["postcode"],
"require": {
"psr/http-message": "^1.0",
"php-http/client-implementation": "^1.0",
"php-http/httplug": "^1.0",
"php-http/message-factory": "^1.0",
"php-http/discovery": "^1.0"
},
"require-dev": {
"psy/psysh": "^0.8.0",
"phpspec/phpspec": "^3.2",
"squizlabs/php_codesniffer": "^2.7",
"php-http/mock-client": "^1.0",
"php-http/message": "^1.0",
"guzzlehttp/psr7": "^1.0"
},
"license": "MIT",
"authors": [
{
"name": "Matthew Daly",
"email": "matthewbdaly@gmail.com"
}
],
"autoload": {
"psr-4": {
"Matthewbdaly\\Postcode\\": "src/"
}
}
}

Note that we don’t install an actual HTTPlug client, other than the mock one, which is only useful for testing. This is deliberate - we’re giving developers working with this library the choice of working with whatever HTTP client they see fit. We do use the Guzzle PSR7 library, but that’s just for the PSR7 library.

Then we install our dependencies:

$ composer install

We also need to tell PHPSpec what our namespace will be. Save this as phpspec.yml:

suites:
test_suite:
namespace: Matthewbdaly\Postcode
psr4_prefix: Matthewbdaly\Postcode

Don’t forget to update the namespace in both files to whatever you’re using, which should have a vendor name and a package name.

With that done, it’s time to introduce the next component.

Introducing HTTPlug

In the past I’ve usually used either Curl or Guzzle to carry out HTTP requests. However, the problem with this approach is that you’re forcing whoever uses your library to use whatever HTTP client, and whatever version of that client, that you deem appropriate. If they’re also using another library that someone else has written and they made different choices, you could have problems.

HTTPlug is an excellent way of solving this problem. By requiring only an interface and not a concrete implementation, using HTTPlug means that you can specify that the consumer of the library must provide a suitable implementation of that library, but leave the choice of implementation up to them. This means that they can choose whatever implementation best fits their use case. There are adapters for many different clients, so it’s unlikely that they won’t be able to find one that meets their needs.

In addition, HTTPlug provides the means to automatically determine what HTTP client to use, so that if one is not explicitly provided, it can be resolved without any action on the part of the developer. As long as a suitable HTTP adapter is installed, it will be used.

Getting started

One advantage of PHPSpec is that it will automatically generate much of the boilerplate for our client and specs. To create our client spec, run this command:

$ vendor/bin/phpspec desc Matthewbdaly/Postcode/Client
Specification for Matthewbdaly\Postcode\Client created in /home/matthew/Projects/postcode-client/spec/ClientSpec.php.

Now that we have a spec for our client, we can generate the client itself:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
11 - it is initializable
class Matthewbdaly\Postcode\Client does not exist.
100% 1
1 specs
1 example (1 broken)
14ms
Do you want me to create `Matthewbdaly\Postcode\Client` for you?
[Y/n]
y
Class Matthewbdaly\Postcode\Client created in /home/matthew/Projects/postcode-client/src/Client.php.
100% 1
1 specs
1 example (1 passed)
16ms

You will need to enter Y when prompted. We now have an empty class for our client.

Next, we need to make sure that the constructor for our client accepts two parameters:

  • The HTTP client
  • A message factory instance, which is used to create the request

Amend spec/ClientSpec.php as follows:

<?php
namespace spec\Matthewbdaly\Postcode;
use Matthewbdaly\Postcode\Client;
use PhpSpec\ObjectBehavior;
use Prophecy\Argument;
use Http\Client\HttpClient;
use Http\Message\MessageFactory;
class ClientSpec extends ObjectBehavior
{
function let (HttpClient $client, MessageFactory $messageFactory)
{
$this->beConstructedWith($client, $messageFactory);
}
function it_is_initializable()
{
$this->shouldHaveType(Client::class);
}
}

Note the use of the let() method here. This lets us specify how the object is constructed, with the beConstructedWith() method. Also, note that $this refers not to the test, but to the object being tested - this takes a bit of getting used to if you’re used to working with PHPUnit.

Also, note that the objects passed through are not actual instances of those objects - instead they are mocks created automatically by PHPSpec. This makes mocking extremely easy, and you can easily set up your own expectations on those mock objects in the test. If you want to use a real object, you can instantiate it in the spec as usual. If we need any other mocks, we can typehint them in our method in exactly the same way.

If we once again use vendor/bin/phpspec run we can now generate a constructor:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
18 - it is initializable
method Matthewbdaly\Postcode\Client::__construct not found.
100% 1
1 specs
1 example (1 broken)
281ms
Do you want me to create `Matthewbdaly\Postcode\Client::__construct()` for
you?
[Y/n]
y
Method Matthewbdaly\Postcode\Client::__construct() has been created.
100% 1
1 specs
1 example (1 passed)
50ms

This will only create a placeholder for the constructor. You need to populate it yourself, so update src/Client.php as follows:

<?php
namespace Matthewbdaly\Postcode;
use Http\Client\HttpClient;
use Http\Discovery\HttpClientDiscovery;
use Http\Message\MessageFactory;
use Http\Discovery\MessageFactoryDiscovery;
class Client
{
public function __construct(HttpClient $client = null, MessageFactory $messageFactory = null)
{
$this->client = $client ?: HttpClientDiscovery::find();
$this->messageFactory = $messageFactory ?: MessageFactoryDiscovery::find();
}
}

A little explanation is called for here. We need two arguments in our construct:

  • An instance of Http\Client\HttpClient to send the request
  • An instance of Http\Message\MessageFactory to create the request

However, we don’t want to force the user to create one. Therefore if they are not set, we use Http\Discovery\HttpClientDiscovery and Http\Discovery\MessageFactoryDiscovery to create them for us.

If we re-run PHPSpec, it should now pass:

$ vendor/bin/phpspec run
100% 1
1 specs
1 example (1 passed)
31ms

Next, we want to have a method for retrieving the endpoint. Add the following method to spec/ClientSpec.php:

function it_can_retrieve_the_base_url()
{
$this->getBaseUrl()->shouldReturn('https://api.ideal-postcodes.co.uk/v1/postcodes/');
}

Here we’re asserting that fetching the base URL returns the given result. Note how much simpler and more intuitive this syntax is than PHPUnit would be:

$this->assertEquals('https://api.ideal-postcodes.co.uk/v1/postcodes/', $client->getBaseUrl());

Running the tests again should prompt us to create the boilerplate for the new method:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
23 - it can retrieve the base url
method Matthewbdaly\Postcode\Client::getBaseUrl not found.
50% 50% 2
1 specs
2 examples (1 passed, 1 broken)
40ms
Do you want me to create `Matthewbdaly\Postcode\Client::getBaseUrl()` for
you?
[Y/n]
y
Method Matthewbdaly\Postcode\Client::getBaseUrl() has been created.
Matthewbdaly/Postcode/Client
23 - it can retrieve the base url
expected "https://api.ideal-postcod...", but got null.
50% 50% 2
1 specs
2 examples (1 passed, 1 failed)
72ms

Now we need to update that method to work as expected:

protected $baseUrl = 'https://api.ideal-postcodes.co.uk/v1/postcodes/';
...
public function getBaseUrl()
{
return $this->baseUrl;
}

This should make the tests pass:

$ vendor/bin/phpspec run
100% 2
1 specs
2 examples (2 passed)
34ms

Next, we need to be able to get and set the API key. Add the following to spec/ClientSpec.php:

function it_can_get_and_set_the_key()
{
$this->getKey()->shouldReturn(null);
$this->setKey('foo')->shouldReturn($this);
$this->getKey()->shouldReturn('foo');
}

Note that we expect $this->setKey('foo') to return $this. This is an example of a fluent interface - by returning an instance of the object, it enables methods to be chained, eg $client->setKey('foo')->get(). Obviously it won’t work for anything that has to return a value, but it’s a useful way of making your classes more intuitive to use.

Next, run the tests again and agree to the prompts as before:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
28 - it can get and set the key
method Matthewbdaly\Postcode\Client::getKey not found.
66% 33% 3
1 specs
3 examples (2 passed, 1 broken)
51ms
Do you want me to create `Matthewbdaly\Postcode\Client::getKey()` for you?
[Y/n]
y
Method Matthewbdaly\Postcode\Client::getKey() has been created.
Matthewbdaly/Postcode/Client
28 - it can get and set the key
method Matthewbdaly\Postcode\Client::setKey not found.
66% 33% 3
1 specs
3 examples (2 passed, 1 broken)
43ms
Do you want me to create `Matthewbdaly\Postcode\Client::setKey()` for you?
[Y/n]
y
Method Matthewbdaly\Postcode\Client::setKey() has been created.
Matthewbdaly/Postcode/Client
28 - it can get and set the key
expected [obj:Matthewbdaly\Postcode\Client], but got null.
66% 33% 3
1 specs
3 examples (2 passed, 1 failed)
52ms

Next, add our getter and setter for the key, as well as declaring the property $key:

protected $key;
public function getKey()
{
return $this->key;
}
public function setKey(string $key)
{
$this->key = $key;
return $this;
}

That should make the tests pass:

$ vendor/bin/phpspec run
100% 3
1 specs
3 examples (3 passed)
38ms

With that done, our final task is to be able to handle sending requests. Add the following imports at the top of spec/ClientSpec.php:

use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\StreamInterface;

And add the following method at the bottom of the same file:

function it_can_send_the_request(HttpClient $client, MessageFactory $messageFactory, RequestInterface $request, ResponseInterface $response, StreamInterface $stream)
{
$this->beConstructedWith($client, $messageFactory);
$this->setKey('foo');
$data = json_encode([
'result' => [
"postcode" => "SW1A 2AA",
"postcode_inward" => "2AA",
"postcode_outward" => "SW1A",
"post_town" => "LONDON",
"dependant_locality" => "",
"double_dependant_locality" => "",
"thoroughfare" => "Downing Street",
"dependant_thoroughfare" => "",
"building_number" => "10",
"building_name" => "",
"sub_building_name" => "",
"po_box" => "",
"department_name" => "",
"organisation_name" => "Prime Minister & First Lord Of The Treasury",
"udprn" => 23747771,
"umprn" => "",
"postcode_type" => "L",
"su_organisation_indicator" => "",
"delivery_point_suffix" => "1A",
"line_1" => "Prime Minister & First Lord Of The Treasury",
"line_2" => "10 Downing Street",
"line_3" => "",
"premise" => "10",
"longitude" => -0.127695242183412,
"latitude" => 51.5035398826274,
"eastings" => 530047,
"northings" => 179951,
"country" => "England",
"traditional_county" => "Greater London",
"administrative_county" => "",
"postal_county" => "London",
"county" => "London",
]
]);
$messageFactory->createRequest('GET', 'https://api.ideal-postcodes.co.uk/v1/postcodes/SW1A%202AA?api_key=foo', [], null, '1.1')->willReturn($request);
$client->sendRequest($request)->willReturn($response);
$response->getStatusCode()->willReturn(200);
$response->getBody()->willReturn($stream);
$stream->getContents()->willReturn($data);
$this->get('SW1A 2AA')->shouldBeLike(json_decode($data, true));
}

This test is by far the biggest so far, so it merits some degree of explanation.

Note that we don’t make a real HTTP request against the API. This may sound strange, but bear with me. We have no control whatsoever over that API, and it could in theory become inaccessible or be subject to breaking changes at any time. We also don’t want to be shelling out for a paid service just to test our API client works. All we can do is test that our implementation will send the request we expect it to send - we don’t want our test suite reporting a bug when the API goes down.

We therefore typehint not just the dependencies for the constructor, but a request, response and stream instance. We mock our our responses from those instances using the willReturn() method, so we have complete control over what we pass to our client. That way we can return any appropriate response or throw any exception we deem fit to test the behaviour under those circumstances. For the message factory, we specify what arguments it should receive to create the request, and return our mocked-out request object.

Also, note we use shouldBeLike() to verify the response - this is effectively using the == operator, whereas shouldBe() uses the === operator, making it stricter.

Let’s run the tests, and don’t forget the prompt:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
38 - it can send the request
method Matthewbdaly\Postcode\Client::get not found.
75% 25% 4
1 specs
4 examples (3 passed, 1 broken)
55ms
Do you want me to create `Matthewbdaly\Postcode\Client::get()` for you?
[Y/n]
y
Method Matthewbdaly\Postcode\Client::get() has been created.
Matthewbdaly/Postcode/Client
38 - it can send the request
expected [array:1], but got null.
75% 25% 4
1 specs
4 examples (3 passed, 1 failed)
56ms

Now we can implement the get() method:

public function get(string $postcode)
{
$url = $this->getBaseUrl() . rawurlencode($postcode) . '?' . http_build_query([
'api_key' => $this->getKey()
]);
$request = $this->messageFactory->createRequest(
'GET',
$url,
[],
null,
'1.1'
);
$response = $this->client->sendRequest($request);
$data = json_decode($response->getBody()->getContents(), true);
return $data;
}

We first build up our URL, before using the message factory to create a request object. We then pass the built request to our client to send, before decoding the response into the format we want.

This should make our tests pass:

$ vendor/bin/phpspec run
100% 4
1 specs
4 examples (4 passed)
307ms

Our client now works, but there are a couple of situations we need to account for. First, the API will raise a 402 if you make a request for a real postcode without having paid. We need to catch this and throw an exception. Add this to spec/ClientSpec.php:

use Matthewbdaly\Postcode\Exceptions\PaymentRequired;
...
function it_throws_an_exception_if_payment_required(HttpClient $client, MessageFactory $messageFactory, RequestInterface $request, ResponseInterface $response, StreamInterface $stream)
{
$this->beConstructedWith($client, $messageFactory);
$this->setKey('foo');
$messageFactory->createRequest('GET', 'https://api.ideal-postcodes.co.uk/v1/postcodes/SW1A%202AA?api_key=foo', [], null, '1.1')->willReturn($request);
$client->sendRequest($request)->willReturn($response);
$response->getStatusCode()->willReturn(402);
$this->shouldThrow(PaymentRequired::class)->duringGet('SW1A 2AA');
}

With that done, run the tests again:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
87 - it throws an exception if payment required
expected exception of class "Matthewbdaly\Postcode\Exc...", but got
[exc:Prophecy\Exception\Call\UnexpectedCallException("Method call:
- getBody()
on Double\ResponseInterface\P15 was not expected, expected calls were:
- getStatusCode()")].
80% 20% 5
1 specs
5 examples (4 passed, 1 failed)
130ms

Let’s amend the client to throw this exception:

use Matthewbdaly\Postcode\Exceptions\PaymentRequired;
...
public function get(string $postcode)
{
$url = $this->getBaseUrl() . rawurlencode($postcode) . '?' . http_build_query([
'api_key' => $this->getKey()
]);
$request = $this->messageFactory->createRequest(
'GET',
$url,
[],
null,
'1.1'
);
$response = $this->client->sendRequest($request);
if ($response->getStatusCode() == 402) {
throw new PaymentRequired;
}
$data = json_decode($response->getBody()->getContents(), true);
return $data;
}

And let’s re-run the tests:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
87 - it throws an exception if payment required
expected exception of class "Matthewbdaly\Postcode\Exc...", but got [obj:Error] with the
message: "Class 'Matthewbdaly\Postcode\Exceptions\PaymentRequired' not found"
80% 20% 5
1 specs
5 examples (4 passed, 1 failed)
389ms

It fails now because the exception doesn’t exist. Let’s create it at src/Exceptions/PaymentRequired.php:

<?php
namespace Matthewbdaly\Postcode\Exceptions;
class PaymentRequired extends \Exception
{
}

That should be enough to make our tests pass:

$ vendor/bin/phpspec run
100% 5
1 specs
5 examples (5 passed)
89ms

We also need to raise an exception when the postcode is not found, which raises a 404 error. Add the following spec:

use Matthewbdaly\Postcode\Exceptions\PostcodeNotFound;
...
function it_throws_an_exception_if_postcode_not_found(HttpClient $client, MessageFactory $messageFactory, RequestInterface $request, ResponseInterface $response, StreamInterface $stream)
{
$this->beConstructedWith($client, $messageFactory);
$this->setKey('foo');
$messageFactory->createRequest('GET', 'https://api.ideal-postcodes.co.uk/v1/postcodes/SW1A%202AA?api_key=foo', [], null, '1.1')->willReturn($request);
$client->sendRequest($request)->willReturn($response);
$response->getStatusCode()->willReturn(404);
$this->shouldThrow(PostcodeNotFound::class)->duringGet('SW1A 2AA');
}

Run the tests:

$ vendor/bin/phpspec run
Matthewbdaly/Postcode/Client
98 - it throws an exception if postcode not found
expected exception of class "Matthewbdaly\Postcode\Exc...", but got
[exc:Prophecy\Exception\Call\UnexpectedCallException("Method call:
- getBody()
on Double\ResponseInterface\P20 was not expected, expected calls were:
- getStatusCode()")].
83% 16% 6
1 specs
6 examples (5 passed, 1 failed)
538ms

This time we’ll create the exception class before updating the client. Create the following class at src/Exceptions/PostcodeNotFound.php:

<?php
namespace Matthewbdaly\Postcode\Exceptions;
/**
* Postcode not found exception
*
*/
class PostcodeNotFound extends \Exception
{
}

And update the client:

use Matthewbdaly\Postcode\Exceptions\PostcodeNotFound;
...
public function get(string $postcode)
{
$url = $this->getBaseUrl() . rawurlencode($postcode) . '?' . http_build_query([
'api_key' => $this->getKey()
]);
$request = $this->messageFactory->createRequest(
'GET',
$url,
[],
null,
'1.1'
);
$response = $this->client->sendRequest($request);
if ($response->getStatusCode() == 402) {
throw new PaymentRequired;
}
if ($response->getStatusCode() == 404) {
throw new PostcodeNotFound;
}
$data = json_decode($response->getBody()->getContents(), true);
return $data;
}

Re-run the tests:

$ vendor/bin/phpspec run
100% 6
1 specs
6 examples (6 passed)
103ms

And our API client is feature complete! You can find the source code of the finished client here.

Summary

Personally, I find that while PHPSpec isn’t appropriate for every use case, it’s particularly handy for API clients and it’s generally my go-to testing solution for them. It handles producing a lot of the boilerplate for me, and it results in a much better workflow for test-driven development as it makes it very natural to write the test first, then make it pass.

HTTPlug has been a revelation for me. While it takes a bit of getting used to if you’re used to something like Guzzle, it means that you’re giving consumers of your library the freedom to choose the HTTP client of their choice, meaning they don’t have to fight with several different libraries requiring different versions of Guzzle. It also allows for easy resolution of the HTTP client, rather than having to explicitly pass through an instance when instantiating your client. I’m planning to use it extensively in the future.

Recent Posts

Adding React to a Legacy Project

Do You Still Need Jquery?

An Approach to Writing Golden Master Tests for PHP Web Applications

Understanding the Pipeline Pattern

Replacing Switch Statements With Polymorphism in PHP

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Django, Phonegap and Angular.js.