Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

2nd January 2018 12:12 pm

A Laravel Package Boilerplate

The second package I’ve been working on recently is Laravel Package Boilerplate. It’s a basic starter boilerplate for building your own Laravel packages.

It’s not meant to be installed as a project dependency. Instead, run the following command to create a new project boilerplate with it:

composer create-project --prefer-dist matthewbdaly/laravel-package-boilerplate <YOUR_NEW_PACKAGE_DIRECTORY>

This will create a new folder that includes a src folder containing a service provider, and a tests folder containing a preconfigured base test case, as well as a simple test case for tests that don’t need the full application instantiated, in order to help keep your test suite as fast as possible.

In addition, it includes configuration files for:

  • PHPUnit
  • PHP CodeSniffer
  • Travis CI

That way you can start your project off the right way with very little effort.

I’ve also added my Artisan Standalone project as a dependency - that way you can access any Artisan commands you need to generate files you need as follows:

$ vendor/bin/artisan

Hopefully this package should make it a lot easier to create new Laravel packages in future.

2nd January 2018 12:01 pm

Using Artisan from Standalone Laravel Packages

Recently I’ve been building and publishing a significant number of Laravel packages, and I thought I’d share details of some of them over the next few days.

Artisan Standalone is a package that, when installed in a standalone Laravel package (eg, not in an actual Laravel install, but in a package that you’re building that is intended for use with Laravel), allows you to use Artisan. It’s intended largely to make it quicker and easier to build functionality as separate packages by giving you access to the same generator commands as you have when working with a Laravel application. It came about largely from a need to scratch my own itch, as when building packages I was having to either run Artisan commands in a Laravel app and move them over, or copy them from existing files, which was obviously a pain in the proverbial.

You can install it with the following command:

$ composer require --dev matthewbdaly/artisan-standalone

Once it’s installed, you can access Artisan as follows:

$ vendor/bin/artisan

Note that it doesn’t explicitly include Laravel as a dependency - you’ll need to add that in the parent package to pull in the libraries it needs (which you should be doing anyway). It’s possible that there are some commands that won’t work in this context, but they’re almost certainly ones you won’t need here, such as the migrate command. As far as I can tell the generator commands, which are the only ones we’re really interested in here, all work OK.

1st January 2018 4:06 pm

Creating Artisan Tasks That Generate Files

While the documentation for creating Artisan tasks is generally pretty good, it doesn’t really touch on creating tasks that generate new files. The only way to figure it out was to go digging through the source code. In this case, I was building an Artisan command to create Fractal transformers as part of a package I’m working on.

There’s a specialised class for generating files at Illuminate\Console\GeneratorCommand, which your command class should extend instead of Illuminate\Console\Command. In addition to the usual properties such as the signature and description, you also need to specify $type to give the type of class being generated. Also, note that the constructor is different, so if you use php artisan make:console to create the boilerplate for this command, you’ll need to delete the constructor.

<?php
namespace Matthewbdaly\MyPackage\Console\Commands;
use Illuminate\Console\GeneratorCommand;
use Symfony\Component\Console\Input\InputArgument;
class TransformerMakeCommand extends GeneratorCommand
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'make:transformer {name : The required name of the transformer class}';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Create a Fractal transformer';
/**
* The type of class being generated.
*
* @var string
*/
protected $type = 'Fractal transformer';
/**
* Get the stub file for the generator.
*
* @return string
*/
protected function getStub()
{
return __DIR__.'/stubs/transformer.stub';
}
/**
* Get the console command arguments.
*
* @return array
*/
protected function getArguments()
{
return [
['name', InputArgument::REQUIRED, 'The name of the command.'],
];
}
/**
* Get the default namespace for the class.
*
* @param string $rootNamespace
* @return string
*/
protected function getDefaultNamespace($rootNamespace)
{
return $rootNamespace.'\Transformers';
}
}

Note the getDefaultNamespace() method. If your class will live directly under the app folder this is not necessary. Otherwise, it needs to return the root namespace, with the folder structure you want after it. Here my class will live under app\Transformers, so I’ve set it to reflect that.

Also, note the getStub() method. This tells Artisan that it should use the specified stub file as the basis for our class. Below you’ll find the stub file I used for my transformer:

<?php
namespace DummyNamespace;
use Matthewbdaly\MyPackage\Transformers\BaseTransformer;
use Illuminate\Database\Eloquent\Model;
class DummyClass extends BaseTransformer
{
public function transform(Model $model)
{
return [
'id' => (int) $model->id,
];
}
}

Note that the DummyNamespace and DummyClass fields will be overwritten with the correct values.

Once this Artisan command is registered in the usual way, you can then run it as follows:

$ php artisan make:transformer Example

And it will generate a boilerplate class something like this:

<?php
namespace App\Transformers;
use Matthewbdaly\MyPackage\Transformers\BaseTransformer;
use Illuminate\Database\Eloquent\Model;
class Example extends BaseTransformer
{
public function transform(Model $model)
{
return [
'id' => (int) $model->id,
];
}
}

You can then replace the model with your own one as necessary, and add any further content to this class.

29th December 2017 6:01 pm

Using Uuids As Primary Keys With Laravel and Postgresql

For many applications, using UUID’s as the primary keys on a database table can make a lot of sense. For mobile or offline apps, in particular, they mean you can create new objects locally and assign them a primary key without having to worry about it colliding with another object that was created in the meantime once it gets synchronised to the server. Also, they are less informative to nefarious users - an autoincrementing value in a URL tells a user that that value is the primary key, and means the app may potentially allow gathering of information via user enumeration (eg calling /api/v1/users/1, /api/v1/users/2 etc).

It’s fairly straightforward to use UUID’s as primary keys on your models when using PostgreSQL. First, you need to set up your migrations to use the uuid-ossp extension and set up the id field as both a UUID and the primary key. You also need to set a default value manually so that if it’s left empty it will generate a UUID for it.

DB::statement('CREATE EXTENSION IF NOT EXISTS "uuid-ossp";');
Schema::create('items', function (Blueprint $table) {
$table->uuid('id')->primary();
$table->text('text')->nullable();
$table->timestamps();
});
DB::statement('ALTER TABLE items ALTER COLUMN id SET DEFAULT uuid_generate_v4();');

Then, in the model definition, you need to tell Laravel to cast the id field to a string, and explicitly set the primary key to id:

class Item extends Model
{
protected $casts = [
'id' => 'string',
];
protected $primaryKey = "id";
}

Once this is done, the model should generate the primary keys for you as usual, except as UUID’s. If your application needs to accept UUID primary keys that were created offline, such as in a mobile app, you will probably want to add the id field to the $fillable array on the model to allow this.

2nd December 2017 11:30 pm

Full Text Search With Laravel and Postgresql

I’ve touched on using PostgreSQL to implement fuzzy search with Laravel before, but another type of search that PostgreSQL can handle fairly easily is full-text search. Here I’ll show you how to use it in a Laravel application.

An obvious use case for this kind of search is a personal blogging engine. It’s unlikely something like this is going to have enough content for it to be worth using a heavier solution like Elasticsearch, but a LIKE or ILIKE statement doesn’t really cut it either, so Postgres’s full text search is a good fit. Below you’ll see a Laravel migration for the blog posts table:

<?php
use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;
class CreatePostsTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('posts', function (Blueprint $table) {
$table->increments('id');
$table->string('title');
$table->datetime('pub_date');
$table->text('text');
$table->string('slug');
$table->integer('author_id');
$table->timestamps();
});
DB::statement("ALTER TABLE posts ADD COLUMN searchtext TSVECTOR");
DB::statement("UPDATE posts SET searchtext = to_tsvector('english', title || '' || text)");
DB::statement("CREATE INDEX searchtext_gin ON posts USING GIN(searchtext)");
DB::statement("CREATE TRIGGER ts_searchtext BEFORE INSERT OR UPDATE ON posts FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger('searchtext', 'pg_catalog.english', 'title', 'text')");
}
/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
DB::statement("DROP TRIGGER IF EXISTS tsvector_update_trigger ON posts");
DB::statement("DROP INDEX IF EXISTS searchtext_gin");
DB::statement("ALTER TABLE posts DROP COLUMN searchtext");
Schema::dropIfExists('posts');
}
}

Note that after we create the basic layout of our posts table, we then have to drop down to raw DB statements to achieve the next steps:

  • We add a column called searchtext with a type of TSVECTOR (unfortunately Laravel doesn’t have a convenient method to create this column type, so we need to do it with a raw statement). This column will hold our searchable document.
  • We use the to_tsvector() method to generate a document on each row that combines the title and text fields and store it in the searchtext column. Note also that we specify the language as the first argument. This is because Postgres’s full text search understands so-called “stopwords”, which are words that are so common as to not be worth bothering with at all, such as “the” - these will obviously differ between languages, so it’s prudent to explicitly state this so Postgres knows what stopwords to expect.
  • We create a GIN index on the posts table using our new searchtext column.
  • Finally we create a trigger which, when the table is amended, regenerates the search text.

With that done, we can now look at actually performing a full-text search. To facilitate easy re-use, we’ll create a local scope on our Post model. If you haven’t used scopes in Laravel before, they essentially allow you to break queries into reusable chunks. In this case, we expect our scope to receive two arguments, the query instance (which is passed through automatically), and the search text:

<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class Post extends Model
{
protected $fillable = [
'title',
'pub_date',
'text',
'slug',
'author_id'
];
public function scopeSearch($query, $search)
{
if (!$search) {
return $query;
}
return $query->whereRaw('searchtext @@ to_tsquery(\'english\', ?)', [$search])
->orderByRaw('ts_rank(searchtext, to_tsquery(\'english\', ?)) DESC', [$search]);
}
}

If $search is empty, we just return the query object as is. Otherwise, we first of all construct a WHERE clause that matches our search text against the searchtext column. Note the syntax used here:

searchtext @@ to_tsquery('english', 'foo')

We use the to_tsquery() method to match our text against our search document. As before, note that we specify the language.

Finally, we specify an order - we want the highest ranked matches to appear first, and this section of the query does that:

ts_rank(searchtext, to_tsquery('english', 'foo')) DESC

Here we use ts_rank() to ensure we get our results in the appropriate order. Note that for both queries, we passed the arguments through as parameterized queries, rather than constructing a raw string - we have to watch out for SQL injection when we’re writing raw queries, but we can use PDO’s parameterized queries from Eloquent in a raw statement, which makes things a bit easier.

Now we can call our new search scope as follows:

$posts = Post::search($search)->get();

Because the scope receives and returns a query builder instance, you can continue to add the rest of your query, or paginate it, as necessary:

$posts = Post::search($search)->where('draft', false)->simplePaginate(5);

If you’re working in a language that makes heavy use of accents, such as French, you might also want to install the unaccent extension (you can do this in the migration with CREATE EXTENSION unaccent). Then, any time we call to_tsvector(), you should pass any strings through the unaccent() method to strip out the accents.

Do we need the migrations?

Technically, we could do without the additional changes to the database structure - we could create a document on the fly inside a subquery and use that to query against, which would look something like this in SQL:

SELECT *
FROM
(SELECT *,
to_tsvector('english', posts.title) || to_tsvector('english', posts.text) AS document
FROM "posts") search
WHERE search.document @@ to_tsquery('Redis')
ORDER BY ts_rank(search.document, to_tsquery('english', 'Redis')) DESC;

However, the performance is likely to be significantly worse using this approach as it has to recreate the document, and doesn’t have an existing index to query against. It’s also a pig to write something like this with an ORM.

I’m currently working on a more generic solution for implementing full text search with Postgres and Laravel, however so far it looks like that solution will not only be considerably more complex than this (consistently producing a suitable query for unknown data is rather fiddly), but you can’t create a column for the vector ahead of time, meaning the query will be slower. This approach, while it requires more work than simply installing a package, is not terribly hard to implement on a per-model basis and is easy to customise for your use case.

Recent Posts

How Much Difference Does Adding An Index to a Database Table Make?

Searching Content With Fuse.js

Higher-order Components in React

Creating Your Own Dependency Injection Container in PHP

Understanding Query Objects

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Zend Framework, Django, Phonegap and React.js.