Matthew Daly's Blog

I'm a web developer in Norfolk. This is my blog...

29th April 2018 8:59 pm

Console Applications With the Symfony Console Component

Recently I’ve had the occasion to add a series of console commands to a legacy application. This can be made straightforward by using the Symfony console component. In this post I’ll demonstrate how to write a simple console command for clearing a cache folder.

The first step is to install the Console component:

$ composer require symfony/console

Then we write the main script for the application. I usually save mine as console - note that we don’t want to have to type out a file extension, so instead we use the shebang:

#!/user/bin/env php
<?php
require __DIR__.'/vendor/autoload.php';
use Symfony\Component\Console\Application;
define('CONSOLE_ROOT', __DIR__);
$app = new Application();
$app->run();

In this case, I’ve defined CONSOLE_ROOT as the directory in which the console command is run - that way, the commands can use it to refer to the application root.

We can then run our console application as follows:

$ php console
Console Tool
Usage:
command [options] [arguments]
Options:
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
Available commands:
help Displays help for a command
list Lists commands

This displays the available commands, but you’ll note that there are none except for help and list. We’ll remedy that. First, we’ll register a command:

$app->add(new App\Console\ClearCacheCommand);

This has to be done in console, after we create $app, but before we run it.

Don’t forget to update the autoload section of your composer.json to register the namespace:

"autoload": {
"psr-4": {
"App\\Console\\": "src/Console/"
}
},

Then create the class for that command. This class must extend Symfony\Component\Console\Command\Command, and must have two methods:

  • configure()
  • execute()

In addition, the execute() method must accept two arguments, an instance of Symfony\Component\Console\Input\InputInterface, and an instance of Symfony\Component\Console\Output\OutputInterface. There are used to retrieve input and display output.

Let’s write our command:

<?php
namespace App\Console;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
class ClearCacheCommand extends Command
{
protected function configure()
{
$this->setName('cache:clear')
->setDescription('Clears the cache')
->setHelp('This command clears the application cache');
}
protected function execute(InputInterface $input, OutputInterface $output)
{
$dir = CONSOLE_ROOT.DIRECTORY_SEPARATOR.'cache';
$this->deleteTree($dir);
$output->writeln('Cache cleared');
}
private function deleteTree($dir)
{
$files = array_diff(scandir($dir), array('.','..'));
foreach ($files as $file) {
(is_dir("$dir/$file")) ? $this->deleteTree("$dir/$file") : unlink("$dir/$file");
}
return rmdir($dir);
}
}

As you can see, in the configure() method, we set the name, description and help text for the command.

The execute() method is where the actual work is done. In this case, we have some code that needs to be called recursively, so we have to pull it out into a private method. Once that’s done we use $output->writeln() to write a line to the output.

Now, if we run our console task, we should see our new command:

$ php console
Console Tool
Usage:
command [options] [arguments]
Options:
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
Available commands:
help Displays help for a command
list Lists commands
cache
cache:clear Clears the cache

And we can see it in action too:

$ php console cache:clear
Cache cleared

For commands that need to accept additional arguments, you can define them in the configure() method:

$this->addArgument('file', InputArgument::REQUIRED, 'Which file do you want to delete?')

Then, you can access it in the execute() method using InputInterface:

$file = $input->getArgument('file');

This tutorial is just skimming the surface of what you can do with the Symfony Console components - indeed, many other console interfaces, such as Laravel’s Artisan, are built on top of it. If you have a legacy application built in a framework that lacks any sort of console interface, such as CodeIgniter, then you can quite quickly produce basic console commands for working with that application. The documentation is very good, and with a little work you can soon have something up and running.

22nd April 2018 11:50 pm

Rendering Different Views for Mobile and Desktop Clients in Laravel

This was a bit of a weird post to write. It started out explaining how I resolved an issue years ago on a CodeIgniter site, but amended to work for Laravel. In the process, I realised it made sense to implement it as middleware, and I ended up pulling it out into a package. However, it’s still useful to understand the concept behind it, even if you prefer to just install the complete package, because your needs might be slightly different to mine.

On web development forums, it’s quite common to see variants of the following question:

How do I redirect a user on a mobile device to a mobile version of the site?

It’s quite surprising that this is still an issue that crops up. For many years, it’s been widely accepted that the correct solution for this problem is responsive design. However, there are ways in which this may not be adequate for certain applications. For instance, you may have an application where certain functionality only makes sense in a certain context, or your user interface may need to be optimised for specific environments.

The trouble is that a dedicated mobile site isn’t a good idea either. Among other things, it means that users can’t easily use the same bookmarks between desktop and mobile versions, and can result in at least some of the server-side logic being duplicated.

Fortunately, there is another way - dynamic serving allows you to render different content based on the user agent. You can also easily enable users to switch between desktop and mobile versions themselves if their client isn’t detected correctly or they just prefer the other one. I’ve implemented this years ago for a CodeIgniter site. Here’s how you might implement it in Laravel, although if you understand the principle behind it, it should be easy to adapt for any other framework.

Don’t try to implement mobile user agent detection yourself. Instead, find an implementation that’s actively maintained and install it with Composer. That way you can be reasonably sure that as new mobile devices come onto the market the package will detect them correctly as long as you keep it up to date. I would be inclined to go for Agent, since it has Laravel support baked in.

We could just use Agent to serve up different content based on the user agent. However, user agent strings are notoriously unreliable - if a new mobile device appears and it doesn’t show up correctly in Agent, users could find themselves forced to use the wrong UI. Instead, we need to check for a flag in the session that indicates if the session is mobile or not. If it’s not set, we set it based on the user agent. That way, if you need to offer functionality to override the detected session type, you can just update that session variable to correct that elsewhere in the application. I would be inclined to use a button in the footer that makes an AJAX request to toggle the flag, then reloads the page.

You also need to set the HTTP response header Vary: User-Agent to notify clients (including not only search engines, but also proxies at either end of the connection, such as Varnish or Squid) that the response will differ by user agent, in order to prevent users being served the wrong version.

Middleware is the obvious place to do this. Here’s a middleware that sets the session variable and the appropriate response headers:

<?php
namespace App\Http\Middleware;
use Closure;
use Jenssegers\Agent\Agent;
use Illuminate\Contracts\Session\Session;
class DetectMobile
{
protected $agent;
protected $session;
public function __construct(Agent $agent, Session $session)
{
$this->agent = $agent;
$this->session = $session;
}
/**
* Handle an incoming request.
*
* @param \Illuminate\Http\Request $request
* @param \Closure $next
* @return mixed
*/
public function handle($request, Closure $next)
{
if (!$this->session->exists('mobile')) {
if ($this->agent->isMobile() || $this->agent->isTablet()) {
$this->session->put('mobile', true);
} else {
$this->session->put('mobile', false);
}
}
$response = $next($request);
return $response->setVary('User-Agent');
}
}

Now, you could then work with the session directly to retrieve the mobile flag, but as you may be working in the view, it makes sense to create helpers for this:

<?php
if (!function_exists('is_mobile')) {
function is_mobile()
{
$session = app()->make('Illuminate\Contracts\Session\Session');
return $session->get('mobile') == true;
}
}
if (!function_exists('is_desktop')) {
function is_desktop()
{
$session = app()->make('Illuminate\Contracts\Session\Session');
return $session->get('mobile') == false;
}
}

Now, if you want to serve up completely different views, you can use these helpers in your controllers. If you instead want to selectively show and hide parts of the UI based on the user agent, you can instead use these in the views to determine what parts of the page should be shown.

Agent offers more functionality than just detecting if a user agent is a mobile or desktop device, and you may find this useful as a starting point for developing middleware for detecting bots, or showing different content to users based on their device type or operating system. If you just need to detect if a user is a mobile or desktop client, this middleware should be sufficient.

12th April 2018 11:57 pm

Making Wordpress Less Shit

I’m not going to sugarcoat it. As a developer, I think Wordpress is shit, and I’m not alone in that opinion. Its code base dates from a time before many of the developments of the last few years that have hugely improved PHP as a language, as well as the surrounding ecosystem such as Composer and PSR-FIG, and it’s likely it couldn’t adopt many of those without making backward-incompatible changes that would affect its own ecosystem of plugins and themes. It actively forces you to write code that is far less elegant and efficient than what you might write with a proper framework such as Laravel, and the quality of many of the plugins and themes around is dire.

Unfortunately, it’s also difficult to avoid. Over a quarter of all websites run Wordpress, and most developers will have to work with it at some point in their careers. However, there are ways that you can improve your experience when working with Wordpress somewhat. In this post I’m going to share some methods you can use to make Wordpress less painful to use.

This isn’t a post about the obvious things like “Use the most recent version of PHP you can”, “Use SSL”, “Install this plugin”, “Use Vagrant/Lando” etc - I’m assuming you already know stuff like that for bog standard Wordpress development. Nor is it about actually developing Wordpress plugins or themes. Instead, this post is about bringing your Wordpress development workflow more into line with how you develop with MVC frameworks like Laravel, so that you have a better experience working with and maintaining Wordpress sites. We can’t solve the fundamental issues with Wordpress, but we can take some steps to make it easier to work with.

Use Bedrock

Bedrock is still Wordpress, but reorganized so that:

  • The Wordpress core, plugins and themes can be managed with Composer for easier updates
  • The configuration can be done with a .env file that can be kept out of version control, rather than putting it in wp-config.php
  • The web root is isolated to limit access to the files

In short, it optimizes Wordpress for how modern developers work. Arguably that’s at the expense of site owners, since it makes it harder for non-developers to manage the site, however for any Wordpress site that’s sufficiently complex to need development work done that’s a trade-off worth making. I’ve been involved in projects where Wordpress got used alongside an MVC framework for some custom functionality, and in my experience it caused a world of problems when updating plugins and themes because version control would get out of sync, so moving that to use Composer to manage them instead would have been a huge win.

Using Bedrock means that if you have a parent theme you use all the time, or custom plugins of your own, you can install them using Composer by adding the Git repositories to your composer.json, making it easier to re-use functionality you’ve already developed. It also makes recovery easier in the event of the site being compromised, because the files outside the vendor directory will be in version control, and you can delete the vendor directory and re-run composer install to replace the rest. By comparison, with a regular Wordpress install, if it’s compromised you can’t always be certain you’ve got all of the files that have been changed. Also, keeping Wordpress up to date becomes a simple matter of running composer update regularly, verifying it hasn’t broken anything, and then deploying it to production.

Bedrock uses WPackagist, which regularly scans the Wordpress Subversion repository for plugins and themes, so at least for plugins and themes published on the Wordpress site, it’s easy to install them. Paid plugins may be more difficult - I’d be inclined to put those in a private Git repository and install them from there, although I’d be interested to know if anyone else uses another method for that.

If you can’t use Bedrock, use WP CLI

If for any reason you can’t use Bedrock for a site, then have a look at WP CLI. On the server, you can use it to install and manage both plugins and themes, as well as the Wordpress core.

It’s arguably even more useful locally, as it can be used to generate scaffolding for plugins, themes (including child themes based on an existing theme), and components such as custom post types or taxonomies. In short, if you do any non-trivial amount of development with Wordpress you’ll probably find a use for it. Even if you can use Bedrock, you’re likely to find WP CLI handy for the scaffolding.

Upgrade the password encryption

I said this wouldn’t be about using a particular plugin, but this one is too important. Wordpress’s password hashing still relies on MD5, which is far too weak to be considered safe. Unfortunately, Wordpress still supports PHP versions as old as 5.2, and until they drop it they can’t really switch to something more secure.

wp-password-bcrypt overrides the password functionality of Wordpress to use Bcrypt, which is what modern PHP applications use. As a result, the hashes are considerably stronger. Given that Wordpress is a common target for hackers, it’s prudent to ensure your website is as secure as you can possibly make it.

If you use Bedrock, it uses this plugin by default, so it’s already taken care of for you.

Use a proper templating system

PHP is a weird hybrid of a programming language and a templating system. As such, it’s all too easy to wind up with too much logic in your view layer, so it’s a good idea to use a proper templating system if you can. Unfortunately, Wordpress doesn’t support that out of the box.

However, there are some third-party solutions for this. Sage uses Laravel’s Blade templating system (and also comes with Webpack preconfigured), while Timber lets you use Twig.

Use the Wordpress REST API for AJAX where you can

Version 4.7 of Wordpress introduced the Wordpress REST API, allowing the data to be exposed via RESTful endpoints. As a result, it should now be possible to build more complex and powerful user interfaces for that data. For instance, if you were using Wordpress to build a site for listing items for sale, you could create a single-page web app for the front end using React.js and Redux, and use the API to submit it, then show the submitted items.

I’m not a fan of the idea the Wordpress developers seem to have of trying to make it some kind of all-singing, all-dancing universal platform for the web, and the REST API seems to be part of that idea, but it does make it a lot easier than it was in the past to do something a bit out of the ordinary with Wordpress. In some cases it might be worth using Wordpress as the backend for a headless CMS, and the REST API makes that a practical approach. For simpler applications that just need to make a few AJAX calls, using the REST API is generally going to be more elegant and practical than any other approach to AJAX with Wordpress. It’s never going to perform as well or be as elegant as a custom-built REST API, but it’s definitely a step forward compared to the hoops you used to have to jump through to handle AJAX requests in Wordpress.

Summary

Wordpress is, and will remain for the foreseeable future, a pain in the backside to develop for compared to something like Laravel, and I remain completely mystified by the number of people who seem to think it’s the greatest thing since sliced bread. However, it is possible to make things better if you know how - it’s just that some of this stuff seems to be relatively obscure. In particular, discovering Bedrock is potentially game-changing because it makes it so much easier to keep the site under version control.

10th March 2018 3:10 pm

Using Stored Procedures in Your Web App

In the last few days I’ve done something I’ve never done before, namely written a stored procedure for a web app. Like most web developers, I know enough about SQL to be able to formulate some fairly complex queries, but I hadn’t really touched on control flow functions or stored procedures, and in my experience they tend to be the province of the dedicated database administrator, not us web devs, who will typically delegate more complex functionality to our application code.

In this case, there were a number of factors influencing my decision to use a stored procedure for this:

  • The application was a legacy application which had been worked on by developers of, shall we say, varying skill levels. As a result the database schema was badly designed, with no prospect of changing it without causing huge numbers of breakages
  • The query in question was used to generate a complex report that was quite time-consuming, therefore the optimisations from using a stored procedure were worthwhile.
  • The report required that data be grouped by a set of categories which were stored in a separate table, which meant the table had to be pivoted (transformed from rows to columns), resulting in an incredibly complex dynamic query that had to be constructed on the fly by concatenating different SQL strings. In PostgreSQL, this can be done fairly easily using the crosstab function, but MySQL doesn’t have native support for anything like this.

Historically, one issue with using stored procedures has been that it kept business logic out of the application code, meaning they are not stored in version control. However, most modern frameworks provide some support for migrations, and since they are intended to be used to make changes to the database, they are the obvious place to define the stored procedure. This particular application was built with an older framework that didn’t come with migrations, so we’d installed Phinx to handle those for us. Initially, I defined the stored procedure inside a migration that ran a raw query to create the stored procedure, as in this example:

public function up()
{
$query = <<<EOF
CREATE PROCEDURE IF NOT EXISTS foo
BEGIN
SELECT * FROM foo;
END
EOF;
$this->execute($query);
}
public function down()
{
$this->execute('DROP PROCEDURE IF EXISTS foo');
}

Once this is done, you can then use your framework’s particular support for raw queries to call CALL foo() whenever your stored procedure needs to be executed.

However, we soon ran into an issue. It turns out mysqldump doesn’t export stored procedures by default, so there was a risk that anyone working on the code base might import the database from an SQL file and not get the migrations. I’d used the Symfony Console component to create a simple command-line tool, reminiscent of Laravel’s Artisan, so I used that to create a command to set up the stored procedure, amended the migration to call that command, and placed a check in the application where the procedure was called so that if it was not defined the command would be called and the procedure would be created. In most cases this wouldn’t be an issue.

Having now had experience using stored procedures in a web application, there are a number of issues they raise:

  • It’s hard to make queries flexible, whereas with something like Eloquent it’s straightforward to conditionally apply WHERE statements.
  • While storing them in migrations is a practical solution, if the database is likely to be imported rather than created from scratch during development it can be problematic.
  • They aren’t easily portable, not just between database management systems, but between different versions - the production server was using an older version of MySQL, and it failed to create the procedure. It’s therefore good practice for your migrations to check the procedure was created successfully and raise a noisy exception if they failed.

Conversely, they do bring certain benefits:

  • For particularly complex transactions that don’t change, such as generating reports, they are a good fit since they reduce the amount of data that needs to be sent to the database and allow the query to be pre-optimised somewhat.
  • If a particular query is unusually expensive, is called often, and can’t be cached, it may improve performance to make it a stored procedure.
  • Doing a query in a for loop is usually a very big no-no. However, if there really is no way to avoid it (and this should almost never happen), it would make sense to try to do it in a stored procedure using SQL rather than in application code since that would minimise the overhead.
  • If multiple applications need to work with the same database, using stored procedures for queries in more than one application removes the need to reimplement or copy over the code for the query in the second application - they can just call the same procedure, and if it needs to be changed it need only be done once.

Honestly, I’m not sure I’m ever likely to again come across a scenario where using a stored procedure in a web application would be beneficial, but it’s been very interesting delving into aspects of SQL that I don’t normally touch on and I’ve picked up on some rarely-used SQL statements that I haven’t used before, such as GROUP_CONCAT() and CASE. With the widespread adoption of migrations in most frameworks, I think that the argument that using stored procedures keeps application logic out of version control no longer holds any water, since developers can generally be trusted to store changes to database structure in their migrations and not start messing them around, so the same applies for stored procedures. Report generation seems to be the ideal use case since this invariably involves complex queries that run regularly and don’t change often, and this is where I expect it would be most likely I’d have cause to use them again.

25th February 2018 5:22 pm

Check Your Code Base Is PHP 7 Ready With PHP Compatibility

I’ve recently started a new job and as part of that I’m working on a rather substantial legacy code base. In fact, it was so legacy that it was still in Subversion - needless to say the very first thing I did was migrate it to Git. One of the jobs on our radar for this code base is to migrate it to from PHP 5.4 to 5.6, and subsequently to PHP 7. I’ve been using it locally in 5.6 without issue so far, but I’ve also been looking around for an automated tool to help catch potential problems.

I recently discovered PHP Compatibility which is a set of sniffs for PHP CodeSniffer that can be used to detect code that will be problematic in a particular PHP version. As I use CodeSniffer extensively already, it’s a good fit for my existing toolset.

To install it, add the following dependencies to your composer.json:

"require-dev": {
"dealerdirect/phpcodesniffer-composer-installer": "^0.4.3",
"squizlabs/php_codesniffer": "^2.5",
"wimg/php-compatibility": "^8.1"
},

Then update your phpcs.xml to look something like this:

<ruleset name="PHP_CodeSniffer">
<description>The coding standard for my app.</description>
<file>./</file>
<arg value="np"/>
<rule ref="PSR2"/>
<rule ref="PHPCompatibility"/>
<config name="testVersion" value="7.2-"/>
</ruleset>

As you can see, it’s possible to use it alongside existing coding standards such as PSR2. Note the testVersion config key - the value specified is the PHP version we’re testing against. Here we’re specifying PHP 7.2.

Obviously, the very best way to guard against breakages in newer versions of PHP is to have a comprehensive test suite, but legacy code bases by definition tend to have little or no tests. By using PHP Compatibility, you should at least be able to catch syntax problems without having to audit the code base manually.

Recent Posts

Better Strings in PHP

Forcing SSL in Codeigniter

Logging to the ELK Stack With Laravel

Full-text Search With Mariadb

Building a Letter Classifier in PHP With Tesseract OCR and PHP ML

About me

I'm a web and mobile app developer based in Norfolk. My skillset includes Python, PHP and Javascript, and I have extensive experience working with CodeIgniter, Laravel, Django, Phonegap and Angular.js.