Powering up git bisect with the run command
Published by Matthew Daly at 19th June 2019 9:00 pm
The bisect
command in Git can be very useful when trying to catch any regressions. If you know that a bug was not present at some point in the past, and now is, you can often use bisect
to track it down quickly and easily.
The basic functionality is fairly simple. You start the process by tracking down a known "good" commit in the past, and a known "bad" commit, which will usually be the head of the branch. Then, you start bisecting:
$ git bisect start
You then specify your bad commit:
$ git bisect bad HEAD
And your good commit
$ git bisect good fe0616f0cd523455a0e5bc536c09bfb1d8fd0c3f
And it will then step through the commits in between. Note that not every commit is loaded - it instead picks a commit between those you entered, and from there quickly narrows down the range. For each commit, you test it and mark it as good or bad with git bisect good
or git bisect bad
as appropriate. Once it's tracked down the commit that introduced the problem, it will tell you what that commit was, making any remaining debugging much easier. There are situations that are more difficult to handle, such as when database migrations have been created and run in the intervening period, but for many cases bisect
can be a very valuable tool.
However, it can still be a chore to step through those commits manually. Fortunately, in situations where you can produce some sort of script to determine if the issue is present or not, there's an easy way to automate it with the bisect run
command.
One of the personal projects I have on the go right now is a micro-CMS intended primarily for brochure-style sites. It includes an AJAX search that uses Fuse.js on the front end, the index for which is generated by a console task built on top of the Symfony Console component. Recently I noticed that although the unit tests still passed, the console task to generate the index no longer worked as expected due to an issue with Flysystem. Since it threw an error in the console, that could be used as input to git bisect
. I was therefore able to automate the process of finding the bug by running this command:
$ git bisect run php console index:generate
This was pretty rare in that it was an ideal situation - the problem was the console command throwing an explicit error, which was perfect as input to bisect run
. A more likely scenario in many cases is that if you want to automate catching the error, you'll need to create an automated test to reproduce that error, and run that test with git bisect run
. Given that TDD already recommends writing a test to reproduce a bug before fixing it, it's prudent to write the test first, then use it to run the bisect command, before fixing the bug and committing both the fix and the new test, so as to not only minimise the manual work required, but also ensure it won't crop up again.
Certain classes of issues are more difficult to automate in this way - for example, visual regressions in CSS. If you're using a library like React or Vue, snapshot testing may be a good way to automate the bisect process for HTML rendered by components, or you could try the approach I've mentioned before for snapshot testing PHP applications. For legacy applications that can't create and tear down a database for testing purposes due to gaps in the migration history, it can also be tricky and time-consuming to ensure consistency between runs. However, if you can do it, automating the bisect command makes it much quicker, and leaves you with a test you can retain to ensure that bug never returns again.