Fixing Shortcomings in the Composer Update Process with Drupal-Composer

PHP is going through a renaissance, and a big part of it is the Composer package manager, which makes it easy to share common libraries of code between projects. With Drupal 8 embracing Composer to pull in core components like Symfony2 and key libraries like Guzzle, many developers are eager to use Composer to manage their Drupal sites more fully, and best practices are beginning to emerge. For more background see my previous blog post comparing Composer with Drush Make, and check out drupal-composer/drupal-project for some useful tools to start managing a Drupal site with Composer.

However, we have not yet reached dependency heaven. When using Composer with Drupal 8, the common pattern is to manage only the core directory with Composer. This works very well with the Composer workflow, but what about tracking the other parts of Drupal that live outside of core?

The Composer Update Process

The great thing about Composer is that it makes it very easy to keep your projects up-to-date.  As new versions of your dependencies are released, one simple command—composer update—will cause Composer to go to the work of ensuring that all of your dependencies, including your dependencies’ dependencies, are updated in a self-consistent way. Composer is, after all, a dependency manager; this is what it does best. Composer also provides a command for starting new projects; if you run composer create-project, and provide the URL of a project template, then Composer will copy and rename this project for you, giving you a fresh place to begin writing code. 

The Problem with Scaffolding Files

In any Composer-managed project, some of the code will be specific to the project, and some will be provided by dependencies, as defined in the composer.json file.  The create-project command may provide project-specific files in addition to the files provided by the newly-created project’s dependencies. These files are called scaffold files, because they are intended to be used as the initial scaffolding for a project; these files are typically filled in by the project implementer, and, as such, they are specific to and owned by the project. Scaffold files, therefore, typically are not updated by Composer, because they are not expected to change upstream.

On Composer-managed Drupal sites, though, some of the project-specific files are provided upstream, such as the index.php file, robots.txt, and sites/default/default.settings.php. These files may occasionally change from release to release, and, while it likely wouldn’t cause any problems to miss an update to default.settings.php file, changes to index.php could be important, and should not be missed. As previously mentioned, though, composer update does not update scaffold files.  What to do?

Solving Update Problems with Composer Script Hooks

Fortunately, Composer has anticipated that some projects might need special handling during the update process, and has provided a number of scripting hooks that can be used to perform additional tasks before or after an update. For example, the following composer.json snippet will run an update script at the end of every composer update operation:

    "scripts": {
        "post-update-cmd": "sh ./scripts/post-update.sh"
    }, 

This script can do whatever operations are necessary to fetch and update the scaffolding files. For example, drupal-composer/drupal-project already has an update-scaffold script that fetches the latest scaffold files from Drupal.org, and copies them into the project. In this simple example, a pre-update command and a post-update command work in concert to update the scaffold files, but only in instances where the version of Drupal core is also updated.  You should already be in the habit of committing your composer.lock file in your git repository; now, whenever you run composer update, you should also use git status to check and see if any of the scaffold files have also changed, and, if so, ensure they are added to the repository in the same commit as the updated composer.lock.

Improving Our Scripts with Robo

Short bash scripts are a very quick way to add some simple functionality to your Composer update process, but script maintenance can start to become an issue when processes get more complicated, and multiple scripts are in use. A better strategy is one that allows scripts to be written in PHP, and use all of the code available to the project via the Composer autoload file.

Robo is a PHP task runner that fits this bill nicely.  If you require codegyre/robo from your project’s composer.json file, then the Robo task runner will be available in vendor/bin, and it will load the Composer autoloader whenever it is used to run a script. It contains a number of very useful library of tasks to do File and FileSystem operations and more. For example, you could place the following snippet in your composer.json file:

    "require-dev": {
        "codegyre/robo": "^0.6.0"
    },
     "scripts": {
        "robo": "robo --load-from $(pwd)/scripts/drupal",
        "pre-update-cmd": "composer robo version:dump",
        "post-update-cmd": "composer robo update:if-newer-version"
    },

The “robo” line in the scripts section tells Composer what to do when composer robo is executed. This is a handy way to add macros to your project for a variety of purposes; we use it to provide a common entrypoint for the pre and post-update commands, so that the location of the RoboFile.php (the file that holds your Robo tasks) can be set in a single location. Robo uses a simple heuristic to convert php method names into Robo commands available to be used from the command line; version:dump and update:if-newer-version are defined by the php methods versionDump and updateIfNewerVersion, respectively.  

Here is an example Robo script that updates the Drupal scaffold files; this is functionally equivalent to the bash script, but the organization is better. We can also get the current Drupal version using \Drupal::VERSION, which is exposed to our scripts by virtue of the fact that Robo includes our project’s autoload file. This allows us to use any of the classes provided by any of our dependencies directly from our script. Using Composer dependencies from a standalone script is usually awkward; with Robo, it is easy.

Packaging Our Update Scripts in a Composer Installer

Custom Robo scripts are a great technique to use when your update actions are very specific to your particular project. In instances where the actions are commonly needed by many different projects, then it makes sense to factor out the script into a custom Composer installer. The drupal-composer/drupal-scaffold project does exactly that; now, if you are using the recommended drupal-composer/drupal-project to manage your Drupal 8 site with Composer, all you need to do is require drupal-composer/drupal-scaffold in your composer.json file, and your Drupal scaffold files will be updated whenever the version of drupal/core changes. If you examine the implementation of drupal-composer/drupal-scaffold on GitHub, you will see that it also uses Robo internally to make the update. This project would therefore make a good example starting point for creating any similar plugins to update scaffold files for other sorts of projects.

There is currently an open pull request in drupal-composer/drupal-project that uses the drupal-scaffold project to set up and update the scaffold files on the branch for Drupal 8. This PR will probably be merged in to the 8.x branch shortly, but you can take a sneak peak at it today. Examining the techniques it uses will add tools to your reputare that you can use to solve other problems.

Acknowledgements

I am deeply grateful for the amazing amount of work that Florian Weber (webflo), Johannes Haseitl (derhasi) and David Barratt (davidbarratt) have done to create and maintain the suite of Drupal Composer tools and Drupal Packagist, which makes Composer-Managed Drupal sites possible.