How Pantheon Tests WordPress Updates

In a previous blog, we described how Pantheon tested Drupal 8 updates using a CircleCI feature that allowed job steps to be stored directly in the CircleCI admin interface. This capability was very important to our use-case, as we cannot commit any extra files to the master branch of our repository, as is explained below. In CircleCI 2.0, though, jobs may only be defined in configuration files stored in the repository, usually in .circleci/config.yml.  In this blog, we will explore how Pantheon uses features of CircleCI 2.0 and GitHub to test WordPress updates without the test configuration files getting in the way. 

Testing WordPress Updates

First, a brief recap of our requirements are in order. Pantheon maintains a copy of the WordPress master repository, customized with changes particular to the Pantheon platform. Most of these changes involve only the addition of new files, so merging in the commits from a new upstream release rarely presents conflicts.

This Pantheon repository is itself used as the upstream repository for all of the WordPress sites on the platform. Whenever a change is committed to the master branch of the Pantheon WordPress repository, every user is notified that their WordPress sites now have updates available. These updates may be applied with the click of a button on the site’s dashboard.

Merge conflicts can be very confusing for some users, and are at the very least an inconvenience for anyone. Pantheon therefore has a policy of avoiding any change that might cause conflicts. Since some of our users add CircleCI configuration files to their repositories to test their own sites directly, we do not want to have a copy of any of those same files in our upstream. This leaves us with the challenge of figuring out how to test our code without having any CircleCI configuration files on the master branch.

This first thing we did to get this working was to create a new default branch named, naturally enough, “default”. We put the CI test scripts on the default branch, and keep the master branch clean.

Default & Mastering

If an outside party visits our GitHub repository, they will be presented with the default branch rather than the master branch. This makes it natural that any pull request contributed will be forked from the default branch where the tests are.

We have a separate automation process that regularly checks for new releases to become available. When this happens, a pull request is created automatically. This pull request is also created from the default branch, so it contains the CircleCI test scripts.

PR on default

The CircleCI 2.0 tests are equipped with a suite of Behat tests that cover basic WordPress functionality. Once these tests have passed, we have a high degree of confidence that the new version may be safely released to our customers. Once this is confirmed, a human presses the GitHub “Rebase and Merge” button, merging the pull request onto the default branch without creating any merge commits.

At this point, the Circle scripts will run again. This time, though, once they pass a second job will run. This job examines the commits on the default branch one at a time, and cherry-picks each one that contains no changes in the .circleci directory, depositing the results back on the master branch. This happens automatically, without any human intervention.

Merge to default and cherry pick

Once this is done, the default branch is rebased against the new master and tagged with the same semver version that the initial pull request was created from.

Rebase after cherry picking

The script that does the merge is stored in our docker container. The basic operations are shown below:

# Commits on the 'default' branch not yet on master in reverse order
# ignoring any commit that modifies only files in .circleci
commits=$(git log master..HEAD --pretty=format:"%h" -- . ':!.circleci' | sed '1!G;h;$!d')

git checkout master
for commit in $commits ; do
  git cherry-pick $commit

git checkout -
git rebase master

The term --pretty=format:"%h"” causes “git log” to print out only the sha-1 hash of each commit. Commits that contain changes in the .circleci directory are filtered out via the “:!.circleci” term, and sed is used to reverse the order of the output.

It would be quite inconvenient if the CircleCI scripts were ever merged to the master branch, though; if this ever happened, the resulting commits would immediately be sent to customer dashboards. Users who accepted this release might experience merge conflict errors if their sites already contained a .circleci/config.yml file. When Pantheon later corrected this problem by removing the CircleCI configuration file, we would then cause merge conflicts again with any site that had already resolved the merge conflicts.

It is best to put in protections that absolutely prevent this sort of error from happening in the first place. To do so, we employ the GitHub branch protection feature:

Branch protection for Master

We require pull request reviews before merging to the master branch, and furthermore, we require that the approval be given by a designated code owner. However, we do not define any code owners, thereby making it impossible to merge pull requests to the master branch through normal means. With this configuration in place, merging to the master branch can only be done by a repository member with “admin” permissions. All humans users are given “write” permissions at most; this gives enough access to allow authorized users to merge to the default branch. The automation bot user is given permission to push to the master branch, allowing the automation scripts to keep the master branch in sync with the default branch as described above.

With all of these automation steps in place, Pantheon is well positioned to quickly and safely deploy new WordPress updates to our customers. While your workflows might not look quite like this, you might still use similar techniques to insert build automation steps into a project that needs to keep a branch in conformance with certain project constraints. With the power of CircleCI 2.0 and GitHub, there are a lot of things that you can do.

You may also like:

Topics Development, Testing & Optimization, WordPress