Test Terminus Plugins with CircleCI 2.0

Come August 31st, CircleCI version 1.0 will be discontinued. That’s just around the corner, so it is important to plan on upgrading shortly. In this article, we will look at the CircleCI 2.0 test suite in the Terminus Plugin Example, and show how the CircleCI 2.0 configuration in this project can be used to test your own Terminus plugins. The topics covered here will also be of interest to other types of projects, although more effort will be required to customize the build steps.

If you followed the example of one original Terminus plugins to set up your tests, you can start off by deleting the following directories from disk and your .gitignore file (if applicable):

• ​bats

• bin

• libexec

• share

• circle.yml

These things are now going to be stored in a different location. If your Terminus plugin does not have any tests yet, no problem! Just continue on to the next step.

Copy the following things from the Terminus Plugin Example into your Terminus plugin project:

• Copy the .circleci directory.

• Copy the tests directory, if you do not already have one.

• From the example project’s composer.json file, copy the “scripts” section into the composer.json file in your project.

• Add the lines “tools” and “vendor” from the example project’s .gitignore file to your project’s .gitignore file.

Note that in the Terminus Plugin Example project, the functional tests can be found in the directory tests/functional. If you have an existing set of tests in the tests directory with filenames ending in .bats, move them into tests/functional.

Once you have done this work, you’ll be most of the way there. A little bit of customization might also be needed. We’ll go over each part one step at a time, and then break down what is inside each section.

We will start by explaining the CircleCI 2.0 configuration that we will use for these tests. CircleCI 1.0 used a file called circle.yml in the root of the repository to store configuration settings. The preferred location for configuration in CircleCI 2.0 is now .circleci/config.yml, and its structure has changed. Open the version of this file that you copied into your project in the previous step.

At the top of the file, it is traditional to define default configuration sections that will be used in multiple places in your CircleCI configuration. Most CircleCI 2.0 configuration files define a section called “defaults”; the defaults from the Terminus Plugin Example looks like this:

defaults: &defaults
  docker:
    - image: quay.io/pantheon-public/terminus-plugin-test:1.x
  working_directory: ~/work/terminus_plugin
  environment:
    BASH_ENV: ~/.bashrc
    TZ: "/usr/share/zoneinfo/America/Los_Angeles"
    TERM: dumb

This is actually not a CircleCI-specific feature, but is actually a standard characteristic of yml files called structures. The most important setting here is the Docker image to use, defined in the “docker: image” attribute. For our tests, we will use an image provided by Pantheon that contains Terminus and a number of test tools pre-installed.

Another big change in CircleCI 2.0 is that tests may be divided up into multiple jobs. To start off, we will define all of our test steps in a single job:

version: 2
jobs:
    test:
        <<: *defaults
        steps:
            - checkout
            - run:
                name: Set up environment
                command: ./.circleci/set-up-globals.sh
            - run:
                name: Dependencies
                command: composer install
            - run:
                name: Lint
                command: composer lint
            - run:
                name: Unit
                command: composer unit
            - run:
                name: Functional
                command: composer functional
            - run:
                name: Style
                command: composer cs

We’ll break down the operations in each step later. If a single step in a job fails, then the steps that come after it will not run. This often is exactly what you want: it does not make sense to run the unit tests if the lint tests fail, and similarly, it is usually not necessary to run the functional tests if the unit tests are not passing (although this is more a matter of opinion). In other cases, though, you may want to see the results of some of the later tests even if you get failures in earlier test steps. For example, it is still interesting to see the results of the code style checks if the tests are failing, and visa-versa. We can achieve this by splitting our test steps into two separate jobs that run in parallel. We do this by inserting the configuration lines to define a new job in the jobs section right above the existing Style step. The result looks like this:

Note that the new job uses <<: *defaults, just as the initial job did. This statement injects the default configuration that we defined above. It is possible to use a different docker container for every job, but here we will use the same one for both.

code-style:
        <<: *defaults
        steps:
            - checkout
            - run:
                name: Set up environment
                command: ./.circleci/set-up-globals.sh
            - run:
                name: Style
                command: composer cs

Once we have more than one job, we need to define how our jobs relate to each other. We will create a workflows section that lists both jobs, but does not define any dependencies between them. This will allow the jobs to run in parallel.

workflows:
  version: 2
  build_test:
    jobs:
      - test
      - code-style

Now, both the test and code-style jobs will run whenever we push a new commit to our project. Both of these jobs still run in a little less than 30 seconds each, but since they run in parallel, and start at about the same time (usually within a few seconds of each other), the overall build time is very similar. Another advantage of splitting up the tests like this is that now we can see the results of the test job and the code style job independently:

Backing up a bit, we’ll examine how the actions in each job are defined.

Checkout

The “checkout” statement causes the code from the project being tested into the location defined by the working_directory defined in the defaults section. Most jobs will begin by checking out the project code.

Set up Globals Script

The set-up-globals.sh script defines any variables needed to allow our tests to run. The most important thing done by this script is to set up the $PATH environment variable. The way that CircleCI 2.0 supports environment variables is a little indirect. CircleCI 2.0 projects may write script commands into the file identified by the environment variable $BASH_ENV. This file is sourced by CircleCI 2.0 at the beginning of every run statement.

Composer Scripts

After the globals have been set up, our CircleCI configuration file runs a number of Composer scripts to run our actual tests. These are defined in the scripts section of the composer.json file that you copied over earlier.

• composer install: Installs all of our dependencies (if any) and generates the files used by the Composer autoloader.
• composer lint: Runs a syntax checker on all of our php files.
• composer unit: Runs the phpunit tests.
• composer functional: Runs the bats functional tests.
• composer cs: Runs a code style check on all of our php files.

See the composer.json file for the definition of each of these commands; it should not be necessary to customize these scripts in most instances. Note that you may also run these commands locally if you’d like to see the results of your tests before shipping your code off to CircleCI. There’s even a master command, composer test, that will run all of the tests.

Next we’ll review how to write more unit tests and more functional tests, and then we’ll be done.

Unit Tests

The best way to write testable code is to always write functions that take parameters and return results without creating any side effects. Let’s take the following section of code as an example. Below is a simplified implementation of the “hello” command:

 public function sayHello()
    {
        $name = $this->getAuthenticatedUserName();
        $this->log()->notice("Hello, {name}!", ['name' => $name]);
    }

This function takes no parameters and returns no values, and it writes a message to the logger, which is a side effect. In order to write a unit test for this method, we would need to mock the authenticated session object and the logger, and check to see whether our mocks were used as expected. While it is certainly possible to proceed like this, there is a better way to design our tests. Let’s factor out the main work being done by this method into a testable service class:

public function sayHello($options = ['type' => 'hello'])
    {
        $name = $this->getAuthenticatedUserName();
        $greeter = new Greeter($options['type']);
        $this->log()->notice($greeter->render($name));
    }

The sayHello method has all of the same characteristic side effects as before; we therefore simply will not test this method in our unit tests, and will instead focus our efforts on testing the Greeter class.

class Greeter
{
    public function __construct($greetingType)
    {
        $this->greetingType = $greetingType;
    }

    public function render($name = 'World')
    {
        $template = $this->getGreetingTemplate($this->greetingType);
        return strtr($template, ['{name}' => $name]);
    }

    protected function getGreetingTemplate($greetingType)
    {
        … 
    }
}

We now have a simple service class that can be created via parameters, and it contains a public method that takes parameters and returns a result. We will not concern ourselves with the implementation of the getGreetingTemplate method; it is an internal (protected) function, so we will test it indirectly by ensuring that our coverage of the class’ public interface is adequate.

Our test for the Greeting class looks like this:

    /**
     * @dataProvider greeterTestValues
     */
    public function testGreeter($expected, $c_param, $name)
    {
        $greeter = new Greeter($c_param);
        $this->assertEquals($expected, $greeter->render($name));
    }

This is a good test, because all that it does is exercise the public interface of our class under test, and assert that the result of the method, given the provided inputs, matches the expected outcome. Tests like this are easy to write and easy to understand.

It is generally insufficient to test a public method with a single call; most code that requires tests will contain conditionals and other behaviors that will cause the actions of the function—and ergo its results—to vary depending on its inputs. This need is often satisfied by duplicating the same test code over and over again, using different parameters each time. If the tests are structured as shown in the example above, though, then a phpunit @dataProvider annotation can be used to run the same tests multiple time using values provided by another function. Our greeterTestValues method looks like this:

public function greeterTestValues()
    {
        return [
            ['Hello, World!', 'hello', 'World', ],
            ['Good morning, Vietnam!', 'morning', 'Vietnam', ],
            ['Good evening, Mr. Bond!', 'evening', 'Mr. Bond', ],
        ];
    }

In the case of this example, the testGreeter method will be called three time, once for each set of parameters returned by the data provider method. If you can use this pattern over and over again, with one test method and data provider for every public interface of your classes under test, then you will be winning at unit testing.

What then of the methods with side effects that we skipped? These methods are best tested functionally, as shown below.

Functional Tests

We use Bats for writing our Terminus functional tests. Bats is ideal for our needs, as it allows us to write simple scripts to call our Terminus plugin using Terminus, and check the result.

An example Bats test looks like this:

@test "run hello command" {
  run terminus hello
  [[ $output == *"Hello, World!"* ]]
  [ "$status" -eq 0 ]
}

The run statement causes Bats to run the following bash statement and populate the variables $output and $status with the results. We can then use ordinary bash comparison functions such as [[ ]] and [ ] to test the results against our expectations. The operator == *"string"* can be used to test if the command output contains the given string. Since Bats tests are essentially just bash scripts, they are easy to write. If you need a reference to a test fixture, such as a Pantheon site to test your Terminus extension with, you may define environment variables in the CircleCI administration interface. 

In the example above, we add the environment variable TERMINUS_TOKEN, and set its value to the machine token for the user our tests will log in as. Once this has been set up, we can then use it in a functional test as shown below:

@test "Attempt to log in with a Terminus machine token" {
  run terminus auth:login --machine-token="$TERMINUS_TOKEN"
  [[ $output == *"[notice] Logging in via machine token"* ]]
  [ "$status" -eq 0 ]
}

Following these patterns, you should be able to quickly add your own functional and unit tests to confirm the behavior of your Terminus plugins. Need more help? See the documentation on migrating to CircleCI version 2.0.

You may also like: 

• [BLOG] Highest / Lowest Testing with Multiple Symfony Versions

• [GUIDE] Pantheon Platform Integrations

• [VIDEO] Bringing Drupal Site Updates with Composer to the Masses

Photo credits:
Train on bridge by GSWRHS Collection https://www.flickr.com/photos/gcargeeg/28316262089/
Helicopter by Jack Snell https://www.flickr.com/photos/jacksnell707/3061928150/