Agencies: How to Implement Software Automation on a Budget

Software automation, if you've never done it before, it can seem like an incredibly daunting task. Where do you start? What’s important to get right? What isn't that essential, but more of a nice-to-have feature?

These are some of the many questions that you're likely to ask yourself after you've made a commitment to introduce software automation to your organization.

squeezed piggy bank

If you were to feel a sense of stress and anxiety, I'd not blame you. Sure, you've heard about the multitude of benefits of implementing a software automation workflow, including:

  • Less human intervention required to roll out software releases

  • Shorter deployment cycles

  • Identification of bugs earlier in the software development process, where they're less costly to address

  • Increased accuracy of testing

These are all compelling benefits, especially if you're a development agency with tight deadlines and budgets. The status quo is having to meet deadlines which enterprise software normally wouldn't, with budgets far smaller in size. All the while, the client still expects the same level of quality and reliability which an enterprise product offers.

To be able to reduce the human cost, and the time involved in releasing software means that you can either take on more projects or produce the same level of projects in less time. Both of these outcomes results in a higher ROI and a healthier bottom line.

The challenge is that implementing a software automation strategy requires an investment of time and money. And until it's running smoothly, it will also introduce an element of doubt and uncertainty as to its validity.

Will it work? Will it produce the oft-cited benefits? Will we see improvements in quality, reliability, client happiness, and turnover? Perhaps most importantly: How long will it take? When will you start to see these gains?

There is no exact answer to these questions, as every organization is different. So, let's set these issues aside, and instead look at how you can start, even when you have no prior experience.

Where Do You Start When You Have No Experience?

If you're just starting out, the first thing to do is this: assess what your needs are.

How, you may ask. Start by determining the kind(s) of software which your agency creates and ships. Are you a WordPress shop? Are you a Drupal shop? Are you a mixture of both, or something entirely different?

Are you not especially focused on a given technology, but rather focused on implementing solutions which the client requests?

After you've done that, of the sites which your agency creates, determine if there are any that are created more often than others. For example, perhaps you receive most requests for Drupal sites. Perhaps they even form up to 65% of all the work that you do.

You then find that, of the remaining 35%, 20% are for applications built using Zend Expressive, and the final 15% are created using WordPress.

Given that, it makes sense to consider implementing software automation primarily around these three tools, based on the percentage of the income which they generate for your agency.

Develop An Initial Strategy

With the initial review now complete, you're in a good position to start mapping out an initial software automation strategy. As Drupal provides the lion's share of your agency's income, the next step is to review what, if any, automated testing infrastructure it provides.

Fortunately—especially with the 8.0 release—Drupal provides a very competent testing infrastructure, making use of PHPUnit in core and Behat used by much of the community.

With this information in hand, the next step is begin assessing the level of experience your developers have with creating automated tests. Perhaps they don't have any, and testing comes down to one (or more) dedicated QA people who continuously test various parts of the site.

While a good start, it's time to begin building up your developers' proficiency with testing and test automation. That way, no matter what they develop they can start to self-check their work, as well as have other developers independently assess the quality of their testing efforts.

Have Patience While Implementing The Strategy

Like anything new, at first there may be pushback, even open distrust of the quality of the results provided by automated tests. This is perfectly reasonable and normal.

It's common for developers to doubt that which they don't know—especially if it requires getting out of their comfort zones, and expending effort to become proficient.

For what it's worth, I don't offer silver bullet solutions, nor quick fixes. I encourage the slow and steady uptake of solutions which provide tangible results over the course of time.

Given that, you're going to need a company advocate, someone tasked with championing the initiative of bringing automated testing into the organization.

As well as being the technical point of call, and training the developers and other staff, this person will also have to assuage any fears and concerns about this new direction.  

This, likely more than anything else, will be where they initially spend most of their time.

Regularly Report Back on Implementation Progress

Now that you've begun training your staff in creating automated tests, they have to receive regular feedback, feedback which is as active and encouraging as possible.

This can be done in a variety of ways. For example, you could have a scoreboard of some kind, which reports the current build status of all projects, such as this one from Codeship:

Codeship build status dashboard

Keeping it as concise as possible, have it report metrics such as:

  • The project name

  • The build id

  • The number of tests

  • The level of test coverage

That way, it's always readily available to everyone in the organization. After that, give positive feedback to the team each time the percentage increases. Conversely, be critical every time the rate declines, or when someone breaks the build.

If you're not familiar with the phrase "break the build", it's where changes are made which causes one, or several, tests to fail.

I'm not suggesting to berate your team members, rather to provide them with a clear and straightforward measure of success, and then to positively reinforce gains made in the improvement of that success ratio.

Begin Automating the Tests

Now that the team is actively engaged in writing tests, they need to be automated. If they're left up to manual intervention, it's inevitable that they will eventually be skipped, which may in time result in them falling out of date and becoming irrelevant, otherwise known as Test Rot.

To run your build and tests, you could setup a local build server, such as Jenkins or Travis CI. Pantheon's Multidev cloud environments are a great place to deploy sites meant for automated testing and review.

Regardless of what you choose, here's how the process will work; sticking with our Drupal frame of reference. A developer makes a change to one of the code files, in a custom module they've written, and commits the change to version control.

As a result, the continuous integration server, or remote service, will then initiate all of the tests which have been written for the application.

When the process is complete, you'll be able to see the build status, whether by viewing the test dashboard, receiving an email with the build status, or both.

From that information, you'll know if the site still works as expected, or if a bug has been introduced. If the build failed, the developer who broke the build then has to fix the issue which they created and commit and push a subsequent change, in turn triggering another build.

When the build succeeds, and only then, can a deploy to one of the staging or production environments be made.

At all times, all staff members can know what's going on, what changed, when, by whom, and whether the application still works or not. I hope you see by now just how much confidence such a workflow can deliver!

workflow graphic  

The Definitive Guide to Agency Optimization
This guide will help you increase efficiency and set standard workflows that allow your agency to take on bigger projects and deliver more value to clients.


The Downside of Poorly Implemented Automation Strategies

While it's good to talk about the upsides of implementing automation strategies, what about the downsides? To my mind at least, there aren't a lot—but they're important. Chief amongst them is that it can breed a sense of complacency or ill-placed trust.

Just because tests are in place and the build is consistently passing, doesn't mean that the application works faultlessly. What's more, it's not a blank cheque to assume that no fault will ever be discovered in the future—especially if your organization is new to automated testing.

An article by talks about several common mistakes junior developers make when writing tests, which are worth noting; these are:

1. Writing the wrong type of tests

2. Writing tests only for successful and well understood paths

3. Tests that are poorly named

4. Tests that aren't specific enough

5. Tests that don't actually test everything

To summarize, while the developers have written a vast array of tests which are saying that the build is successfully passing, for all intents and purposes, this information may be next to meaningless.

Why? Because having tests which imbue a false sense of confidence is worse than having no tests at all. Given that, it's important to mentor developers, especially in the early stages, so that they create tests which are meaningful and valuable.

There needs to be a mindset within the development team that not only are automated tests the right approach, but that the tests which are being written and run truly test the application in its entirety. That way, the organization can continue to ship software with confidence, leading to satisfied clients.

To get you off on the right foot, here is a selection of excellent resources:

In Conclusion

Implementing software automation introduces a range of tangible benefits, regardless of the type of project, or the size of the development team involved. However, it's not without its challenges.

But, when done correctly, even in organizations where the deadlines and budgets are quite tight, such as digital agencies, software quality can be increased, bugs can be reduced, and customers can become increasingly confident that what they deliver will work, and work reliably.

You may also like: 

Topics Agencies, Website Technology, Digital Agencies, Development, Agency Partners, Drupal, WordPress