The Definitive Guide to Agency Optimization

Every Day I’m Shufflin’: Continuous Integration, Delivery, and Deployment

Continuous Integration, Continuous Delivery, and Continuous Deployment progressively build on one another. Continuous Integration (CI) is the process of automatically turning the commits you make in your codebase into a useable build (website). Continuous Integration also requires some level of testing to verify that the build process succeeded. With CI you define how your code is transformed into a working site; often living in a sandbox environment.

Continuous Delivery introduces the concept of a “deployment pipeline.” Continuous Delivery forces you to define in detail how your deployment pipeline takes changes from that sandbox and gets them the production environment. Sites on Pantheon use a deployment pipeline that goes through our Dev, Test and Live environments. Doing Continuous Delivery requires adding on to the automated tests created to fulfill Continuous Integration. When your changes move through the deployment pipeline, your Continuous Delivery process should validate that the site works in each environment.

Continuous Deployment is a business decision to send all changes straight through the deployment pipeline once tests have passed on those changes. With a strong deployment pipeline created in Continuous Delivery, Continuous Deployment is a simple business question. Ask your stakeholders “Do you want each change sent through the deployment pipeline to live as soon as the change is approved/merged? Or do you want changes held so that they are deployed in bunches at the end of each sprint (or other schedule)?” For many web agencies and their clients it is preferable to stick with scheduled releases.

Dave Ross

Dave Ross, Associate Director of Engineering, 10up


Continuous Integration puts our work in front of clients and teammates early on, so we can address poor user experiences and malfunctioning code while an engineer is focused on the project. CI also helps merge together independently written code as soon as possible. This way, conflicts can be identified and remedied (or opportunities to share code can be exploited to the benefit of the project). Our engineers can find issues on staging and fix them before they result in QA showstoppers or costly production issues.

I used to work in environments with weekly deployments. A change finished on Wednesday went out on the following Tuesday, alongside newer changes that just passed Monday night’s QA process. If something is ready now, there's no sense making 10up's clients or their customers wait for it. Additionally, in the rare event something goes wrong during deployment, we only need to roll back a small changeset rather than a big bundle of features.

Abandoning the weekly "deploy day" doesn't mean adopting undisciplined "cowboy coding." CI and CD depend on standards and rigor. Code reviews, automated testing, and formal QA help ensure that the work we push live meets 10up's standards for performance and security (while still delivering the functionality our clients need).

Before you start chasing the holy grail of Continuous Deployment, ask your team what is currently standing in the way of a solid CI process or deployment pipeline.

4 Questions for Getting Started with CI

  • What manual steps we are doing repeatedly in our workflow?

  • How do we know when our site is working or broken? How can we answer that question with a script?

  • Is everyone on the team team compiling their changes (to Sass, CoffeeScript or Composer) in exactly the same way?

  • What is the unit of work on which we want to perform builds? Pull requests, each commit, or specially named branches?

Teams often create long wish lists of all the steps they want to automate. It is important to remember that all of the automation you do should add value for your client and your team.  With each task you want to automate, think about the value that automation will bring. Here are some of advantages you can get from each step you script:

  • Trust: You can trust your co-workers more when they say a new feature worked on their machine. The development process is less opaque, so non-dev stakeholders have more confidence in the project.

  • Speed: Setting up a CI process can be time-consuming at first. But over time it pays huge dividends, as your team no longer spends time resolving merge conflicts.

  • Lower Mental Overhead: With an automated CI process, developers don’t have to remember and manually execute a complex series of steps, freeing up their mental RAM for more important tasks.

  • Clarity: CI canonizes your preferred approach to adding and updating modules and plugins. With the process codified into a script, it’s clear to everyone what the best practices are.

  • Lower risk of regression: Teams doing continuous integration are less likely to discover the task they thought was done is no longer done.

Here are some popular tools for CI/CD:

Weston Ruter

Weston Ruter, CTO, XWP


We have developed a workflow whereby when a pull request is merged into master on GitHub, Travis CI runs automated code quality checks and tests using our wp-dev-lib, and if these pass then Travis pushes the code to the repo mirror on Pantheon. This works great because we can be assured that only code which meets our standards will ever get deployed to any environment.

Any agency that deals with new clients on a regular basis, and any agency that hopes to scale up, should have in place workflows that take the guesswork out of getting new client projects set up and ready for delivery.

5 Essential Tasks for Implementing Continuous Integration & Continuous Deployment:

  1. Automate the build: "Continousness" really happens when developers don't need to do extra work. CI/CD systems will kick off a build—including a testing pass—with every commit without needing any intervention. This ensures that builds happen every time, and feedback comes quickly.

  2. Fix broken builds immediately: Respect the sanctity of the build and fix issues right away. This is vastly preferable to having the last mile of a sprint or project turn into a surprise death march. This saves time and helps create a culture of quality.

  3. Make it transparent: Build status should be readily available to the whole team, and easy to investigate. The CI/CD system should be a shared set of facts, not a specialized area of secret DevOps arcania. Taking the time to make your build process clearly visible and accessible is key to making it a standard part of your practice.

  4. Test in a clone of the production environment: Nobody likes surprises when going live. Testing isn't about preventing mistakes, it's about creating confidence and velocity. Make sure your test environment matches production as closely as possilbe, and that you are testing with a current snapshot of production data.

  5. Automate deployment: You don't need to be robotically deploying after every build to see huge benefits from a scripted deployment process. Even if you're waiting for final human acceptance from a stakeholder or QA team before a release, setting up that acceptance testing environment should be automagic. Likewise the final "greenlight" should be a single click or command, nothing more. This keeps you moving fast and prevents human error from taking you down at the last minute.

An Agency’s Most Valuable Tool

Pantheon lets you do more with your resources. The platform is absolutely free for agencies—just invite clients to pay when their sites go live.

Contact Sales or call us at 855-927-9387