Ebook Page Header

WordPress Multisite - Option 3 - Multiple WordPress Sites

Multiple WordPress Sites Managed by Automation

Option 3 - Multiple WordPress Sites Managed by Automation

If you have multiple sites or brands, and the goals and functionality, and accordingly the code and plugins used on each site are very different, and/or there are different teams working on each site and each of them need to develop different functionality at the code level - this is not a Multisite. You should build it out as multiple WordPress sites, and manage them using automation and modern DevOps practices. Automatic deployment and updates will be needed to handle the complexity and large volume of changes across the multiple sites, as well as issues like robustness, resilience and scale.

Why Automation?

Even for one mission-critical WordPress site, automation is essential, for several reasons:

Ongoing updates can become a major task and human error can pose serious risk.

Testing is something that, if done only manually, can be spotty at best.

Dev, test and production environments are going to deviate over time (what is known as "configuration drift") because they’re not being replicated automatically. This will create changes in the transition between environments ("it worked on my machine").

Opportunity cost - most companies will assign tasks such as version control, deployment, backup, monitoring, user management - to their most experienced developers. If it can be done automatically, that could free time for those star developers to improve your websites.

Assurance - When you automate you know every task gets done every single time. That means less risk, less need to oversee tasks - computers don’t forget.

Perfecting the architecture - by automating you have an opportunity to fine-tune the infrastructure. Find out over time what is the perfect server config to run your sites on, deploy that consistently across all sites and environments, and continually improve it keeping all environments consistent using "infrastructure as code".

Automation gives you “peace of mind” - you don’t have to waste energy by continually answering questions about the performance or functionality of the website. Any improvement you make - for example, a performance optimization, is put into code and automatically propagates across all your sites, in every deploy on every applicable environment, from that point onwards.

To take another example, say you define in your test suite that a broken link fails the test, and we set up the automatic deployment to never deploy when that test fails. In this case you know that you'll never deploy a broken link. And then you'll never get a ticket to your support team saying “your link is broken”, which is a waste of time for the team, for your customer, and provides a bad user experience. Over time you can get these issues out of the way automatically, allowing you to focus on building the functionality of the website.

Importance of Automation for Multiple Sites

Across multiple large scale WordPress sites, the problems detailed above are compounded by an order of magnitude.

Multiplied opportunity cost - Multiple sites multiply the opportunity cost across multiple development teams, and we multiply the risk of manual updates and changes. The challenge of hosting a matrix of dev/test/prod environments for numerous sites, and having to install, update and maintain each of those environments, can become a nightmare, not to mention the detrimental effect to your software quality and visibility over the process.

Dangers of a single dev/test environment - In reality, most shops resort to setting up a single development or testing server for all the teams. This creates a single point of failure, so that when that server goes down development is stalled across all your sites.

Standardization and on-boarding - Another important benefit of automation is standardization - on-boarding new people to the team becomes easy, as they don’t need to figure out what’s different about each different website in your organization. It's much easier to move developers between teams, websites and projects, compared to a bespoke project-specific architecture.

The Dev/Test/Deployment Workflow

Here are the main building blocks required to have a fully automated development lifecycle for your WordPress sites, which is crucial to operate multiple sites at large scale.

First of all, you need to ensure each WordPress instance is scalable and performant as we laid out in Option #1 - read up there.

Beyond that, there must be version control for the codebase, and you'll also need an automated deployment workflow, and monitoring tools. Optionally, by adding Continuous Integration you will gain a more streamlined dev cycle with faster feedback for developers. Below we explain each one of these requirements in detail, with a quick guide to get you started in your real WordPress environment.

Version Control for the Codebase

An easy way to get started with version control in WordPress is the SourceTree plugin, which works with BitBucket. There are many other source control systems you could use, and you can also do this the geeky way with git.

Using the SourceTree plugin, moving your WordPress theme code into a BitBucket repository is as easy as:

  1. Setting up a BitBucket repository.
  2. Creating local repository in SourceTree.
  3. In repository settings, add the remote BitBucket repo.
  4. Select "unstaged" files, type a commit message, Commit, then Push.

And your code is now in version control! For example, if you commit something that breaks your site, you can easily roll back to a previous committed version: Right-click the version in the left nav and select Reverse Commit. And you're back to the previous version.

The preceding workflow is based on the excellent post by Lucy Beer - read it to get all the details + screenshots.

Deployment Automation

This will require rolling up your sleeves a bit. A method we recommend for automating deployment on WordPress sites is to create a local dev environment using Vagrant, and use Ansible to automatically provision the server for development and production, with the appropriate differences between them.

A tried and tested tool you can use is Trellis - it provides a ready made stack for a WordPress server (based on Ubuntu, NGINX, MariaDB and Memcached), and provides ready-made Ansible playbooks that allow you to automatically setup this server environment. Check out the Trellis Documentation for more information.

If that's not cool enough for you, check out this Sitepoint tutorial explaining how to deploy WordPress with Docker, but be warned that we have not had experience with this in production.

Backup and Restore - Without Plugins

It's crucial to have an automated method for backing up and restoring your WP sites. At large scale, you don’t want to rely on a WordPress backup plugin, because typically such plugins will use the PHP environment to perform your backups. PHP is running the site and delivering pages to your users, and running a backup, which is a costly operation, within PHP, will slow down the site.

So running site backups in a way that consumes PHP resources will threaten the performance and availability of your site - or it will limit your ability to do regular backups to times when there is low traffic or enough free resources. With multiple sites and large traffic, you don't want to have this constraint - you should be able to back up on a regular schedule with no risk.

Therefore our recommendation is to do backup and restore outside the PHP/WordPress environment. Generate a SQL dump of your database using a CLI tool (you can write a script which automates the required commands), and save the result of the SQL dump on a third-party storage service such as Amazon S3. In addition, save an archive of your code, and the wp-content/uploads folder which is frequently updated with multiple versions of user-uploaded files.

To restore, you would write a script that pulls these three things from third-party storage and uses them to reconstruct the site. You would need to drop the existing database, import the database archive into MySQL, remove existing uploads and code and replace them with the archives you had backed up. It will take time to get this working, but it will result in clean backups and restores every time.

To learn more about WordPress backups and ways of achieving them without relying on plugins, see this detailed post by Laurence Bradford of SkillCrush.

Monitoring

For a large scale WordPress deployment with multiple sites, you'll probably need several layers of monitoring:

  • Basic uptime monitoring - being notified when a site goes down so you can do something about it. There are numerous WordPress plugins that do this, but a more industrial-strength option is a dedicated service like Pingdom.
  • Application Performance Monitoring - monitoring the health of your WordPress application, with application-specific metrics that can help you identify and diagnose problems. A popular option is New Relic, which provides data on WordPress MySQL performance; a metric indicating how WordPress is performing for end users; and visibility into external service calls such as plugins and APIs, to understand what might be slowing down pages on the site.
  • Network monitoring - if you're a bit more advanced and want much more granular visibility as to what's happening on the network as WordPress serves your websites, you can add an enterprise-grade monitoring tool like Nagios. This post by Nick Bettison (somewhat dated) explains how to setup Nagios with a WordPress installation.

Continuous Integration

Continuous Integration is a widespread development practice in which developers frequently commit their changes to a central code repository, and an automated process is run to build, test and possibly deploy the software. This provides an important feedback loop to developers, and helps them find issues early, preventing "integration hell" that happens after numerous developers commit complex changes to a program or website.

In the context of WordPress, we highly recommend setting up a CI server that sets up the WordPress environment with your latest code changes, runs at least some basic automated tests, and provides a quick response saying if your current version is working or not (i.e. if the "build is broken").

Here is how to achieve this relatively simply with CircleCI, a lightweight Continuous Integration server. If your WordPress installation is not super-complex, this could also make do as a deployment automation strategy:

  1. Include the circle.yml file in your project - this is instructions for the CI environment.
  2. In circle.yml, define your "machine" and specify your DB under "dependencies".
  3. Use cURL to fetch the WP-CLI binary.
  4. Still in the "dependencies" section, use WP-CLI to download WordPress.
  5. Use WP-CLI to generate wp-config.php, including your database credentials.
  6. Clone your Git repository locally, and use WP-CLI to install and activate your themes & plugins.

Prepare at least some basic "smoke tests" that automatically verify if your WordPress installation is running and functional. CircleCI can then accept the result of these tests and give developers instant feedback on whether your latest build is working or not.


tipsCOMMUNITY TIPS

A different point of view from Jim Nelson of Tripawds Blogs Community hosting over 1000 blogs:

  • My primary disagreement is that there are basically two options when it comes to large scale WordPress deployment, not three: Single Install(s), or Multisite Network.
  • What you describe as Option 3, multiple WordPress sites managed by automation, can certainly be done with a WordPress Multisite Network. Stating that WP Multisite is "not suitable" for a network of vastly different sites is just wrong. An efficient network manager could easily deploy various different plugins and theme scenarios for an unlimited number of sites.
  • I do agree that automation is imperative when hosting multiple independent WP installs, and a central dashboard for managing updates could be beneficial.

Want to Get it Out of the Box?

Want to get the benefits of an automated dev/test/deploy workflow for multiple WP sites, without working hard?

See our secret Secret Option - How to do it on Pantheon.