On Pantheon’s enterprise-grade WordPress hosting, you can get a WordPress site with Varnish pre-configured, making it much easier to scale up to huge traffic.
When WordPress Cache Plugins Are Not Enough
Reverse Proxy and Object Caching - Performance at Any Scale
WordPress cache plugins can help your site go faster - improving page load times and your ability to serve much more traffic without degradation in performance. But there will come a time when a cache plugin will not be enough.
All it takes is a huge peak of traffic. If content on your site makes it big and you get sudden attention on social media or other channels - the concentrated burst of traffic can bring your site to its knees. A huge success can turn into a performance disaster.
This page will answer the following questions:
- Why are WP cache plugins not enough for large scale?
- How do the pros handle WordPress caching?
- What additional tools do they have beyond the regular cache plugins?
- What benefits can you get from these additional tools?
- How can you get super-scale caching built into WordPress?
Note: We’re the makers of Pantheon, the high-performance, enterprise-grade hosting platform for WordPress. The info below is based on our experience hosting tens of thousands of demanding WordPress sites and helping them scale up.
Image
Why Cache Plugins are Not Enough - Meet the Reverse Proxy
What’s the fastest WordPress cache plugin on the planet? Probably Batcache, a blazing-fast plugin used to speed the world’s biggest WordPress server, WordPress.com.
Now see this chart from Joe Hoyle comparing Batcache with a different option, Varnish. The green graph shows what happens to even this industrial-strength plugin as you increase the number of concurrent users on your site.
As the load increases to 20, 50, 100 and more concurrent users, response times start climbing to 10, 25, 40 and even 70 seconds. It only happens at the higher levels of traffic, but that’s where we want to be, right?
This applies to many popular WordPress cache plugins - such as WP Rocket and W3 Total Cache - which serve web pages using WordPress and its underlying PHP engine. They rely on compression, CSS minifying and other techniques to reduce page load time, and this works at the small scale, but suffers when WordPress itself starts to choke on big traffic.
What is the orange graph that remains stable even when the number of concurrent users goes through the roof? That’s a reverse proxy.
A reverse proxy (Varnish in this case) is an intermediary service that runs independently, not as part of WordPress, and accepts requests to the WordPress server. Like a cache plugin, it provides a pre-saved copy of a WordPress response for a limited time.
The difference is that while cache plugins are driven by the WordPress server, which can get tired as load increases, a tool like Varnish is specialized at serving cache requests and can be literally up to 100x more efficient than a PHP web server.
That generates the amazing flat graph as traffic increases to mega-site levels.
If you want to get started with Varnish, this post provided by the makers of Varnish explains how to setup Varnish on WordPress.
Static-File-Based Plugins vs. Varnish
Some WordPress cache plugins work differently - they save static files of your dynamic WordPress pages and reroute web visitors to those static versions.
This is an approach that circumvents the WordPress and the PHP engine, and in theory, can be as fast as Varnish under high traffic volumes. Some examples are WP Super Cache and WP Fastest Cache.
However, there are a number of problems we’ve seen with cache plugins based on static files:
File system trouble - One of the downsides of file based caches like WP Super Cache is that on certain file systems (especially a networked file system like NFS) the disk can significantly hurt performance when you write/delete/change many files at once. This can cause your site to either not cache data (making it slow) or serve badly cached data (making it look broken). Varnish keeps cached page data in RAM, which avoids this problem.
Plugin incompatibility - Due to the number of different ways that data can be rendered/assembled using WordPress, not all plugins can take advantage of different caching solutions and run into compatibility problems. With Varnish, caching is done on the edge and only individual elements are cached. This means you remain fully compatible with plugins and dynamic content.
Web server configuration required- A number of plugins, like WP Rocket, require specific settings on the web server to get the cache to work. In many cases, the webmaster can’t change web server settings or has a non-Apache web server like NGINX.
Inefficient compression handling - Some WordPress cache plugins compress most of your files using GZIP every time they're requested. Varnish caches the compressed pages and delivers them as-is for web browsers that support GZIP (99%), and uncompresses the files on the fly, which is less efficient, only for browsers that don’t support GZIP. The same approach is taken by CDNs. This means Varnish’s compression handling is better optimized for the vast majority of users.
No failure isolation - Varnish and CDNs can recognize when something is wrong on your server, and help you bridge over this time by delivering slightly staler (but still functional) content. This can significantly reduce the impact of downtime. Most WP page caching plugins do not do this, making web server downtime full downtime.
Disclaimer: We aren’t affiliated with Varnish, but at Pantheon, we feel strongly that it’s the best option for large-scale WordPress. That’s why we’ve implemented it as part of our WordPress hosting platform.
Get a WordPress site with Varnish Pre-Configured
Image
Object Caching - Speed Up Dynamic Pageviews
This graph from NewRelic shows what can happen when you integrate a tool called an Object Cache - one popular tool is Redis. This relates to another aspect of larger-scale performance - an internal object cache.
An object cache is a way of storing data from the database in memory, to prevent unnecessary queries on the database. WordPress comes with its own internal object cache but many experts say it is quite inefficient.
Like with the reverse proxy, here too using a dedicated external service for object caching can make a dramatic difference, as you can see in the graph above - the most timely database operations that slow down a site can drop to almost zero.
Popular options for external object caching are Redis or Memcached (or on the cloud, AWS ElastiCache or Azure Managed Cache), which can dramatically speed up execution and reduce DB load.
Here are some resources to get you started with Redis and Memcached on WordPress:
- This post from Flynsarmy provides a tutorial and script for running Redis as a cache backend for WP.
- This detailed post by Scott Taylor explains how Memcached works with WordPress and how to set it up.
Get a WordPress site with Redis Pre-Configured
On Pantheon’s enterprise-grade WordPress hosting, you can get a WordPress site with Redis pre-configured, making it much easier to scale up to huge traffic.
Pantheon - The Easy Way to Get WordPress with Pro Caching Built In
We hope you’re convinced by now that a reverse proxy like Varnish and object caching like Memached/Redis is super important to prepare for larger scale traffic. However, setting up these things can be a bit complex. That’s where we come in!
Pantheon is an enterprise-grade WordPress hosting service that provides high performance and advanced DevOps features, using an automated container-based platform. Among other performance features, we provide Varnish and Redis built in and preconfigured.
So if you set up your WordPress site on Pantheon, you’ll enjoy all the benefits of reverse proxy and advanced object caching, without having to work hard to get everything working.
On top of that, every site on Pantheon comes with our Global CDN pre-configured which distributes your site over a global network of POPs for subsecond pageloads and airtight security.
Sign up for a free Pantheon account and see how easy it is to get a WordPress installation ready for very large scale.
Community Tips
A different point of view from Aigars Silkalns of Colorlib:
- You mention in your post: "Web server configuration required - A number of plugins, like WP Rocket, require specific settings on the web server to get the cache to work. In many cases, the webmaster can’t change web server settings or has a non-Apache web server like NGINX." In my experience, the vast majority of WordPress users won't need to change server configuration in order to get cache plugins to work.
- NGINX is currently a WordPress hosting standard used by popular providers such as HostGator, SiteGround and more. On the other hand, setting up Redis and/or Varnish on your own would require more than just a few modifications to your server and none of the shared and managed hosting solutions will allow you to do that outside of their own setup.
Tips from Mobilunity - cache plugin challenges:
Despite the fact that cache plugins are a popular WordPress website optimization solution and are even recommended by shared hosting solutions, these plugins may still cause some issues. Here are a few examples of bugs Mobilunity experienced with some cache plugins, while managing our customers' WordPress sites:
- Blank login screen after a WordPress update or website relocation
- Broken layout on websites built with Visual Composer
- Incorrect display of mobile version on websites with device identification
- Different caches on websites with mixed protocols (HTTP and HTTPS)
- Random pages displayed blank
- Cookies blocked after WordPress update
All such issues require additional checkup of conflicting plugins and expert troubleshooting.
Tips from Zac Johnson of blogging.org - improving your site's speed and caching performance:
In addition to the great notes and references in this article, it's important to consider the following options when trying to improve the overall speed and caching performance of your site.
- WordPress Themes: While many sites use custom WordPress themes, not all designers and coders actually optimize for peak performance. Verify that your site is not only coded properly but is also optimized and responsive at the same time. Sloppy coding and functionality on any site are a huge damper on its overall performance.
- Hosting Provider: Another issue that may arise when trying to increase the speed and caching of a site, is the actual hosting provider. Not all web hosts are created the same, nor is their data accessed from different locations. Perform the necessary speed tests to make sure your hosting provider is delivering your site at the fastest speeds possible.
- 3rd Party Plugins: A huge data and loading crunch can occur as a result of pulling third-party plugins from other sites. This is especially true if you are pulling multiple codes at the same time, such as from Facebook Fan Pages, Google Analytics, and/or Ad tracking platforms. The same is true for WordPress plugins, if you are using too many or have unused plugins in your dashboard area.