Using ElasticSearch for Monitoring and Debugging in Drupal

Nick Stielau , Director of Engineering June 22, 2015 Reading estimate: 1 minutes

Thanks to everyone who came to see me and Howard Tyson of Zivtech present our session on ElasticSearch at Drupalcon LA. We decided to nerd out, both doing our slides in code with Bespoke.js and focusing on getting a working demo, searching through Drupal core commit history with ElasticSearch.

There was strong interest in ElasticSearch in LA, and there’s already an ElasticSearch session proposed for DrupalCon Barcelona, so it’s pretty clear ElasticSearch will have a growing presence in the next few years.

That shouldn’t be a surprise, given how it hits the sweet spot of an incredibly powerful tool that’s easy to use. At Pantheon, we send over 3 million log messages through ElasticSearch to our ELK-stack logging cluster. These logs help us monitor the platform, ensure audit trails for security purposes, and debug customer issues.

Let’s use ElasticSearch to dig in to one of everybody’s favorite things to do on Pantheon: clear caches!

In the past day, the clear-cache button has been clicked 1498 times. Yikes! That’s a lot of cache bustin’.

Looking at the histogram over time, we can see a fairly typical diurnal spread:

Breaking that down by success, failure, and aborted workflows. That’s a failure rate of 36/1498, or ~2%. In fact, engineers at Pantheon are leveraging ElasticSearch and this log data right now to exactly what these errors are, and which customers they are affecting.

We can also easily see who is kicking of these Cache Clear workflows. Most are from users with the general “Team Members”, although some are from the site owners, one triggered by the "super" user (the platform API itself kicking off the workflow). One cache-clear was triggered by a user with the "developer" permissions (a developer who does not have access to test/live environments).