Parsing Nginx Access Logs with GoAccess

Learn how to parse the nginx-access.log file with GoAccess to gather information on your visitors and referral traffic.

Contributors: Albert Causing, Sarah German.

Tweet

Pantheon runs nginx web servers for optimal performance. Your site's nginx access logs record web server events and activities that can help you identify potential issues and gather information about users.

 Note

Requests served by the Pantheon Global CDN will not hit the nginx webserver and will not be logged in nginx-access.log.

GoAccess is a free, open source utility that creates reports by parsing nginx-access.log files. Use it to quickly identify the most used browsers and operating systems, visitor IPs, or most frequent 404s — all from the command line.

Before You Begin

Be sure that you have:

This guide is written for the latest stable release of GoAccess as of this writing, which is version 1.3 (release notes).

Edit GoAccess Configuration

To parse your nginx-access.log files with GoAccess, you'll need to configure GoAccess to read Pantheon's log formats.

The configuration file is located under ~/.goaccessrc or %sysconfdir%/goaccess.conf where %sysconfdir% is either /etc/, /usr/etc/ or /usr/local/etc/ (read more).

Add the following lines to the goaccess.conf file:

goaccess.conf
time-format %T
date-format %d/%b/%Y
log-format %h - %^ [%d:%t %^]  "%r" %s %b "%R" "%u" %T "%^"

Create a report

  1. Download your nginx log files from Pantheon via SFTP.

  2. From the directory containing your nginx-access.log file, run GoAccess:

    goaccess nginx-access.log

    You can use the arrow keys on your keyboard to scroll down to view more of the report, or hit q to exit.

    Alternatively, you can generate an HTML report:

    goaccess nginx-access.log > report.html

  3. View the report in your browser by opening report.html. For MacOS:

    open report.html

    For Linux:

    xdg-open report.html

Troubleshooting "goaccess.conf Not Found"

In certain MacOS Homebrew installations of GoAccess, goaccess.conf is not found by the binary. To resolve:

  1. Display the default config path:

    goaccess --dcf

  2. Move goaccess.conf from /usr/local/Cellar/goaccess/[VERSION]/conf/etc/goaccess to /usr/local/Cellar/goaccess/[VERSION]/conf/etc:

    mv /usr/local/Cellar/goaccess/[VERSION]/conf/etc/goaccess /usr/local/Cellar/goaccess/[VERSION]/conf/etc

An issue has been filed on the GoAccess repo.

Automate GoAccess Reports

  1. Copy the general log retrieval script from Automate Downloading Logs, and use this to download logs from all application containers on the desired environment.

  2. Add the following to either collect-logs.sh or a separate file:

    # Unpack archived log files (optional).
gunzip */nginx-access.log-*

# Create a GoAccess report and open it in a browser.
goaccess */nginx-access.log* > goaccess.html && open goaccess.html # Or xdg-open for Linux

See Also