AWS S3 Setup for Drupal

Add the ability to integrate with AWS S3 to a Drupal 7 site on Pantheon

Contributors: Peter Sawczynec, Alex Fornuto


Pantheon's Global CDN makes it unnecessary to add a third party CDN for most use cases. While technically possible, stacking another CDN on top of the Global CDN adds potentially unnecessary complexity. Confirm whether your needs are met by the Global CDN before considering stacking another CDN on top of it.

Amazon Web Services (AWS) offers Simple Storage Service (S3) for scalable storage and content distribution, which can be integrated with Drupal sites running on Pantheon.

Before You Begin

Be sure that you have:

Configure S3 within the AWS Console

Before integrating S3 with Drupal, you'll need to configure the service within your AWS Management Console.

Create a New AWS S3 Bucket

If you do not have an existing bucket for your Drupal site, create one:

  1. From your AWS Console, click S3.
  2. Click Create Bucket.
  1. Enter a bucket name. The bucket name you choose must be unique across all existing bucket names in Amazon S3.
  1. Select a region and click Create.
  2. Select Permissions within the bucket properties and click Add more permissions.
  3. Choose a user and tick the boxes for Read and Write access for both Objects and Permissions, then click Save.

Create an Identity and Access Management Policy

Identity and Access Management (IAM) allows you to manage all user access to AWS resources and services. Creating a policy allows you to explicitly set limited privileges on your specific bucket. This strategy offers long-term flexibility for organizing and managing users and their privileges.

  1. From your AWS Console, click the IAM link.
  2. Go to Policies and click Create Policy.
  3. Select Create your Own Policy.
  4. Give it a name and use the code example code provided in Amazon's Policy Documentation.
  5. Choose Amazon S3 for the AWS Service and select All Actions. Provide the Amazon Resource Name for your bucket, and click Next Step.
  6. Edit the policy name and description (optional).
  7. Click Create Policy.

For details, see Example Policies for Administering AWS Resources.

Create an Identity and Access Management Group

We recommend that you do not access an S3 bucket using your AWS root user credentials. Instead, create an IAM group and user:

  1. From your AWS Console, click Identity & Access Management.
  2. Click Groups, then Create New Group.
  3. Enter a descriptive group name and click Next Step.
  4. Filter policies by Customer Managed Policies and select your recently created policy.
  5. Click Next Step, then Create Group.
  6. Go to Users and click Create New Users.
  1. Provide a user name and click Create, then view the new user security credentials by clicking Show User Security Credentials.
  1. Click Download Credentials. Make sure you save the credentials in a secure location before leaving this page.
  2. Go to the group created in step 5 and select Add Users to Group.
  3. Select your newly created user and click Add Users.

Integrate S3 with Drupal

You will need to install the appropriate Drupal module(s) and the AWS SDK library.

The following instructions use Terminus, Pantheon's CLI which allows you to call Drush remotely without using a local installation.

These steps require Drush 8, which is run by default on Pantheon for newly created Drupal sites. Sites created prior to November 4, 2015 run 5.x by default.

Before you begin:

  • Set your site’s Drush version to Drush 8 if needed.
  • Either copy the default.settings.php file to settings.php or create an empty settings.php file within the sites/default directory if you have not done so already.
  • Set the site's connection mode to SFTP within the site Dashboard or via Terminus:

    terminus connection:set <site>.<env> sftp
    

S3 File System

Install the Libraries API and S3 File System modules:

terminus drush <site>.<env> -- en libraries s3fs -y

Get the AWS SDK Library 2.x:

terminus drush <site>.<env> -- make --no-core code/sites/all/modules/s3fs/s3fs.make code/
  //or if you have a contrib subfolder for modules use:
  //terminus drush <site>.<env> -- make --no-core code/sites/all/modules/contrib/s3fs/s3fs.make code/

The above command will add the AWS SDK version 2.x library into the sites/all/libraries/awssdk2 directory.

S3 File System CORS

Use the S3 File System CORS Upload module to enhance your Drupal media handling and interface with your S3 bucket by having your file uploads go directly to your S3 bucket.

Install s3fs_cors module using Drush:

terminus drush <site≥.<env> -- en jquery_update s3fs_cors -y

Before you begin:

  • Install Composer locally.
  • Set the site's connection mode to Git within the site Dashboard or via Terminus:

    terminus connection:set <site>.<env> git
    
  • Create a local clone of your site code, and navigate to it in your terminal.

S3 File System

Install the S3 File System module and AWS SDK version 3.x library using Composer.

  1. Ensure that the Composer for your site will first look to Drupal's preferred package source to find modules:

    composer config repositories.drupal composer https://packages.drupal.org/8
    
  2. Install s3fs module from the preferred package source:

    composer require drupal/s3fs --prefer-dist
    

S3 File System CORS

Use the S3 File System CORS Upload module to enhance your Drupal media handling and interface with your S3 bucket by having your file uploads go directly to your S3 bucket.

Install s3fs_cors module using Composer:

composer require drupal/s3fs_cors --prefer-dist

Drupal Module Configuration

S3 File System User Credentials

You can configure the settings for the S3 File System module via the Drupal admin after the installation (admin/config/media/s3fs/settings).

Enter credentials created for the user in the previous section and your bucket name.

You can optionally use a CNAME to serve files from a custom domain if desired. However, you will need to create a corresponding CNAME record with your DNS host.

Configure Download and Upload Destinations

Go to admin/config/media/file-system and set Default download method to Amazon Simple Storage Service. You can set the Upload destination to S3 File System within the Field Settings tab.

S3 File System CORS Upload (s3fs_cors)

From /admin/config/media/s3fs/cors, set CORS Origin to your domain. There's an individual max file path length of 250 characters.

Synchronizing the S3 Bucket and Drupal Files

Periodically, you'll need to run Actions provided by the S3 File System module either via the admin or Terminus to sync Drupal with your S3 bucket.

If you have files on S3 already that are not known to Drupal, refresh the files metadata cache:

terminus drush <site>.<env> -- s3fs-refresh-cache

If you have files in Drupal that need inclusion with S3 run:

terminus drush drush <site>.<env> -- s3fs-copy-local

If you receive an access denied error message from Amazon, check the permissions on your bucket and policies. Verify all your configuration settings in Drupal are complete and accurate.