Drupal 8 File Migrations

In my last blog post about migrating to Drupal 8, I focused on setting up a connection to your source Drupal 6 or 7 database. The drush migrate-upgrade command takes the credentials for your source database and configures migrations that can be run separately. With the database alone, you can move all of your Drupal nodes, users, taxonomy terms and other information held in the database. That last post did not detail how to ensure that your uploaded files (images, pdfs, etc.) also migrate successfully to Drupal 8.

To migrate those uploaded files to Drupal 8 you will need to use the --legacy-root flag on drush migrate-upgrade. This parameter takes two types of values, the absolute URL of your source site or a relative path to source Drupal 6 or 7 site.

Option One: Move source files individually over HTTP during the migration by referencing the absolute URL of the source site.

terminus drush "migrate-upgrade --legacy-db-key=drupal_7 --configure-only  --legacy-root=http://dev-migrate-pantheon-d7.pantheonsite.io/ "

By telling the Drupal 8 migration system the source site URL, it can then determine the URLs of individual files. For instance, the source site may be http://dev-migrate-pantheon-d7.pantheonsite.io/ and the file to be migrated is recorded in the database as public://field/image/drops.png. The Drupal 8 migration system can then guess that this file can be downloaded from http://dev-migrate-pantheon-d7.pantheonsite.io/sites/default/files/field/image/drops.png

I list this option first because it requires the least effort. By telling Drupal 8 where to find your source site, the downloading of files to Drupal 8 should just work. This option works well when you don’t mind waiting for the migration to download each file individually. If you are going to be running your migration repeatedly, or if you are measuring your source files directory in gigabytes, then you might want option two.

Option Two: Move source files to the Drupal 8 site before running the migration and reference the source files directory with a relative path.

If you have a very large set of files, it will almost certainly be faster to rsync them from the source Drupal 6 or 7 site Drupal 8.

First rsync down from Drupal 6/7 to your local machine:

# Set $ENV to the name of the environment from which you are downloading (dev, test, live)

# Set $SITE to the uuid of your source Pantheon site.
rsync -rlvz --size-only --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/ ~/drupal_7_files

Next, on the Drupal 8 site, use SFTP to first create the destination directory: files/drupal_7_files/sites/default/files

Then rsync to that directory:

# Set $ENV to the name of the environment to which you are migrating (dev, test, live)

# Set $SITE to the uuid of your new Drupal 8 Pantheon site.

rsync -rlvz --size-only --ipv4 --progress -e 'ssh -p 2222'  ~/drupal_7_files/  $ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/drupal_7_files/sites/default/files

The idea here is that we are recreating part of the Drupal 7 file system on the Drupal 8 infrastructure so that we can give Drush a relative location, as if the Drupal 7 and Drupal 8 sites were on the same server:

terminus drush "migrate-upgrade --legacy-db-key=drupal_7 --configure-only  --legacy-root=~/files/drupal_7_files"

We are essentially telling this command that there is a Drupal 7 site located at ~/files/drupal_7_files. This top-level “~/files” directory is where Pantheon symlinks sites/default/files on Drupal sites. You will have the contents of your Drupal 7 sites/default/files folder inside your Drupal 8 sites/default/files folder, so your Drupal 7 files will be available at sites/default/files/drupal_7_files/sites/default/files during migration. Once your Drupal 8 site is launched, you should be able to remove the extra drupal_7_files directory.

Now the Drupal 7 to Drupal 8 migration process (run through drush migrate-import --all) will run faster as it does not have to transfer files individually over HTTP. Be aware with rsync you may have to re-run the sync commands as new files get added to the still live Drupal 6 or 7 site. For more information on using rsync with Pantheon, see our documentation.

If you are interested in seeing more details of a migration to Drupal 8 on Pantheon, check out this repository of continuous integration scripts. This script runs every night to verify that a very basic migration completes successfully using the latest dev versions of migration contrib module. That repository and the examples in this blog post use the --legacy-db-key flag the drush migrate-upgrade command that helps your keep your source database password hidden. That flag comes from my patch which is now committed to Drupal core.