Importing Large WordPress sites into Pantheon

Recently I wrote about how to prepare your site and import it into Pantheon’s new WordPress platform. This process works well for smaller sites. However, if you have a large WordPress installation, the process may not work. The Pantheon Importer has a limit on the size of files it can process. If you are uploading the file to Pantheon, the limit is 100MB. If you are giving the Pantheon Importer a URL, the file cannot be larger than 500MB. Therefore, let’s discuss what it takes to import a larger site.

DANGER: Here There Be Dragons!
If you are not a developer, this article is not for you. This article is full of command line tools and assumptions that you know how to use them. This article will not spend a lot of time explaining the tools or how to use them, if you are the target of this article, you will know. If you are unsure at all, go pull your nephew out of class. He probably understands.

Before we get started, we are going to need a few things.

• An account at pantheon.io
• Access to the database and sourcecode of your existing site.
• terminus
• mysqldump
• An sftp client.

Note: It's fastest to run these commands on the server where your existing WordPress installation is hosted. Copying everything down to a computer on consumer grade Internet connectivity - or worse yet, wifi - can slow things down tremendously and may cause timeouts. Going from existing host direct to Pantheon is much faster.

Ok, let's get started.

Step 1: Back everything up!

This is good advice for any tech project but it’s vital for this project as we will be using that backup.

1a: Create a zip or tarball of your entire file system.

1b: Use mysqldump to backup your database.

mysqldump --opt -u USER_NAME -p DATABASE_NAME > database.sql

Make sure you use the --opt flag when creating the backup. This will, among other things set the --extended-insert flag which you want. In my initial testing, a dump made without the --extended-insert flag took 65 minutes to import. (127,000+ insert statements) A dump done with --extended-insert took less than 2 minutes. You wantthe --opt flag. The --opt flag will also enable --drop-table which will add a Drop Table; before each Create Table command.

Step 2: Let’s get started

If you haven't already, log into your Pantheon dashboard. Create a blank WordPress site. This will be the site you build your site in. Unlike the previous tutorial, we are not using the Pantheon auto importer.

Step 3: Git not SFTP

Once the site is created, swap it from sftp to git mode. We are going to do the bulk of our work in git.

Step 4: GIT your site

Using the “Connectivity” tab on your dashboard, get the parameters provided for git clone. Switch to the command window you have open on your server clone for your new WordPress site. This will give you a repo to work with. When we are done, you can delete this repo as we only need it for the import.

Step 5: Moving Day

Now it is time to copy your existing site over to the new repo. Most of the files in the new repo are fine to overwrite. There is one exception however and we need to handle it first.

5a Copy the file named wp-config.php from the new repo into a holding directory. Pantheon uses a specially crafted wp-config.php. Since your database name and password will change when you get it onto Pantheon, the information in this file is not really necessary. If you have any customizations to your wp-config.php, you will need to manually merge them as you cannot keep your existing file.

5b Copy all the files from the root directory of your blog - and all of it's subdirectories - into your newly created repo.

5c Copy the wp-config.php that you put in your holding directory back into the newly created repo.

Step 6: Don't be afraid of COMMITment

6a In your newly created repo issue this command.

git add -A .*

This will add all the new files to your repo as well as any files that have been modified.

6b Issue a git commit with a meaningful, if not cute message. Something about the beginnings of life of “42” are always appropriate.

git commit -m "Insert cute message here."

Step 7: Push It

Issue a git push origin master to push everything up to Pantheon.

git push origin master

If you have your Pantheon Dashboard open, you will notice that it sees the push and will begin updating everything.

Step 8: All Your dataBases Are Belong To Us

Take the data dump you made in 1b and import it into your Pantheon site. The easiest way to do this is to go to the Connectivity tab on your dashboard and get the mysql command. Paste that into your command window and at the end add < /path/to/your/ddatabase.sql

This will pump your database dump into your newly created database.

Step 9: Database Cleanup

At this point, your site will most likely come up and look correct. There is just one problem. Your database contains a lot of references to your old site. Graphics are still being pulled from your old site. If your old site isn’t up, they are just missing. You will need to use terminus to cleanup your database.

That last switch is important. Without it terminus doesn’t know which site to do the conversion on.

Step 10: SFTP to the Rescue!

After you finish the last step, your site will be broken again. Most, if not all of your graphics will no longer be appearing. The issue is that we used git to upload your site to Pantheon. In the .gitignore file is wp-content/uploads. This is there because you generally do not want to version user content. The final thing we need to do is put those files back in place.

To connect to your Pantheon site with SFTP, go back to your dashboard and click on the "Connection Info" tab again. At the bottom will be your SFTP info. The first thing is the command line command to connect. If you are using your OS's built in sftp cli clinet, that is all you need. However, if you are using something more feature-rich, like FileZilla, you will need to put the connection info into the correct place.

Unless you have a non-standard setup, you will put your files in /code/wp-content/uploads.

Once you have pushed all your content back up, your site should be working correctly again. Make sure you clear the cache from your dashboard before testing, just in case.

PRO Tip:Prefer rsync over sftp? Try this command. Just make sure that you replace ENV with dev and UUID with the UUID of your site.

If you don't know the UUID of your site, the easiest way to get it is to look at the url of your dashboard. Below is a sample. If you see a # ignore it and anything after it.

https://dashboard.getpantheon.com/sites/UUID

Step 11: Crank it up to 11!

Test everything. Seriously, everything. Every plugin, every subsystem, every theme feature and every widget. Once you are satisfied, clone it to test and hand it out to friends, family, co-workers or your stable of intern minions. Get more eyeballs on it than just yours. Make sure you can post, delete, upload, etc. You really can’t spend too much time testing it to make sure everything is working. Once everything is working, push it into live.

Step 12: Congratulate yourself on choosing Pantheon

Even a large site with a lot of content is easy to get setup with Pantheon. Once you have it setup, testing and “go live” are also “push button” operations. Pantheon makes managing your workflow so easy, you won’t have to get your nephew out of class to help you.