Convert a Standard Drupal 8 Site to a Composer Managed Site

Upgrade a standard Drupal 8 site, by converting it to a Composer-managed Drupal 8 site on the new Integrated Composer framework.

Contributors: Dustin LeBlanc, Greg Anderson, Tom Stovall.

Discuss in our Forum Discuss in Slack

In this guide, we'll convert a standard Drupal 8 site to use Composer to manage deployments and dependencies, then switch from drops-8 to the new Integrated Composer drupal-composer-managed upstream while remaining on Drupal 8.

If this is not your scenario, see Drupal 9 Migration Guides for additional upgrade paths.

Overview

Drupal 9 sites on Pantheon have Integrated Composer built-in to manage site dependencies.

The goals of this conversion are:

  1. Remove dependencies that Composer will manage from the existing Drupal 8 site's Git repository, and have Composer manage those dependencies instead.

  2. Switch to the drupal-composer-managed Integrated Composer upstream.

The drupal-composer-managed Integrated Composer upstream works with Drupal 8, and following the drupal-composer-managed upstream will help keep your site up to date with any general configuration changes recommended by Pantheon.

Add Drupal 8 core dependency instructions to drupal/core-recommended, to keep the site on Drupal 8 until you are ready to upgrade to Drupal 9.

Will This Guide Work for Your Site?

Confirm that your site meets the following requirements before you continue:

  • Ensure your site has the Pantheon drops-8 repo in its upstream.

    • Use Terminus to Confirm the drops-8 Upstream

      Run the command terminus site:info $SITE to display the site's basic information and properties.

      The following is an abridged example of the output for a site upstream set to drops-8:

      terminus site:info $SITE
      ------------------ -------------------------------------------------------------------------------------
      ID                 abdc3ea1-fe0b-1234-9c9f-3cxeAA123f88
      Name               anita-drupal
      Label              AnitaDrupal
      Created            2019-12-02 18:28:14
      Framework          drupal8
      ...
      Upstream           8a129104-9d37-4082-aaf8-e6f31154644e: https://github.com/pantheon-systems/drops-8.git...
      ------------------ -------------------------------------------------------------------------------------

      The following values indicate that a site is using a drops-8 upstream:

      • The Framework is drupal8

      • The Upstream includes https://github.com/pantheon-systems/drops-8.git

    • The site cannot be set to use an empty upstream.

  • The site does not use a nested docroot.

    • The process outlined in this guide will not work if the site repository has a /web folder at its root.

    • Refer to the documentation on Serving Sites from the Web Subdirectory for information about nested docroots.

  • The site does not use another package and library manager like Ludwig.

Before You Begin

  • This guide is written for users with access to Pantheon's Multidev feature. Pantheon support is not available to users who avoid the Multidev steps.

  • The site owner should ensure the trusted host setting is up-to-date. Refer to the Trusted Host Setting documentation for more information.

 Note

The steps in this process migrate a site, so the new site will no longer maintain its existing commit history.

Prepare the Local Environment

  1. Review our documentation on Git, Composer, and Terminus, and install and configure them on your local computer. Pantheon requires Composer 2 at minimum.

    • Mac users can use Homebrew to install Git, Composer, and PHP, along with their required dependencies. Note: Terminus 3 should be used for PHP >= 8.0. Restart the shell or terminal environment after entering the following command:

      brew install git composer php
    • Windows users can install Composer and Git, and may need to install XAMPP or similar to satisfy some dependencies.

  2. Set the temporary variable $SITE in your terminal session to match the name of your site.

    This doc uses several commands that use the temporary command line alias, $SITE, to make entering command examples easier. Refer to the steps further in this doc to see which sites should be aliased (it may be more than one), then replace anita-drupal in this example:

    export SITE=anita-drupal && echo "New alias set as $SITE"
  3. Create a new folder to use while working on the migration. This folder will contain two subdirectories that you'll create in the next sections, one for the site on the former platform, and one for the Pantheon site.

Apply All Available Upstream Updates

Update the site to the latest Pantheon Drops 8 Upstream and apply all available updates.

  1. Use Terminus to list all available updates:

    terminus upstream:updates:list $SITE.dev
    [warning] There are no available updates for this site.
  2. Apply any available updates using the command line or the Pantheon Dashboard:

    terminus upstream:updates:apply $SITE.dev --updatedb

Add the Integrated Composer Upstream in a New Local Branch

This process involves significant changes to the codebase that may take some time to complete, and can be complicated to roll back.

To minimize issues, these steps make the codebase changes in a new branch:

  1. In your local terminal, change directories to the site project. For example, if you keep your projects in a folder called projects in the home directory:

    cd ~/projects/$SITE/
  2. Add the Pantheon Drupal Project upstream as a new remote called ic, fetch the ic upstream, and checkout to a new local branch based on it called composerify:

    git remote add ic git@github.com:pantheon-upstreams/drupal-composer-managed.git && git fetch ic && git checkout --no-track -b composerify ic/main
    Switched to a new branch 'composerify'

    If you prefer, you can replace composerify with another branch name. If you do, remember to adjust the other examples in this doc to match.

Set Drupal Core Version

Set the Drupal core version, to ensure the site remains on Drupal 8 for now:

composer require --no-update drupal/core-recommended:^8.9
composer require --dev drupal/core-dev:^8.9
git add composer.*
git commit -m "Remain on Drupal 8"

Add Upgrade Status Module

This step is optional; you may wait and add the Upgrade Status module to your site later.

The Upgrade Status module will help to determine whether or not your site is ready to upgrade to Drupal 9.

Add the Upgrade Status module to your site with Composer:

composer require drupal/upgrade_status
git add composer.*
git commit -m "Add Upgrade Status module"

When you are ready to begin upgrading your site to Drupal 9, you may enable this module and view the status report it provides to find things that need to be done before upgrading.

Copy Existing Configuration

Copy any existing configuration from the default branch. If no files are copied through this step, that's ok:

git checkout master sites/default/config
git mv sites/default/config/* config
git rm -f sites/default/config/.htaccess
git commit -m "Pull in configuration from default branch"

Copy pantheon.yml

  1. Compare the old codebase's pantheon.yml to the new pantheon.upstream.yml:

    git diff master:pantheon.yml pantheon.upstream.yml

    Press q on your keyboard to exit the diff display.

  2. Copy the old pantheon.yml to preserve settings:

    git checkout master pantheon.yml
    git add pantheon.yml
    git commit -m 'Copy my pantheon.yml'

    Remove any values from pantheon.yml that you prefer to keep listed in pantheon.upstream.yml. Then add build_step: true to pantheon.yml if it is not already included.

    In the pantheon.yml file, the api_version: 1 and build_step: true values are required.

Add in the Custom and Contrib Code Needed to Run Your Site

What makes your site code unique is your selection of contributed modules and themes, and any custom modules or themes your development team has created. These customizations need to be replicated in your new project structure.

Contributed Code

Modules and Themes

The goal of this process is to have Composer manage all the site's contrib modules, contrib themes, core upgrades, and libraries (we'll call this contributed code). The only things that should be migrated from the existing site are custom code, custom themes, and custom modules that are specific to the existing site.

The steps here ensure that any modules and themes from drupal.org are in the composer.json require list.

Once Composer is aware of all the contributed code, you'll be able to run composer update from within the directory to have Composer upgrade all the contributed code automatically.

Begin by reviewing the existing site's code. Check for contributed modules in /modules, /modules/contrib, /sites/all/modules, and /sites/all/modules/contrib.

  1. Review the site and make a list of exactly what versions of modules and themes you depend on. One way to do this is to run the pm:list Drush command from within a contributed modules folder (e.g. /modules, /themes, /themes/contrib, /sites/all/themes, /sites/all/themes/contrib, etc.).

    This will list each module followed by the version of that module that is installed:

    terminus drush $SITE.dev pm:list -- --no-core --fields=name,version  --format=table
  2. You can add these modules to your new codebase using Composer by running the following for each module in the $SITE directory:

    composer require drupal/MODULE_NAME:^VERSION

Composer might pull in a newer version than what is specified (depending on version availability), if the machine name for the module is MODULE_NAME, and the version of that module is VERSION. You can read more about the caret (^) in the Composer documentation.

Some modules use different version formats.

  • For older-style Drupal version strings:

    Chaos Tools (ctools)  8.x-3.4

    Replace the 8.x- to convert this into ^3.4

  • Semantic Versioning version strings:

    Devel (devel)  4.1.1

    Use the version directly, e.g. ^4.1.1

Other Composer Packages

If you have added non-Drupal packages to your site via Composer, use the command composer require to migrate each package. You can use the following command to display the differences between the master and your current composer.json:

git diff master:composer.json composer.json

Libraries

Libraries can be handled similarly to modules, but the specifics depend on how your library code was included in the source site. If you're using a library's API, you may have to do additional work to ensure that library functions properly.

Custom Code

Next, manually copy custom code from the existing site repository to the Composer-managed directory.

Modules and Themes

To move modules, use the following commands:

git checkout master modules/custom
git mv modules/custom web/modules/
git commit -m "Copy custom modules"

To move themes, use the following commands:

git checkout master themes/custom
git mv themes/custom web/themes/
git commit -m "Copy custom themes"

Use the above commands with any of the custom code.

settings.php

Your existing site may have customizations to settings.php or other configuration files. Review these carefully and extract relevant changes from these files to copy over. Always review any file paths referenced in the code, as these paths may change in the transition to Composer.

We don't recommend that you completely overwrite the settings.php file with the old one, as it contains customizations for moving the configuration directory you don't want to overwrite, as well as platform-specific customizations.

git status # Ensure working tree is clean
git show master:sites/default/settings.php > web/sites/default/original-settings.php
diff -Nup --ignore-all-space web/sites/default/settings.php web/sites/default/original-settings.php
# edit web/sites/default/settings.php and commit as needed
rm web/sites/default/original-settings.php

The resulting settings.php should have no $databases array.

Additional Composer Configuration

Any additional Composer configuration that you have added to your site should be ported over to the new composer.json file. This can include configurations related to repositories, minimum-stability, or extra sections.

You can use the diff command to get the information you need to copy:

git diff master:composer.json composer.json

Deploy

You've now committed the code to the local branch. Deploy that branch directly to a new Multidev (called composerify in the steps below) and test the site in the browser.

Deploy to a Multidev

  1. Push the changes to a Multidev called composerify to safely test the site without affecting the Dev environment:

    git push -u origin composerify && terminus env:create $SITE.dev composerify
  2. Make a small change to pantheon.yml:

    pantheon.yml
    database:
     version: 10.4
    
    # add a comment to trigger a change and build
  3. Commit and push the change to trigger an Integrated Composer build on the Multidev:

    git commit -am "trigger composer build"
    git push origin composerify

Since the commit history of the composerify Multidev has no commits in common with the master branch, you cannot view the Multidev commit history from the Dashboard or the Integrated Composer logs.

If the site is not working, try this Composer command on the local composerify branch:

composer --no-dev --optimize-autoloader --no-interaction --no-progress --prefer-dist --ansi install

If Composer runs into an error or if any files have been changed (files that are not ignored by .gitignore), resolve those issues before you continue. See the Integrated Composer Troubleshooting section for more information about troubleshooting Integrated Composer.

Move composerify to the Main Dev Branch

Once you have confirmed that the site works in the Multidev, replace the master branch and its commit history with the composerify Multidev's commit history.

  1. Retrieve the most recent commit hash from the local composerify branch:

    git log --format="%H" -n 1

    This will give you a commit hash like fd3636f58f5b275b998bb1c9267bff8808353840.

  2. Reset the master branch to match that commit then force push that to the Dev environment:

    git checkout master
    git reset --hard fd3636f58f5b275b998bb1c9267bff8808353840
    git push --force origin master

Your site's Dev environment is now set up to use the Drupal 9 Integrated Composer upstream.

Troubleshooting: Inspect Site Logs

If the site doesn't load properly, before you do too much work to investigate issues, clear the cache and try again.

Use Terminus to inspect the site's logs;

terminus drush $SITE.composerify -- wd-show

See our logs collection documentation for more information.

Troubleshooting: Provided host name not valid

If you receive the error message "The provided host name is not valid for this server.", then update your settings.php file with a trusted host setting. Refer to the Trusted Host Setting documentation for more information.

Change Upstreams

Your Pantheon site is now set up to use the Drupal 9 Integrated Composer upstream. To continue tracking additional changes to the Pantheon upstream, change the upstream your site is tracking with Composer:

terminus site:upstream:set $SITE drupal-composer-managed

Following the drupal-composer-managed upstream will help keep your site up to date with any general configuration changes recommended by Pantheon. The dependency you added above on drupal/core-recommended will keep you on Drupal 8 until you are ready to upgrade to Drupal 9.

Working With Dependency Versions

Version compatibility issues can occur when packages pulled by Composer are updated along with their dependencies. If this happens, you will need to manually alter the version constraints on a given package in the require or require-dev section of composer.json to update the packages. Refer to the updating dependencies section of the Composer documentation for more information.

Troubleshoot package updates by running composer update. This updates composer.lock to the latest available packages. Package updates are constrained by version requirements in composer.json.

See Also