My projects all use DDEV for local development. Regarding end-to-end testing, as part of my continuous integration process, I also use DDEV! This way, my scripts for running Cypress are the same locally and in my CI. Leveraging DDEV in your CI is especially useful if your project has multiple domains, which is harder to replicate using good old php -S 127.0.0.1:8080 -t web. Recently I wanted to speed up my end-to-end tests by avoiding full site installs for each job. A full site install could take a few minutes with the default content creation. But I knew a database restore could take a fraction of that time. Luckily, DDEV provides snapshots that can be used to restore your environment from a backup!

The approach was this:

• Create a workflow against the main branch to generate the snapshot, but only on a cache miss.
• Add if statements to the end-to-end testing workflow only to perform a site install on a cache miss. Otherwise, perform a snapshot restore.
• Manually purge the snapshot cache weekly, or build a job on a schedule to delete or recreate the cache.

The cached directory is .ddev/db_snapshots.

This was set up for GitHub Actions but should work on any continuous integration system and its caching capabilities.

Generating the DDEV snapshot

First, there needs to be a workflow for generating that snapshot so that it can be cached and re-used. Since there may be a lot of consecutive merges, I made sure to put the workflow behind a concurrency check only to allow one of these jobs to run at a time.

name: DDEV snapshot cache
# Only run on pushes to `main`
on:
  push:
    branches:
      - 'main'
# Do not allow multiple runs.
concurrency: cache_ddev_snapshot
jobs:
  setup_cache:
    name: Set up snapshot cache
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      # Download cache
      - name: Download DDEV snapshot
        id: ddev-snapshot
        uses: actions/cache@v3
        with:
          path: .ddev/db_snapshots
          key: ddev-db_snapshots

      # Install DDEV if cache miss and generate snapshot.
      - name: Install ddev
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: curl -LO https://raw.githubusercontent.com/drud/ddev/master/scripts/install_ddev.sh && bash install_ddev.sh
      - name: Start ddev
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: ddev start
      - name: Install Drupal
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: ddev site-install
      - name: Take DDEV snapshot
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: ddev snapshot --name ci

The actions/cache action provides the cache-hit output to determine if the cache was a hit or miss. This allows you to skip or perform certain steps. In the above example, I check for cache misses to perform the site install and snapshot generation.

if: steps.ddev-snapshot.outputs.cache-hit != 'true'

Once the job is completed, on a cache miss, the .ddev/db_snapshots directory will become cached for other workflows to leverage.

The concurrency check prevents the cache entry from being created multiple times if several consecutive jobs have a cache miss. Any following runs will skip their steps since they will have cache hits.

Using the snapshot in end-to-end testing workflow

Here is the end-to-end testing workflow. It downloads the cached DDEV snapshot. The ddev snapshot restore command is used alongside drush deploy if the snapshot is available from cache. If the cache is not available, then a site install is performed.

name: E2E Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  test:
    name: Cypress
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      # We always use DDEV, so download it.
      - name: Install ddev
        run: curl -LO https://raw.githubusercontent.com/drud/ddev/master/scripts/install_ddev.sh && bash install_ddev.sh
      - name: Start ddev
        run: ddev start

      # Download cache
      - name: Download DDEV snapshot
        id: ddev-snapshot
        uses: actions/cache@v3
        with:
          path: .ddev/db_snapshots
          key: ddev-db_snapshots

      # If there is a cache miss, install the site.
      - name: Install Drupal
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: ddev drush site-install --account-pass=admin -y
        
      # On cache miss, take a snapshot to populate branch-specific cache.
      - name: Take DDEV snapshot
        if: steps.ddev-snapshot.outputs.cache-hit != 'true'
        run: ddev snapshot --name ci

      # If there was a cache it, restore from the CI.
      - name: Restore from DDEV snapshot
        if: steps.ddev-snapshot.outputs.cache-hit == 'true'
        run: ddev snapshot restore ci
      
      # Run drush deploy to perform schema and config updates before testing.
      - name: Run updates
        if: steps.ddev-snapshot.outputs.cache-hit == 'true'
        run: ddev drush deploy

      # Execute tests...

Again, steps are controlled based on the cache hit or miss output.

if: steps.ddev-snapshot.outputs.cache-hit == 'true'

In the event of a cache miss, the job will perform its own site install and snapshot creation. This creates a cache specific for that branch.

Invalidating/purging the cached snapshot

I admittedly didn't get that far yet. GitHub has documented how to delete caches using the gh-actions-cache cli in a workflow. Caches live until they're evicted after 7 days of inactivity or storage limits are reached.

Caches are immutable. They will not be saved and updated if there is a cache hit. But there is a documented way to update a cache to be saved even if a cache hit occurs. Another workflow on a weekly schedule could be used to ensure the cache is updated.

Quirks and cautions

Drupal's install process will ensure various directories exist – such as the public files directory, private files directory (if configured), and temporary directory. My project uses object storage, and I overrode the file_temp_path value to provide a custom directory relative to the project's code base. I had to ensure these directories existed. Otherwise, PHP raised a notice that tempnam had to fall back to the system's temporary directory. For some reason, this caused a bigger error than it should have.

With DDEV, I had an error appear after my snapshot restore. DDEV sets the appropriate transaction isolation for the database when the project starts.

Container ddev-router  Running
unable to SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED: stdout='', stderr='', err=ComposeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-PROJECT docker-compose -f /home/runner/work/OWNER/REPO/.ddev/.ddev-docker-compose-full.yaml exec -T db bash -c set -eu && ( mysql -e "SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;" 2>/dev/null)', action='[exec -T db bash -c set -eu && ( mysql -e "SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;" 2>/dev/null)]', err='exit status 1', stdout='', stderr=''

This is superficial and will be fixed in DDEV 1.22.0. See https://github.com/ddev/ddev/issues/4933. 

I'm available for one-on-one consulting calls – click here to book a meeting with me 🗓️