Managing your Drupal platform with Drush

Published onTuesday 27, January 2015

Drush - the Drupal command line utility. One of the benefits to using Drupal is harnessing the command line tool Drush. If you don’t know about Drush, I advise heading over the the repository and also checking out this guide. Drush is powerful because its commands allow you to simplify your workflow - from clearing caches to migrating databases across environments. One of the best features, in my opinion, is the make command. Using Drush you can build Drupal with specific libraries and modules from a makefile. To make it even more awesome you can patch these projects through the makefile.

When I first started with Drupal I didn't really touch makefiles - I actually had no idea about them or their use. I stuck to "oh sweet, I can flush caches and enable modules!" Then the time came when working on a client's site uncovered a bug in a module being used. Usually I would just do a blind update to the latest development version and hope for the best. Generally this worked, but it also meant I was pulling in other fixes (changes) I didn't want. Then there was the dreaded situation where the patch wasn't committed yet, leaving me in patch purgatory. There isn't exactly an easy way to note which modules have been patched along the way - aside from a manually curated text list, or moving them into a patched file (leading to registry rebuilds each time.)

Originally this post started out to reflect how Drush make aids in project management plus integrity at a platform level. Then I realized this is for anyone who manages Drupal sites - regardless if they're built off of profiles or cobbled together client sites.

Drush make, and why it's awesome

Imagine creating a simple file and running a single command to have a full Drupal code base. The best way to dive into the anatomy of a makefile is to review one inside of a Drupal profile distribution. For those who didn't know (I was one of them) that's how distributions are handled on Drupal.org. It's a make file in the repository and then compiled on Drupal.org for end users to download as a zip or tarball.

To get started you need to define the makefile's API version, which is 2. Then specify what version of Drupal core you're using - like 7.x. You'll add lines to pull in core and then your projects. Here's an example portion from Panopoly's Core module (adapted slightly for this article)

; Panopoly Core Makefile

api = 2
core = 7.x

; Drupal Core
projects[drupal][type] = core
projects[drupal][version] = 7.34

; Panels and Chaos Tools Magic

projects[ctools][version] = 1.5
projects[ctools][subdir] = contrib
projects[ctools][patch][2312505] = http://drupal.org/files/issues/ctools-views-pane-more-link-2312505-1.patch
projects[ctools][patch][1978378] = http://www.drupal.org/files/ctools-page-title-check-plained-twice-1978378-1.patch
projects[ctools][patch][1565782] = http://www.drupal.org/files/1565782-obey-view-display-defaults-5.patch
projects[ctools][patch][2012188] = http://www.drupal.org/files/issues/2012188-9-ctools_entity_field_content_type_admin_title.patch
projects[ctools][patch][2055785] = http://drupal.org/files/issues/Modal-window-top-and-left-style-values-2055785-3.patch

projects[panels][version] = 3.4
projects[panels][subdir] = contrib
projects[panels][patch][2283719] = http://drupal.org/files/issues/panels-icon-text-renderer-2283719-1.patch
projects[panels][patch][2280797] = http://drupal.org/files/issues/panels-ipe-keyboard-focus-2280797-2.patch

If this was saved as "mysite.make" you could simply run drush make mysite.make /path/to/build and have Drupal core with Chaos Tools and Panels, patched appropriately.

Why am I referencing Panopoly? Because I learned my workflow by referencing the maintainer's methods. Modules carefully evaluated and not just updated to latest dev, only pulling patches that properly fix bugs. 

I made a "profile" for client sites. I use quotes because I manually built the profile and put the code in the modules folder and had the whole thing in version control. Life would have been a lot easier if I only needed to maintain a single file and build as needed into new client site directories.

For anyone reading this thinking "well I don't want to build a whole profile to use..." you don't need to. It's my experience that most client's get a site up and have a little maintenance here and there. Then some time down the road there is heavy work to be done, and chances are you may have forgotten what modules were patched. Drush will create a PATCHES.txt in the module's directory (if you didn't keep original makefile around.)

The following patches have been applied to this project:
- http://drupal.org/files/issues/ctools-icon-text-renderer-2280875-5.patch
- http://drupal.org/files/fix-autocomplete-581670.patch
- http://drupal.org/files/issues/ctools-views-pane-more-link-2312505-1.patch
- http://www.drupal.org/files/ctools-page-title-check-plained-twice-1978378-1.patch
- http://www.drupal.org/files/1565782-obey-view-display-defaults-5.patch
- http://www.drupal.org/files/issues/2012188-9-ctools_entity_field_content_type_admin_title.patch

If you ever wonder why people put issue node IDs in patch file names, I hope you're seeing why this is important - hint, it's so you can easily reference the issue via patch.

Defining Projects and Libraries

In general you can define a project as simply as

projects[views] = 3.8

Cool. But that means it'll just dump into the sites/all/modules folder, not something like sites/all/modules/contrib.

; Place Views inside the contrib folder.
projects[views][version] = 3.8
projects[views][subdir] = contrib

; Or make all projects go into contrib folders - all/themes/contrib, all/modules/contrib
defaults[projects][subdir] = contrib

This makes it easier to tell custom work apart from contributed work. Take a Drupal Commerce site, for example. I'll define the default to be contrib, but Commerce and all Commerce contribs go under a commerce subdir, to make things a little cleaner/easier to find.

You also have control of downloading projects from a specific revision. If you wanted to blindly download the latest development version, simply drop #.x-dev as the version and Drush will go all out for you. I highly recommend, however, you pick a revision you're satisfied with, or at least the current one so it doesn't randomly get updated on you (and break, leading to head marks on your desk.)

projects[redirect][download][type] = git
projects[redirect][download][revision] = 0b7b8dc2d58cb277874d87c91c45f0a361e148f7
projects[redirect][download][branch] = 7.x-1.x

In this case you'll use the download settings. Define the type as Git, give it a revision and branch. Drush will check out the project via Git at the specified revision for you. This is also how you'd grab a sandbox project, or even a project not hosted on Drupal.org (like GitHub.)

; Media Multiselect
projects[media_multiselect][type] = module
projects[media_multiselect][subdir] = contrib
projects[media_multiselect][download][type] = git
projects[media_multiselect][download][url] = http://git.drupal.org/sandbox/fangel/1652676.git
projects[media_multiselect][download][branch] = 7.x-1.x
projects[media_multiselect][download][revision] = 1b4fc64

; GitHub hosted theme
projects[material_admin][type] = theme
projects[material_admin][download][type] = git
projects[material_admin][download][branch] = master
projects[material_admin][download][url] = https://github.com/mglaman/material_admin.git

Just be sure to use the SSH URL for your GitHub/BitBucket/etc repos if they are private.

And, then, there are your good ole libraries. Drush can grab your FontAwesome, Colorbox, etc, etc ,etc.

; Font Awesome
libraries[fontawesome][download][type] = get
libraries[fontawesome][download][url] = http://github.com/FortAwesome/Font-Awesome/archive/v4.2.0.zip

; COLORBOX
libraries[colorbox][download][type] = git
libraries[colorbox][download][url] = git://github.com/jackmoore/colorbox.git
libraries[colorbox][download][revision] = 124ec40
libraries[colorbox][directory_name] = colorbox

Patch Management

The central theme here has tried to pin down on patch management. Patches can be grabbed from any URL or from a local source (the latter is a huge benefit which I'll cover.) When defined in the makefile patches are an array value. Let's revisit the snippet from Panopoly Core once more.

projects[panels][patch][2283719] = http://drupal.org/files/issues/panels-icon-text-renderer-2283719-1.patch
projects[panels][patch][2280797] = http://drupal.org/files/issues/panels-ipe-keyboard-focus-2280797-2.patch

Each patch is using the issue node ID as the key value. You don't have to do this, you could keep it simple as [], but I highly recommend following this pattern. There will be times you come across patch missing the node ID in the file name - or maybe even typed incorrectly. Coming back to a makefile three months later and unable to track down an issue is pretty frustrating. This is a shining example of why using makefiles is a great form of project management - you can easily visualize the status of your codebase. You know the exact versions and patches, along with a simple way to go back to those issues for review.

There's one major problem with patches - sometimes they just don't apply. A great scenario is the use of a module that's had a lot of work of development done since the last release. You may find a patch, or have to write a patch against develop's HEAD) and it doesn't apply to the release. You have two options: upload a second patch against the release to the issue, or reference a local patch. To be honest, I think you're safer with the first, but the latter can de-clutter the issue comments.

projects[panels][patch][XXXXXXX] = path/relative/to/make/my-special-patch-to-fix-the-things.patch

Local patches are referenced to the relative path of the makefile. Simply add the path instead of a URL and Drush will pull it in to patch the project. A good example would be #2415427: Avoid loading FPP from database in fieldable_panels_panes_preprocess_panels_pane(). Obviously the second patch against 1.5 was uploaded so Panopoly could pull it in, but this helps visualize the use case. 

Another usefulness of local patches is you don't rely on Drupal.org for hosting. Of course we need Drupal.org to download projects, but Drush can be pretty reliable for digging items up out of its cache. Drush caches everything it downloads, including patches. However some might feel safer knowing patches are in VCS and always available.

Here's where local patches really matter. Why? Because it lets you break the golden rule: NEVER HACK CORE (& CONTRIB).

We've all been there. We need some very specific functionality out of a module that isn't part of the project's plan and we can't just hook our away through. Well, after you have exhausted all other options you can feel at ease by falling back to patches. I stress this as a last resort, because it can become easily to just start writing private patches to solve problems that might be valid upstream.

Some people are going to debate this. But the fact is we sometimes need things to work in a certain way. This way you don't feel the need to fork a module privately, or keep the source code version controlled to live in a Frankenstein'd hell. You're able to keep up to date with upstream while preserving needed modifcations.

Why would I ever hack a modules source code?!

I feel 35% of the community would lambaste me over the above comment. Some others might actually have no idea, or hopefully these examples help a few readers solve existing management issues.

  • API implementations: Up until recently you couldn't tell Commerce Authorize.net to use different gateway URLs. A lot of payment gateways emulate Authorize.net. Instead of having to hack the module a local patch can be made to change these URLs.
  • Security/Senstivity: I don't mean security in the software vulnerability sense. I mean the business's practices could be exposed. Going back to an API example, the platform or customer could need specific alterations to the API requests not available through a hook. This can be a soft hack until the module provides proper hooks to change requests.
  • Minimally maintained modules: I've had a few cases where I've run into a minimally maintained module that needed a lot of work. I wrote a bunch of patches, but in the end there was so much work only one patch would apply. A solution is to keep a local patch of the fully updated module until upstream gets a few things committed. 

To touch on that last point, I would prefer maintaining a large patch over directly version controlled files. It's a personal preference that keeps the refs clean.

Other tools

At work I was originally tracking all of our patches through a wiki page in a table. It was pretty rough and difficult to maintain. So on News Years Day 2015 I tossed together my first Chrome app - Drupal Issue Tracking. It's grown a bit, but the original use case was to give myself an interface for keeping an eye on patches we were using. It also gave me a way to keep tabs on important issues affecting our platform.

Drupal.org has a "My Issues" section, but chances are you, like me, are active on multiple issues that aren't just patches or critical changes. The goal of the app is to make this management simpler.

That's all folks.

Hopefully this opened a few people up to using Drush. I wrote this article as something I wish I would have found a year and a half ago when I first went into the Drupal world. Hell, I think it's something project managers at firms using Drupal should learn about to help support their teams. It's even a selling point for ongoing maintenance - reduce patch list, keep things up to date. It also helps highlight technical debt. The more patches the more fragile your platform becomes. Balance time between building your platform and getting those patches committed.