Dave Hall Consulting logo

Drupal Puppies

Over the years Drupal distributions, or distros as they're more affectionately known, have evolved a lot. We started off passing around database dumps. Eventually we moved onto using installations profiles and features to share par-baked sites.

There are some signs that distros aren't working for people using them. Agencies often hack a distro to meet client requirements. This happens because it is often difficult to cleanly extend a distro. A content type might need extra fields or the logic in an alter hook may not be desired. This makes it difficult to maintain sites built on distros. Other times maintainers abandon their distributions. This leaves site owners with an unexpected maintenance burden.

We should recognise how people are using distros and try to cater to them better. My observations suggest there are 2 types of Drupal distributions; starter kits and targeted products.

Targeted products are easier to deal with. Increasingly monetising targeted distro products is done through a SaaS offering. The revenue can funds the ongoing development of the product. This can help ensure the project remains sustainable. There are signs that this is a viable way of building Drupal 8 based products. We should be encouraging companies to embrace a strategy built around open SaaS. Open Social is a great example of this approach. Releasing the distros demonstrates a commitment to the business model. Often the secret sauce isn't in the code, it is the team and services built around the product.

Many Drupal 7 based distros struggled to articulate their use case. It was difficult to know if they were a product, a demo or a community project that you extend. Open Atrium and Commerce Kickstart are examples of distros with an identity crisis. We need to reconceptualise most distros as "starter kits" or as I like to call them "puppies".

Why puppies? Once you take a puppy home it becomes your responsibility. Starter kits should be the same. You should never assume that a starter kit will offer an upgrade path from one release to the next. When you install a starter kit you are responsible for updating the modules yourself. You need to keep track of security releases. If your puppy leaves a mess on the carpet, no one else will clean it up.

Sites build on top of a starter kit should diverge from the original version. This shouldn't only be an expectation, it should be encouraged. Installing a starter kit is the starting point of building a unique fork.

Project pages should clearly state that users are buying a puppy. Prospective puppy owners should know if they're about to take home a little lap dog or one that will grow to the size of a pony that needs daily exercise. Puppy breeders (developers) should not feel compelled to do anything once releasing the puppy. That said, most users would like some documentation.

I know of several agencies and large organisations that are making use of starter kits. Let's support people who are adopting this approach. As a community we should acknowledge that distros aren't working. We should start working out how best to manage the transition to puppies.

Distros are Drupal's future!

Steve Purkiss wrote:

Thanks for highlighting this important area of the Drupalsphere Dave, I've always believed distros are Drupal's future, specifically verticals. There are thousands of people with the same business models who are getting sold cobbled-together sites mixing various SAAS offerings because the level they're being sold to at is the design level as opposed to the engineering. So you end up with sites that have the latest fashion, whether a scrolly uppy-downy or a scrolly lefty-righty but when you actually try to do anything functional it fails. You get transported between one underlying system and another, for which many aren't linked, don't follow your brand identity - look and feel, through and often break or change because the SAAS business model didn't work for that company or the SAAS service got bought out and so on.

Drupal could excel here, but it needs to have the communities to come together around it, so say for example business coaches, or astrologers, or whatever. Find out who's got the biggest problems you can solve with the easiest workflows in Drupal, find 10 or 100 customers in that industry and build.

In technical and sustainability terms, the work Nedjo and team have been doing on Drutopia (gitlab.com/drutopia) is quite incredible. Aimed at grass-roots campaigns, NGOs and so on, they also have good concepts around providing an OpenSAAS service for a modest monthly fee where you have your choice of host, regular updates, and help fund ongoing development. I've been encouraged by the work there and have high hopes for Drutopia, I believe it will be of great value to a world that Drupal(tm) seems to have left behind in the dust of late. Viva la GPL, Free Software, Free Society and all that!

Last example I have is one I've been supporting for a while now as I like it in technical terms although it needs a lot more / some documentation, support, and so on. It's had over £650k of investment from client projects been put into it but is a mystery to use unless you know what it does - native Drupal CRM. These guys have been building CRM "the Drupal way" for 7+ years now, and whilst projects like CRM Core and RedHen provide solutions for simple CRM problems, the minute you get into anything more complex they become troublesome if not impossible to use. These guys have built some amazing systems on the back of their distro and are working on the 8 version now, there's one upload of a kickstart distro which does install and works, but of course without any documentation or support it's not getting much traction, and I don't have the resources myself to help more than i do already - managed to get them into a couple of Drupal events this year so there's a couple of videos on youtube if you search "native drupal crm", the kickstart is at dgo.to/contacts_kickstart

Distros, they're Drupal's future I tell ya - and I wrote about it here: https://purkiss.com/blog/steve-purkiss/2016/11/18/end-web-cms-brass-era

Added Sun, 2017-09-24 22:13

I totally agree with your

dawehhner wrote:

I totally agree with your point that installation profiles should be a starting pack based around composer.

There are many things in install profiles currently which lead to bad decisions:

* Install profiles are active module * Install profiles ship with modules directly * Install profiles are code

In my ideal future many of those features should be basically build upon composer. As result install profiles end up being initial configuration + a composer.json file.

Stuff like continues updates of configuration etc. can all be solved on the module level ...

Added Mon, 2017-09-25 02:29

I totally agree with you that

Mohammed Razem wrote:

I totally agree with you that developers and companies creating a distribution should commit supporting it and "raising" their puppies.

Take Varbase for example. The things that allowed us to build, grow and keep maintaining it are:

  1. We dedicated 2 developers - non-billable, full time working on the distro.
  2. We implemented automated functional testing - a must to keep the distro in shape as more requirements are implemented.
  3. Other teams/developers inside Vardot contribute patches/ideas - since we use Varbase for our projects (we eat our own dog food).
  4. We chose a clear path for what is this starter kit for - that is a "coupled" Bootstrap-based starter kit.

In terms of updating, even though Varbase is a stater kit, I believe users should not be responsible, or even try to update the distro's modules/components on their own. It is still the responsibility of the maintainer. This is because stater kits have patches, integration/functional tests can easily break if you update, and there's definitely a reason why a maintainer would tag a dependency to specific release or even a commit hash in composer. That's how we're doing it right now for Varbase (and I guess Lightning are doing the same).

Also, it's important to note that Drupal 8's config management made it extremely super resilient to manage update paths. For example we categorize updates as:

  1. Forced update: We have a new module that became a dependency, or a configuration "fix". In the update_hook we check if this setting remained the same before we do it so we don't override user's setting.
  2. Optional update: A nice enhancement that we recommend you to use. We usually communicate this in Release notes. This should have UI such as checklistapi. There's no hook_update for this.
  3. No update: Only new installed would get this change.

At the end, the user uses his/her own composer.json in a template project that requires varbase (as a whole). An update would be as easy as to require the newer version.

Added Mon, 2017-09-25 16:35

What a shame...You have

Anonymous wrote:

What a shame...You have wasted occasion to paste some images of puppies! IMHO it would be perfectly justified! ;)

Otherwise, I agree completely with what you stated.

Added Tue, 2017-09-26 00:34

I like the "puppies" analogy!

balsama wrote:

I like the "puppies" analogy! I do disagree though that users "should never assume that a starter kit will offer an upgrade path from one release to the next". As dawehhner wrote, updates and migrations can be handled at the module level, which includes modules that are bundled with Distros.

Similar to what Mohammed described in Varbase, Lightning does provide an update path between releases. Our CI gives a glimpse into that as we test updates from the last five Lightning releases on each commit.

This, of course, causes problems when we want to make a change to previously supplied config that the site now owns. Again like Varbase, we break our updates into two categories: Required (hook_update) updates that don't change config, and Optional updates that typically reconfigure something we previously supplied.

In Lightning 2.1.8 Lightning started providing a Drupal Console command that will automate the configuration changes in the Optional updates: update:lightning.

I like the idea of Distributions limiting their scope and looking at them as a starting point on which something can be built. Getting the Inherited Profiles patch to state where it would be committed would go a long way towards people using them like that. But none of that precludes a Distro from providing update paths for its users.

Added Tue, 2017-09-26 03:43