Dist::Zilla is a tool that lets you work how you want, and produce consistent, CPAN-compatible distributions irrespective of your particular personal preferences.

A while ago (a "3 years" sort of while), Dist::Zilla was broken most of the time. CPAN installations of it would fail, and using it would put the distribution in a state whereby if you couldn't get at Dist::Zilla, you couldn't work on your code. This was a big turn-off, and it was hard to forgive it for being new, because it was exciting and we all wanted to play with it.

Recently, advocates of Dist::Zilla started making enough noise within my light-cone that the signals reached me, and I decided to have another look. Our adolescent temper-tantrum module has matured into adulthood! It was time to start taking it seriously again.

Pre-zilla

I've been at OpusVL for just over a year, and one of the first things I noticed when I joined was that a typical customer application was separated into a large number of discrete modules.

Architecturally, this is the preferable way, but newly emerged and blinking in the light of the Perl world, I had to be reminded quite how granular your code can be when you have a decent toolchain and a real import mechanism. At previous jobs I've instigated a sort of slow mudslide from gargantuan applications to a more modular style, but that was PHP. There was only so much that could be done.

The upshot of this was that Makefile.PL was the most common filename in existence and I had no idea what to do with a module once I'd edited code.

The problem was not an architectural one but a social one: simply that there were many internally-written tools and procedures that needed to be known. It turned out the thing I was supposed to do with the module was release it. Upload it to our internal CPAN mirror, so that deployment targets can install it from the same mirror. So, dutifully, I bumped the version numbers, committed the files with the commit message "Bumped version number", created the git tag, uploaded to the local CPAN mirror and pushed all my changes to the internal git repository for that module.

It was a while before I learned about the release.sh script in a git repository designed to hold all the sorts of scripts we often use for git-related things.

I probably don't need to censure the vices of internal scripts that Do Things. They have probably evolved cankerously over time; originally intended to support just one method of Doing Things, they now support the old way, and the new way, and try to guess which is which; and woe betide if there are more than two historical ways, or the ways are indistinguishable except for entries in the wiki.

Obviously, not all such scripts have such barnacles. Nevertheless, surely many people around the world have similar development practices? Surely, since we're effectively authoring modules and releasing them to CPAN, we should be able to use external tools?

Now, yes. But 3 years ago, when Dist::Zilla was in its high-entropy state, hand-rolling automation of certain processes was necessary. Those battle-scars your internal scripts have accumulated have been seen time and again across the world.

The outcome of all these hand-rolled automation scripts is, naturally, a community effort to abstract and contain common practice. That accretion disc eventually coalesced into the Dist::Zilla of today.

Post-zilla

I can't go off on painting a vibrant picture of the marshmallow wonderland that is this Dist::Zilla utopia, because our present usage of it is limited to the lesser-used modules—easily branched and easily contained. However, I can comments on the success and future optimism that we've already received from a minimal input of effort¹.

Release process

It should be apparent, from my cursing of the Internal Script Methodology, that one of the major benefits of Dist::Zilla is the eradication thereof.

That's pretty much the case. Because our internal development process is functionally identical to the public PAUSE/CPAN authoring process, Dist::Zilla has all the tools we need to replace our existing scripts with Dist::Zilla plugins.

The Dist::Zilla::Plugin namespace contains myriad modules designed to work with the standard CPAN toolchain. These include auto-generating the files the PAUSE indexer needs, auto-generating the files the CPAN clients need, and doing the whole send-file-to-CPAN-for-indexing HTTP process in the first place.

Some of the release process is kind of personal - the version scheme, when you bump the version, how you deal with git, etc. But it's not so personal that other people haven't done that, too. The plugin namespace has things for git, for versioning, for the README and LICENSE and Changes files... it's a build-your-own-bear workshop of goodies.

With some choice selection of plugins, all of the internal script can be farmed off to modules that are either flexible enough to accommodate our style, or mature enough to impose a sensible-and-acceptable way of doing things on us.

The upshot of it all was that the only thing we actually had to write anew was a very specific release mechanism to upload the tarball to the type of mirror we're using. Since we are using CPAN::Mini::Inject, by means of CPAN::Mini::Inject::REST, the equivalent Dist::Zilla plugin Dist::Zilla::Plugin::CPAN::Mini::Inject::REST is now available on CPAN.

The other thing we wrote was the module that told Dist::Zilla how to behave - called a plugin bundle. This is actually available on CPAN if you want to work like we do; the only difference between this and any other release process is that the target is a CPAN::Mini(::Inject::Rest) server, as mentioned.

Finally, being that Dist::Zilla is entirely configured by the dist.ini file in the project's root directory, it is possible to have each project built and released in entirely different ways. No longer must a single script support multiple historic mechanisms; instead, each mechanism exists as its own Dist::Zilla::PluginBundle module, selected by the project and not loaded by any other.

It should go without saying that using modules written for the toolchain that has existed for literally a decade, written by and for people using the toolchain on a daily basis, will produce much more reliable and consistent results than trying to do the same thing manually, without understanding of the possible problems with this approach.

Building and Testing

Dist::Zilla separates the build/release cycle of a distribution into several stages. The first is creation, of course, but there are several steps that can be run independently—the actual act of pushing your release up to CPAN is on the other side of a [y/N] prompt, so you're unlikely to do this by accident.

Two useful steps in this are dzil test and dzil build.

dzil test will run all of your tests. The nice thing about Dist::Zilla is that testing is a first-class part of the process - traditionally, testing is something that's done by volunteer systems in spare cycles, after a push to CPAN.

dzil smoke allows you to run smoke tests. These are tests designed to run quickly and tell you whether anything obvious has gone wrong, e.g. compilation errors. Thanks to the plugin architecture of Dist::Zilla, you can select modules that recognise different ways of separating smoke tests from full tests.

It also allows you to specify author tests versus installation tests versus smoke tests. Author tests are those which are run when you run dzil test, but not when you later install the module with the CPAN toolchain. This ensures that you don't accidentally prevent the installation of a working module just because you had a spelling mistake in the POD.

dzil build produces a tarball suitable for uploading to CPAN. There are several stages involved in the actual build process, and they can all be hooked into via plugins. Primarily, these plugins are focused on collecting the correct files and putting them into the build directory—the one that is then tar'd up into the distribution tarball.

The benefit of the separation of this stage is the amount of automation that can be done. In our process, it actually uses git to determine what files should be in the distribution. That means that we can remove files from the build without removing them from the working tree.

There are also plugins that automatically generate files. These are usually to allow the rest of the toolchain to actually install the module - META.yml, the Makefile.PL itself, and such things. We also auto-generate the README.pod from the main module's documentation.

This all sounds neat, allowing the developer to do as much as possible with as little knowledge as possible, but the benefits really shine when you realise that, if you don't need any knowledge to build, test, and release a module, you don't need a developer to do it at all. Indeed, you could instigate a process whereby Jenkins or similar CI will build, test, release every time the master branch is pushed to.

For existing modules, the switch to Dist::Zilla has more benefits than even we've got around to actually making the best of, yet.

The only boilerplate in any of our distributions is the dist.ini itself, now:

name = Some-Module
license = Internal
[@Author::OpusVL]
mcpani_host = internal

This four-line file is the only thing that's guaranteed to be the same in all our distributions. It contains enough information to be able to autogenerate every single part of the distributed module.

This means that new developers, or even old developers six months later, are presented with a working directory almost entirely representative of the distribution. That is to say, editing any file besides dist.ini is going to have a real and valuable contribution to the module, and all files they can open will provide information about the module.

Minting process

The release.sh script was one that I - perhaps unfairly - picked on to make a point; the point being that when you work within a toolchain, there's rarely a reason to hand-roll parts of your working process, because you didn't write the toolchain either.

Another illustrative part of this principle is that the Dist::Zilla folk have also provided us with ways of creating distributions, called minting.

During my investigations, I got the impression that minting was a road much less travelled within the dzil community. While minting profiles do exist, they don't seem to roll with the levels of complexity that the build process carries; there are noticeably fewer minting profiles on the market, and the differences between them often seem to be whose author bundle the dist.ini uses, or what license goes in it by default.

Experimentally, I tried creating a couple of minting profiles that would create new modules to work with our framework—much like Ruby on Rails allows you to create a new model or controller. Except, our framework actually encourages you to have entirely separate distributions for the different sections of the site; that is, the entire site is made out of plugins.

It seemed like the minting process is perfect for this, because it covers both sets of behaviour: you can create an entire distribution, or you can add files to an existing one.

However, I did discover that there were some features lacking from the core Dist::Zilla plugins that I really needed to get the most out of this feature. Primarily, the minting process lets us create new files—but the options for doing this made it somewhat awkward to actually do so.

I did write a patch for this, but we seem to be in a stalled state on that.

The facility to generate parts or the entirety of new distributions is very powerful, however. We've already got ideas for creating minting profiles for Catalyst and DBIx::Class distributions.


¹ Most conversions to Dist::Zilla were really as simple as creating the dist.ini and extracting the dependency list out of Makefile.PL and into a cpanfile—then deleting all the crufty files that Dist::Zilla will autogenerate.

Posted by Alastair McGowan-Douglas on 13/10/2015