Production is an Artifact of Development

Presented by Michelle Krejci (@dev_meshev)

TL;DR

development

Development tools

artifact

Shipment box

development


section {
    height: 100px;
    width: 100px;

    .class-one {
       height: 50px;
       width: 50px;

       .button {
           color: #074e68;
       }
    }
}
            

artifact


section {
    height: 100px;
    width: 100px;
}

section .class-one {
    height: 50px;
    width: 50px;
}

section .class-one. button {
    color: #074e68;
}
            

development

Drupal development tools

artifact

Drupal Icon

development


Vagrantfile
behat.yml
circle.yml
composer.json
composer.lock
PROJECT.info
PROJECT.module
README.md
build/
features/
cnf/
bin/
www/
vendor/
modules/
themes/
            

artifact


CHANGELOG.txt
COPYRIGHT.txt
INSTALL.mysql.txt
INSTALL.pgsql.txt
INSTALL.sqlite.txt
INSTALL.txt
LICENSE.txt
MAINTAINERS.txt
README.txt
UPGRADE.txt
authorize.php
cron.php
includes/
index.php
install.php
misc/
modules/
profiles/
robots.txt
scripts/
sites/
themes/
update.php
web.config
xmlrpc.php
            
This is not production.

The Agenda

  1. How project repository got to the state it is in now.
  2. Why production should be different.
  3. What does separating development from production look like in practice.
  4. Implications.

@dev_meshev

Palantir.net Pays me, too!
  • Engineer, Palantir.net
  • Repository of strong opinions, weakly held

A

brief, opinionated, subjective

History of a shippable Drupal

circa 2001 - 2009

CHANGELOG.txt

COPYRIGHT.txt

INSTALL.mysql.txt

INSTALL.pgsql.txt

INSTALL.sqlite.txt

INSTALL.txt

LICENSE.txt

MAINTAINERS.txt

README.txt

UPGRADE.txt

authorize.php

cron.php

includes

index.php

install.php

misc/

modules/

profiles/

robots.txt

scripts/

sites/

themes/

update.php

web.config

xmlrpc.php

Great Deployment Problems in Drupal's history

  • Everything in the database.
  • When and how to apply changes to the database.
  • Making the build explicit and executable.
  • Coming up with excuses not to test things.
  • Building on a known state that mimics production.
  • Lugging around the Drupal core and contrib codebase inside the repository.
  • Dealing with Drush make.
  • Building Drupal root with Composer.
  • Installing dependencies.

Problem

Everything in the database \o/

Solution

features!

circa 2009

CHANGELOG.txt

COPYRIGHT.txt

INSTALL.mysql.txt

INSTALL.pgsql.txt

INSTALL.sqlite.txt

INSTALL.txt

LICENSE.txt

MAINTAINERS.txt

README.txt

UPGRADE.txt

authorize.php

cron.php

includes

index.php

install.php

misc/

modules/

profiles/

robots.txt

scripts/

sites/

themes/

update.php

web.config

xmlrpc.php

Problem

When and how to apply changes to the database.

Solution

Move the drupal directory back one directory, add a script.

circa 2009 - 2011

drupal-root/

deploy.sh

Problem

How do we build this thing?

Solution

Make it explicit and executable.

circa 2009 - 2011

drupal-root/

deploy.sh

build/install.sh

build/update.sh

Problem

Every LAMP WAMP MAMP XAMP was different and none matched production.

Solution

Vagrant

circa 2011

drupal-root/

build/

Vagrantfile

provisioning/

Problem

We have run out of excuses for not testing our features.

Solution

Start testing.

circa 2013

drupal-root/

build/

Vagrantfile

provisioning/

behat.yml

features/

phpspec.yml

.travis.yml

circle.yml

Problem

Drupal is just a dependency.

Solution

Drush make.

circa 2013

drupal-root/

build/

Vagrantfile

provisioning/

behat.yml

features/

phpspec.yml

circle.yml

MY_PROJECT.make

Problem

  • Slow build when downloading dependencies
  • Does the bare minimum
  • Is a singleton

Solution

Composer icon

circa 2014

drupal-root/

build/

Vagrantfile

provisioning/

behat.yml

features/

phpspec.yml

circle.yml

MY_PROJECT.make

composer.json

composer.lock

vendor

Problem

Composer doesn't know how to build Drupal.

Solution(s)

circa 2015

drupal-root

build/

Vagrantfile

provisioning/

behat.yml

features/

phpspec.yml

circle.yml

composer.json

composer.lock

bin/rootcanal

Problem

Installing dependencies

...authentication tokens and dependencies and timeouts and malformed packages...

Solution

Leverage automation tools and ship only the product of development.

The Future

CHANGELOG.txt

COPYRIGHT.txt

INSTALL.mysql.txt

INSTALL.pgsql.txt

INSTALL.sqlite.txt

INSTALL.txt

LICENSE.txt

MAINTAINERS.txt

README.txt

UPGRADE.txt

authorize.php

cron.php

includes

index.php

install.php

misc/

modules/

profiles/

robots.txt

scripts/

sites/

themes/

update.php

web.config

xmlrpc.php

Why is development and production code different?

We are going to chat about Composer in 5 slides.

1.

Development code should be readable as instructions for humans.

Production code should be readable by a web service.

2.

We should depend on our platform hosts to host the thing,

not the build the thing.

3.

Devs (and shops) should be opinionated about development.

Production environments (PaaSs) should be opinionated about production.

4.

No dev dependency should be installed on a live server.

5.

Especially composer

(which you should absolutely use)

  • Retrieving meta-data and dependencies eats a ton of ram
  • Composer will fetch dependencies from a lot of sources
  • Rate limits on github
  • Your deployment could take longer than expected
  • Your dependencies could disappear
Further reading

Demo

What you are about to see.

  1. Project will build (with Composer and build scripts), tests will run.
  2. If successful, an artifact will be build.
  3. Artifact will be committed to the deployment environment.
  4. Database against which the changes have been made will be pushed.
For folks unable to see the presentation, you can view the deployment here. Take a look at the development code here.

Implications

For developers

For PaaSs

For Drupal 8

Michelle Krejci

Palantir.net

Let's make something good together

Keep tabs on our work at @Palantir