Elegant Software Delivery Pipelines: Practice
How frequently and easily can you take developers code from commit to production? If the answer is more than an hour, then your company is probably spending orders of magnitude more per feature delivered.
There are a lot of things that you can do to make a difference, but one of the most fundamental is in automating how software is built and deployed.
This is the second part of a series detailing theory and practice of software delivery pipelines.
Everything that is described in this series is intended to help you think about driving improvements in reliability and predictability. The task of actually writing software can’t be automated, but the process of delivering software is like a manufacturing production line, and the mechanics and operation of production lines is something that is well understood.
In this article, I’ll talk about developer workflow and the practical mechanics of how feature delivery can be rapidly accelerated. Your software developers will thank you for it. I’ll also talk about the role of Operations in this new world. The operations team will thank you as well.
One Quick Thing, Though…
A long time ago, I said that the journey starts with the tooling. That’s absolutely right in the context of cultural change, but it’s also over-simplifying.
It’s really tempting to think that this tool or that is cool, and you absolutely have to use it. Maybe you think that way about everything-on-containers or a particular cloud provider.
But you have to resist the temptation! Just like you can’t design a house by first picking the brand of toilet seat or refrigerator, you also can’t design continuous delivery by picking a specific tool.
It’s really vital to think about the needs of the developers and the organization, about workflow. About the story. Figure out what you need before you decide how to do it.
Also, think about who is responsible for this change: asking traditional operations folks to deliver will almost certainly give you the wrong answer: this is a software development lifecycle matter, not a new flavor of operations.
Overview
Okay, so what do we need?
The pattern starts with a developer committing code, and that code (eventually) running in production.
This process is broken into two distinct phases: Continuous Integration (“CI”) which is where source code is turned into something that is ready to deploy, and the actual deployment phase which is called Continuous Deployment (“CD”). Together, these two are known as “CI/CD”. They are segregated by somewhere to put the stuff (called the artifact) that was created during continuous integration phase. This keeps things nice and clean.
The process is triggered when a developer commits upstream to the git server. Generally from there, the process will be automated, with usually some manual intervention along the way to delivery to production.
But there’s a fourth step that’s often missed: we also have to think about how things get initialized (such as the code repository and the automated unattended provisioning of infrastructure and supporting service).
Together, these things should be incredibly simple to run and require minimal cognitive load. Otherwise, you won’t have a compelling solution, and compelling solutions are vital if you want anyone to actually use this stuff.
Think About Outcomes
When all of this is done well, all of the silos between developers and production resources have been eliminated. There are no more silos to navigate in terms of deploying infrastructure, production code, or security and compliance checks.
“But, but… that’s reckless! It will never work here!”
If this is your initial response (and trust me, it’s not uncommon), then please start thinking about how you can make it work rather than the assumptions that you’re using that are driving your initial response. Fear and business-as-usual thinking have no place in this changing world.
So what will the Operations Team do, given that most operations are now automated or silos have been removed and teams are self managing in a super scalable way?
Software Rot is a very real thing, and it affects everything we do. Even beyond treading water, continuous improvement to enable faster and faster software delivery is essential. So there is always room for the operational folks who herd cats all day to become SREs or DevOps Delivery engineers driving continuous improvement in how the deployment tools, infrastructure, and applications perform.
There’s also a likelihood that the operations team knows more about running infrastructure than anyone else: coaching will be vital.
Certainly, it’s a better full time job than performing work that would better be handled through automation, and — bluntly — those folks who are happy with the status quo and don’t want things to be better are only hurting you.
Closing
So now you have the high level:
- think about outcomes rather than focus on specific implementation, and those outcomes must be developer-centric
- the process is divided into three “regular” phases of continuous integration, artifact management, and continuous deployment, plus one additional “getting stuff up and running” provisioning phase.
Next up is a deeper dive on those phases. Plus almost certainly there will be some other stuff to talk about.
