When should software teams set up deployment pipelines?

Stop me if you’ve heard this one before.

Enterprise Software Team gets new project. Team rushes into requirements analysis and development. Team gets development environments set up, and team gets closer to deadlines. With the deadline looming, the team realizes they don’t have a deployment pipeline set up — Deploying to a shared development environment was always ‘good enough’.

In a recent example a team was modernizing a system through a rewrite (things you should never do), one of the new team members asked, “How do we deploy this system?”

The answer was that they manually copied files to the shared development environment, and manually ran database scripts for updates, and that there was no pipeline set up to deploy to any test, staging, or eventual production environment.

When that team finally got their application to a staging or production environment, they realized they still had weeks worth of work to do to be able to deploy.

If you’re a software developer or former software developer, that line should shock you.

But, if you’ve worked on an enterprise team, or an R & D software team, that may feel normal.

In reality, not having a deployment pipeline set up first of the riskiest parts of any enterprise project.

In Enterprise software projects (moreso than in product development) deadlines are driven by executives that have an older-school mindset about work. You set a deadline, and work is done by that date and the project is delivered. It doesn’t help that waterfall is easy to understand and therefore easy to gravitate towards.

Software isn’t a process that lends itself to deadlines well. Unless the team is doing the exact work, in the exact environment, in the exact political structure they were working before, any estimate they give is as good as a guess, and about as accurate as throwing a dart against the board.

Take this example: Hangfire (a technology for running automated tasks) has a particular version that has a bug against a particular version of SQL Server, and in certain environments (like an enterprise environment) it causes unintended behavior (in this case, a proliferation of xp_userlocks). No one could plan for that, even if they’ve used Hangfire before. No one could have planned that the corporate IT team would strike the use of hangfire, causing the development team to scramble to rearchitect their solution to make up for this.

But, you know what would have caught that error sooner? The deployment pipeline the team didn’t have.

Enterprise environments (and organizations growing towards a more rigid governance model) have these separation of environments to ensure their data and business remains secure. Software Development is necessarily new every time a project is created. The software stacks change, the third party libraries change, and the practices change. As I write this, Angular (a common client side web application framework) has iterated on 6 major versions (Version 2-8) in the last three years alone. They release a new major version every 6 months!

Because things are changing so fast and because data security is important, it’s critical that enterprise software teams set up their deployment pipeline first. Without integrating their work into the IT Governance infrastructure, there’s no way to tell what risks will come to fruition without it.

In the beginning it may seem like a waste; but it is a bit like insurance: You’re paying early so that when the deadline gets closer, you have less risk for delivery, and due to the way that software works, that decision will pay for itself many times over.