The History of Waterfall
Whenever I teach classes, I almost always talk about the history of waterfall. It’s one of the least-understood elements of corporate process: we follow a process that dictates how we do work, but hardly anyone knows what was actually originally proposed. So how did we get the process that we have today? And how useful is that process today? Let’s hit the WABAC (way-back) machine and see what life was like in 1970. Take it away, Mr. Peabody…
The Age of Aquarius
In the late 1960s, Dr Winston Royce was a Director of Engineering at TRW and trying to find ways to cut costs on his projects and make them more predictable. He was working on space-flight systems and his job – as he stated in his paper – was to try to deliver projects “at an operational state, on-time, and within costs”1. In his paper he described what we know today as the “waterfall” process:
This is what we’ve been used to in software development for about thirty-five years. It’s also the basis for a lot of stage-gate lifecycle and financial processes, many of which are used to support continued project funding. If a project fails to meet the requirements at any gate, we can stop funding that effort. This is also the second diagram in Royce’s paper (the first being a simple diagram showing Analysis and Coding boxes only – a process to deliver small computer programs).
We also need to think about what the computing world of 1970 like. People tend to forget what it was like to write software.
The Dawn of Waterfall
So what did computing look like in the 1960s and 1970s? If you raised your hand and said “OO! OO! OO! Mistah Kott-air!” you probably remember that it involved magnetic tapes, Hollerith (punch) cards, and, possibly, custom-built circuitry. Computing was difficult and it could be both expensive and time-consuming writing code and testing code. When we look at the different stages of the Waterfall cycle, the most money was being spent in the coding/test cycles, so that was a great place to look at cost savings. And how can we reduce our costs in those areas? Do most of our work in the upfront stages to make sure that we’re building things once and only once (or as close to once as possible). It makes a lot of sense. But it doesn’t actually work…
What Was Actually Proposed
The most interesting thing I find about that second diagram (the one above) that Royce actually advised against thinking that way. The statement immediately below the diagram reads:
I believe in this concept, but the implementation described above is risky and invites failure.
- Dr. Winston Royce, 1971
His actual proposal was far more complex and contained numerous feedback loops from one state to previous states. In other words, it was iterative in nature and he stated that we should do each step in the process at least twice. And the fifth step in the process was to involve the customer. If you’ve looked at the Agile Manifesto, that should sound vaguely familiar.
The Fall of Waterfall
The general problem with a methodology like Waterfall is that it expects us to have more foreknowledge than what we can learn as we work. That means we’re going to spend time trying to think of solutions to all of our possible future problems today rather than try to figure out solutions to problems we actually have when they crop up. It also means that, in general, we aren’t delivering anything until after our process is complete. On my typical projects, we had almost twelve months to do our work. Meaning that we wouldn’t even be starting coding until month four or five. In an Agile space, though, we work to complete work in short timeboxes that allow us to give something to our customers faster and get their feedback to help direct our future efforts. By the time we deliver something in a Waterfall project, we may be way off the mark. Plus we have to deal with all that “scope creep” that shows up because the customer changes their mind or the market conditions change. So the value we deliver may actually be less than what we envisioned because the optimal solution has changed from when we first envisioned it.
Waterfall isn’t a bad process – there are times and projects where it’s a very good methodology, forcing us to consider possible scenarios and prepare for those in advance. But for run-of-the-mill software (and most, if not all, cyber-physical projects) that we have in the 21st century, using Waterfall can be like a millstone around our necks, tying us into suboptimal solutions and designs and delaying our delivery until it’s potentially too late. And remember – it was never intended to be used the way we use it at all. The “Father of Waterfall” was anything but that.
Notes
- Royce, Dr Winston W. Managing the Development of Large Software Systems. 1970, IEEE / Wescon.