Traditional waterfall methods for developing software are rapidly declining in popularity as more recently developed Agile methodologies are increasingly adopted.
What is waterfall?
The waterfall model is one in which each phase of a product’s lifecycle takes place in sequence, so that progress flows steadily downwards through these phases like a waterfall.
Nobody invented the waterfall method. Rather it was inherited by enterprise software developers from other industries where once a particular phase of production is complete (like laying the foundations of a building, for example), it was incredibly costly or impractical to go back and make changes. The waterfall was only codified when people subsequently realised that it wasn’t the only way of doing things. (Winston W. Royce is commonly credited with the first formal description in an article from 1970 in which he described a flawed software development model.)
In waterfall methodologies, all the requirements gathering and design work are done before any coding takes place.
There are several well-known and widely implemented waterfall methodologies that are used on IT projects. These include PRINCE2 which was created by the UK government and remains popular in the UK public sector and PMI PMP which is more internationally recognised.
In general, these methodologies have stages that deal with what you need to do before a project, during a start-up phase, a planning phase, an execution phase and a closing phase. They also then have a series of processes for managing work packages, exceptions, reporting, risks and issues.
Pros of the waterfall method
Potential issues that would have been found during development can be researched and bottomed out during the design phase. If appropriate meaning an alternate solution is selected before any code is written.
The development process tends to be better documented since this methodology places greater emphasis on documentation like requirements and design docs. Many organisations find this reassuring.
Because the waterfall process is a linear one it is perhaps easier to understand, especially for non-developers or those new to software development. Often teams feel more comfortable with this approach.
Cons of the waterfall method
Often the people we’re building software for (the client) don’t know exactly what they need upfront and don’t know what’s possible with the technology available. This way of working doesn’t handle this well.
Solution designers often aren’t able to foresee problems that will arise out of the implementation of their designs.
Changes to requirements (for example, those resulting from new technologies, changes in a market or changes to business goals) can’t easily be incorporated with the waterfall method and there are often laborious change control procedures to go through when this happens
The process doesn’t have its own momentum
What is agile?
An Agile software development methodology – such as Scrum – is one which eschews a linear, sequential approach in favour of an incremental, iterative one.
Instead of extensive planning and design up front, Agile methodologies allow for changing requirements over time by using cross-functional teams – incorporating planners, designers, developers and testers – which work on successive iterations of the product over fixed time periods (timeboxes). The work is organised into a backlog that is prioritised in exact priority order based on business (or user) value.
These teams are self-organising and include a representative of the business (the product owner). The emphasis is on efficient face-to-face communication and short feedback loops.
The goal of each iteration is to produce a working product, which can be demonstrated to stakeholders. Feedback can then be incorporated into the next or future iterations.
Agile evolved out of a number of different lightweight software philosophies which developed in the 1990s in counterpoint to heavyweight methodologies like waterfall.
Pros of Agile methods
Working software is delivered much more quickly and successive iterations can be delivered frequently, at a consistent pace.
There is closer collaboration between developers and the business.
Changes to requirements can be incorporated at any point of the process – even late in development.
It gives the opportunity for continuous improvement of live systems
It's highly transparent
Cons of Agile methods
Agile methodologies (e.g. Scrum, XP, Kanban, Crystal etc) are often more difficult to understand than linear, sequential ones – at least initially.
Because of the emphasis on working software, there can be a perception that documentation can sometimes be neglected. The focus should be on appropriate documentation for the audience that needs it but, if not implemented well, this isn’t always the case.
When implemented badly Agile can introduce extra inefficiencies in large organisations or can be working against long-standing organisational processes.
When is Agile better than waterfall?
I’m a huge proponent of Agile software methodologies. I believe that technology, businesses and markets change so fast these days that software development needs to be adaptable above all other qualities.
Agile methods are more flexible than the waterfall method which means that customers’ requests are more likely to be met. Even when things change the constant focus on value means what you’ve already delivered should have been worthwhile. And the rhythm that Scrum creates helps build highly motivated teams where productivity increases over time.
Having said all that, there are still circumstances in which the waterfall method can be suitable – for example, where requirements are guaranteed to be unchanging and there is very little uncertainty or if the project is very simple – but those circumstances are becoming fewer and farther between.
Also, if an organisation and the people involved in the project are not in a mature enough state for Agile, it may be more appropriate to use traditional project management methods.
What are your thoughts? Have you tried using Agile – how was it received in your organisation and what challenges did you face?
Our recent insights
FAIR data - what is it and why should you care?
One of our senior data consultants, Dr Alasdair Gray, explains what FAIR data is, who’s using it, why it’s so useful and some common misconceptions around it.
Big Data or Machine Learning — What's best for your organisation?
What's the difference between big data and machine learning, and which should you be using to satisfy your business requirements?
How to get started with low code
When should you consider low code solutions, what are the benefits, and how to begin?