The Evolution of Waterfall to Agile & DevOps
DevOps has undergone quite the transformation in the 2010s. But the most dominant change has been a departure from waterfall and the massive adoption of Agile as teams strive toward full DevOps.
Keep reading to learn about the fundamental differences of waterfall vs. Agile and DevOps, and how these methodologies have evolved to meet today’s DevOps demands.
Agile vs. Waterfall
Waterfall is a development method that is linear and sequential and once dominated software teams. While waterfall is structured, Agile is flexible. The Agile method is based off continuous iterative sprints which are more conducive to change than waterfall’s rigid process. Teams today favor the Agile method over waterfall.
Transitioning From Waterfall to Agile & DevOps
At the start of this decade, the majority of organizations explored ways to enhance software delivery productivity, learning about Agile, Scrum, and related methodologies. But in reality, they were implementing the traditional waterfall software development methodology.
With waterfall methods, organizations moved slowly with strict dependencies between software development lifecycle phases and releasing products in a very long iteration (months long).
Post 2013 saw a breakthrough in terms of maturity and awareness of Agile.
Agile started evolving as a way for organizations to deliver value to customers faster, with greater quality, better predictability, and better control over changes in requirements. As deemed by the Agile Manifesto, the idea was to remove obstacles and barriers from the developers and program managers as they develop software to release it faster.
Effects of the Agile Transition
As a result of the Agile transformation, the CI/CD pipeline and DevOps started invading organizations. This wave carried with it a few key technologies that have since become standard — including cloud computing and Microservices.
Organizations understood that in order to implement Agile and DevOps, it was necessary to address cross-team collaboration and break software requirements down into smaller chunks. They also needed to adopt tools and re-design software architecture to make it adaptable to change and easier to ensure quality.
Today, 17% of legacy organizations are stuck in the old-fashioned method of waterfall software development. The majority are already in an Agile/DevOps way of working.
In order for these changes to scale and continuously keep up with the evolution of market technologies (mobile, web, others), they require even smarter tools and processes. This is where we’ll see more AI/ML algorithms and tools being implemented and adopted to support such growth.
To summarize the evolution over the past decade, we can investigate the following visuals.
As we can see from the three software development methods, there’s a clear transition. More and more requirements have been added to the plates of developers.
With the adoption of “shift left” and “shift right,” developers and testers test earlier in the development cycle and share the responsibility for software quality.
The above is very good in theory and it works better in some organizations than others. But as a market average at the end of 2019, the level of automation within the entire DevOps pipeline activities is still low, error-prone, flaky, and not always efficient. Teams still struggle with automating key tasks that include code reviews, security audits, testing, environment setup, deployment, and more.
Other DevOps Practices of the 2010s
In the context of Agile and DevOps it’s also important to bring up a few key terms that became popular over the past decade and also became integral part of the engineering team tool chain: source control management tools, CI servers, pipelines, and Value Stream Maps.
While each of these are used in different stages of the development cycle, they are all critical to the success of teams. If there’s no well-structured and gated way to manage code branches and accept changes between feature teams and individuals prior to integrations, the entire project will become a mess.
If teams cannot identify software waste through a high level VSM (value stream map) and see where there are the bottlenecks and pains, the overall activity will fail. Same goes for the continuous integration (CI) and pipeline management aspects.
These are the main vehicles for software development and when they have hiccups and instabilities, that’s when trouble begins. We see too many broken software builds, testing that fails from the wrong reasons within CI, failures to properly merge and integrate changes within large pipelines, and more.
In 2020 and beyond, the goal for DevOps organizations will be to enhance automation within the DevOps chain. They will also seek to reduce flakiness, ensure flawless pipelines, and make sense out of the key production and pre-production data to continuously improve for the future.