Agile Software Development: What It Is and How the Cloud Can Help
More and more businesses—big and small—are embracing Agile as a means to innovate quickly, streamline software development and stay ahead of the competition.
But with so much information available, it’s hard to know where to start.
In this blog, we’re providing a breakdown of Agile software development: what it is, why it works, key practices and how it relates to DevOps and the cloud.
What Is Agile Software Development?
Agile software development is guided by the principles in the Agile software development manifesto, where teams collaborate to uncover better ways of developing software by doing it and helping others do it.
History has shown that we don’t know what we are doing when it comes to software development. Either we start with the wrong assumptions and do the wrong thing, or we are paralysed by fear of doing the wrong thing that we don’t get started at all.
I like to frame Agile software development as walking the path to the unknown.
The principles in the Agile manifesto are the guides to help us walk this path to the unknown, by collaborating and responding to change
Agile Vs Waterfall
The alternative—and more traditional—approach to software development adopts the waterfall method.
In Waterfall style development, we don’t do anything until we know everything and then assume what we have built is correct.
In contrast to Agile, waterfall style software development takes a linear approach—by spending time defining requirements upfront before building working software. The lag between gathering requirements and building software can be months, and by that time the requirements will have changed or the market will have moved on.
Accepting that we have doubt and uncertainty affects how we go about doing software development. This brings us to the nature of Agile software development.
The Essence of Agile
To walk the path of uncertainty, we need to see what we are doing from multiple perspectives. To do this, we must work collectively:
- We lean on the interactions we have with individuals and take advantage of the connected human.
- We focus on the real and tangible output of our working together (working software) so that we can see if we are doing the right thing.
- We collaborate and focus on the interactions we have with the individuals who will benefit from what we are building – our customers.
- We continuously evaluate our ability to change when we realise we are going in the wrong direction, and we keep continuously sensing our way.
Doing all of the above requires focussing on the nature of the interactions between us and exposing ourselves to the feedback loops which tell us if we are doing the right thing or not.
This is the essence of Agile software development.
The Definitive Guide to Cloud Migration in the Enterprise: Here Be Dragons!
The road to cloud migration is full of DRAGONS...
We explore the four dragons that are most likely to burn your transformation to the ground and the tools to slay them!
Agile Software Methodologies – Deconstructed
When you hear people refer to ‘Agile methodologies’, you’ll likely think of Scrum, Kanban and Lean.
However, rather than listing these methodologies (which can be a contentious topic), it’s much more valuable to consider what they should achieve and how.
The Agile methodology we choose has to help us do the following:
- Help the team building the software become better at their craft (be it engineering, design or testing)
- Help the team have healthier interactions with each other and with the customer
- Help the team create feedback loops so that they can course-correct
- Help the team understand the domain and the problem they are trying to solve
Any Agile or non-Agile development methodology that ignores the above—and focuses only on delivering features—delivers mediocre software, which frustrates the people who use it and causes a team to burn out.
Iteration and Reflection Cycle
There are two important methods that help teams to achieve the four goals listed above.
- Iteration: Iterating on what we have built helps the team reflect and get better at it. Iteration helps us get better at software development.
- Reflection: Between iteration, the team needs to step back and reflect on what they have built, on the process used to create it, and on the interactions the team had while they were working on it.
When we iterate and reflect, we learn from the difference between what we expected and what actually happened. The smaller the iteration steps, the smaller the difference is, so we are not overwhelmed with the amount we need to learn before we take the next step.
When walking the path of uncertainty, smaller steps are less risky.
This reflective iteration is similar to a Deming Cycle—otherwise known as the Deming Wheel or the PDSA Cycle (Plan-Do-Study-Act). This is a systematic process for gaining valuable learning and knowledge for the continual improvement of a product, process, or service.
Iterations
In Agile we have iterations–you can call them sprints. Shorter iterations are better.
At the start of the iteration, we plan for what we will attempt to do in the iteration. The iteration isn’t a deadline. It’s a time box to run an experiment to see whether we can achieve what we set out to do at a sustainable pace.
Retrospectives
At the end of the iteration, we look at if we achieved what we set out to do, and reflect on what stopped us from achieving what we set out to do?
- Is the team happy?
- Is the customer happy?
- Is the software working?
- What did we learn about the problem we are trying to solve?
- What did we learn about the problem domain?
- Have we improved our craft?
- What do we need to change in order to do better in the next iteration?
This reflection is what we call a retrospective, the process of reflecting on what we have learned, and taking the learning to the next iteration.
At its core, an Agile methodology is a system to help a group of people learn.
Backlog
Before we start an iteration we need to have an idea of what we need to do. We write down the list of things that we intend to do. This is called the backlog.
At the start, this list of things can be small, and that’s ok. As we continue our iterations and learn about the problem we are trying to solve, this list will grow and shrink. The backlog items start to become well defined and have more details, as we ask more questions.
User Stories
We capture the answers to the problems we are trying to solve in the form of user stories. User stories aren’t just answers, they are the on-going result of the team’s collaboration with the user. They tell the story of what the user wants to achieve, instead of what needs to be built. This helps the team achieve outcomes rather than building software that is not aligned to user needs.
Within the iteration and reflection cycle, there are two key practices that make Agile software development succeed, which we as an industry have learned: XP Practices and Continuous Delivery.
eXtreme Programming (XP) Practices
Extreme programming (XP) is a discipline of software development based on values of simplicity, communication, feedback, courage, and respect. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation.
Despite what the name suggests XP practices are not so extreme anymore. They are part of the muscle memory that Agile teams should have in order to succeed. XP practices help us improve our engineering skills, and make sure we are focussing our engineering efforts on the right thing.
Let’s look at a few of the core XP practices:
- TDD - Test Driven Development. Red, Green, Refactor. Write the test, see it fail, write the simplest code to make it pass, make it better
- Refactoring - Change the structure of the code without changing its external behaviour. Keeping the code amenable to change continuously
- Continuous Integration - Continuously integrate your code with everyone else's, and keep it deliverable
- Pair Programming - Two devs share the same keyboard. Ping pong style, driver navigator, mob programming. Pair programming is not watching someone else type.
- Sustainable Pace - Commit to what you can sustainably deliver, no overtime, no weekends.
- Collective Ownership - The whole team owns the process and the code
Continuous Delivery
If someone thinks of a good idea, how do we deliver it to users as quickly as possible?
Continuous delivery ensures your software is deployable throughout its lifecycle and when we prioritise keeping the software deployable over working on new features:
- Anybody can get fast, automated feedback on the production readiness of their systems any time somebody makes a change to them
- You can perform push-button deployments of any version of the software to any environment on demand
XP combined with Continuous Delivery creates the feedback loops needed to course correct, safely and quickly when we need to. Together, these practices help the team practise listening to the feedback loops that tell them if they are headed in the right direction.
How Does DevOps Relate to Agile?
In the past—before cloud and programmable infrastructure—Agile software development teams weren’t able to bring their skills to bear on the systems that would run their software.
Agile teams excluded the infrastructure teams that would ultimately run and operate what was built. Running the software was someone else’s problem.
DevOps enables Agile teams to build and deliver tangible working software that delights the customer.
It brings everyone who needs to build, run and operate the software together into one team. Agile and XP practices like Continuous Integration, Continuous Deployment, Test Driven Development are applied to infrastructure as well.
With cloud and programmable infrastructure, infrastructure can be treated like software. This means we can write code to build our infrastructure and pay for only what we use.
How Does the Cloud Help?
Agile software development thrives on fast feedback loops, and the most important feedback loop is from the user who uses the software.
The faster we can get working software into the hands of the user, the faster we can learn about what the user wants, and if we are solving the right problem.
Traditionally, infrastructure provisioning has been a bottleneck in the path to the user. Even when software was built and ready to be consumed, getting the infrastructure to build and run it needed to be purchased months in advance. Often the infrastructure wasn’t fit for purpose.
The public cloud removes this bottleneck. Agile teams now have access to on-demand infrastructure, within minutes, and pay only for what they use.
So what are the key benefits of the cloud when it comes to agile software development?
Reduces the cost of change
Agile software development thrives on fast feedback loops, and methods to reduce the cost of change. The public cloud reduces the cost of getting access to compute, and reduces the cost of changing our minds when we find that our user needs have changed. Agile teams are not constrained by infrastructure anymore. Changing infrastructure is a matter of changing a few lines of code, in contrast to changing contracts with multiple vendors in the on-prem world.
Infrastructure on-demand
Prior to the advent of the public cloud, even though teams could iterate faster, and change the software they were building, access to compute to run the software needed up-front fixed investment. We needed to plan up-front for the infrastructure we needed. This became a bottleneck and very un-agile. It also created a dependency on infrastructure teams who were outside the purview of the software development team.
The public cloud enables agile teams to request infrastructure on-demand. When the software changes, the infrastructure can change too. You don’t have to pay upfront; you pay just for what we use.
More time to focus on your customer
PaaS services take away the grunt work of building infrastructure for the most common scenarios. By using the public cloud, agile teams can focus on solving the problems for their customer, and be in control of how the service they are building runs.
More feedback loops
In addition, the public cloud provides more feedback loops. Teams have access to granular cost management, so that they can see how much it costs to solve the problem, and make informed decisions. Public cloud tells the team immediately if the infrastructure they need is available, without having to rely on an infrastructure team to tell them.
Why Do Agile Software Development?
How we build software needs to adapt, as we discover new knowledge about the problem we are trying to solve.
The key benefits that Agile software development provides is:
- A framework for us to change when we gain new knowledge.
- We can stop early. We can decide at each iteration if the customer is happy.
- We deliver working software in each iteration.
- We get to fail fast, instead of failing a year later with millions spent.
All of this means that software development teams are more empowered to deliver high quality products, faster. While business leaders benefit from greater transparency, flexibility and efficiency and less risk.
