Michael Gill, Agile Business Analyst, discusses the key tenets of the Agile Manifesto for software development and what they could mean for business.
It's nearly fifteen years since seventeen IT professionals met at a ski resort in Utah, to explore an alternative to documentation driven, heavyweight software development processes. What emerged was the Agile Manifesto espousing four values and twelve principles. Here we look at what this might mean for an organisation either seeking to transition to or already embarked on an agile methodology.
Before I do that, however, let's think about what's happened since the Manifesto's launch, at least in technology terms. The internet was still in its infancy. Some technology experts and organisations weren't convinced of its longevity. Mobile phones were pretty much just that, a mobile phone, and desktop computing was dominated by Windows software. Not even the much vaunted 'Windows XP' had been released. And the terms 'social media' and 'Google' hadn't been invented.
Since then there has been a revolution. More fundamental than perhaps the early years of the industrial revolution. The internet is part of the very fabric of all our lives. Smartphones, or more accurately, pocket computers are integral to the way we communicate and gather information. There are over 3.5 billion Google searches every day1, and to many the concept of a life without Facebook or Twitter is incomprehensible. New devices such as tablets or Kindles are part of our everyday fabric, and for many of us, Windows remains a distant memory as other players have established themselves in the market.
We live in a world where someone can wake up on a Monday morning with a great idea and weeks later millions of people across the world are running her software.
Yet despite this transformation, many organisations are stuck in the past as far as their development methodologies go. It's as if the last fifteen years never happened.
So what does the Agile Manifesto give us and what do the values actually mean?
Here's the opening statement along with the four core values in their entirety:
"We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more."
Let's explore how this might play out by examining each of the values in turn.
1. Individuals and interactions over processes and tools
For many organisations, this value can be one of the most challenging. The emphasis on people and communication over rigidly sticking to a process is an uncomfortable place to be.
By putting communications at the heart of everything you do, there is a greater likelihood of solutions being found to problems, issues aired and confronted, conflicts resolved and consequently forward momentum.
Where adherence to a process dominates, on the other hand, issues and conflicts are more likely to be hidden and glossed over. More emphasis is placed on conformity rather than achieving the end goal of building software. 'We all know our place' might be another way of putting it.
Many financial institutions, for example, have a very hierarchical, even feudal management structure. This is one reason why these sorts of organisations struggle to adopt and benefit from agile.
Emphasis is placed on the process, so we change from one form of box ticking in waterfall to another form in agile without ever truly understanding the philosophical shift necessary to realise that structure and hierarchy just aren't that relevant anymore.
The truly self-organising team can be a real challenge to the status quo. The idea that a group of individuals plan and innovate outside what is the norm can be seen as a threat. And many times the diktat will be 'you can be agile, but only so far.'
Adopting agile takes courage, honesty and a willingness to break with the past. The emphasis on people and communications is vital to making this a success.
2. Working software over comprehensive documentation
Hand in hand with an emphasis on the process is the production of documentation. After all, that's how we know the process is working, isn't it?
What we need to concentrate on, however, is the reason we're developing in the first place - working software.
I was working on a regulatory project some years back and we were legally obliged to deliver to a fixed deadline. Despite this, the project was starved of resources and instead the focus was on a bright shiny new system for another aspect of the business. Two years in, they were due to go live and informed the new incoming head of IT that there was a problem with the new system. 'Is it a problem in testing?' he naively asked, 'no,' came the reply 'we're still in analysis.' But at least there was lots of documentation.
Ask the question what does the documentation contribute to the goal of producing working software? Does it provide intrinsic value? Is anyone going to read it? If so, who and why?
Other factors to consider are the location of the team(s). Co-located teams with good communication will negate the need for reams of documents. Multiple teams in different locations and possibly time zones might require a degree of more formal documentation though this can still be done with a light touch.
As for the working software itself, this is production quality code, able to be released when it makes sense to do so. Make this the focus and eliminate unnecessary and costly documentation cottage industries.
3. Customer collaboration over contract negotiation
You would think that collaboration as a concept would be a no-brainer, after all without it, pretty much most things involving two or more people won't have much chance of success. The central aspect of this value though is in the involvement of the business.
Embedded, involved stakeholders are pivotal to the success of an agile project. Able to make quick decisions, constant involvement in re-planning, (re)prioritisation, guiding the team as they witness the growth of the (working) software week by week. Sometimes it's uncomfortable for them to be put on the spot, but rather a poor decision than no decision at all.
Their involvement in ceremonies such as the 'stand up', and 'retrospective' fosters a greater degree of honesty. Helping to write or writing acceptance criteria, provides them with a real stake in the development and an investment in the project's success.
Of course collaboration extends to all the team members, some used to handing over documentation down the lines to other disciplines (contract negotiation), can struggle initially with regular interaction, but as they witness the organic growth of the system, they wonder how they ever did things differently.
If you have to deal with third parties, then things can get a little trickier. Almost invariably you will be saddled with contract negotiation, and many companies make money on the number of change requests raised.
If you can, you need to negotiate contracts where you can buy the third party's time for use as you see fit, harnessing them into your sprints for example or if they don't work in an agile manner letting them develop up front with incremental deliveries. This is a subject in its own right, but when negotiating with third parties ensure you have flexibility so you are not constrained by prehistoric working practices and philosophies.
4. Responding to change over following a plan
Eisenhower once said, "In preparing for battle I have always found that plans are useless, but planning is indispensable." Who am I to argue with the great General? A detailed Gantt chart is a complete waste of time and money.
I used to love creating long, complex project plans. Structuring detailed lists of tasks, assigning resources, creating weekly time-sheets, capturing actuals, it was like producing a work of art. It took a degree of courage for me to realise it had pretty much zero value.
The reason? As soon as the plan was 'finished' it was out of date. Often, even before it had been distributed to the willing or unwilling participants. And keeping it up to date (yes I was one of the few statistically, who updated their plans) was a huge maintenance overhead.
And of course, I've been on the receiving end of so many unrealistic plans. I remember one time when regardless of progress, the plan end date always remained the same. Any slippage was accounted for by shaving off time from subsequent 'not started' tasks, regardless of the effort necessary to complete them. Problems only surfaced when there was no time left to shave, we were reaching the end date but had lots more still to do. But up until then it was a success, management was happy as things were 'going to plan'.
Unfortunately this is the problem. Plans become a method of appeasing layers of management, never reflect the reality on the ground and everyone's left scratching their heads when nothing's delivered.
So my advice, forget your Gantt charts, with the exception of a simple, easy to read milestone plan, that's about all they're fit for.
Plans fail because of no control over scope and change. Traditionally waterfall has struggled with the very concept of change, after all, the requirements need to be locked down early so that everyone knows what they're building. Change has to be negotiated through a typically convoluted change management or change request process. And there is often fierce resistance to any change to the plan (the plan that doesn't reflect reality), as that will jeopardise the delivery.
Agile, on the other hand, recognises that change is inevitable and accommodates it. It's part of the regular planning activity (note the word 'planning', not 'plan', Eisenhower would be proud). Change is put entirely in the hands of the business where it should be. Remove some piece of equivalently sized scope to accommodate the change, or accept a longer delivery time. Nothing new or revelatory here, but it leads to tight control over scope, and that control leads to working software within desired time frames.
What can Agile do for my business?
Keeping focussed on the left side of the values provides the opportunity to transform the way an organisation works and delivers software, leading to genuine change, and change for the better.
And it hasn't stopped at software development. The Manifesto has evolved into a philosophy for business management in general, with executives latching on to agile practices such as its heavy emphasis on flexibility and collaboration. Which, when done properly, can be transformational for any organisation.
Similarly, other business disciplines are taking the spirit of agile and creating their own models and values, again derived from the original Manifesto, for example ‘agile marketing', 'agile UX' and so the list keeps growing...
You can find the full manifesto text, values and principles at agilemanifesto.org. I'll take a look at the twelve principles in a future post.
1 http://www.internetlivestats.com/google-search-statistics/
Before I do that, however, let's think about what's happened since the Manifesto's launch, at least in technology terms. The internet was still in its infancy. Some technology experts and organisations weren't convinced of its longevity. Mobile phones were pretty much just that, a mobile phone, and desktop computing was dominated by Windows software. Not even the much vaunted 'Windows XP' had been released. And the terms 'social media' and 'Google' hadn't been invented.
Since then there has been a revolution. More fundamental than perhaps the early years of the industrial revolution. The internet is part of the very fabric of all our lives. Smartphones, or more accurately, pocket computers are integral to the way we communicate and gather information. There are over 3.5 billion Google searches every day1, and to many the concept of a life without Facebook or Twitter is incomprehensible. New devices such as tablets or Kindles are part of our everyday fabric, and for many of us, Windows remains a distant memory as other players have established themselves in the market.
We live in a world where someone can wake up on a Monday morning with a great idea and weeks later millions of people across the world are running her software.
Yet despite this transformation, many organisations are stuck in the past as far as their development methodologies go. It's as if the last fifteen years never happened.
So what does the Agile Manifesto give us and what do the values actually mean?
Here's the opening statement along with the four core values in their entirety:
"We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more."
Let's explore how this might play out by examining each of the values in turn.
1. Individuals and interactions over processes and tools
For many organisations, this value can be one of the most challenging. The emphasis on people and communication over rigidly sticking to a process is an uncomfortable place to be.
By putting communications at the heart of everything you do, there is a greater likelihood of solutions being found to problems, issues aired and confronted, conflicts resolved and consequently forward momentum.
Where adherence to a process dominates, on the other hand, issues and conflicts are more likely to be hidden and glossed over. More emphasis is placed on conformity rather than achieving the end goal of building software. 'We all know our place' might be another way of putting it.
Many financial institutions, for example, have a very hierarchical, even feudal management structure. This is one reason why these sorts of organisations struggle to adopt and benefit from agile.
Emphasis is placed on the process, so we change from one form of box ticking in waterfall to another form in agile without ever truly understanding the philosophical shift necessary to realise that structure and hierarchy just aren't that relevant anymore.
The truly self-organising team can be a real challenge to the status quo. The idea that a group of individuals plan and innovate outside what is the norm can be seen as a threat. And many times the diktat will be 'you can be agile, but only so far.'
Adopting agile takes courage, honesty and a willingness to break with the past. The emphasis on people and communications is vital to making this a success.
2. Working software over comprehensive documentation
Hand in hand with an emphasis on the process is the production of documentation. After all, that's how we know the process is working, isn't it?
What we need to concentrate on, however, is the reason we're developing in the first place - working software.
I was working on a regulatory project some years back and we were legally obliged to deliver to a fixed deadline. Despite this, the project was starved of resources and instead the focus was on a bright shiny new system for another aspect of the business. Two years in, they were due to go live and informed the new incoming head of IT that there was a problem with the new system. 'Is it a problem in testing?' he naively asked, 'no,' came the reply 'we're still in analysis.' But at least there was lots of documentation.
Ask the question what does the documentation contribute to the goal of producing working software? Does it provide intrinsic value? Is anyone going to read it? If so, who and why?
Other factors to consider are the location of the team(s). Co-located teams with good communication will negate the need for reams of documents. Multiple teams in different locations and possibly time zones might require a degree of more formal documentation though this can still be done with a light touch.
As for the working software itself, this is production quality code, able to be released when it makes sense to do so. Make this the focus and eliminate unnecessary and costly documentation cottage industries.
3. Customer collaboration over contract negotiation
You would think that collaboration as a concept would be a no-brainer, after all without it, pretty much most things involving two or more people won't have much chance of success. The central aspect of this value though is in the involvement of the business.
Embedded, involved stakeholders are pivotal to the success of an agile project. Able to make quick decisions, constant involvement in re-planning, (re)prioritisation, guiding the team as they witness the growth of the (working) software week by week. Sometimes it's uncomfortable for them to be put on the spot, but rather a poor decision than no decision at all.
Their involvement in ceremonies such as the 'stand up', and 'retrospective' fosters a greater degree of honesty. Helping to write or writing acceptance criteria, provides them with a real stake in the development and an investment in the project's success.
Of course collaboration extends to all the team members, some used to handing over documentation down the lines to other disciplines (contract negotiation), can struggle initially with regular interaction, but as they witness the organic growth of the system, they wonder how they ever did things differently.
If you have to deal with third parties, then things can get a little trickier. Almost invariably you will be saddled with contract negotiation, and many companies make money on the number of change requests raised.
If you can, you need to negotiate contracts where you can buy the third party's time for use as you see fit, harnessing them into your sprints for example or if they don't work in an agile manner letting them develop up front with incremental deliveries. This is a subject in its own right, but when negotiating with third parties ensure you have flexibility so you are not constrained by prehistoric working practices and philosophies.
4. Responding to change over following a plan
Eisenhower once said, "In preparing for battle I have always found that plans are useless, but planning is indispensable." Who am I to argue with the great General? A detailed Gantt chart is a complete waste of time and money.
I used to love creating long, complex project plans. Structuring detailed lists of tasks, assigning resources, creating weekly time-sheets, capturing actuals, it was like producing a work of art. It took a degree of courage for me to realise it had pretty much zero value.
The reason? As soon as the plan was 'finished' it was out of date. Often, even before it had been distributed to the willing or unwilling participants. And keeping it up to date (yes I was one of the few statistically, who updated their plans) was a huge maintenance overhead.
And of course, I've been on the receiving end of so many unrealistic plans. I remember one time when regardless of progress, the plan end date always remained the same. Any slippage was accounted for by shaving off time from subsequent 'not started' tasks, regardless of the effort necessary to complete them. Problems only surfaced when there was no time left to shave, we were reaching the end date but had lots more still to do. But up until then it was a success, management was happy as things were 'going to plan'.
Unfortunately this is the problem. Plans become a method of appeasing layers of management, never reflect the reality on the ground and everyone's left scratching their heads when nothing's delivered.
So my advice, forget your Gantt charts, with the exception of a simple, easy to read milestone plan, that's about all they're fit for.
Plans fail because of no control over scope and change. Traditionally waterfall has struggled with the very concept of change, after all, the requirements need to be locked down early so that everyone knows what they're building. Change has to be negotiated through a typically convoluted change management or change request process. And there is often fierce resistance to any change to the plan (the plan that doesn't reflect reality), as that will jeopardise the delivery.
Agile, on the other hand, recognises that change is inevitable and accommodates it. It's part of the regular planning activity (note the word 'planning', not 'plan', Eisenhower would be proud). Change is put entirely in the hands of the business where it should be. Remove some piece of equivalently sized scope to accommodate the change, or accept a longer delivery time. Nothing new or revelatory here, but it leads to tight control over scope, and that control leads to working software within desired time frames.
What can Agile do for my business?
Keeping focussed on the left side of the values provides the opportunity to transform the way an organisation works and delivers software, leading to genuine change, and change for the better.
And it hasn't stopped at software development. The Manifesto has evolved into a philosophy for business management in general, with executives latching on to agile practices such as its heavy emphasis on flexibility and collaboration. Which, when done properly, can be transformational for any organisation.
Similarly, other business disciplines are taking the spirit of agile and creating their own models and values, again derived from the original Manifesto, for example ‘agile marketing', 'agile UX' and so the list keeps growing...
You can find the full manifesto text, values and principles at agilemanifesto.org. I'll take a look at the twelve principles in a future post.
1 http://www.internetlivestats.com/google-search-statistics/
