Management of Uncertainty in Agile Projects

One of the biggest advantages of an Agile approach is the ability to manage higher levels of uncertainty. Agile is based on an empirical process approach – the word “empirical” means “based on experiment or observation”. When you use an empirical process approach, you accept that you don’t know everything you might want to know about a project before you start and the process is designed to adapt both the solution and the process to discover the solution to learning as the project progresses.

A “Defined Process” is repeatable and doesn’t change significantly from one project to the next and is very predictable in the results it produces while an “Empirical Process” is specifically intended to favor adaptability over predictability. For that reason, an empirical process is much better suited for a project with high levels of uncertainty.

A very powerful concept for understanding uncertainty is the “Stacey Complexity Model” which is shown below:

Stacey Diagram

There are two dimensions of uncertainty in this model:

  • One dimension is requirements uncertainty – How well understood are the goals and requirements for the project and how certain are the customers that they know what they want?
  • The other dimension is technology uncertainty – How well understood is the technology solution to the problem and what is the level of risk associated with the technology solution?

This is a very important concept because the ability to handle uncertainty is so important in today’s most critical projects and heavily plan-driven projects are not well-designed to deal with high levels of uncertainty. What typically happens in a plan-driven project is the project manager tries to reduce the level of uncertainty to an acceptable level before starting the project by:

  • Trying to resolve any uncertainties in the requirements as much as possible before the project starts, and
  • Trying to eliminate as much technology risk as possible

This often results in using tried-and-proven technology and doesn’t push the envelope too far in terms of going into areas of new and undefined user requirements. The downside of that, of course, is that the technology approach may wind up being obsolete within a relatively short amount of time after it is released and it may also result in a very mediocre user experience with the solution.

Let me clarify what I mean by “managing uncertainty”. To some people, uncertainty is like a cancer that attacks a project and can cause it to fail and conventional project management thinking has reinforced this approach to reduce the risk and uncertainty in a project as much as possible. I don’t see it that way uncertainty can also represent opportunity to go beyond what is expected and the value a project produces is many times directly related to the risk and uncertainty in the project. If you force a project to fit into a plan-driven model by reducing the risk and uncertainty, you may be maximizing the predictability of the project to meet cost and schedule goals but minimizing the value that the project produces.

That doesn’t mean that uncertainty should be ignored and not managed at all and fortunately, this is not a black-and-white, either/or decision between a totally rigid, plan-driven approach with almost zero uncertainty and a totally adaptive approach with an extremely high level of uncertainty. The right thing to do is to fit the approach to the level of uncertainty in the project rather than force-fitting a project to some kind of canned approach (whatever it might be). It takes more skill to develop an intelligent approach for managing uncertainty – it requires:

  • The ability to objectively assess the level of uncertainty in a project
  • An understanding of both empirical and defined process models
  • A deeper understanding of the principles behind these approaches to know how to blend the two together to fit the situation

The ability to do that is the essence of what I believe is an effective Agile Project Management approach.