Agile is Only a Tool – It’s Not a Religion

I was reviewing one of Dean Leffingwell’s slide presentations and I particularly like a comment he made that “Agile is only a tool – it’s not a religion”. It inspired me to write this blog post. (Thanks Dean!)

There is a lot of religious fervor about Agile and there’s a lot of polarization between proponents of Agile and more traditional plan-driven project management approaches. There’s a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically – becoming Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation.

Don Reinertsen has written a good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.

The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. In the real world, I don’t think very many people would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles, at least at a high level, is very valuable and it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized the eight principles here:

  1. Economics – Take an Economic View
  2. “Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”

    I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself and it reaches a point of diminishing returns at some point and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.

    Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because if they have a significant economic impact.

  3. Queues – Actively Manage Queues
  4. “Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”

    Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:

    • The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or
    • Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned

    But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).

  5. Variability – Understand and Exploit Variability
  6. “There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”

    Reducing variability will many times improve efficiency but that isn’t always the case. For example, Breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point further attempts to reduce variability do not have economic value.

    For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.

  7. Batch Size – Reduce Batch Size
  8. “Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”

    Examples of batch size inefficiencies include:

    • Project scope – taking on more in a single project than is truly necessary
    • Project Funding – The entire project is conceived and funded as a single large batch proposal
    • Requirements definition – the tendency to define 100% of the requirements before the project starts
  9. WIP Constraints – Apply WIP Constraints
  10. Use Work in Process (WIP) constraints to manage overall flow. For example,

    • Control the number of projects taken on at any one time to avoid over-saturating development resources
    • Use specialized resources wisely to maximize their impact on overall flow
  11. Cadence, Synchronization, and Flow Control – Control Flow Under Uncertainty: Cadence and Synchronization
  12. “Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”

    Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:

    • Concurrent development on multiple paths at the same time
    • Concurrent testing of multiple subsystems
  13. Fast Feedback – Get Feedback as Fast as Possible
  14. Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.

    Fast feedback combined with selecting appropriate measures of performance enables rapid learning and ongoing continuous improvement.

  15. Decentralized Control – Decentralize Control
  16. “Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”

    Source: Reinertsen, Don, The Principles of Product Development Flow, Celeritas Publishing, 2009

    I recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.

Learning to Live with Uncertainty with Agile

Effective management of uncertainty and risks is probably the single-most important factor in the success of any project (Agile or not). Prior to Agile, there may have been a false sense of security from thinking that you could completely eliminate or significantly reduce uncertainty by developing a very detailed requirements document for a project upfront and then managing changes against those requirements to control the scope and direction of the project as the project was in progress. That sort of approach is often taken in order to gain some level of predictability over the costs and schedule of a project; however, it turns out in many cases that a good deal of that feeling of security may be an illusion and it is very difficult to accurately predict all the requirements, costs, and schedules of a software development project.

With Agile, in many situations, the pendulum has swung almost completely in the opposite direction – a lot of people in the Agile community seem to feel that any attempt to try to reduce uncertainty upfront in order to estimate the costs and schedules of a project is totally futile because there might be a lot of uncertainty involved and the requirements are only going to change anyway. For that reason, there is a tendency to not do any significant amount of upfront planning at all. I think that’s an over-reaction and it can lead to significant problems.

Here’s an example of the impact that it can have…I was recently involved in helping to rescue an Agile project that had gone on for about two years with no end in sight. One of the basic problems was that the project had started without a sufficient level of upfront planning to assess the scope of the effort and it was just assumed that it could be completed in some reasonable amount of time by a single agile team. After about two years into the effort, we stepped back and did some re-planning and found that the scope of the effort was much larger than a single Agile team could accomplish in any reasonable amount of time. If more upfront planning had been done prior to the start of the project, perhaps that probably would have been discovered much earlier.

There are two primary problems I have often seen in this area:

  1. Analysis Paralysis – Delaying making commitments or delaying making any decision at all until the information required to make the decision has a very high level of certainty about it and all of the risks associated with a project are well-known and completely understood. This is often associated with the traditional, “Waterfall” style of project management. Unfortunately, this mode of operation has been the norm in some companies that are very control-oriented and insist on accurately knowing the cost and schedule of a project before making a significant commitment to it.
  2. Cavalier Approach – The opposite of that can be to take a very “cavalier” approach to not worrying about managing uncertainty and risk at all, just start doing a project with very little or no upfront planning, and assume that whatever uncertainty and risk is inherent in the project will be discovered and somehow work itself out as the project proceeds. This is the approach I’ve commonly seen on a number of Agile projects.

Neither of those approaches is ideal and each can lead to significant problems – a better approach is to make informed decisions as best you can based on the level of risk and uncertainty in the situation. Very few decisions in life are based on either 100% known or 100% unknown information and learning to deal with uncertainty is a skill that any good Project Manager learns over time. That skill is still very important in an Agile environment and shouldn’t be ignored. (In other words, don’t throw the baby out with the bath water when you convert from a traditional project management approach to a more Agile approach)

You often have to make informed decisions based on very uncertain information because waiting for the information to become more certain might significantly delay the project or might not make sense at all. The approach that seems to work best is to separate the “known” from the “unknown”, make some hypotheses or assumptions about the unknowns if necessary, and then evaluate the risks of going forward based on those assumptions or hypotheses. If you do decide to go forward, you shouldn’t forget that those assumptions were only assumptions and may need to be revalidated later on. (Many people often forget to do that) In a traditional project management environment, a risk register or something equivalent to that might be used for that purpose – that level of formality may or may not be necessary in an Agile environment, but the principle is nonetheless important.

There’s an art and a skill associated with making decisions in an uncertain environment. Anyone who has done any gambling where you have to bet on the outcome of an unknown event (like what card you’re going to be dealt next) will understand that. Here’s an example: most people are familiar with the game of “Blackjack” where the players play against the dealer and both try to win by getting the highest possible hand without going over 21. Suppose I have been dealt two cards and have a total of 16 points in my hand and the dealer is showing one card as a “10” with the second card face down. There are several very significant things I don’t know:

  1. I don’t know what the dealer’s second card is because it is turned face-down
  2. I don’t know what card I might be dealt next

Both of those are significant unknown’s but I have to make a decision anyway. I can make an assumption that the dealer’s second card is either a “10” or a face card (Jack, Queen, or King) for a total of 20 points. That assumption seems to make a lot of sense because there are more ways for that to happen (Ten, Jack, Queen, or King) than any other individual card. If I make that assumption, I know that my 16-point hand is going to lose against his 20-point hand and the best course of action would be for me to take a hit and risk going over 21 because I’m going to lose anyway if I don’t take a card or I don’t do anything at all.

That’s an example of an informed decision – I separated the known’s from the unknown’s and then made a reasonable assumption about the unkown’s based on the risks I saw in the situation. The alternative would have been to make no decision at all because the unknown’s were too significant or to make a “shoot from the hip”, uninformed decision to do something without doing any real analysis or really thinking it through at all and without taking advantage of the facts I do know in order to make a decision. I think that one of the most important factors for doing that effectively is to make an honest and objective assessment of the situation. There are several errors I have seen often with this:

  1. Overestimate the level of confidence in what is known and make a poor decision based on incomplete information
  2. Make an assumption about incomplete information, make a decision based on that assumption, press forward, and then forget that it was an assumption and never retest or revalidate it later
  3. Ignore whatever known information is available for making a decision and just proceed forward based on an uninformed decision without taking advantage of what is known

Is Agile Project Management an Oxymoron?

A few years ago, I attended a presentation by a well-known agilist who said that “Agile Project Management is an Oxymoron”. That kind of perception is based on a lot of stereotypes about Project Management that need to be changed. Here are some of those stereotypes that exist:

  1. Project managers are very command-and-control oriented
  2. Project managers are heavily associated with plan-driven “Waterfall” processes and document-centric methodologies where you need a plan for almost every aspect of any project and you also need a document, checklist, or a form for almost anything you do; and there is no role for a Project Manager in an Agile environment that requires a much more adaptive approach

There are certainly project managers who have some of those characteristics, but it’s unfair to make a judgment that all project managers are that way or to write-off the whole project management profession as being irrelevant to Agile because some project managers may be that way. However, we in the project management profession need to recognize the impact of Agile and recognize that a shift in thinking is needed to broaden our perspective on project management to correct these stereotypes that do exist. Here are some of those shifts in thinking that I believe need to take place.

  • Command-and-Control Orientation

Project managers are noted for getting things done and as a Project Manager, I’ve been in numerous situations where a customer has expected me to “ramrod” a project to get it done to meet time and budget constraints. That’s the way Project Managers have operated for many years and it naturally leads to a very assertive role where the Project Manager takes total responsibility for the success of the project and drives everyone on the project team to deliver results to achieve success. Both project managers and their companies need to recognize the limitations in that approach.

The Project Management profession needs to go through some changes that I saw the Quality Management profession go through a long time ago. In the early 1990’s I worked in the Quality Management profession. What I learned at that time was that it is self-defeating for a Quality Manager to take on too much ownership for quality. An effective Quality Manager has to be somewhat of a “missionary” and engage others in the effort to own responsibility for “quality” to be successful. If the only people who are responsible for “quality” are the people in the Quality Management department or the Quality Assurance department, it’s not going to be very effective.

The same thing also applies to project management. The project management responsibility for an Agile project should be shared by everyone on the team – even a developer has to be a little bit of a project manager to plan, organize, and manage the tasks that he/she is responsible for. Operating in this environment calls for more leadership than management – a leader sets the vision as well as goals and objectives to be accomplished; puts in place the people, process, and tools to get it done; and then steps out of the way as much as possible.

That approach is counter-cultural for many project managers because project managers are noted for getting things done and many companies expect that of their project managers. So it requires a shift in thinking in both project managers and the companies who manage them to bring about this change in orientation. There is also a natural resistance to this in some instances because if you do it successfully and really empower the teams you manage, you can eliminate the need for a project manager and put yourself out of a job and that might be the right thing to do – there is no defined role for a Project Manage in an Agile team that is working as it should be. The key thing is that it requires a change in thinking in project managers to let go of that role that they may have played in the past and move up to playing a higher-level role that provides more value-added. It’s ironic, that as I think back on some of the recent projects I’ve managed, if I’m asked what I did to make it successful, the answer is “I did as little as possible”.

  • Association with “Waterfall” Methodology

    Project managers are also heavily associated with plan-driven, document-centric approaches like the “Waterfall” methodology that can be extremely cumbersome and bureaucratic and there is no need for a project manager in an Agile environment. Unfortunately, I think this stereotype is somewhat legitimate and I know a number of project managers who are still holding on to a traditional PMBOK-style, plan-driven, document-centric, “Waterfall” approach to project management. I think there are a number of possible reasons for that:

    • Job Security – Many project managers have been heavily trained in traditional project management skills and PMBOK and it’s a big risk to give that up because that’s what their knowledge is based on and that’s the value-added that they have been trained to deliver. The role of an Agile Project Manager is also very undefined and uncertain at this point and might require a significant amount of retraining. It’s understandable that some project managers might not feel comfortable making that leap of faith to re-orient their career around Agile Project Management
    • Binary, All-or-nothing Thinking – A lot of people on both sides of the fence have the perception that it is a binary choice between a totally plan-driven and heavily controlled approach (e.g. “Waterfall” and a totally unplanned approach with no control (e.g. “Agile”) and that is not the case. I’ve previously written a couple of articles on that subject:

      Many project managers and many companies do not see that there is a lot of middle ground between extreme Waterfall and extreme Agile…those alternatives may not be well-defined and it can take more skill to figure out how to develop an approach that blends the right proportion of Agile and traditional project management principles and practices to fit the situation but it definitely can be done. Unfortunately, many people make the mistake of force-fitting their business and their projects to one of these extremes rather than fitting the methodology (or combination of methodologies) to their business and to each particular project.

What is the solution to this? This is basically a change management problem and I’ve learned over the years that there are three elements that are essential to any significant change management initiative:

  1. “Burning Platform” – There has to be a sufficient level of “pain” associated with the current situation to recognize that it is untenable and making a change is crucial. People have to get past the “denial” stage and recognize that Agile has a profound impact on the project management profession and in the not-too-distant future, project managers who only know how to do traditional project management PMBOK-style Project Management and don’t know how to take a more Agile and adaptive approach when it makes sense are going to be at a serious disadvantage in the job market.
  2. Vision for the Future – The next step in any successful change management initiative is that there has to be a vision for what the future looks like after the change has taken place. To do that, we need to better define what an Agile Project Manager is and what role he/she will play. I’ve tried to help develop that vision with the two books I’ve published but there’s still work to be done in that area to better define that vision. I’m currently working on developing a graduate-level course on Agile Project Management with a major university in the Boston area that I’m hoping will also help fill this need.
  3. Progress in the Right Direction – In any change management initiative, there will always be some hold-outs and laggards who will wait on the sidelines to see if the change is going to be successful or not before jumping on board. I’m sure that there are a number of project managers who don’t want to take the risk of becoming Agile Project Managers until that role is much more well-defined and proven and they will wait until more people have actually demonstrated success in that role.

That will obviously take time and isn’t going to happen overnight, but the first step is to recognize the problem. I hope this blog post helps others see this problem as I do. I welcome your comments and thoughts on this!

What is The Role of a PMO in an Agile/Lean Organization?

I recently saw a question on a LinkedIn discussion group asking:

What is role of a PMO in an Agile/Lean organization? 

There are many people in the Agile community who might say that there is no role for a PMO in an Agile/Lean environment and that the whole concept of a PMO is inconsistent with Agile.  That opinion is based on a stereotype that the role of the PMO is heavily associated with controlling and enforcing rigid waterfall-style policies for selecting and managing the execution of projects and programs.  In that kind of environment, a PMO might require:

  • Very thorough and detailed upfront planning to justify the ROI on projects to support rigorous project/product portfolio management decisions
  • Rigid control of project execution to ensure that projects meet their cost and schedule goals and deliver the expected ROI

There is no doubt that some PMO’s have played that kind of role to some extent in the past; however, it is a stereotype to believe that is the only possible role for a PMO to play.  A related stereotype is that many people also see the choice between Waterfall and Agile methodologies for projects as a binary decision – that stereotype is that an organization is either:

  • Totally “Waterfall” with extensive upfront project planning as well as rigid policies and controls over how projects are executed, or
  • Totally “Agile” with zero upfront planning and no policies and controls over project execution

Neither of those extremes is realistic in actual practice and you have to use good common sense to fit the right level of upfront planning and control to the project and business environment that the company operates in. 

Many people make the mistake of seeing this as a binary decision and force-fit projects to one of those two extremes when the right solution is to fit the methodology (or combination of methodologies) to the project and business environment.  It takes more skill to do that – it requires knowledge of a broader range of methodologies as well as a deeper understanding of the principles behind them to know how to blend traditional project management and Agile principles and practices together in the right proportions and customize them as necessary to fit a given situation.

I believe the role of any PMO is to align the selection and execution of projects and programs with the organization’s business goals which includes Project/Product Portfolio Management, providing oversight of project execution and the overall interface for management and reporting of projects and programs to senior management and the business, and finally to provide coordination, guidance, and training to project teams as needed in the organization’s methodologies and standards for project management. 

Those general functions probably don’t change in an Agile/Lean project environment, but how a PMO performs those functions may change significantly depending on the organization’s overall strategy for implementing an Agile transformation. 

  • Some organizations may choose to implement a relatively complete top-to-bottom Agile transformation for their business – Dean Leffingwell’s Scaled Agile Framework (SAFe) is an example of such a model. 
  • However, that can be a very ambitious and gut-wrenching change for many organizations. It also may not be the best solution.  I think it is a mistake to believe that you have to force a company to do an extensive, top-to-bottom Agile transformation in order to adopt an Agile process at the development level.

Organizations typically have different layers of management as shown in the diagram below; and, at each level, there is a choice of taking more of an Agile approach or more of a traditional, plan-driven approach:

Enterprise Agile 3

At the business management level, the approach should be designed around what makes the most sense for the company’s business and that may or may not be exactly the same as the approach used to manage projects at the development level. 

  • An Agile development process is easiest to apply in companies whose primary business is developing software products (Such as Intuit QuickBooks and TurboTax) or companies where software development has a very direct and significant leverage effect on the company’s business (Such as Amazon.com).  In those companies, there is a fairly direct alignment between a company’s overall business management goals and an Agile development process. 
  • In companies where that is not the case, the alignment may be less direct.  For example, it may or may not be totally realistic for a company to adopt a complete, top-to-bottom Agile approach for their entire business and a more traditional, plan-driven approach may be appropriate at the higher levels (at least as an interim solution).  However, that doesn’t preclude implementing a totally Agile or hybrid Agile development process.  The Managed Agile Development process is an example of a hybrid Agile approach that can be used in that kind of environment.

The role of the PMO should be aligned with supporting whatever that overall strategy is.  For example,

  • The PMO may still be the focal point for Project/Product Portfolio Management, but a more agile approach might be used to perform that function.  Instead of very rigorous upfront planning that might be required to analyze project ROI to support a more traditional, plan-driven project/product portfolio management approach, a more dynamic decision-making process might be used at that level with a much more limited amount of upfront planning and less detailed ROI analysis.
  • In the other functions related to managing the execution of projects, the PMO probably would probably delegate more responsibility to project teams and  play more of a facilitative and consultative role to support the project teams rather than playing a controlling role.

In summary,

  • Agile certainly forces some rethinking of the role of a PMO but it doesn’t necessarily make the whole concept of a PMO obsolete and irrelevant, and
  • There are a wide range of strategies an organization can choose for implementing an Agile transformation at an enterprise level and it isn’t necessarily a binary choice between a pure “Waterfall approach from top-to-bottom or a totally “Agile” approach from top-to-bottom.  You have to choose the right approach to fit the business rather than attempting to force-fit the business to some kind of “textbook” approach. 

The important thing to recognize is that this is not a “one size fits all” decision.  What is the right approach for one company may not be the best approach for another.  It’s kind of like a chess game to choose the fit the right strategy to each level of the organization as shown in the diagram below:

Enterprise Agile 2

Will There Ever Be an Agile Version of PMBOK?

I’ve been working on developing a graduate-level training course on Agile Project Management based on the two books I’ve published in this area for a major university.  In developing a course like this, there are two aspects of the work to be done:

  1. Synthesizing or developing the body of knowledge that you need to communicate, and
  2. Designing the course materials to teach that body of knowledge

If the body of knowledge associated with Agile Project Management were available in some well-documented form like PMBOK, the task of developing a course to teach that body of knowledge would be much easier, but I’m not sure if the body of knowledge associated with Agile Project Management will ever exist in that form.

There’s a fundamental problem with PMBOK that it attempts to boil down everything you could ever possibly want to know about project management and put it in the form of some checklists that you can use for managing almost any aspect of a project.  I don’t think that approach works very well with Agile Project Management at all…Agile Project Management requires a much more adaptive approach to fit the methodology to the project and to the business environment and checklists just aren’t generally very useful for doing that.  To do that effectively, requires a deeper understanding of the principles behind why you would do things in a given situation and not just an understanding of the mechanics of how to do things as a checklist might provide.  It’s also difficult to boil down Agile Project Management principles and practices into a concise form like that.

This is a fundamental problem that I think PMI is a long way from solving and I’m not even sure what the solution is:

  • Is the solution to make PMBOK more “agile” or to create another version of PMBOK that embodies everything you might want to know about Agile Project Management?  PMBOK version 5 sprinkles a few words about Agile here and there but doesn’t scratch the surface of what needs to be done in my opinion.
  • The PMI-ACP exam and materials are a step in the right direction but don’t go far enough in my opinion.  The PMI-ACP exam as it exists today is mostly a test of terminology and doesn’t really test that you know how to blend Agile and traditional project management principles and practices in the right proportions to fit a given situation – that’s the real challenge facing many project managers today.

This problem is made even more difficult because the principles and practices associated with Agile Project Management are still rapidly evolving and are even somewhat controversial:

  • It’s rapidly evolving because much of the knowledge behind Agile principles and practices are based around the notion of a single Agile team and typically there really is no defined role for a Project Manager at that level.  The role of a project manager, if any, would typically come into play for larger and more complex enterprise-level solutions and much of the knowledge associated with how to scale an Agile approach to enterprise levels is not well-defined or understood at this point
  • It’s somewhat controversial because many people see traditional project management and Agile principles and practices as separate areas that are in opposition to each other and don’t mix very well (like oil and vinegar).  The idea that you can blend those principles and practices together as needed to fit a given situation is not widely-accepted yet by many people and there are even some people on both sides of the fence that regard it as “heresy” to even entertain the idea that you can blend these principles and practices together.

My opinion is that PMI has an enormous challenge to figure out a solution to this problem and it may even require a major rethinking of how PMI synthesizes and communicates knowledge like this because I just don’t think extending PMBOK or creating another version of PMBOK is likely to be an effective solution.  PMI has a huge vested interest in PMBOK and any changes to PMBOK typically take a long time to build consensus on and produce which makes this an even more difficult problem to solve.

Enterprise-level Product Backlog Organization

A lot of thinking about Agile seems to be based on small, single-team projects rather than large, complex enterprise-level initiatives and there is a limited amount of information on what needs to be done to scale small, single team projects to large, complex enterprise-level initiatives.

One of those areas that often needs to be done differently on large, complex, enterprise-level projects is Product Backlog organization.  On small single-team Agile projects, the predominant thinking seems to be that you only need to plan the backlog a few sprints in advance, you just prioritize the stories in the backlog, and then pull them off as needed to start a sprint.  That method starts to break down as you scale projects to large, complex enterprise levels.  There are a number of problems I’ve seen with that approach:

  • Without planning the backlog further in advance, it’s difficult to assess the overall scope of the project and determine the resources required for the project.  For example, I’ve seen a project that just started development with a small, single Agile development team and after two years of development still had no end-in-sight of when the project would be completed.  It was a major surprise when we stepped back and re-planned the entire backlog at a high level to find that the project was going to take almost another two years to complete with the current level of resources.
  • When you have a large product backlog with hundreds or perhaps even over 1,000 stories, understanding the inter-relationship of the stories becomes important.  When you make priority changes among stories in the Product Backlog, it often doesn’t make sense to do it the level of individual stories…if you move one story up in the Product Backlog, what about the other stories that are inter-related with it?  Wouldn’t you want to move all of those stories together as a group?  Organizing the Product Backlog into Epics and perhaps even themes provides a way to make priority decisions at a higher level that is more appropriate for enterprise-level projects.  At that level, priority decisions are often more based on a higher-level of epics and themes rather than at the level of individual stories within an epic or theme.
  • Another major problem area is architecture. If you take a piecemeal approach to doing individual stories without planning and considering the entire solution, it can lead to poor architectural decisions and possibly significant rework later on.
  • Finally, when a project grows to the point that multiple teams are involved, it becomes even more essential to organize the work to be done in a way that it can be segmented among multiple teams without requiring excessive amounts of coordination overhead among the teams.

In another article I recently wrote on “Enterprise-level Agile Implementation“, I discussed the other levels of management that are typically found at an enterprise-level in larger companies that need to be integrated such as Program Management and Product/Project Portfolio Management.  In a large company, organizing the Product Backlog into a hierarchical structure can be essential to support effective decision-making at those higher levels of management above the level of a single Agile team and that is often critical to ensure that all of the projects in an organization are well-aligned with the company’s overall business strategy.

Enterprise-level Agile Implementation

I was recently asked to help a consulting firm who is working with a client having trouble with a very large enterprise-level Agile implementation.  It seems that the company had gone head-over-heels into Agile across the whole company and the senior executives were unhappy with the way it was going.  Many projects were going off track and senior management didn’t feel like they had much visibility and predictability to see if all the development efforts were really well-aligned with the company’s business strategy.  I think this is a typical problem that many companies face for large-scale, enterprise-level Agile implementations.

The problem is that large companies typically have some kind of management infrastructure such as a PMO  for managing projects as well as some kind of project portfolio management approach to align projects with the company’s business strategy and that existing management infrastructure probably isn’t totally compatible with an Agile development approach.  The choices are:

  1. Dismantle the existing management infrastructure and simply implement Agile at a development team level with no guiding management infrastructure.  That typically results in problems  such as projects going out of control and not being well-aligned with the company’s business strategy.
  2. Implement a new top-to-bottom Agile management approach such as the Scaled Agile Framework.  This is a good solution but requires a major redefinition of the company’s management infrastructure and some companies are just not ready to make that kind of gut-wrenching change.
  3. Implement a “bridge” between the existing management infrastructure and the Agile development approach using a hybrid management approach overlaid on top of the Agile development process.

The most important point is that you can’t ignore the need for these higher levels of management and implement Agile as a development process without defining some way that it integrates with the company’s overall business strategy. The choices look something like this:

Enterprise Agile 1

This can be a difficult thing to do because standard Agile methodologies such as Scrum do not provide much guidance above the development team level and there are a number of choices at each of these levels.  At each level, there is a choice of implementing a more Agile or a more traditional, plan-driven management approach.  It is somewhat like a chess game as shown in the diagram below:

Enterprise Agile 2

Here’s how I would position some of the frameworks for filling this need:

Enterprise Agile 3

The three frameworks shown above are:

  1. My own Managed Agile Development model
  2. Scott Ambler’s Disciplined Agile Delivery model
  3. Dean Leffingwell’s Scaled Agile Framework (SAFe)

 

 

Product Development versus Project Development

Agile has been most widely used in “product” development environments and less widely used in “project” development environments.  The difference between those two environments is not widely recognized.  Of course, this is not a totally universal, black-and-white distinction; but, in general:

  • Products are less deterministic and the business model is usually a little more open-ended.  For example, a company might say that we want to develop a product to satisfy “X” market need (where that market need may only be generally defined and might need to be validated) and we’re going to invest $X to fund a team for ongoing development to support that product development initiative.

For many products, it’s an effort that simply goes on-and-on without end to provide ongoing support and enhancements for the life of the product.  Products are generally somewhat speculative and might require a significant amount of innovation particularly if it is something that has never been done before.  For that reason, a product development effort is very well-suited to an Agile development process.

The business model behind a product development effort is typically based on a projected return on investment (ROI) that the decision to invest $X in the ongoing development effort will provide an acceptable return from the profitability that the product will generate over a period of time.

The decision process associated with a  product development effort is generally focused on prioritizing what features should be added to the product to provide the highest level of customer satisfaction and profitability.  That decision process is ideally suited to an agile development process.

  • Projects are typically more deterministic and less speculative.  Very few development teams are given a “blank check” to do some kind of project without having some expectations of what the project will accomplish, what it’s going to cost, and what the schedule will be.

The business model behind projects is also typically very different.  A company typically has a given amount of funding to invest in projects and some kind of project portfolio management approach is generally needed to determine the appropriate mix of projects that will provide the greatest overall benefit.  In order to make that decision, something must be known about the expected results, costs, and schedules of the projects in that portfolio and there is an ongoing need to monitor the performance of those projects to see if they really are going in the right direction to provide the return that was expected.

The process associated with managing a “project” is also typically different…the general nature of the features the “project” will provide would generally be much more well-defined prior to the start of the project, there would typically be some expectations about what the cost and schedule of the project would be, and it probably wouldn’t be completely open-ended to add features as a “product” might be.   These are key reasons why it is more difficult to apply an Agile development model to projects than it is to products.

Applying Agile principles and practices in a “project” development environment rather than a “product” development environment can be a bit more challenging but it definitely can be done.  Agile works best where there are no constraints on costs and schedules and the primary goal is to add features to maximize customer satisfaction (product model).  When you introduce constraints on costs and schedules (project model), this is the area where it is many times appropriate to use some kind of hybrid managed agile approach to meet the competing demands of a highly flexible and adaptive development approach and the predictability of meeting cost and schedule constraints that is often demanded in a project environment.

The Hybrid Agile Development Approach I’ve described in one of my other posts is an example of how this can be done.  It involves wrapping a “shell” around an Agile development process that can be as thick or thin as you want it to be to provide a sufficient level of planning and predictability to meet the needs of the business environment while retaining a lot of flexibility and adaptivity within the development process.