Modifying and Extending Agile/Scrum

I recently participated in a discussion on LinkedIn that was initiated by someone who suggested several possible roles for a Business Analyst in Agile/Scrum that didn’t seem consistent with Agile principles at all. I believe that Agile/Scrum can and should be modified and extended as necessary to fit the situation, but it has to be done intelligently and I think it takes some skill to figure out what makes sense and what doesn’t.

We all know that there are Agile “zealots” who insist on rigid adherence to doing Agile/Scrum “by the book” without any deviation. On the other hand, there are people who “wing it” and treat Agile/Scrum practices like a “cafeteria menu” where you can pick and choose the principles and practices you want to adopt and which ones to ignore. Neither one of those approaches makes sense in my opinion but there’s a lot of “gray area” between those extremes. So, how do you determine what makes sense and what doesn’t make sense? I don’t think there’s a clear answer to that question but here’s some guidelines that I think are useful.

  • There’s a big difference between:
    1. Taking a proven framework like Scrum and modifying it and extending it in a way that is consistent with Agile principles and practices, and
    2. Just starting from scratch ignoring Scrum and all other Agile methodologies, principles, and practices and attempting to put together some kind of ad-hoc approach
  • There’s an analogy to the martial arts that I think fits pretty well. There are a variety of different kinds of martial arts but they all have some similarity and they all require some level of knowledge, proficiency, and discipline in how they’re practiced to be good at it. You don’t just go out and start doing martial arts without any training and experience to know how to do it. Check out this article I wrote on “Stages of Mastery in Agile”.

    http://managedagile.com/2014/06/13/levels-of-mastery-in-agile/

    It is based on a model of stages of maturity in martial arts called “Shu-ha-ri”:

    http://managedagile.com/2013/07/17/agile-and-lesssons-learned-from-martial-arts/

    The essence of the “Shu-ha-ri” martial arts philosophy is that you should be at a level of proficiency before you start improvising and “improvisation without knowledge and proficiency is just amateurism”. I think that is also very applicable to Agile. It takes a considerable amount of skill to figure out how to modify and extend Scrum and other Agile methodologies to fit a given situation.

The key message is that people shouldn’t underestimate the level of skill it takes to modify and extend an Agile/Scrum approach to fit a given situation. That’s a key advantage of some predefined frameworks like SAFe but, on the other hand, even with some of the predefined frameworks, it takes some skill to adapt an Agile approach to fit a business and there is no “silver bullet”.

Business Process Reengineering and Agile

I recently wrote an article on a “Business-centric Approach to Agile“. Have you ever thought about how similar an enterprise-level Agile transformation is to “Business Process Reengineering (BPR)”? The similarities are amazing but I suspect that many people don’t think of any relationship between BPR and Agile.

Business Process Reengineering (BPR) was very hot in the 1990’s. One of the catalysts that precipitated the need for BPR was the advent of new Enterprise Resource Planning (ERP) systems. ERP systems enabled many companies to much more completely automate their business processes but it was a gut-wrenching change for many companies because implementing an ERP system in many cases required rethinking their business processes to take a much more cross-functional approach to their business. Another important catalyst was “lean manufacturing” which seeks to eliminate the use of any resource that does not create value for the end consumer. Does that sound like an Agile enterprise-level transformation?

Here’s how Bain and Company defines “Business Process Reengineering”:

“Business Process Reengineering involves the radical redesign of core business processes to achieve dramatic improvements in productivity, cycle times and quality. In Business Process Reengineering, companies start with a blank sheet of paper and rethink existing processes to deliver more value to the customer. They typically adopt a new value system that places increased emphasis on customer needs. Companies reduce organizational layers and eliminate unproductive activities in two key areas. First, they redesign functional organizations into cross-functional teams. Second, they use technology to improve data dissemination and decision making”

Source: Bain & Company: Insights – Management Tools, Business Process Reengineering

Let’s take this definition one step at a time:

  • The first statement is “Business Process Reengineering involves the radical redesign of core business processes to achieve dramatic improvements in productivity, cycle times and quality” – there’s no question in my mind that that statement could apply to an Agile transformation, but do companies really realize that and do it that way?
  • The next statement is “In Business Process Reengineering, companies start with a blank sheet of paper and rethink existing processes to deliver more value to the customer.” There’s also a good fit with that statement. You may not start with a “blank sheet of paper” and throw out all your existing management processes, but it is definitely important to rethink many existing stereotypes and misconceptions that exist about both Agile and traditional management approaches before you launch into an Agile transformation.
  • The statement that “They typically adopt a new value system that places increased emphasis on customer needs” is very relevant to an Agile transformation but is probably not given the attention that it deserves. When a company implements an Agile transformation, it is often done from a limited development perspective focused on how it improves the development process but that needs to be done in a larger context of how it improves the customer value that the company delivers to its customers.
  • The last statement is absolutely very relevant to an Agile transformation: “Companies reduce organizational layers and eliminate unproductive activities in two key areas. First, they redesign functional organizations into cross-functional teams. Second, they use technology to improve data dissemination and decision making”

I’m not defending BPR, there were definitely some problems in the way it was implemented, but there’s a lot we can learn from it (both good and bad). If more companies realized how similar to Business Process Reengineering is to an Agile transformation and treated it that way, the probability of success would probably be significantly higher. It expands your thinking to see an Agile transformation in an overall business context rather than a very limited development-centric perspective.

I’ve developed a new online training course called “Making Agile Work for Your Business” that is designed to help companies see this perspective and to take a business-centric approach to successfully integrate an Agile development approach into their business.

A “Business Centric” Approach to Agile

Many Agile coaching and consulting companies take what I would call a “developer-centric” approach to Agile. It is heavily focused on team-level capabilities and is primarily oriented around improving the development process. There’s nothing wrong with that, in itself; however, people often make the mistake of assuming that whatever is good for the development process must be good for the business as a whole and that is not necessarily the case.

What I’ve seen frequently is that people have the belief that any kind of traditional management approach is bad, Agile is good, and there is a binary and mutually-exclusive choice between “Agile” and what’s commonly called “Waterfall”. That over-simplifies what I believe is a much more complicated decision and the result of that is that people often try to force-fit a company’s business into an Agile approach. The right solution, in my opinion, is to go in the other direction and fit the approach to the company’s business and sometimes that may require blending an Agile approach with a more traditional management approach in the right proportions to fit the company’s business rather than force-fitting the whole company to an Agile approach.

Becoming agile for the sake of becoming agile may not be an appropriate goal for all companies. You have to ask “what problem will it solve?” and “how will it really benefit the company?” and the answer to those questions may be very different depending on the nature of the company’s business. Check out my article on “Where Does it Hurt?” for more on that. Agile works best in companies that are in the business of developing software products like Intuit who develops Quicken, QuickBooks, and TurboTax and other companies where software development has a very direct relationship to their business objectives like an Amazon.com which is very technology-driven.

In those companies, there is a strong and direct alignment between an Agile development process and the company’s business objectives and it is relatively easy to implement an Agile development approach in that environment. In companies that are not in the primary business of developing software products, the relationship between the development process and the company’s business objectives is typically much more indirect and there is less of a natural alignment between an Agile development approach and the company’s business objectives.

The key to developing a more business-centric approach is that you have to recognize that at an enterprise level, the overall approach must be designed to satisfy the critical success factors of the company’s business. A good model to look at to understand this better is the idea of “value disciplines”. Check out my article on “Agile and Corporate Culture” for more on that. For example, if a company like Walmart is in a business that demands “operational excellence” as the primary value discipline, one of the most important critical success factors in their business is going to be reducing costs to be most competitive. How does an Agile development approach contribute to achieving that objective? The answer isn’t necessarily obvious.

What’s needed in this situation in many cases is more of a “top-down” business analysis to identify potential areas for process improvement so that those initiatives are really well-aligned with the critical success factors that are most important to the company’s business. That should be one of the first steps in an Agile transformation for this kind of company. Before you jump to the conclusion that Agile is a good solution to any problem they might have and start working on making them more agile, you have to figure out how its really going to benefit the company and make the company more competitive in the business that they’re in.

It also doesn’t necessarily require throwing out any existing management processes that they may have and making the whole company more Agile. There may be a legitimate reason for some of those management processes that are already aligned with the critical success factors in their business and it may require some compromise to adapt an Agile development approach to that environment. In the 1990’s and early 2000’s, I did a lot of work with companies on large-scale process improvement and business process reengineering initiatives and even though that effort had nothing to do with Agile at the time, the methodology you would use to do a business-centric Agile transformation would be very similar.

This is a great example of what I call the “Program du Jour”. When a new management approach like Agile comes along, we often “throw the baby out with the bath water” and consider anything that came before it as obsolete and passé. I saw a similar thing when Six Sigma was hot in the early 2000’s. Everyone wanted to jump on the Six Sigma bandwagon, there was a lot of hoopla about it (like green belts and black belts, etc.), any other process improvement methodology that came before it was considered out-of-date, and people got lost in the mechanics of Six Sigma without understanding how it really helped their business.

I published my first book at that time in 2003 and did quite a bit of research before writing my book. What I found was that the companies I considered most successful in implementing Six Sigma had so thoroughly integrated it into the way they did business that it might not have been easily identifiable as Six Sigma and they may not have even called it “Six Sigma”. I think a similar thing is needed with Agile today…companies need to go beyond the mechanics of simply implementing “Agile” and figure out how to really integrate it into the way they do business and there isn’t just one canned way to do that. The approach for doing that may be very different depending on the nature of the company’s business.

To do this, requires a broader approach for implementing an enterprise-level Agile transformation that blends a top-down business-centric approach with a bottom-up developer-centric approach in the right proportions. The basic approach for doing that top-down “business-centric” approach to identify areas of process improvement for the business might not be a lot different from what we did 10 years ago for some of the business process improvement initiatives and business process reengineering initiatives I was involved in at that time that had absolutely nothing to do with Agile.

In fact, an Agile transformation is very similar to what you would do in a major business process reengineering effort. Of course, there are many ways to do this top-down business centric analysis depending on the nature of the company’s business and the company’s appetite for making a significant change but this can be a way to keep an Agile transformation well-aligned with the company’s business objectives rather than simply becoming Agile for the sake of becoming Agile.

Making Agile Work for Your Business

Background

There is widespread knowledge that exists about almost every possible aspect of how to optimize an Agile development process at a team level; however, the knowledge about how to make Agile work at an enterprise level is much more limited. There have been numerous failures in trying to make Agile work at an enterprise level and I believe that there are some significant misconceptions behind these failures:

  • Agile versus Waterfall – At the project management level, there is a big misconception that there is a binary and mutually-exclusive choice between an Agile approach and a traditional plan-driven project management approach (or what people many times refer to loosely as “Waterfall”). The result of this misconception is that people often try to force-fit projects to one of those extremes when a much better solution is to fit the approach to the project and sometimes a hybrid of the two approaches is the best fit. (See my online training course “Learning the Truth About Agile versus Waterfall” for more on that)
  • Aligning Agile With a Business Strategy – Another big misconception is that whatever is good for the development process must be good for the company as a whole and that is also not necessarily the case. At the business management level, the approach should be designed around what makes the most sense for the company’s business and that may or may not be exactly the same as the approach used to manage projects at the development level. The people designing the enterprise-level strategy need to be able to understand the business strategy as well as the development strategy and fit the two together. It isn’t necessarily just a matter of forcing the entire company to become more agile.
    • An Agile development process is easiest to apply in companies whose primary business is developing software products (Such as Intuit QuickBooks and TurboTax) or companies where software development has a very direct and significant leverage effect on the company’s business (Such as Amazon.com). In those companies, there is a fairly direct alignment between a company’s overall business management goals and an Agile development process.
    • In companies where that is not the case, the alignment may be less direct. For example, it may or may not be totally realistic for a company to adopt a complete, top-to-bottom Agile approach for their entire business and a more traditional, plan-driven approach may be appropriate at the higher levels (at least as an interim solution). However, that doesn’t preclude implementing a totally Agile or hybrid Agile development process. The Managed Agile Development process is an example of a hybrid Agile approach that can be used in that kind of environment.
  • Enterprise-level Agile Transformation Strategies

    There are a number of different potential strategies at an organizational level for implementing an Agile transformation:

    • Some organizations may choose to implement a relatively complete top-to-bottom Agile transformation for their business – Dean Leffingwell’s Scaled Agile Framework (SAFe) is an example of such a model. However, that can be a very ambitious and gut-wrenching change for many organizations and it also may not be the best solution.
    • Fortunately, there are other alternatives companies can select to fit an Agile approach with their business

    Organizations typically have different layers of management as shown in the diagram below; and, at each level, there is a choice of taking more of an Agile approach or more of a traditional, plan-driven approach:

    Enterprise Agile 3

    Overall Summary

    The important thing to recognize is that this is not a “one size fits all” decision. What is the right approach for one company may not be the best approach for another. It’s kind of like a chess game to choose the fit the right strategy to each level of the organization as shown in the diagram below:

    Enterprise Agile 2

    It should be apparent that making Agile work at an enterprise level isn’t necessarily as simple as it might seem and requires a broad understanding of both the business strategy and the development strategy to fit the two together. For more information on this subject, see:

What is an Enterprise-level Agile Coach?

When people use the term “Agile Coach” it is often not exactly clear what they mean. Most often, what they’re talking about is what I would call a team-level Agile Coach. That is someone who works at a tactical level with individual members of an Agile team to help them become more proficient in executing an Agile development process such as Scrum. There is very little standardization or certification for what it takes to become an “Agile Coach” at that level and almost anyone could claim to be an “Agile Coach”.

Beyond that; however, there is a different kind of Agile coaching role at an enterprise-level that needs to be better-defined and differentiated from a normal team-level “Agile Coach” role. The enterprise-level role is someone who works at a more strategic level to integrate an Agile development process with a company’s business. That role is not well-defined at all and the difference between the team-level role and the enterprise-level role is also not clearly differentiated. It is assumed that someone who has “Agile Coach” on his/her resume can do all of that and I don’t believe that is necessarily correct.

What often happens in applying Agile at an enterprise level is that an “Agile Coach” attempts to plan and organize an enterprise-level agile transformation. However, that person is probably only trained in Agile from a team-level development process perspective and makes the assumption that whatever is good for the development process must be good for the business as a whole. They also may assume that it is a binary and mutually-exclusive decision to be either “Agile” or “Waterfall” and attempt to force-fit the entire company into an Agile model when the right solution is to fit the approach to the company’s business. I don’t believe that either of those assumptions is necessarily correct.

The problem is this – Agile works very well in companies that are in the primary business of developing products (particularly software products – Intuit is an example that develops TurboTax, Quicken, and QuickBooks). In those companies, there is a strong and natural alignment between an Agile development process and the overall business goals of the company and it is very easy to apply an Agile development process in that environment. It is much more difficult to apply an Agile development process in a company who is not in the primary business of developing products and is in some other kind of business where the relationship of an Agile development process to the company’s overall business strategy is much more indirect.

In companies that are not in the primary business of developing products, you can’t just force the company to be “Agile” in order to make the company more amenable to an Agile development process. The company’s overall culture and business strategy needs to be optimized around whatever the critical success factors are for the business that they are in. For example, if a company is in a business that requires operational excellence, it needs to focus its overall culture and business strategy primarily on efficiency of operations and reducing costs and that doesn’t necessarily align with just becoming more “Agile”. In that kind of environment, you have to develop a strategy that considers both the company’s business strategy and the requirements of an Agile development process to develop a well-integrated approach. The implementation of that strategy often requires fitting the approach to the company’s business environment rather than trying to force-fit the company to some kind of overall Agile approach.

The approach that you might wind up with in that kind of environment also could be a blend of Agile and traditional plan-driven management principles and practices blended together in the right proportions to fit the situation. That is a lot more difficult thing to do and requires a lot more skill than a typical team-level Agile coach would normally have. It requires an understanding of:

  • Agile principles and practices as well as
  • Traditional project management principles and practices

and a deeper understanding of the principles behind both of them (not just the mechanics) to know how to blend them together as necessary to fit a given situation. Beyond that; however, it also requires the ability to look at a very complex, broad-based, enterprise-level business from both a more strategic high-level business management perspective as well as a more tactical product development process perspective to develop a strategy for integrating the two.

What often seems to happen is someone who is trained in “Agile” from a development process perspective attempts to steer the company in the right direction and that person typically doesn’t have the breadth of business management experience and agile development process experience to know how to successfully integrate the two. Is it any wonder why some of these “Agile transformations” are not successful?

I’ve developed some on-line training that should be helpful to people who want to understand this better:

  • Agile Project Management Overview for Project Managers – addresses this from a project management perspective to help project managers see Agile and traditional project management principles and practices as complementary rather than competitive and to learn how to blend the two together to fit any given situation.
  • Agile Project Management Overview for Executives – addresses this from a business management perspective and provides some essential principles and guidelines of how to successfully develop a well-integrated enterprise-level approach for any business.

What is The Role of a PMO in an Agile/Lean Organization?

I recently saw a question on a LinkedIn discussion group asking:

What is role of a PMO in an Agile/Lean organization? 

There are many people in the Agile community who might say that there is no role for a PMO in an Agile/Lean environment and that the whole concept of a PMO is inconsistent with Agile.  That opinion is based on a stereotype that the role of the PMO is heavily associated with controlling and enforcing rigid waterfall-style policies for selecting and managing the execution of projects and programs.  In that kind of environment, a PMO might require:

  • Very thorough and detailed upfront planning to justify the ROI on projects to support rigorous project/product portfolio management decisions
  • Rigid control of project execution to ensure that projects meet their cost and schedule goals and deliver the expected ROI

There is no doubt that some PMO’s have played that kind of role to some extent in the past; however, it is a stereotype to believe that is the only possible role for a PMO to play.  A related stereotype is that many people also see the choice between Waterfall and Agile methodologies for projects as a binary decision – that stereotype is that an organization is either:

  • Totally “Waterfall” with extensive upfront project planning as well as rigid policies and controls over how projects are executed, or
  • Totally “Agile” with zero upfront planning and no policies and controls over project execution

Neither of those extremes is realistic in actual practice and you have to use good common sense to fit the right level of upfront planning and control to the project and business environment that the company operates in. 

Many people make the mistake of seeing this as a binary decision and force-fit projects to one of those two extremes when the right solution is to fit the methodology (or combination of methodologies) to the project and business environment.  It takes more skill to do that – it requires knowledge of a broader range of methodologies as well as a deeper understanding of the principles behind them to know how to blend traditional project management and Agile principles and practices together in the right proportions and customize them as necessary to fit a given situation.

I believe the role of any PMO is to align the selection and execution of projects and programs with the organization’s business goals which includes Project/Product Portfolio Management, providing oversight of project execution and the overall interface for management and reporting of projects and programs to senior management and the business, and finally to provide coordination, guidance, and training to project teams as needed in the organization’s methodologies and standards for project management. 

Those general functions probably don’t change in an Agile/Lean project environment, but how a PMO performs those functions may change significantly depending on the organization’s overall strategy for implementing an Agile transformation. 

  • Some organizations may choose to implement a relatively complete top-to-bottom Agile transformation for their business – Dean Leffingwell’s Scaled Agile Framework (SAFe) is an example of such a model. 
  • However, that can be a very ambitious and gut-wrenching change for many organizations. It also may not be the best solution.  I think it is a mistake to believe that you have to force a company to do an extensive, top-to-bottom Agile transformation in order to adopt an Agile process at the development level.

Organizations typically have different layers of management as shown in the diagram below; and, at each level, there is a choice of taking more of an Agile approach or more of a traditional, plan-driven approach:

Enterprise Agile 3

At the business management level, the approach should be designed around what makes the most sense for the company’s business and that may or may not be exactly the same as the approach used to manage projects at the development level. 

  • An Agile development process is easiest to apply in companies whose primary business is developing software products (Such as Intuit QuickBooks and TurboTax) or companies where software development has a very direct and significant leverage effect on the company’s business (Such as Amazon.com).  In those companies, there is a fairly direct alignment between a company’s overall business management goals and an Agile development process. 
  • In companies where that is not the case, the alignment may be less direct.  For example, it may or may not be totally realistic for a company to adopt a complete, top-to-bottom Agile approach for their entire business and a more traditional, plan-driven approach may be appropriate at the higher levels (at least as an interim solution).  However, that doesn’t preclude implementing a totally Agile or hybrid Agile development process.  The Managed Agile Development process is an example of a hybrid Agile approach that can be used in that kind of environment.

The role of the PMO should be aligned with supporting whatever that overall strategy is.  For example,

  • The PMO may still be the focal point for Project/Product Portfolio Management, but a more agile approach might be used to perform that function.  Instead of very rigorous upfront planning that might be required to analyze project ROI to support a more traditional, plan-driven project/product portfolio management approach, a more dynamic decision-making process might be used at that level with a much more limited amount of upfront planning and less detailed ROI analysis.
  • In the other functions related to managing the execution of projects, the PMO probably would probably delegate more responsibility to project teams and  play more of a facilitative and consultative role to support the project teams rather than playing a controlling role.

In summary,

  • Agile certainly forces some rethinking of the role of a PMO but it doesn’t necessarily make the whole concept of a PMO obsolete and irrelevant, and
  • There are a wide range of strategies an organization can choose for implementing an Agile transformation at an enterprise level and it isn’t necessarily a binary choice between a pure “Waterfall approach from top-to-bottom or a totally “Agile” approach from top-to-bottom.  You have to choose the right approach to fit the business rather than attempting to force-fit the business to some kind of “textbook” approach. 

The important thing to recognize is that this is not a “one size fits all” decision.  What is the right approach for one company may not be the best approach for another.  It’s kind of like a chess game to choose the fit the right strategy to each level of the organization as shown in the diagram below:

Enterprise Agile 2

Where Does it Hurt?

I’ve seen a lot of people jump into Agile just because it’s the latest and coolest thing to do and there’s a lot of momentum behind it.  It’s like the “Program du jour” for some companies – I can remember a similar phenomenon when Six Sigma was initially coming into vogue in the early 2000’s.  At that time I worked with a number of companies on process improvement projects and I interviewed a number of companies for the first book I published, “From Quality to Business Excellence:

  • I saw many companies that got wrapped up in the rituals and hoopla of Six Sigma (green belts, black belts, etc.) and, in many cases, their implementation was superficial and didn’t last.
  • On the other hand, there were a small number of companies that took the time to understand Six Sigma at a deeper level, blend it together with other process improvement methodologies, and really integrate it into the way they did business. In many cases, it was such an integral part of their business that it was difficult to recognize it as Six Sigma. Naturally, those were the companies I thought were really excellent.

I met with a company recently who is considering adopting a more agile approach to help them determine if it made sense for them and a question I like to ask in that kind of situation is “Where does it hurt?”  In other words, what are the aspects of your current process that are causing some amount of “pain” to the organization that you would like to improve on?

Many people skip that step of defining what problem they want to solve and just jump into Agile and I think it’s important to more clearly define what you want to get out of it.  There are at least a couple of reasons why that is important:

  • Agile is not a panacea for every problem, and
  • Many businesses make the mistake of force-fitting their business to some methodology (Agile or whatever)…the right solution, in my opinion, is to fit the methodology (or combination of methodologies) to the business

An Agile transformation may also not get much momentum in a company if the problem it addresses and the goals to be accomplished aren’t defined.  An Agile transformation typically involves some change management and I learned a long time ago that there are three very important elements for any significant change management effort to be successful:

  1. Burning Platform – There needs to be a “burning platform”…there needs to be a sufficient level of pain with the current situation to motivate people to make a change particularly a difficult change that has some risks associated with it.  Specifically, with regard to an Agile transformation, if you don’t clearly identify what is wrong with the current state, the transformation may not gain too much momentum.
  2. Vision for the Future – There needs to be a vision for the future – what does the end state look like after the change?  With regard to an Agile transformation, how will the overall enterprise-level model change as a result of implementing Agile?  Will it be a pure Agile approach from top-to-bottom or is some other kind of framework more appropriate?
  3. Progress in That Direction – In any significant change, there are always naysayers who will stand on the sidelines and wait for it to fail, but it becomes increasingly difficult to do that if you demonstrate progress and benefits to the organization.  With regard to an Agile transformation, a lot of people advocate “just start” without a lot of upfront planning – I’m not an advocate of that extreme, but there certainly is value in starting to make demonstrable progress as quickly as possible.

I think Agile is entering a new stage of maturity where it is no longer perceived as a fad, a lot of the hype religious zealotry about Agile is fading away, and a more pragmatic approach is emerging focused on doing what’s right for the organization.  A good article I recently read called that “Post-Agilisim”:

http://www.jenrickblog.co.uk/2013/11/post-agilism-world-gary-sage/?utm_medium=email&utm_source=Jenrick+Servcies+Ltd&utm_campaign=3285069_The+Digest+-+Nov+(week+1)+-+RE-SEND&utm_content=postagilismgarysage&dm_i=12QQ1YERX,BQ45HU,7120M,1