I’m spending quite a bit of time these days helping organizations implement Agile methodologies.  As such I hear the same set of questions and see the same set of issues over and over again.  One of the issues I see quite often is the “long sprint.”  To explain what I mean by this I’ll use a hypothetical conversation with a Team Lead new to Agile.

Team Lead: “How do you deal with the fact sprint planning and reviews take so long?”

Me: “How long is long?”

Team Lead: “Sometimes a week!”

Me: “How long are your sprints?”

Team Lead: “Usually 3-6 weeks.”

Me: “How did you determine that 3-6 weeks was a good length?”

Team Lead: “Because we couldn’t spend 2 weeks out of every month not coding!  We use longer Sprints to avoid spending so much time in review and planning.”

For those of you that are experienced in Agile, you should see a few problems here.  For now, let’s just focus on the “long sprint” concept.  The “long sprint” seems to manifest in organizations that are always sprinting.  First off, this is in direct conflict with the name sprint.  Sprints should be just that, a focused exertion of energy over a short period of time…2-3 weeks max.  So how do we solve our Team Lead’s issue with spending too much time in review and planning and not enough time writing code?  Enter the “Release.”

A Release is a means of building larger blocks of functionality in multiple Sprints. (usually 4-6)  Some Agile methodologies (XP) implement this concept directly, others do not. (Scrum)  There are many reasons to use the concept of a Release, a select few are:

• Some features just might not fit into a two week Sprint.
• Allow team to perform work in parallel to development.  In general team members can do work other than sitting in sprint review and planning sessions.  This is because these sessions are lighter weight and involve fewer people.  You simply review implemented sprint functionality ensuring it meets the needs of the stakeholder and grab the next chunk of the prioritized Product Backlog. (ok, a bit more than that, but you get the point) These two sessions should not take more than 3 days.
• Reduce the overhead of delivering software to production every 2 weeks.
• Reduce the information overload caused by releasing every 2 weeks.
• Define a delivery pace more inline with that of the organization. (the rope in the Drum-Buffer-Rope of Throughput Accounting)

The next logical question is “Do we create a Release Backlog?”  I agree with Mike Cohn in this case and would say no.  I do however, use the concept of a Release Plan or Release Roadmap.  According to Mike a Release Plan contains:

• Graph showing historical velocity.
• Prioritized Product Backlog. (including some big user stories, "epics")
• A predicted range of where we will finish. This should be either a date-range for a fixed-scope project or a functionality-range.
• Personnel assumptions. (Team members and availability)
• Velocity assumptions. ("we’ll go about half speed during December because of holidays and time off")

I would also add the following:

• A Vision Statement.  (“We want to add the shopping cart functionality and connect it to PayPal”)
• Estimated Release Value as I discuss here.

Using the concept of a Release in Agile organizations can be an extremely effective way to increase efficiency through the elimination of waste.  I will try to post more on this topic in the future.  As always, let me know your thoughts.

Further Reading:

Extreme Programming Release Planning:

http://www.extremeprogramming.org/rules/planninggame.html

Mike Cohn on Release Planning:

http://blog.mountaingoatsoftware.com/why-do-release-planning

Mike Cohn on why there should not be a release backlog:

http://blog.mountaingoatsoftware.com/why-there-should-not-be-a-release-backlog

Kelly Waters on Release Planning:

http://www.agile-software-development.com/2008/02/agile-release-planning.html

I have a couple of custom-written apps that allow teams to track current progress on a projector using TFS.  I just found a community project on CodePlex that looks like a promising replacement, Scrum Sprint Monitor.

Scrum Sprint Monitor provides the Agile team with hands-off, always up-to-date status of the current Sprint, both at the individual and team level. It is designed to run either on a large LCD screen located in a public area, or as a desktop application.

Looks like a great product!  If you try it, let me know what you think.

In much the same way Michael Kunze coined the acronym LAMP, I propose the use of a new industry acronym for Agile. Scrum, Lean, Kanban & Kaizen. (SLKK, pronounced slick)  I have successfully used this process which incorporates techniques from all of these areas and felt it needed a name.  I also feel that it will be very useful to many of you that are already using or just beginning to use Agile.

Scrum – Describes the artifacts, meetings, backlog and iteration terms, etc. A good intro here. (Hat tip Ken Schwaber and Jeff Sutherland) 

Lean – Describes mechanisms for the elimination of waste, systems thinking and pull concepts. Wikipedia description here. (Hat tip Tom and Mary Poppendieck)

Kanban – Introduces the card wall, reduction of work-in-progress(WIP) and resource workflows.  Personal Kanban also useful. A great description by Kenji Hiranabe here. (Hat tip Taiichi Ohno)

Kaizen – Company and culture. Also speaks to elimination of waste using the scientific method. Using actionable items found in Scrum retrospectives, teams perform “kaizen blitzes” to improve processes and performance. Personal Kaizen also useful. (Hat tip TPS, W. Edwards Deming and Joseph M. Juran)

I have read numerous articles and blog posts discussing one technique vs. another and have found it amazing that very few discuss the how they compliment one another. This is what I hope to begin to do in this post.  If you would like to learn more about any of the four pieces, click on some of the links above or books below. First a picture to graphically show my thoughts.

Figure 1 – Scrum, Lean, Kanban, Kaizen Relationship

In a nutshell, here is what the process looks like:

1. Initial Backlog prioritized by business value.  Value drawn from market (client) needs.  This is the pull in Lean.
2. Map your value stream.  Make sure you understand how you will measure partially done work, task switching, defects, etc.
3. High level estimates.
4. Build release roadmap based on value and estimates.  Refactor if necessary.
5. Release planning session for release.  Make sure everyone is clear on the vision of the release.
6. Sprint planning session.  Limit Work-in-progress (WIP) as much as possible.  Prioritized by business value.  Make sure work is buffered and balanced. 16 hour SBI’s.  Make sure everyone is clear on the vision of the Sprint.
7. Commitment on Sprint Backlog.
8. If not already done, automated build and test environment setup.
9. Kanban card wall goes up for Sprint.  An electronic version of this on a 60” LCD works well!  :)
10. Daily Scrums review the card wall deltas.  If electronic, task cards added to personal Kanban walls.  If manual, use resource swim lanes.  How have we improved on our Kaizen item?
11. Sprint Review.  Demo new value features. What was the Throughput in dollars? Review PBI’s contained in the next sprint and verify correct priority order.  Sprint Review minutes published via email to all stakeholders.
12. Sprint Retrospective.  What are we doing well?  What could we do better?  Pick and actionable item for improvement during the next Sprint. How will we measure our improvement?  How did we do on our last item?
13. Ready for Release?  If yes, go to #14, if not go back to #6.
14. Software released to production.  Release notes sent to all stakeholders including version number, release CVE, de-scoped features, added features, impediments and any open discussion items.
15. Is the dollar value (revenue potential) for the next release greater than the cost?  If not, the project is ended or put on hold.
16. Does this project have the highest IRR of all the potential projects for this team?  If not, change projects.
17. Begin next release, go to #5.

While I hope to go into more detail on this technique in future posts, but for now, here are some great primer books to get you started:

Agile Software Development with Scrum (Series in Agile Software Development)

Agile Estimating and Planning (Robert C. Martin Series)

Lean Software Development: An Agile Toolkit (Agile Software Development Series)

Scrumban - Essays on Kanban Systems for Lean Software Development

Agile Retrospectives: Making Good Teams Great

Kaizen and the Art of Creative Thinking - The Scientific Thinking Mechanism

The Kaizen Pocket Handbook

Now that we have some estimates which include our buffers, we have to see what can realistically get done in the release. This is the process I use:

1. Figure out what individual resource loading looks like.
2. Establish any dependencies.
3. Try to distribute work away from the most loaded resource. (Elevate the constraint)
4. Assess Risks. (I’ll discuss this in future articles)
5. Discuss alternatives.
6. Verify validity of estimates and adjust if necessary.
7. With the new information learned from 1-6, repeat the sequence at least once.

This process should be the primary focus of your Release Planning meeting.  In the following sections I will talk about each of these steps individually.

Figure out what individual resource loading looks like

Use your tools here.  Most Agile tools have a way to examine individual developer loading for a Sprint and Release.  In lieu of that you can use an Excel Pivot Table of your Product or Sprint Backlog to create the numbers.  To do this, highlight your estimate table and click the PivotTable button on the Insert ribbon and click PivotTable. (Excel 2007)  This should give you something that looks like Figure 1.

Figure 1 – Excel 2007 PivotTable Empty

From here, drag the developer into the Row Labels box and Estimate,Min and Max into the Values box.  You should now have something that looks like Figure 2.

Figure 2 - Excel 2007 PivotTable Basic

You will notice that our values all say “Count” and that’s not what we are looking for, we want the sum.  To change this click on each of the items in the Values box and select “Value Field Settings…” and change it to “Sum.”  You should now see something like Figure 3.

Figure 3 - Excel 2007 PivotTable Resource Loading

This shows us that Joe has the most work with 3 hours, followed by Bob then Chris.  While this is just an example, in step 3 we will want to try to reduce Joe’s load by attempting to shift work to Bob or Chris.

Establish any dependencies

Don’t spend too much time on this one.  What you want to do is take a look at the priority order of the backlog and see if you need to make any changes due to dependencies.  Obviously you won’t be able to write data to the database if the database has not yet been built.  In my experience, most teams tend to do this intuitively.  Just to be sure, discuss the dependencies of each item as part of the Release and Sprint Planning meetings.  The goal here is to sequence items in such a way that there will not be any resource downtime during a Sprint.

Try to distribute work away from the most loaded resource

In accordance with the Theory of Constraints, we want to try to elevate a constraint by shifting some of its work to other resources.  This step should be fairly obvious with regards to what to do.  A word of caution here.  You may find that suboptimal loading creates greater throughput than a perfectly balanced load.  For example your Team Lead may require slightly less loading due to his need to guide the team.  You may also find the need to overload some of your team members that tend to over-estimate items.

Assess Risks

Due to the short time frames of most Agile projects, you may choose to do this step as an informal part of the discussion of each item.  If your project requires it you may need to build a risk register.  In terms of the PMBOK this is covered extensively in Chapter 11 (Third Edition) and I will cover it in more detail in later posts.

Discuss alternatives

Sometimes building multiple solutions to the same problem yields better results in less time.  In the book Lean Software Development, Mary and Tom Poppendieck discuss this in detail in Chapter 2/Tool 6.

If there is a need for an optimal solution to a given feature, try developing 2 or 3 solutions during a Sprint and let the user decide which one is best.  Often you will find the user wants some combination of the solutions. Obviously doing this is going to add work to the schedule and as such we consider it here.

This is also the point at which the team should ALL have an understanding of the approach that will be used to build out each item.  Often in this dialogue, new ideas are formed that create better solutions.

Verify validity of estimates and adjust if necessary

Give the steps above, you will need to adjust your user stories, estimates and schedule.  You may have found that you can add more features to a given Sprint or that you grossly underestimated what it was going to take to get it done.  Either way, you will probably need to make some adjustments here.  Additionally, check the validity of the new estimates using this technique and add buffers if necessary. 

With the new information learned from 1-6, repeat the sequence at least once

With your adjustments in hand, go through the process again.  This time it will probably go much quicker.  Follow the same steps as before but also make sure the team discusses the adjustments that were made and the associated impact. 

Conclusion

I have found this process usually takes about 3-4 days with a typical Scrum team. (2-6 people)  The first day is all about estimating with the second day being performing the steps above.  The 3rd and 4th days involve backlog lockdown and team commitment.  In the next post I will talk about how this process ties into section 6.5 of the Third Edition PMBOK.  Until then, good luck and let me know what you think!

In my post Planning the Project – Part 1, I talked about the use of financial metrics to determine if we should execute a project or not. In my last post I empirically showed that slight variations in a group of dependent processes can have very dramatic effects on the outcome. So the question is, how do we use these two bits of knowledge to help us estimate? In simplest terms, we want to use some estimation techniques that take statistical variation into account and use NPV and IRR to gauge financial impact. We can use a number of techniques to estimate time and therefore cost. A few of the more popular ones are:

1. Proxy based estimation. (PROBE)
2. Parametric estimation.
3. The Planning Game or Planning Poker.
4. Putnam model based estimation. (SLIM)
5. Evidence-based Scheduling. (EBS)
6. Other various algorithm based models. (COCOMO, SEER-SEM, etc.)

Before we start using any of these techniques, it's good to have a couple of releases under your belt to gauge the velocity of your team. Depending on the Agile methodology, we may want to estimate features, Product Backlog Items, Scenarios, etc. We will want to start by having the teams make a guess at how long it will take to complete these items. I recommend breaking down any item that is estimated for more than 2 days into smaller chunks. Then you will also need to monitor the actual time needed to complete the item. Over a couple releases you should get a better understanding of how good certain individuals are at estimating and how quickly they can actually get things done. Given that information I like to build a "risk factor" or deviation for each developer that is providing estimates. Using this risk factor I can produce min/max estimates more accurately.

Now that you have some data in hand it's time to generate some estimates. I like to use a combination of Evidence-based Scheduling (EBS) and Proxy based estimation. Joel Spolsky has an EBS primer here. Proxy based estimation is simply the act of using similar pieces of completed code as the basis for current estimates, but here again we need data. Let's look at the widget example from Part 1 and look at the specifics. A 6 man-month project is a two month project for a team of three developers. Assuming we use Scrum, we are looking at three development sprints followed by a release sprint. The two week release sprint will be used to stabilize the software by eliminating all the bugs that are deemed unacceptable for release. Each Product Backlog Item (PBI) will be broken into one or more Sprint Backlog Items (SBIs) and used to generate the estimate. I'm just going to use the first PBI which will take the first week of the first Sprint for purposes of discussing estimation. All the other PBIs will be estimated the same way.

Figure 1 - PBI/SBI Estimation Example

Using the historical estimation data you can produce a deviation for each developer that will give you a pretty good range. You can see from the Figure 1 that we have a margin of error of 6.6 days, this is 5.5% of a 120 man-day project. I would be nice if we could reduce this, but let's review what we are trying to do. We are building an ESTIMATE, not an EXACTAMATE! What these numbers really tell you is the confidence you can have in completing the sprints and the release on-time.

I can hear people screaming now, "But Chad! You shouldn't do this in Agile!" Really? I disagree. Agile is not an open checkbook that allows development to just keep working on projects indefinitely, it is a mechanism to Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.¹ We still need to create estimates that allow the business to plan on a software release by a certain date. Remember, we can choose to add or remove scope as necessary but we MUST meet our release dates. We estimate to gain a level of confidence that we can, in fact, meet those dates. As we move forward in the release and learn more we will continue to plan and execute as we monitor the project during the execution phase. This is the "Plan-Do-Check-Act" cycle prescribed by the PMBOK.

If you haven't already figured it out, I'm working my way through the Planning Process Group section of the PMBOK. (3.2.2 in the Third Edition) This article covers .3 through .8 and a bit of .9. I'm going to go into a bit more detail on Schedule Development in my next post.

1. Principles behind the Agile Manifesto - http://agilemanifesto.org/principles.html

There are many facets to initiating a project according to the PMBOK.  Questions like, “Should we do this project?” and “Will we be able to deliver results?” all need to be answered.  To answer these questions we need to run the core process of this step, Scope Initiation.  The Scope Initiation process has, as do most PMBOK processes, Inputs, Tools and Techniques and Outputs.  Inputs include things like the product or service description, statement of work, strategic plan and historical information.  The Tools and Techniques include things like project selection methods and expert judgment. Finally, the Outputs are the Project Charter and the Preliminary Project Scope Statement.  I will examine the relevance of these documents in Agile Management one at a time.  Additionally, given the fractal nature of Agile development, this step can be used at the release level or at the individual iteration level.

The Project Charter is used to authorize the project and document its business need.  In Agile this step is very important as it helps create a unifying vision for the product or service.  It is easy in an Agile environment to lose track of the original intent, this document can help remind the team of what and why of the project.

The Project Scope Statement defines the deliverable requirements, boundaries of the product or service, methods of acceptance, etc.  This statement is present either implicitly or explicitly in most modern Agile methodologies and is usually represented by one or more artifact.

Agile correlation for these two documents are shown below.

In my last article I established that we have a shot at using PMBOK tools in Agile Management.  Today I want to take a look at how the PMBOK Process Groups map to Agile. As any PMP will tell you the five phases of a project (according to PMI) are:

1. Initiating
2. Planning
3. Executing
4. Controlling
5. Closing

While I will examine each in detail, let’s take a high level look at the above and compare it to one of the more prevalent Agile methodologies, Scrum.

Figure 1 - PMBOK Project Management Process Groups

Figure 2 - Simplified Scrum Process

From Figures 1&2 you the correlation between the PMBOK Process Groups and the steps of Scrum should be obvious.  For clarity, I am defining a project in terms of Scrum as a Release.  Some organizations may choose to define the project as the Sprint itself for the purpose of controlling products or services that don’t have the concept of a Release.

Here is the Process Group alignment matrix for some of the Agile Methodologies:

So again we see that PMBOK concepts hold up as we examine Agile in more detail.