I talk quite a bit about the concept of intrinsic motivation in my presentations and workshops. Intrinsic motivation describes our satisfaction in doing something simply for the sake of doing it.  Think of playing an instrument, solving a puzzle or painting a picture.  The activity is a reward in itself.  Daniel Pink’s “Drive” is great book that discusses this concept.  If you haven’t read it yet, checkout the video below.

(Hat tip Aaron Bjork)

Over the last few years I have found myself drawing the same Scrum diagram on the whiteboard over and over again.  A simple, no-frills, diagram that shows the basic ideas of Scrum.  This is that diagram.

I just finished reading Sam Guckenheimer and Neno Loje’s new book “Agile Software Engineering with Visual Studio.”  Great job guys!

Some observations:

• They appear to provide a fairly clear picture of Scrum.
• I love the fact they cover mura, muri and muda.
• I don’t like their explanation of technical debt.
• Great discussions on “Analysis Paralysis.”
• They pull in Lean and Kanban very nicely.
• Tacit knowledge is mentioned.
• Inclusion of Product Backlog grooming which many Scrum books overlook.
• Good discussion on Kano Analysis. (nice picture of Stephanie Cuthbertson and team!)
• Good discussion on Empirical vs. Defined Process Control.
• They present some great information on metrics and agile.
• Finally! A good description and discussion on Emergent Architecture!
• They talk about using branching sparingly then dive into a head-on discussion of a mature branching schema. I would have liked to have seen a discussion of lighter-weight methods.
• Sam and Neno advocate for integration as often as possible.  Awesome!
• The handling of testing is a bit dicey and appears to target the use of TFS/Visual Studio over good Agile practices.
• Good discussion on exploratory testing!
• Don’t care for their “Handling Bugs” section.

Quotes I really like:

• “you do not measure planned tasks completed as the primary indicator of progress; you count units of value delivered. “ (pg. 8)
• “Lean turns governance on its head, by trusting teams to work toward a shared goal, and using measurement transparency to allow teams to improve the flow of value and reduce waste themselves.“ (pg. 14)
• “Note that this does not mean that all tasks are known on the first day of the sprint. On the contrary, tasks may be added to the sprint backlog whenever necessary.“ (pg. 29)
• “mastery of Scrum is really for the whole team, not just a designated individual.” (pg. 76)
• “Unfortunately, using metrics to evaluate individual performance is often horribly counterproductive” (pg. 81)
• “use branches sparingly and intentionally. If you need to do something temporary, use a shelveset instead. ” (pg. 162)
• “A potential dysfunction is that integration fails. Integration issues, such as merging, are a common source of unhappiness and waste in teams.” (pg. 197)
• (Regarding test automation) “Automation is useful when it achieves high coverage and when the tests will be used many times for many configurations across many changes in the software under test (SUT).” (pg. 219)
• “work should be sequenced to facilitate getting PBIs through acceptance testing to done as quickly as possible.” (pg. 235)

This is a very good primer for using TFS and Visual Studio Ultimate within a Scrum environment!  This book coupled with Scrum.org’s Professional Scrum Developer course and you should be up and running in no time!

In a recent discussion with a team, the topic of “unshippable bugs” was brought up.  The team explained to me that there are cases when there are bugs found during a Sprint that cause the software to become unshippable.  Their fix was to extend the Sprint and fix the bugs. For many of us this probably does not seem all that crazy of an idea.  In general, there is a big problem with this idea if you are using Scrum to build software.  Let me explain.  In Scrum we are always shooting to complete PBIs in the order of importance.  We don’t want to “divide and conquer” all of the PBIs in a Sprint but rather work on one at a time, in order of importance.  (as much as possible)  There are numerous reasons for this including the fact that we want to always deliver some “done” value at the end of the Sprint.  If we split up the work we run the risk of not delivering any of them and thus creating no value.  Additionally, if we want to deliver positive, quality value, we need to always including many forms of testing in our definition of done. (unit, integration, functional, performance, regression, etc.)

If we comply with our “definition of done” during the Sprint in a way that makes the PBI shippable, how are teams getting to the end of the Sprint with “unshippable bugs?”  One reason might be that the bug was discovered during testing after the PBI was considered done.  Should the team be writing code in such a way that testing can’t keep up?  Should the testing specialist be the only one testing?  No. Quality and testing should be owned by the entire team such that the software is thoroughly tested as it is written.  This way bugs found after the PBI is considered done are usually minor and easy to fix.

The next issue here revolves around the concept of delivery.  Should the team just extend the Sprint?  No. If the team is shipping every Sprint, this is usually indicative of a fast pace and the Sprints should already be short. (1-2 weeks)  If a bug is found and the Product Owner wants it fixed prior to shipping, the team should simply make it #1 in the order and deliver it as part of the next Sprint.  If the Sprints are longer (2-4 weeks) the team is using working in a Release Cycle and has a number of choices, ship what was delivered a Sprint prior or wait for another Sprint.  Finally, when you extend a Sprint (timebox), the Product Owner and the organization looses the ability to contain risk, measure velocity and establish cadence.

I think we are all well aware that bugs cost more money to fix the longer they live.  What are the reasons for this?  As some of you have heard me discuss in various presentations, a lot of it has to do with context.  As developers move on to new areas of code, they loose context for previous areas that may contain bugs.  Specifically, they loose the fine details on how that area of the code works, thus preventing a quick fix of the bug.  I’ve done some analysis of some of the teams I’ve worked with and have found the time to fix a bug varies with time as shown in Figure 1.

Figure 1

This curve shows us that bugs caught early can be fixed fairly quickly.  However, after a certain point, there is a spike in the amount of time it takes to understand to the code and fix the bug.  I have seen this spike occur somewhere between one and three months, depending on the developer.  Obviously this in and of itself is reason to fix bugs early.  But there is another monster waiting for us in the closet…bugs love to procreate.  (just like in nature)  The longer a bug lives in code, the more chances developers have to write code to accommodate the bug, thus creating another bug or two or three.  Left unchecked, this growth becomes exponential.  (Figure 2) I’m guessing this is no surprise to many of you.

Figure 2

We can look at both Figure 1 and Figure 2 as “cost curves.”  By this I mean that at points low on the curve we incur low cost and at points high on the curve we incur high costs.

In Agile, we seek to change the way teams test software.    We want to eliminate the “let me finish it all, then you can test” mentality with “let’s write a little and test a little.”  This behavior encourages teams to find bugs quickly, before they get into the wild.  Lisa Crispin and Janet Gregory thing of this as “Coding and Testing Progressing Together” in their book Agile Testing.  What is the point of cranking out a bunch of code if it’s riddled with bugs?

There are a ton of advantages to using Agile iterations.  One of which is improved quality.  By driving towards zero bugs every iteration, we can keep all of our costs on the bottom of the “costs curves.”  This is one way that Agile helps limit the risk of ballooning costs late in a project.

Betting on a greyhound race can be tons of fun.  You find a cool sounding greyhound, check the stats, then lay down $20 to win. (or some other bet variation)  We are ecstatic when our greyhound wins, collecting our money and touting the prowess of our keen eye for the stats.  When we lose, we attribute it to the dog having a bad day, the weather, the law of averages, etc.  If we are an avid better, we may begin to use complex formulas that take into account the dog’s past performances, current weather conditions, time of day, etc.  In fact, my wife’s uncle has written a book on such formulas for greyhound racing.  Even after an exhaustive amount of work we are still left with a game of chance each time we place a bet.  As hard as we try, predicting the future is hard, improving our chances takes significant effort and in the end we may be left with nothing more than a 1 in 10 chance of being right.  We’ll come back to this in a second…

Let me now turn the focus to that of software project estimation.  I’ve been spending time lately reviewing all the COCOMO II (B. Boehm et al.) material along with concepts of variable covariance. (as I talked about in this blog post)  What I am realizing is that formulas used by COCOMO II bare significant similarities to those used in gambling, e.g. greyhound racing.  We look at the empirical data (if we have it) from the team’s past projects and try to determine (guess) certain characteristics for our new project.  For example, in COCOMO II’s early design model, Person Months(PM) is estimated using the following formula:

PM = A x Size^B x M

Where A is a local coefficient that depends on things like organizational practices and type of software.  Similarly, B is an exponent that reflects the increased size of the project based on its novelty with Size being measured in thousands of lines of source code.  Finally M is another multiplier based on seven project specific attributes defined as follows:

M = PERS x RCPX x RUSE x PDIF x PREX x FCIL x SCED

I’m not going to go into each of these as the information on this formula is readily available.  The thing to understand is that each attribute is a value from 1 to 6 that represents a guess. (albeit an educated one)

What you will find in any traditional estimation system is something very similar to COCOMO II.  Tons of factors looking to measure this or that, complex formulas and a good mix of probability theory. In many cases today, this type of method simply won’t work.  Why? 

If we look at the challenges in quantifying each factor above we can begin to understand the reasons. 

A - Organizations change practices on a regular basis, fail to keep the necessary empirical data, face frequent significant technology changes.

B – Team membership changes which may impact team cohesion, varying degrees of market understanding and regular changes to the architecture driven by market demands.

M – Platform difficulty not understood early on, team changes effecting personal capabilities along with experience and poorly understood reuse model.

To wrap this all up, I thought it would be fun to consider our greyhound race being run as a modern software development project.  First, we are betting on not who wins the race, but how long it takes each dog to finish.  Further, we are going to change the track during the race (but we’ll announce it to the audience) and swap out dogs here and there to keep it interesting.(the dogs are just running right?)  :)  Anyone want to go to the track?

While I’m by no means advocating for companies to stop estimating, I am advocating for simpler, less risky and more cost effective models such as Planning Poker tied to Release and Sprint Planning coupled with a Product Roadmap.  These tools used with a good Agile approach are showing us a more sensible way forward.  I’ll try to explain some of the ideas around why we “bet on the dogs” along with some tools to help us move away from this in future posts.

As I mentioned, I received numerous questions during my presentation at the WI .NET User Group a few weeks back.  One of the them was: “Is there a way to add the subject of a user story as an explicit field?”  After probing the individual a bit, I discovered he wanted to be able to sort user stories bases on this field.  As a recap a user story is typically in the form:

As a <type of user> I want <some goal> so that <some reason>

In this post I will show you how to add the <type of user> or user story role to the user story in TFS 2010.

First make sure you have TFS Power Tools installed.  For the TFS 2010 RC, download them here.

Second, I’m going to show you how to do it using the MSF for Agile v5.0 User Story Work Item Type (WIT).  While other templates are going to be slightly different, you can use this same method as a general guideline on how to do this.

Step 1:

Open the “User Story” WIT from the Server.

Step 2:

On the Fields tab click New.

Step 3:

Setup the new field similar to that below.  You will probably want to change “MyCompany” in the Reference name to be the name of your company.

Step 4:

Still in the Field Definition dialog, click on the Rules tab.

Step 5:

Click New, select ALLOWEDVALUES and click OK.

Step 6:

In the ALLOWEDVALUES dialog continue to click new and enter the desired roles until you have all the roles you want to use.

Step 7:

Click OK back to the WIT editor.

Step 8:

Go to the Layout Tab.

Step 9:

Right click on the Column node under Group – Classification and select New Control.

Step 10:

Setup the fields similar to the following:

Step 11:

Click Save.

Your’re Done!

Your User Story screen should now resemble the following:

You will also be able to add the Story Role column to queries and sort and filter by it.

Good luck and enjoy!

Last July I posted an article on how to use a histogram to gauge how accurately you or your team members estimate.  I’ve had a few people ask me about this recently so I thought I’d post on how to create these histograms in TFS 2010.  For a quick recap on what we want to accomplish with these histograms, take a look at my July article.  You will need a process template that allows you to capture and Original Estimate and Complete Work values.  (Such as the MSF for Agile v5.0 template)  Assuming you have Excel 2007 and Team Explorer 2010 installed, go ahead and open Excel and follow the steps below:

Step 1:

Click on the Data Ribbon and Select Existing Connections.

Step 2:

You should see TfsOlapReport which is a data connection to the Tfs_Analysis cube.  Select it and click Open. (If you don’t see the connection, go here.)

Step 3:

You should the Import Data Dialog.  Change the location of the data to $A$2 as show below and click OK.

Step 4:

Drag the field “Completed Work” into the Values box as shown below:

Step 5:

Drag the fields “Assigned To” and “State” into the Filter box as shown below:

Step 6:

Drag the field “ID” into the Row Labels box as shown below:

Step 7:

Select the team member you want to look at(in this case Bob Smith) and select the State to be closed in the filter list above the pivot table.  This is show here:

Step 8:

Click the dropdown next to Row Labels, select Value Filter and click on Equals.

Step 9:

Setup the Original Estimate value you want to estimate as shown below.  (In this case we will look at the original estimate being 16 hours.) Click OK.

Step 10:

At this point you are ready to build your histogram.  You can use Excel’s Data Analysis pack to build one for you or you can build you own. I like to build my own since the Data Analysis pack charts are kinda crappy, so this is the method I will show.  Start by clicking the top-left corner of the worksheet to select the entire worksheet.   Press Ctrl-C to copy it.

Step 11:

Select Sheet 2 and press Ctrl-V to paste a copy of the pivot table into the new worksheet.

Step 12:

Select the cell directly to the right of the “Completed Work” column header.

Step 13:

Select the Data ribbon and click the Advanced Filter button.

Step 14:

Set the List Range to the all the data in the Completed Work column and select the "Unique records only” check box.  Click OK.

Step 15:

You should have a list that resembles the following:

Step 16:

Copy the values from the filter Completed Work column and paste them back into Sheet 1.  This should resemble the following:

Step 17:

Label the column data you just pasted in as “Bin” and label the column to the right of it “Count”

Step 18:

In the first data cell of the Count column add the following formula:  =COUNTIFS($B$5:$B$100,"=" &D5)

Step 19:

You will need to modify the first argument of the formula added in Step 18 to be the full range of the Completed Work column and the second argument to point the value in the Bin column.

Step 20:

Copy this formula down in the the empty cells.

Step 21:

Total you Count column.

Step 22:

Select your count column and click a bar chart on the Insert ribbon.

Step 23:

Rick click on your chart and click Select Data.

Step 24:

Select Edit under the Horizontal (Category) Axis Labels text.

Step 25:

Select all the values in your Bin column for the Axis label range. Click OK all the way back to the worksheet.

Step 26:

Select your Bin/Count table and then click the Sort Smallest to Largest button in the Data ribbon.

You’re Done!

This data should be used to help you and your team get better at estimating.  As the goal of this type of exercise is to increase our skills, I would advise against using this as a means of rating individual performance.  This can backfire by creating resistance to entering real data significantly skewing the results.

Good luck and enjoy!

First let’s start with Ward Cunningham’s definition here.  The way I explained this at MADdotNET was poor at best.  ;)  While I tend to include measures of waste in technical debt, this is NOT generally shared by the Agile community.  Simply put, technical debt is the cost (debt) from hasty software design and development.  Steve McConnell expounds on this here.  I hope this clears things up!

I presented “Succeeding With Agile & TFS 2010” for the MADdotNET User Group last night to a full house.  I want to thank everyone for the turnout and the great questions and discussions!  I’m going to try to post some articles as follow-ups to some the the questions that were asked last night that didn’t get the attention they deserve.  Some of these topics will be:  Agile Estimation, Technical Debt, Code Churn and including bugs and refactoring on the Product Backlog.  Let me know if I missed anything you want me to cover.

For those that are interested, here is the presentation: