Microsoft Research has a great tool for automatic unit test generation (Pex) and delegate method replacement (Moles).  These tools can be a great way to get unit testing up and running on your brownfield project.

http://research.microsoft.com/en-us/projects/pex/

In a recent discussion with a team, the topic of “unshippable bugs” was brought up.  The team explained to me that there are cases when there are bugs found during a Sprint that cause the software to become unshippable.  Their fix was to extend the Sprint and fix the bugs. For many of us this probably does not seem all that crazy of an idea.  In general, there is a big problem with this idea if you are using Scrum to build software.  Let me explain.  In Scrum we are always shooting to complete PBIs in the order of importance.  We don’t want to “divide and conquer” all of the PBIs in a Sprint but rather work on one at a time, in order of importance.  (as much as possible)  There are numerous reasons for this including the fact that we want to always deliver some “done” value at the end of the Sprint.  If we split up the work we run the risk of not delivering any of them and thus creating no value.  Additionally, if we want to deliver positive, quality value, we need to always including many forms of testing in our definition of done. (unit, integration, functional, performance, regression, etc.)

If we comply with our “definition of done” during the Sprint in a way that makes the PBI shippable, how are teams getting to the end of the Sprint with “unshippable bugs?”  One reason might be that the bug was discovered during testing after the PBI was considered done.  Should the team be writing code in such a way that testing can’t keep up?  Should the testing specialist be the only one testing?  No. Quality and testing should be owned by the entire team such that the software is thoroughly tested as it is written.  This way bugs found after the PBI is considered done are usually minor and easy to fix.

The next issue here revolves around the concept of delivery.  Should the team just extend the Sprint?  No. If the team is shipping every Sprint, this is usually indicative of a fast pace and the Sprints should already be short. (1-2 weeks)  If a bug is found and the Product Owner wants it fixed prior to shipping, the team should simply make it #1 in the order and deliver it as part of the next Sprint.  If the Sprints are longer (2-4 weeks) the team is using working in a Release Cycle and has a number of choices, ship what was delivered a Sprint prior or wait for another Sprint.  Finally, when you extend a Sprint (timebox), the Product Owner and the organization looses the ability to contain risk, measure velocity and establish cadence.

I think we are all well aware that bugs cost more money to fix the longer they live.  What are the reasons for this?  As some of you have heard me discuss in various presentations, a lot of it has to do with context.  As developers move on to new areas of code, they loose context for previous areas that may contain bugs.  Specifically, they loose the fine details on how that area of the code works, thus preventing a quick fix of the bug.  I’ve done some analysis of some of the teams I’ve worked with and have found the time to fix a bug varies with time as shown in Figure 1.

Figure 1

This curve shows us that bugs caught early can be fixed fairly quickly.  However, after a certain point, there is a spike in the amount of time it takes to understand to the code and fix the bug.  I have seen this spike occur somewhere between one and three months, depending on the developer.  Obviously this in and of itself is reason to fix bugs early.  But there is another monster waiting for us in the closet…bugs love to procreate.  (just like in nature)  The longer a bug lives in code, the more chances developers have to write code to accommodate the bug, thus creating another bug or two or three.  Left unchecked, this growth becomes exponential.  (Figure 2) I’m guessing this is no surprise to many of you.

Figure 2

We can look at both Figure 1 and Figure 2 as “cost curves.”  By this I mean that at points low on the curve we incur low cost and at points high on the curve we incur high costs.

In Agile, we seek to change the way teams test software.    We want to eliminate the “let me finish it all, then you can test” mentality with “let’s write a little and test a little.”  This behavior encourages teams to find bugs quickly, before they get into the wild.  Lisa Crispin and Janet Gregory thing of this as “Coding and Testing Progressing Together” in their book Agile Testing.  What is the point of cranking out a bunch of code if it’s riddled with bugs?

There are a ton of advantages to using Agile iterations.  One of which is improved quality.  By driving towards zero bugs every iteration, we can keep all of our costs on the bottom of the “costs curves.”  This is one way that Agile helps limit the risk of ballooning costs late in a project.