Betting on a greyhound race can be tons of fun. You find a cool sounding greyhound, check the stats, then lay down $20 to win. (or some other bet variation) We are ecstatic when our greyhound wins, collecting our money and touting the prowess of our keen eye for the stats. When we lose, we attribute it to the dog having a bad day, the weather, the law of averages, etc. If we are an avid better, we may begin to use complex formulas that take into account the dog’s past performances, current weather conditions, time of day, etc. In fact, my wife’s uncle has written a book on such formulas for greyhound racing. Even after an exhaustive amount of work we are still left with a game of chance each time we place a bet. As hard as we try, predicting the future is hard, improving our chances takes significant effort and in the end we may be left with nothing more than a 1 in 10 chance of being right. We’ll come back to this in a second…
Let me now turn the focus to that of software project estimation. I’ve been spending time lately reviewing all the COCOMO II (B. Boehm et al.) material along with concepts of variable covariance. (as I talked about in this blog post) What I am realizing is that formulas used by COCOMO II bare significant similarities to those used in gambling, e.g. greyhound racing. We look at the empirical data (if we have it) from the team’s past projects and try to determine (guess) certain characteristics for our new project. For example, in COCOMO II’s early design model, Person Months(PM) is estimated using the following formula:
PM = A x Size^B x M
Where A is a local coefficient that depends on things like organizational practices and type of software. Similarly, B is an exponent that reflects the increased size of the project based on its novelty with Size being measured in thousands of lines of source code. Finally M is another multiplier based on seven project specific attributes defined as follows:
M = PERS x RCPX x RUSE x PDIF x PREX x FCIL x SCED
I’m not going to go into each of these as the information on this formula is readily available. The thing to understand is that each attribute is a value from 1 to 6 that represents a guess. (albeit an educated one)
What you will find in any traditional estimation system is something very similar to COCOMO II. Tons of factors looking to measure this or that, complex formulas and a good mix of probability theory. In many cases today, this type of method simply won’t work. Why?
If we look at the challenges in quantifying each factor above we can begin to understand the reasons.
A - Organizations change practices on a regular basis, fail to keep the necessary empirical data, face frequent significant technology changes.
B – Team membership changes which may impact team cohesion, varying degrees of market understanding and regular changes to the architecture driven by market demands.
M – Platform difficulty not understood early on, team changes effecting personal capabilities along with experience and poorly understood reuse model.
To wrap this all up, I thought it would be fun to consider our greyhound race being run as a modern software development project. First, we are betting on not who wins the race, but how long it takes each dog to finish. Further, we are going to change the track during the race (but we’ll announce it to the audience) and swap out dogs here and there to keep it interesting.(the dogs are just running right?) :) Anyone want to go to the track?
While I’m by no means advocating for companies to stop estimating, I am advocating for simpler, less risky and more cost effective models such as Planning Poker tied to Release and Sprint Planning coupled with a Product Roadmap. These tools used with a good Agile approach are showing us a more sensible way forward. I’ll try to explain some of the ideas around why we “bet on the dogs” along with some tools to help us move away from this in future posts.