To estimate or not to estimate, that is the question

Disparaging those who provide software estimates seems to be a growing sport. At conferences, in blogs and the twitter-verse it seems open season for anyone who dares to suggest a software team should estimate. And heaven help anyone who says that an estimate might be accurate!

Denigrating estimation seems to be the new testosterone charged must-have badge for any Agile trainer or coach. (I’ve given up on the term Agile Coach and call myself and Agile Consultant these days!)

Some those who believe in estimation are hitting back. But perhaps more surprisingly I’ve heard people who I normally associate with the Scrum-Planning Poker-Burndown school of estimation decry estimation and join in the race to no-estimation land.

This is all very very sad and misses the real questions:

• When is it useful to estimate? And when is it a waste of time and effort?
• In what circumstances are estimates accurate? And how can we bring those circumstances about?

These are the questions we should be asking. This is what we should be debating. Rather than lobbing pot-shots at one another the community should be asking: “How can we produce meaningful estimates?”

In the early days of my programming career I was a paid up member of the “it will be ready when its ready” school of development. I still strongly believe that but I also now believe there are ways of controlling “it” (to make it smaller/shorter) and there are times when you can accurately estimate how long it will take.

David Anderson and Kanban may have fired the opening shots in the Estimation Wars but it was Vasco Duarte who went nuclear with his “Story Points considered harmful” post. I responded at the time to that post with five posts of my own (Story points considered harmful? - Journey's start, Duarte's arguments, Story points An example, Breakdown and Conclusions and Hypothesis) and there is more in my Notes on Estimation and Retrospective Estimation essay and Human’s Can’t Estimate blog post - so I’ve not been exactly silent on this subject myself.

Today I believe there are circumstances where it is possible to produce accurate estimates which will not cost ridiculous amounts of time and money to make. One of my clients in the Cornish Software Mines commented “These aren’t estimates, that is Mystic Meg stuff, I can bring a project in to the day.”

I also believe that for many, perhaps most, organisations, these conditions don’t hold and estimation is little more than a placebo used to placate some manager somewhere.

So what are these circumstances? What follows is a list of conditions I think help teams make good estimates. This is not an exhaustive list, I’ve probably missed some, and it may be possible to obtain accuracy with some conditions absent. Still, here goes….

• The team contains more than at least two dedicated people
• Stable team: teams which add and loose members regularly will not be able to produce repeatable results and will not be able to estimate accurately. (And there is an absence of Corporate Psychopathy.)
• Stable technology and code base: even when the team is stable if you ask them to tackle different technologies and code bases on a regular bases their estimates will loose accuracy
• Track record of working together and measuring progress, i.e. velocity: accuracy can only be obtained over the medium to long run by benchmarking the team against their own results
• Track the estimates, work the numbers and learn lessons. Both high level “Ball Park” and detailed estimates need to be tracked and analysed for lessons. Then, and only then, can delivery dates be forecast
• All work is tracked: if they team have to undertake work on another project it is estimated (possibly retrospectively) in much the same manor as the main stream of work and fed into the forecasts
• Own currency: each team is different and needs to be scored in its own currency which is valued by what they have done before. i.e. measure teams in Abstract Points, Story Points, Nebulous Units of Time, or some other currency unit; this unit measures effort, the value of the unit is determined by past performance. In Economists’ lingo this is a Fiat Currency
• Own estimates: the team own the estimates and can change them if need be, others outside the team cannot
• Team estimates: the team who will do the work collectively estimate the work. Beware influencers: in making estimates the teams needs to avoid anchoring, take a “Wisdom of crowds” approach - take multiple independent estimates and treat experts and anyone in authority like anyone else.
• (Planning Poker is a pretty good way of addressing some of these points, I still teach planning poker although there may be better ways out there)
• Beware The Planning Fallacy - some of the points above are intended to help offset this
• Beware Goodhart’s Law, avoid targeting: if the estimates (points) in anyway become targets you will devalue your own currently, when this happens you will see inflation and accuracy will be lost
• Don’t sign contracts based on points, this violates Goodhart’s Law
• Overtime is not practiced; if it is then it is stable and paid for
• Traditional time tracking systems are ignored for forecasting and estimating purposes
• Quality: teams pay attention to quality and stove to improve it. (Quality here equates to rework.)
• The team aim for overall accuracy in estimates not individual estimates; for any given single piece of work “approximately right is better than precisely wrong”
• Dependencies & vertical teams: Teams are not significantly dependent on other groups or teams; they posses the skills and authority to do the majority of the work they need to
• The team are able to flex the “what” the thing they are building through negotiations and discussions. (Deadlines can be fixed, teams members should be fixed, “the what” should be flexible.)
• The team work to a series of intermediate deadlines
• It helps if the team are co-located and use a common visual tracking system, e.g. a white board with cards
• Caveat: even if all the above I wouldn’t guarantee any forecasts beyond the next 3 months; too much of the above relies on stability and beyond 3 months, certainly beyond 6, that can’t be guaranteed

My guess - and it is only a guess - is that when these conditions don’t hold you will get the random results that Duarte described. Sure you might be able to get predictable results with a subset of these factors but I’m not sure which subset.

The more of these factors are absent the more likely your velocity figures will be random and estimates and forecast waste. When that happens you almost certainly are better off dumping estimation - at best it is a placebo.

Looking at this list now I can see how some would say: “There are too many conditions here to be realistic, we can’t do it.” For some teams I’d have to agree with you. Still I think many of these forces can be addressed, I know at least one team that can do this. For others the prognosis is poor, for these companies estimation is worse than waste because the forecasts it produces mislead. You need to look for other solutions - either to other estimation techniques or to managing without.

I’d like to think we can draw the estimation war to an end and focus on the real question: How do we produce meaningful estimates and when is it worth doing so?