The Estimates Land Mine - use and misuse of estimates

After posting my last post - Estimate or #NoEstimate that is the question? - I felt a little as if I’d stepped on a land-mine. That is to say I had a few comments and a bit a mini-twitter storm. If I’m being honest I have been avoiding some of the estimates/#NoEstimates debate until now precisely because it is obvious feelings on the topic run high.

Perhaps the thing that surprised me most was that a post intended to support making human estimates was interpreted by many Tweeters as part of the #NoEstimates movement! Maybe convergence between #NoEstimates and #NoProjects is already happening in the public mind.

In the meantime it seems to me that a lot of the problem with Estimates lies in what they are, what they are not, how they are used and how they are mis-used.

As is so often the case it all depends on what we mean by the word, in this case “Estimate”.

Generally I find it useful to agree with Humpty Dumpty:

“When I use a word it means just what I choose it to mean—neither more nor less." (Through the Looking-Glass, Lewis Caroll).

After all, who can forget Bill Clinton saying:

“It depends upon what the meaning of the word 'is' is.”

While I am sometimes guilty of the use and misuse of words myself I find it helps to keep an open mind on just what someone means when they use a word or phrase. For example, if a developer says “Unit tests” I try not to jump to assumptions about what “Unit tests” actually are. The fact that such language is used is itself interesting but I also want to know what they actually mean by it.

But back to the word “Estimates.”

On occasions like this I like to check my dictionary, in this case the :

“Estimate

        1.        to form an approximate idea of (size, cost, etc.); calculate roughly

        2.        to form an opinion; judge

        3.        submit an approximate price for a job to a prospective client

        4.        an approximate calculation

        5.        a statement of the likely charge for certain work

        6.        an opinion” (Collins Paperback English Dictionary 2001)

My other usual source is Wikipedia which on this occasion gives:

“Estimation is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is derived from the best information available.”

From these definitions I define a sense that an estimate is:

• Approximate
• A statement of possibility
• Is based on available data which may itself be incomplete or variable

Maybe all estimates should be accompanies by a statement of probability but as Kahneman and Tversky described in the planning fallacy, and has been proved repeatedly since, not only do humans underestimate time but humans are over confident in their estimates. Thus any estimate probability statement would probably itself be an over estimate of probability.

Besides, very few of the “estimates” I’ve ever seen are accompanied by a statement of probability so I don’t think this suggest will get very far.

More importantly these definitions also help tell us determine what an estimate is not:

• As estimate is not exact
• An estimate is not a promise, guarantee or commitment
• An estimate is not a target or deadline

And it is not several other things.

Now in my previous blog post I introduced the idea of “Accurate Estimates” so I was actually sneaking in the idea that an estimate could have a high probability and could be an accurate indicator of what will happen. Perhaps I was guilty of something there.

The trap I fell into is one that many fall into, that of accepting the general usage of the word “Estimate”.

In general usage - in the software community - we misuse the word estimate.

Firstly we automatically equate Estimate with “Effort Estimate”: effort (and therefore cost) estimate proliferate in software development but we overlook other estimates that might be useful, in particular Benefit Estimate.

Second the inherently approximate nature of estimation is too often ignore, estimated are endowed with a sense of promise of what will be rather than recognising their inherent approximate nature. (And as noted in the Planning Fallacy, Vierordt’s Law and Hofstadter's Law time estimates will always be under estimates.)

This also leads to too much conversation about “Why was the estimate wrong?” - sometime blame may be implied. The answer to the question is really: “The estimate is not wrong because it was an ESTIMATE” That is to say: An estimate is never wrong because an estimate is an approximation and therefore is not binary “Right” or “Wrong”.

Sure you can have a conversation about why the estimate was very different to what actually played out but the nature of that conversation is going to be different depending on what you will do with the findings of the conversation. For example, if the resulting information is used to refine future estimates it will be a very different conversation to one where the result will be punishment for someone. (Yes, people do get punished, I once saw a company where Project Managers were rewarded/punished based on the variance between estimated time spent on work and actual time spent.)

In short too often an approximate estimates based on variable information is used as some kind of exact promise to meet a deadline.

Software developers love to imagine it is evil managers who take their estimates, massage them to be politically correct, promise them to higher ups and then force poor coders to honour the number they first thought, but, big BUT, managers are not the only ones. Even in everyday life the Planning Fallacy, Vierordt’s Law and Hofstadter’s Law hold. Observe yourself next time you have to catch a bus, train, complete a tax return, hand in course work or do something (almost anything) with your kids.

I would love it if I could wave a magic wand and reset everyones’ understanding of the word Estimate but I don’t see it happening.

And I think - although Woody and Vasco might like to correct me - that a large part of the #NoEstimates movement is motivated by this problem. The way I see the logic is:

• Estimates are seldom recognised for what they really are and honoured as such.
• Estimates are misused and used as a stick to beat people and organizations.
• Therefore estimates have become a problem and it is better of finding a way or working without them.

I’m not saying this is the whole #NoEstimates logic but it is part which strikes a chord with me.

Incidentally, because I believe estimates are not a promise I don’t believe in Scrum commitment and because I believe they are approximate that in Xanpan I focus their use on the near term. And because I believe benefits should dictate deadlines not effort I refuse to use estimates as deadlines.