Getting Good Estimates

Good estimates are hard to come by.  They are typically too optimistic or too pessimistic or aren’t grounded in reality.  Here is my approach to effort estimation.  I’ve used it successfully in a number of roles and have seen engineers go from poor to reasonable to good estimators.

UPDATE: I have gathered some thoughts and comments and included them in this update.

UPDATE 2: I have an update on the methodology and some further insights in this blog post.

What I look for in Estimates

Typically when asked for an estimate, you will get a single value with no qualification.  “The work will take 3 weeks”.  Experience has shown me that the a single value implies a lack of understanding the nuance of the problems and issues that the task might have lurking just below the surface.

When asking for an estimate, I’m looking for two things.  1) A baseline effort, and 2) A confidence interval.  This comes in one of two forms

  1. 4 weeks of effort with 60-70% confidence
  2. 3-6 weeks of effort

Both these values are effectively the same.  I let the engineers choose which ever one they are comfortable with.

Characteristic Curve

I can’t recall when I began to understand the characteristic curve within the methodology I use for engineering. I’d say that a long-term colleague Larry Bonfada was a strong influence in the thought process and I have since seen similar characteristic curves in Waltzing with Bears: Managing Risk on Software Projects by Tom DeMarco and Timothy Lister.  I don’t have sufficient a background in statistics to define the shape.  Feel free to leave a comment to educate me on the distribution type.

The critical sections of the curve in the table below.

Section

Description

Confidence

Absolute Earliest

The absolute earliest date that the task can be complete.

0%

Highest Confidence

The date that represents the highest likelihood of being delivered on or around.

60%

Long Tail

Worst case scenarios, if things go wrong, this date will be hit.

<10%

Typically engineers will choose one of those sections for their estimates.  Optimists will communicate the absolute earliest date, pessimists will go for the long tail and your more experienced realists will go for the point of highest confidence – somewhere in the middle.

Shaping the Curve

Quite possibly you are thinking that to get this curve you have to apply painful or difficult to use models; fortunately, it’s not rocket science.  Most engineers actually have a strong gut feel for the shape of the curve, so it’s a matter of teasing out a good estimate.

The way it works is through a set of questions to the person providing the estimate.

Question

Answer

What’s the lowest effort for this task?

 2 weeks

What’s the likelihood it will task 20 weeks

 1%

What’s the likelihood it will take 10 weeks

 5%

What’s the likelihood it will take 5 weeks

 30%

What’s the likelihood it will take 4 weeks

 50%

What’s the likelihood it will take 3 weeks

 60%

I intentionally use an number of extreme points (10,20 weeks) to drive the shape of the curve.  When graphed, it comes up similar to below.

I find that most engineers will naturally have a strong gut feel for the estimates and in the majority of times will give numbers that result in more or less the same shape.

Now of course, there are a class of engineers who are either so cautious that they always estimate in the long tail – or too optimistic (or naive to the real effort) that they will always resist this sort of analysis.  My advice is to push through with them (or at least work out a way to interpret their estimates).

From the answers to the questions in the examples above, I’d walk away with either of the agreed to estimates of 2½ – 4 weeks of effort or 3 weeks with 60-70% confidence.   Each team or organization will have it’s own sweet spot of acceptable range.  Tightening and getting the estimates to the right shape, usually involves a mixture of analytics and soft management skills.

Tightening the Curve by Managing Unknowns

The uncertainty in the curve is representative of a number of different factors, be it experience, unknown complexity, inter-dependencies and so on.

A hallmark of a large amount of unknowns in this sort of analysis is overtly broad ranges.  I’ve had engineers give a range of 2 weeks to 3 months.  Obviously the estimate isn’t workable by any stretch of the imagination.  The engineer in this case is either being obstructionist or hasn’t, or isn’t willing, to look at the unknowns that would drive such a broad estimate range.

The types of questions that I tend to ask the person giving the estimate will be along the lines of

  1. What could happen that will prevent the absolute earliest time from occurring?
  2. What could happen that would push you from the 60% confidence date to a later date?

As each of these questions are answered with the unknown factors becoming more visible, you can revisit the original estimating questions again after those factors have been determined.  If you are lucky there are some factors that are either issues that can be dealt with or risks that can be mitigated or removed.  In addition, it is worthwhile and discover these issues and risks and have them tracked formally as part of the greater project.

I generally find repeated cycles of this sort of analysis serve to improve the estimates to the point where I am comfortable to accept the estimate into the project.  With each iteration the discovery process either moves the overall curve to the left or the right (smaller or large effort) or tightens the shape of the curve (increasing the confidence).

Feel free to provide feedback below on how you deal with estimates.

Never, Ever Say “Process”

pro·cess

noun \ˈprä-ˌses, ˈprō-, -səs\

a series of actions or operations conducing to an end; especially : a continuous operation or treatment especially in manufacture.

Words are an amazing thing.  Words create images and communicate intent.  They have the power to strengthen and the power to weaken.  With software, the most divisive word is Process.  It truly is the demotivator of teams and can suck the life out of developers in an instant.
Continue reading “Never, Ever Say “Process””

At-Launch Linux Hardware Enablement

The Linux market presents some unique challenges to Independent Hardware Vendors (IHVs) in bringing their products to market with broad support available at the time of launch.  Independent of ideological or pragmatic rationale, both Open Source and proprietary drivers are constrained by similar mechanics.  This article provides a broad outline of the mechanics and considerations that are needed for delivering hardware support at-launch.

Continue reading “At-Launch Linux Hardware Enablement”

All Projects Great and Small

Projects come in all shapes and sizes. However there is a human tendency to begin to look for consistency in the way projects are run. If there is the possibility to make all the projects look the same and be approximately the same size, then we’ll force them to the same size. In my particular domain, software projects will aggregate sub projects until they get to a particular size that warrants an organization’s standard project methodology.  This has a number of possibly unintended consequences that bear consideration.
Continue reading “All Projects Great and Small”

Engaging User Communities: Lessons from webOS

This was presented at the Embedded Linux Conference in April 2010.  This was different than other presentations that I have done in that it is virtually word free and is more of the talking point model.

A PDF is available.

The Five Stages of Benchmark Loss

This presentation was presented at Scale8x in Los Angeles in February, 2010.  It was primarily a vent piece highlighting the way that you can never win when running benchmarks and you can never win when publishing benchmarks.

Presentationad and audio available.