Thursday, June 04, 2009

Improving Your Estimates: Part I

This is Part I of a 3 part series detailing common problems with software estimates and some simple ways to handle them.

Unfortunately, most of us software developers are pretty bad at estimation. Is it because we're incompetent? Of course not. It's because coding is creative work, and creative work is unpredictable. Writing software is not like working on an assembly line in a factory. Workers on an assembly line put together a product based on meticulous specifications and instructions. When it comes to software, that's the compiler's job. A team of programmers on a software project is analogous to the team of engineers that designed the product being put together on the assembly line. We might use a familiar set of tools and idioms, and carry over techniques and design elements from project to project (just like an engineering team does) but make no mistake: every project is different, and we are inventing/designing/specifying a new product each time.

Luckily, despite the unpredictable nature of the work, there are a few techniques we can use to make our estimates more accurate and more useful, and to prevent scheduling disasters.

Today, I'm going to talk about a very common cause of poor estimates: the difference between ideal days and real days.

Problem: Ideal Days vs. Real Days

Early in my programming career, I often found myself surprised, annoyed, or defensive when I estimated some task or feature as “two to three days of work” and someone showed up at my desk asking about it two days later. Eventually I realized the reason for my reaction. When I said “two to three days of work”, that's not what I actually mean. What I actually meant was “this task will take 16 to 24 hours of my uninterrupted time, based on my understanding of the task right now.” The difference between those two phrases caused me, and my managers, many more headaches than were necessary.

There are a lot of differences between the days we're thinking about when we give estimates, and the days that sneak by us so quickly in the run of a work week. When we're thinking about how long a task will take, we consider the details involved in executing that task, but not all the day-to-day distractions that eat up our time but have nothing (or little) to do with that task:
  • Meetings.
  • Customer support calls.
  • Checking email, answering phone calls, and chatting with folks who drop by your desk.
  • Helping the new guy set up his development environment.
  • Participating in whiteboard design sessions, design reviews, or code reviews.
  • Troubleshooting that strange intermittent network or VM issue.
  • Helping a junior developer track down an elusive bug.
  • Answering questions or pairing with a developer who needs your expertise about that part of the system you know better than anyone else.
  • Did I mention checking email?
These kinds of time sinks are easy enough to account for, and most of us get better at factoring them into our estimates as we gain experience in the field. However, Hofstadter's Law (“It always takes longer than you expect, even when you take into account Hofstadter's Law.”) reminds us that we always have room to improve when it comes to providing more accurate estimates.

Solution: Empirical Estimation


Hofstaedter's Law implies that we are bad at estimation, even when we keep in mind that we are bad at estimation, because we are bad at estimating how bad we are at estimation! Fortunately, there's a trick for getting around this sticky problem: basing our estimates on historical data (a.k.a. empirical estimation).

The first step in the process is gathering that data. When assigned a task, record your initial estimate. Then, keep track of the elapsed time you spend on the task, including all the time sinks mentioned above. After a few weeks, you'll have a pretty good data set. Compare your initial estimates with the elapsed times for various tasks. You will probably be surprised by the difference between the two. Ratios of 1:2 and 1:3 are not uncommon.

You'll be tempted to dismiss the results because they make it look like you're a lot less productive than you really are. After all, those elapsed times include time spent on that “emergency” bug fix, the two-hour architecture meeting, and the morning you had to take an hour off to go to the dentist.

Whatever you do, don't give into this temptation to discount the data. All the factors that caused your estimate to differ from the elapsed time for those tasks are likely to crop up again in future tasks. The value in this exercise is forcing yourself to face the facts. All those things contribute to the difference between estimated delivery dates and real delivery dates. In the future, keep this ratio of on-task time to elapsed time in mind, and estimate accordingly.

If you're nervous that these new estimates will make your manager think you're just padding your estimate so you can slack off, explain your thought process. Let him know the task will take 6 hours of intensive work, but empirical measurement shows you that 6 hours of on-task work usually comes with 6 more hours of meetings, assisting other team members, and the like. It may help to record a sample of the types of time sinks you encounter while recording your historical data.




Stay tuned for Part II later this week.

13 comments:

Kelly Waters said...

Personally I prefer a more abstract form of relative estimating, i.e. estimating in points and tracking velocity.

I have now seen dozens of different teams in different circumstances become remarkably reliable and predictable about what they can achieve in a given time period or sprint, even within just a few iterations.

Did they suddenly get good at estimating? No, of course not. But statistically the idea of velocity really does work.

It inherently takes into account the kind of productivity issues you describe in your post. But it also compensates statistically for bad estimating. As long as you're consistently bad :-) it's amazing how accurate you can get at predicting the duration.

If you're interested in finding out more, I've written a blog post on it here:

http://www.agile-software-development.com/2009/04/agile-estimating.html

Kelly Waters
All About Agile

Ryan Cooper said...

Hi Kelly;

I certainly agree: tracking velocity eliminates a whole bunch of estimation problems, and can make project scheduling a heck of a lot easier. It's my preferred way of eliminating estimation woes as well.

The only "problem" with velocity is that it's a team decision. In this series, I really wanted to focus on things an individual programmer can do to improve their estimates, even if they don't work in an "agile" environment.

Thanks for the feedback!

Robert Dempsey said...

Hi Ryan,

We estimate purely in story points because of the many issues with ideal days.

1. When talking time, you are assuming that you are the one who will be doing the work, which may or may not be true.

2. As knowledge emerges from a project, the initial time estimates may be way off, either positively or negatively.

3. Time estimates can take longer to do. Estimating using relative measures can be much faster. It also keeps the implementation out of it, therefore speeding up the process.

Story points get around these issues.

One thing to watch out for is using too much historical data. Unless the project you are estimating is extremely similar to a project you have done in the past, and the skill set of your team has not changed due to previous work, the estimates can be quite inaccurate.

I look forward to your next post. Thanks.

Rolf said...

Hi,

I have a process description for the determination of individual verlocity, as opposed to team velocity: http://planetproject.wikidot.com/measuring-and-estimating-personal-velocity

following this process helped me and some co-workers a great deal, I's say that we are right with our estimates about 90% of the time.

Cheers!

Rolf

Ryan Cooper said...

Thanks for the link Rolf!

Anonymous said...

Money is so intangible, its almost like a promise and a piece of paper.

Sandra Davis said...

Thanks for taking the time to discuss this, I feel strongly about information and love learning more on this. If possible, as you gain expertise, It is extremely helpful for me. would you mind updating your blog with more information.

muebles said...

I believe one and all must glance at it.

pay per head shop said...

Brilliant post and useful information…I think this is what I read somewhere…but

sashweer sailash said...


Wonderful blog & good post.Its really helpful for me, awaiting for more new post. Keep Blogging!

ISTQB Training Institute in Chennai

Anonymous said...

When some one searches for his necessary thing,
so he/she desires to be available that in detail, thus
that thing is maintained over here.

my web site nicotine replacement

Anonymous said...

I know this if off topic but I'm looking into starting my own blog and was wondering what all is required to get setup? I'm assuming having a blog like yours would
cost a pretty penny? I'm not very internet savvy so I'm not 100% sure.
Any tips or advice would be greatly appreciated.
Many thanks

Also visit my web blog - e-cigaretter

Addwin Net said...

Kunbalik di blog saya yang kurang bagus ini tapi tentu sangat bermanfaat di Jual Skripsi