Improving Your Estimates: Part III

This is Part III of a 3 part series detailing common problems with software estimates and some simple ways to handle them. Part I can be found here, and part II can be found here.

Unfortunately, most of us software developers are pretty bad at estimation. Is it because we're incompetent? Of course not. It's because coding is creative work, and creative work is unpredictable. Writing software is not like working on an assembly line in a factory. Workers on an assembly line put together a product based on meticulous specifications and instructions. When it comes to software, that's the compiler's job. A team of programmers on a software project is analogous to the team of engineers that designed the product being put together on the assembly line. We might use a familiar set of tools and idioms, and carry over techniques and design elements from project to project (just like an engineering team does) but make no mistake: every project is different, and we are inventing/designing/specifying a new product each time.

Luckily, despite the unpredictable nature of the work, there are a few tricks we can use to make our estimates more accurate and more useful, and to prevent scheduling disasters.

Today, I'm going to talk about a couple of similar problems: getting a new assignment while in the middle of an important task or project, and realizing midway through a task that there are complexities you hadn't anticipated.

Problem: Gotchas and Reassignments

There are a few classes of time sinks that are harder to predict than those mentioned previously in this series. These can throw a real wrench in your estimates even when you do use empirical estimation and are explicit about what you're estimating, because they occur less frequently, but often eat up a bigger chunk of time when they do occur.

First, we have the “gotchas”. These are the unanticipated but necessary sub-tasks, the technical hurdles hiding in the details, and those subtle complexities in the problem domain that don't become obvious until you're knee-deep in conditional logic. As the saying goes, the devil's in the details; unless the task at hand is trivial, it's impossible to predict these things before you really dig into the code.

Next, we have reassignments. One of the most common delays in completing an important task is being reassigned to work on something even more important (or, at least, something that someone thinks is even more important). A reassignment may come from a different manager than your original task, and it may be disguised as a request that you “multi-task”, but don't be fooled. You can't work on something new without affecting the schedule of your original assignment.

There's not much that can be done to actually prevent these sources of delays, so we have to take a more reactive approach.

Solution: Early Warning System

When disaster strikes, a little time to prepare can make all the difference. That's why we have hurricane early warning systems; we can't stop the hurricane from coming, but we can prepare for the worst and ride out the storm in safety.

The same principle applies in software development. A key team member running behind schedule can wreak havoc on Gaant charts and project schedules. To a project manager, that's a real disaster! But, if you keep an eye on the weather, so to speak, you can mitigate the risk and prevent a catastrophe. (Even better, unlike real weather, sometimes by giving an early enough warning, you can avert the storm altogether!)

“Keeping an eye on the weather” means stepping away from your keyboard every few hours to think about how things are going. Just because you're being fantastically productive doesn't mean things are going well. (And if things are going poorly, it doesn't mean you've done something wrong!) Any of these warning signs should prompt you to alert your manager that the schedule is at risk:

• You find yourself thinking “Well, that part took longer than expected, but I'm sure this next bit will be easy. I'm sure it will all balance out in the end.” You should never depend on a lucky break to meet your schedule.
• You need input/feedback/help from another person and haven't had a chance to verify that they have time to spare.
• You've discovered a sub-task that wasn't accounted for in your original estimate.
• You've run into an “interesting” technical problem. These can be especially insidious, since you'll feel the urge to hack away and not come up for air until you've solved the problem.
• The domain turns out to be more complicated than you had anticipated. If the problem domain is complex, it's likely there are additional subtleties you will discover as you continue working on the task. This makes it unlikely you will make up for lost time later.
• It looks like you will finish just in time. This is usually a sign that you will finish late, but won't realize it until you're almost done.
• You are reassigned to some other work, even if you don't think it will take long. It's easy to get absorbed in your new task and forget how fast the clock is ticking on your original assignment. If the new task was handed to you by someone other than the person who gave you the original task, it's especially important to make sure everyone is aware of the switch in priorities.

There's no law stating that you can't revise an estimate after starting a task. The beginning of the task is the point in time when you have the least information on which to base an estimate. It would be silly not to refine that estimate as you gather more information. It's better to deliver a valid estimate a day late than to stick with an invalid estimate until the end.

---

And so ends the series. I hope my advice helps you avoid the scheduling problems that are so common in our industry. Remember, delivering reliable estimates takes more than just arbitrarily padding your first guess. By using the techniques described in this series, you can avoid nasty surprises for both yourself and your manager. Not only does this ease project scheduling woes and make your manager's life easier, it makes your life easier too.