Why You Should Not Estimate in Hours or Days
Developers don’t like to provide time estimates for implementing a software feature. Management, on the other hand, has a legitimate need for project management estimates. This article explains how the Scrum Agile Project Management framework provides a solution to this conflict.
Author: Stephen Walther, SonicAgile, http://SonicAgile.com
Don’t Make Me Lie
I’ve spent a considerable number of hours of my life in planning meetings for software products. The goal of these meetings is to prioritize product features and estimate the amount of time which each feature will take to implement. Typically, these meetings involve the entire software team including all of the developers, Quality Assurance (QA) engineers, marketing people, and documentation people involved in a project.
For example, I’ve been in planning meetings for building new web applications. The goal might be to build a new e-commerce application in which the application includes features such as a shopping cart, a product catalog, and a product details page.
These meetings are typically broken into two parts. In the first part of the meeting, there is a healthy debate about the priority of different features. The goal of the first part of the meeting is to determine what features must be implemented and in which order. For example, should the product catalog be implemented first or the shopping cart? So far, so good.
Things always start going badly with the second part of the meeting. In the second part of the meeting, everyone on the team is asked to estimate how long it will take to implement the features. I’ve been in planning meetings in which the developers on a team flat out refuse to answer this question. Developers sometimes respond to requests to provide time estimates with ridicule or hostility.
Management wants these estimates because management wants to know how many resources they need to allocate to a project. For example, management wants to know how many developer hours, Quality Assurance engineer hours, and documentation hours a project will consume. This is a reasonable question. If a planned project consumes too many resources, depending on the business value of the project, the project might not be worth pursuing.
Unfortunately, developers almost never want to provide time estimates. The problem is that it is extremely difficult to estimate how long it will take to implement a software feature.
Writing software is more like Finding a Cure to Cancer than Building a Brick Wall. If you ask me how long it will take to Build a Brick Wall, I can provide you with a pretty accurate estimate. From experience, I know it takes a certain amount of time to add one brick to a wall. Adding additional bricks will take about the same amount of time. Therefore, I can give you a somewhat accurate prediction of how long it will take to build an entire wall.
If, on the other hand, you assemble a team of scientists and ask them to estimate how long it would take to Find a Cure for Cancer, you would get a very inaccurate estimate. The problem is that there are far too many unknowns. The scientists would not be able to put together any estimate because they have no way to extrapolate from previous experience.
Writing software, like Finding a Cure for Cancer, is more a research activity than anything else. When implementing a software feature, I often spend more time Googling for an answer than I do actually writing code. Often, I don’t know whether something will work until I actually write the code.
So, with good reason, software developers often respond to a request to provide estimates with frustration. The correct answer is often “I have no idea!”. Pushing a developer to provide an exact number of hours for implementing a feature is interpreted as a demand that the developer lie and make something up.
So what do you do? Management has a reasonable need to know how many resources a project is going to consume. They need estimates to determine whether a project is worth it. However, members of a software team have good reason to not provide time estimates – no one wants to be forced to lie.
Scrum is a project management methodology embraced by the Agile software community. There are three concepts from Scrum which can help solve this problem of estimation:
* Estimate using Story Points
* Work in Sprints
* Break Stories into Tasks
Estimate using Story Points
In Scrum, you estimate the amount of work required to do something using Story Points instead of hours or days. For example, instead of estimating that Creating a Shopping Cart will take 5 days, you estimate that Creating a Shopping Cart will require a particular number of Story Points.
Different teams use different units for Story Points. For example, some teams like to use Shirt Sizes such as Small, Medium, Large, and X-Large. Other teams like to use the Fibonacci series. I, personally, like to use Coffee Cup Sizes such as Short, Tall, Grande, and Venti. For example, Creating a Shopping Cart requires a Grande amount of effort and not a Tall amount of effort.
Developers are much happier to provide estimates using Story Points than time. The advantage of Story Points is that they are a coarser measure than time. For example, a developer might refuse to state the exact number of hours required to Create a Shopping Cart. However, a developer might be willing to say that Create a Shopping Cart is a Grande work item instead of a Venti work item.
Think of it like this. If you are asked to estimate the number of hours required to Create a Shopping Cart, you will most likely be wrong. On the other hand, if you are asked to state whether Creating a Shopping Cart is a big work item or a little work item, you are much more likely to be right. Using Story Points removes the illusion of precision by giving you a courser measurement of effort.
Work in Sprints
According to Scrum, attempting to create an accurate estimate of the amount of time it will take to complete an entire software project is almost always doomed to failure. Instead, Scrum teams work in Sprints. A Sprint is a time interval in which a Development Team commits to get a certain amount of work done.
For example, a Development Team might work in one-week sprints. Each week, the Development Team attempts to complete a certain amount of work measured in Story Points. A team might discover that it can complete 50 Story Points a Sprint.
The average amount of work that a team can complete in a Sprint is called the Team Velocity. You calculate Team Velocity by taking the average number of Story Points that the team completed in the previous three or four Sprints.
Instead of predicting how long it will take to complete a project up front, you get a more and more accurate estimate of the amount of time it will take to complete a project as the project proceeds. At the end of each Sprint, you will have a clearer idea of the Team Velocity and a clearer idea of how much work the team can complete in future Sprints.
Break Stories into Tasks
In Scrum, a distinction is made between User Stories and Tasks. A User Story is a non-technical expression of a requirement for a project. For example, a User Story might be Create a Shopping Cart.
At the beginning of a Sprint, the Development Team commits to completing a set of User Stories. At the beginning of the Sprint, but not before, the User Stories in the Sprint are broken into Tasks.
For example, the Create a Shopping Cart User Story might be broken into the following four tasks:
* Create the Oracle database tables
* Write the MVC controller action
* Design the appearance of the Shopping Cart
* Write the Selenium Tests
Unlike a User Story, a Task contains implementation details. For example, to complete the Create a Shopping Cart Story, the team might need to create certain database objects and write certain QA tests.
Breaking a Story into Tasks forces the Development Team to think through a Story in more detail. The team might not know everything involved in implementing the Create a Shopping Cart Story until the Sprint begins.
In Scrum, you use Story Points to estimate the amount of effort required to complete a User Story and you use hours to estimate the amount of effort required to complete a Task. So, you might estimate that the Create a Shopping Cart User Story is a Grande Story and provide hour estimates for each of the tasks required to complete the Story. For example, you might estimate that it will take 3 hours to complete the Write the Selenium Tests task.
There are two reasons that it makes sense to estimate Tasks, and not Stories, using hours. First, Tasks are always smaller than Stories. A Story typically takes a certain number of days. A Task typically takes a certain number of hours.
Second, unlike Stories, you do not estimate the amount of effort required to complete a Task until just before you start working on the Task. You don’t break a Story into Tasks, and estimate the time required to complete each Task, until you start the Sprint in which the Story is implemented. Your estimate is more likely to be accurate because you are about to undertake the work.
What about Management?
So what does Scrum mean for management? According to Scrum, coming up with an accurate estimate for the amount of time which it will take to complete a complex software project is doomed to failure. The best that you can do is to improve your estimate of Team Velocity from Sprint to Sprint.
What this means is that management cannot accurately estimate the business value of a project until a project begins. Each Sprint, everyone will have a clearer idea of how much work can be done in future Sprints. Each Sprint, management will have a clearer idea of the amount of business value which can be delivered from the project.
This might seem disappointing. If you are a manager, then you might weep in frustration. You might really, really want an estimate of the amount of time which it will take to complete a project. This desire is understandable but, according to Scrum, impossible to satisfy.
Developers don’t like to provide time estimates with good reason. An estimate of the number of hours or days required to complete a software feature is almost always inaccurate. Demanding that a developer provide a time estimate forces the developer to lie and no one likes to lie.
In Scrum, you represent work with User Stories and you do not attempt to estimate the amount of work required to complete a User Story using hours or days. Instead, you estimate by using Story Points. For example, you might estimate using Coffee Cup Sizes and estimate a Story as a Short, Tall, Grande, or Venti Story.
Furthermore, in Scrum, you break the work required to complete a project into Sprints. Each Sprint, you complete a certain amount of work measured in Story Points. At the end of each Sprint, the Development Team has a clearer idea of their Team Velocity and the Development Team can more accurately predict the amount of work that they can complete in the next Sprint.
Finally, in Scrum, a Development Team breaks User Stories into Tasks at the start of each Sprint. Tasks, but not User Stories, are estimated in hours. Estimating Tasks using hours makes sense because Tasks are shorter than User Stories and Tasks are not estimated until the Development Team is ready to begin working on the Task.
To learn more about project management using Scrum, I recommend that you read my article Scrum in 5 Minutes.