“You can’t manage what you don’t measure” is an adage that is popular in project management. However, metrics programs are not easy to implement and have their dark sides. In their book “The Agile Culture: Leading through Trust and Ownership”, Pollyanna Pixton, Paul Gibson and Niel Nickolaisen provides some advice about implementing metrics, the Agile way.
How do we make sure that the metrics we are using are effective and not just vehicles for giving leaders a feeling of power and control while not achieving anything useful? How do we know the metrics we want to create will be of any value?
Let’s look at some general rules of thumb for metrics:
* The fewer metrics, the better. Choose those that will have the most significant impact. When these have made good progress, consider changing the metric in an iterative way
* Minimize negative side effects. Remember that all metrics have strong side effects and that you must be aware of the potential negative impacts of optimizing for a specific goal. Select metrics that complement each other or counterbalance each other. For example, you can measure expense reduction to increase profit margins, but you run the risk of increasing time-to-market or decreasing quality.
* People do what they are measured by. The underlying truth is that people’s behavior is always shaped by how they believe they are measured. Meaningful metrics result in meaningful and productive behavior changes. Meaningless metrics result in meaningless and counterproductive behavior changes.
Source: “The Agile Culture: Leading through Trust and Ownership”, Pollyanna Pixton, Paul Gibson and Niel Nickolaisen, Addison-Wesley
This is a reminder that metrics are not a “natural” ingredient of the software development process. They should be the result of an improvement definition process and not the starting point. The GQM (goal question metric) approach is a reminder that you should start by thinking about what you want to achieve before implementing metrics. As this quote says, the possible negative side effects of the metrics should also be considered carefully and you should think about what you want to avoid.
You should also remind that it is often difficult to assess unique metrics values. Is 100% test coverage a goal that you want to achieve? Is it meaningful? Is it worth to do it? Your answers might vary if you are developing embedded software for a plane or a conference management web site. It could be more meaningful to track and act on the evolution of your metrics than to react on single a value.
Another point about metrics is that the process will be more efficient if there is an automatic data collection. Anyone who had to fill some time reports split on multiple tasks or projects know that this is not the developer favorite task.