March 10th, 2008

13 reasons IT hates metrics

Posted by Michael Krigsman @ 2:37 pm

Categories: CIO issues, IT issues, Project management

Tags: Metrics, Information Technology, Perception, Metrics Initiative, Chances, Strategy, Management, Michael Krigsman

I’ve blogged about the importance of establishing clear metrics to help align IT and business, especially in relation to areas such as project portfolio management. In a great counterpoint, Ed Yourdon has compiled a list of “13 common political problems associated with metrics initiatives.” Ed’s a prolific blogger and author, a world-renowned project failures expert, my personal intellectual sparring partner, and a good guy; anything he writes is worth your attention.

Here’s his list:

1. Metrics are often used to “punish” people — e.g., to criticize them for bugs, or to fire the people with the lowest productivity.
2. A common perception is that newly-introduced metrics will be used to punish people — even if that’s not what management had in mind.
3. Metrics are sometimes used (or misused) as leverage in highly political negotiations about deadlines, budgets, and staffing in high-pressure, risky projects.
4. “Unintended consequences” — the introduction of a metrics initiative is likely to have “feedback loop” consequences that nobody expected or intended.
5. IT organizations sometimes introduce a metrics initiative that measures hundreds of different things, thus overwhelming everyone with a mountain of data.
6. Other IT organizations introduce a metrics initiative that focuses on only one measurement — e.g., programmer productivity measured in lines of code per person-month.
7. Management doesn’t realize that, in many cases, you get what you measure — e.g., if you create the impression that people will be measured by how many lines of code they write, then they’ll write lots of code, even if it’s buggy, stupid code.
8. The Hawthorne Effect.
9. The perception that the metrics data gathered by the newly-introduced metrics initiative will be kept secret, and not shared with the people doing the work.
10. The perception that the metrics data will be completely ignored by management.
11. The perception that, even if management does review the metrics results, they won’t take appropriate action — e.g., they’ll try to hide or bury the problem, or blame someone else for the embarrassing metrics.
12. The perception that the metrics results are not credible (sometimes, again, because management doesn’t want the world to see just how bad the metrics really are).
13. The perception that gathering/recording of metrics data will take too much time, and that it’s not productive — e.g., the reaction from software engineers that “we should be doing our work, not spending all of our time measuring the work that we don’t have time to do!”

If you’re introducing metrics into your organization, look closely at this list. Chances are you’ll eventually run up against at least some of these challenges. Forewarned is forearmed!