I came across Goodhart’s Law over the weekend:
“When a measure becomes a target, it ceases to be a good measure.”
If you are on a journey to be more “data driven” and are gathering metrics from all over the place, you should:
- Fight the tendency to make individual measures into goals.
- Or when you do make a measure into a goal (such as test coverage % as a target), do it deliberately and have an idea of what the possible behaviors that will drive.
Using that example, if you set a goal of 70% test coverage, you should be aware that some people may try to game the metric and write tests to get coverage with bogus assertions just to make sure code is covered. What you are hoping will happen is that code will be tested more thoroughly, but what you may get are useless tests with high coverage.
Another example, from Agile this time, could be “I want to increase team velocity by 10%”. If you set that as a goal, then it’s likely that the value of a point will simply decrease so that the volume of points go up 10%. “More” work isn’t actually being delivered. Instead, you should be focusing on whether or not the work the team is doing is valuable.
In conclusion, Goodhart’s Law help’s us understand that sometimes metrics and measures should be just that; metrics and measures. Goals should be established first, then metrics can be identified and collected to make sure progress is being made on the goals. Doing it any other way is backwards and opens you up to thinking you’re making progress, when you really aren’t.