My Miagi-Do school mentor Matt Heusser placed a blog entry on metrics today. Since I haven’t got the clear problem with metrics, I needed to contact him to fulfill The Rule of Three Interpretations. In our conversation I realized that he was referring to a concept, which I haven’t been experiencing in my three years of working as a software tester.

Generally spoken I was referring to metrics as using FindBugs, PMD or cover coverage tools on your software. For some time we have been using these for the framework that we grew on top of FIT for testing our software product. In combination with Continuous Integration you can see improvements and you can see where your project currently is. Is it in a good shape? Where might be holes? Which codepaths are not tested well enough? This feedback is essential for management to make the right decisions about the risks when delivering the product against your test results.

On the other side Matt refers to metrics on a different level. If your annual reviews and your salary gets decided upon management metrics, these are evil. The struggle I have is, that I am in the situation that I never worked in such a system where my personal performance and salary was based on some metric. Basically I can think of situations where this is evil. Here are a few:

• A software architect getting paid by the number of architectural pages written.
• A software developer getting paid by the number of lines of code.
• A software developer getting paid by the number of software products finished.
• A software tester getting paid by the number of tests executed/automated.
• A software tester getting paid by the number of bugs found.

Speaking as a software tester, I would use a tool for generating the easy test cases that are easy to automated (I have been down this rabbit hole, I just realize, but wasn’t getting paid based on that) or use a spell-checker on our logfiles (I always wanted to do this, but didn’t because there are more severe problems than the correct spelling of some debug log messages). As a colleague uses to put it: When you measure something, you change the thing you’re measuring. Be careful what you measure, because it just might improve. When I understood these different meanings of metrics, I also got the problem.

Some time later I found the origin for the discussion. It is a recent statement from Tom DeMarco reflecting over 40 years of software engineering and measurements. Take the time to read. Here is the portion which I found most interestingly:

So, how do you manage a project without controlling it? Well, you manage the people and control the time and money. You say to your team leads, for example, “I have a finish date in mind, and I’m not even going to share it with you. When I come in one day and tell you the project will end in one week, you have to be ready to package up and deliver what you’ve got as the final product. Your job is to go about the project incrementally, adding pieces to the whole in the order of their relative value, and doing integration and documentation and acceptance testing incrementally as you go.”

I believe that this will work. At least it will keep the team from being micro-managed and over-measured.