newspeak

code coverage – Software development metrics and reporting – Stack Overflow.

I’m giving a lightning talk later this month at the flossmetrics event in Portland, Oregon. I wanted to interrogate metrics–a too-encompassing term–from the developer’s perspective and so ask, What metrics do developers find useful? I found this one Q/A on Stackoverflow relevant.

It also accords with my impression. In open source communities sponsored by corporations, in particular, the measure of community work, health, production and everything else, including the effectiveness of the managers, is subject to anxious measure. The anxiety may be more obvious with open source communities, as they still represent unorthodox ways of doing things–an unorthodoxy that has at its heart the seemingly transgressive notion of giving away (you know, “sharing”) intellectual property. You can see why measuring the value then of the investment can be so fraught. Determining if the community is actually doing anything, let alone anything of value and even more important, better than an orthodox team, is something that from management’s perspective needs to be answered quarterly, in order to justify the risk.

But what metrics actually mean anything? If we were talking about marketing communities, where brand loyalty is enhanced by consumers collaborating together on better ways to consume the product and tips on how to navigate the environment–eBay comes to mind–then I would measure social media activity within the domain, as well as in standard social media vehicles. I’d look too to see if the most regular posters in the chats were also regular buyers and sellers; whether they were “influence” leaders and had large followings (or if not large, whether their followers were heavy spenders, say: quality over quantity). I’d look too to see what I could do to encourage this social consuming (such as make searches that relate to individual posters easier), and routinely try things. And, I could draw graphs and maps of activity and connectedness, relate effectively microeconomic points to macroeconomic conditions. I might learn that the reason there was a spike associated with, say, Fred’s posts on motorbikes had to do with fuel rising in cost.

What I would learn from tracking these activities–these metrics–would help me organize and manage the consumer communities better. (And there are many other kinds of data one can obtain that would make the consumer a better buyer and user of the product and also help other consumers.)

But these sorts of measures don’t really work in the same way when we are looking at productive networks. The demands differ, as does the nature. Yes, there are overlaps, obvious ones. But if I am working in a team, on a difficult project, knowing that Fred is posting a lot and is influential (duh) probably won’t really help me work better–either write better code with others or be happier about the code written.