The corrupting power of numbers

There’s a hopeful mantra going around in evaluation circles – endorsed by many – that “you get what you measure.”  Personally, I don’t believe it; I’ve been trying to measure social justice for years, and I can’t say I’ve seen more of it as a result.  That’s because I have no control over the rewards or punishments given for just or unjust behavior, a critical condition omitted from that would-be mantra.   If I could make rewards flow to individuals or organizations or systems for behaving more justly, maybe I would see more just behavior, at least in the little corner of the world that I influence.

A truly horrible example of how wrong-headed measurement can corrupt a gigantic swath of life in our communities and in institutions we’re obliged to trust is presented in the article, “Why police lie under oath” (NYTimes, 2/3/13), by Michelle Alexander, an Associate Professor of Law at Ohio State University, and author of The New Jim Crow: Mass Incarceration in the Age of Colorblindness (The New Press, 2010).

In it, Professor Alexander reports that “in this era of mass incarceration, the police shouldn’t be trusted any more than any other witness, perhaps less so.”  That’s because lying by police has become the norm, she (and others she cites) says – lying under oath, lying in reports, lying when representing the facts of the case, lying when apprehending completely innocent people — for the sake of making the numbers go up.   And high numbers yield rewards, recognition, and reputation.

Does this mean I’m against measurement with numbers?  No, I’m against the corruption that facile and wrong-headed measurement with numbers can induce.  And too much of what passes for measurement is facile and wrong-headed because of the honor we falsely grant to numbers.

Remember, numbers are only proxies for the real thing.  Words are too, but words suffer the reputation of being imprecise, whereas numbers bask in the reputation of at least looking precise.  In my opinion, numbers are corrupting if they don’t honor the complexity of the underlying thing being measured.  In the reported story, it’s “criminality” that’s tacitly measured by arrest rate.  Banking on one single number, like arrest rate (or achievement test score or quarterly revenue), is investing in an especially poor and thin proxy for the real thing.  Multiple measures from multiple perspectives are required to represent something like criminality (or school achievement or corporate health).

Criminality is a complex concept, not easily reducible to a single number like “arrest rate,” which as the story relates, so easily misrepresents the truth of criminality, even while it is so easily manipulated. Districts with high levels of criminality (high arrest rates) are rewarded by government grants, claims Professor Alexander, allegedly to combat that criminality.  It’s easy to move up in the competition for such grants simply by arresting more people, and making the district look more criminal.

Why not just report high numbers?  The mildly cynical would point out that falsifying records to look good and gain rewards is an age-old practice and certainly not unique to police departments.  Everyone knows stories of this kind of cheating.

But the truth told by Michelle Alexander goes much further, of police not simply reporting false numbers, but going out and falsely arresting more people — even on completely false  charges.  Because they can, with no consequence, she says.   Because of course those people are criminal, police say.  Because no one cares if these innocent people are marked criminal for life, she says.   That’s corruption, and not only are innocent lives recorded as blemished, their entry into wage-earning society is made all the harder, derailing and condemning way too many for life.

You get what you measure?  If you measure someone’s criminal behavior by arresting someone – without regard to their actual behavior, and without regard to their guilt – but just because you can, those measures not only missing the mark by an intolerably large margin, the consequences of such measurement error are severe, creating real damage.

And regrettably, we’re not even able to say that destroyed lives are “unintended consequences,” because in this story those consequences are certainly intended.  The damage to all those maliciously arrested is obvious and incalculable, and as a karmic corollary, so is the damage to the police department.  Major corruption.

# # #

Steven E. Mayer, Ph.D. / Effective Communities Project / February 22, 2013