There’s a hopeful mantra going around in evaluation circles that “you get what you measure.” Personally, I don’t believe it; I’ve been trying to measure social justice for years, and I can’t say I’ve seen more of it as a result. That’s because I have no control over the rewards given for more-just behavior. And I have no control over the punishments given for unjust behavior. If I had the authority to reward and punish individuals or organizations for behaving more justly or less unjustly, maybe that mantra could become more true, and I would “get what I measure.”
Takeaway 1 – We don’t usually get what we measure
A truly horrible example of how wrong-headed measurement can corrupt a gigantic swath of life in our communities and in institutions we’re obliged to trust is presented in the article, “Why police lie under oath” (NYTimes, 2/3/13). The author is Dr. Michelle Alexander, an Associate Professor of Law at Ohio State University, and author of The New Jim Crow: Mass Incarceration in the Age of Colorblindness (The New Press, 2010).
In it, Professor Alexander reports that “in this era of mass incarceration, the police shouldn’t be trusted any more than any other witness, perhaps less so.” That’s because lying by police has become the norm, she (and others she cites) says. She shows that lying under oath, lying in reports, lying when representing the facts of the case, lying when apprehending completely innocent people happens regularly for the sake of making the numbers go up. And high numbers yield rewards, recognition, and reputation.
Takeaway 2 – Measuring complex things with simple measures is insufficient and misleading
Does this mean I’m against measurement with numbers? No, I’m against the corruption that facile and wrong-headed measurement with numbers can induce. And too much of what passes for measurement is facile and wrong-headed because of the honor we falsely grant to numbers.
Remember, numbers are only proxies for the real thing. Words are too, but words suffer the reputation of being imprecise, whereas numbers bask in the reputation of at least looking precise. In my opinion, numbers are corrupting if they don’t honor the complexity of the underlying thing being measured. Relying on one single number like arrest rate to show the alleged criminality of a neighborhood or community is an especially poor and thin proxy for the real thing.
Criminality is a complex concept, not easily reducible to a single number like “arrest rate,” which as the story relates, so easily misrepresents the truth of criminality. Unfortunately, arrest rate is easy to measure and easy to manipulate, making it easy for police departments to move up in the competition for such grants simply by arresting more people, and making the district look more criminal.
Takeaway 3 – Numbers, despite their reputation, are not truth-telling
Why not just report high numbers, no matter the reality? The mildly cynical would point out that falsifying records to look good and gain rewards is an age-old practice and certainly not unique to police departments. Everyone knows stories of this kind of cheating.
But the ugly truth told by Dr. Alexander goes much further, where police, in an effort to raise the numbers, go out and arrest more people, even on completely false charges. Because they can, with no consequence, she says. Because those in charge don’t care if these innocent people are marked criminal for life, she says. That’s corruption, and not only are innocent lives recorded as blemished, their entry into wage-earning society is made all the harder, derailing and condemning way too many for life.
You get what you measure? If you measure someone’s criminal behavior by arresting someone – without regard to their actual behavior, and without regard to their guilt – but just because you can, those measures are not only misstating the truth by an intolerably large margin, the consequences of such measurement create real damage.
Takeaway 4 – If you want to be malicious, numbers work as well as words
And regrettably, we’re not even able to say that the consequences fall under the heading of “unintended consequences,” because in this story those consequences are certainly intended. The damage to all those maliciously arrested is obvious and incalculable, and as a karmic corollary, so is the damage to the police department. Major corruption.
# # #
Published originally by Steven E. Mayer, Ph.D. / Effective Communities Project
February 22, 2013 / gently revised July 8, 2019