There’s a hopeful mantra going around in evaluation circles that “you get what you measure.” Personally, I don’t believe it. I’ve been trying to measure social justice for years, and I can’t say I’ve seen more of it as a result. That’s because I have no control over the rewards given for good, just behavior. And I have no control over the punishments given for bad, unjust behavior. If I did have that power, to reward and punish individuals or organizations or systems based on how well they measured for social justice, maybe then I would get what I measured.
Takeaway 1 – We don’t usually get what we measure
Here’s a truly horrible example of how wrong-headed measurement can corrupt a gigantic swath of life in our communities and in institutions we’re supposed to trust. It’s presented in the article, “Why police lie under oath” (NYTimes, 2/3/13). The author is Dr. Michelle Alexander, an Associate Professor of Law at Ohio State University, and author of The New Jim Crow: Mass Incarceration in the Age of Colorblindness (The New Press, 2010).
In it, Professor Alexander reports that “in this era of mass incarceration, the police shouldn’t be trusted any more than any other witness, perhaps less so.” That’s because lying by police has become the norm, she (and others she cites) says. She shows that lying under oath, lying in reports, lying when representing the facts of the case, lying when apprehending completely innocent people happens regularly just to make the numbers go up. And high numbers yield rewards, recognition, reputation, and oftentimes, big money.
Takeaway 2 – Measuring complex things with simple measures is misleading and dishonest
Does this mean I’m against measurement with numbers? No, I’m against the corruption that facile and wrong-headed measurement with numbers can induce. And too much of what passes for measurement is facile and wrong-headed because of the honor we falsely grant to numbers.
Remember, numbers are only proxies for the real thing. Words are too, but words suffer the reputation of being imprecise, whereas numbers bask in the reputation of at least looking precise. In my opinion, numbers are corrupting if they don’t honor the complexity of the underlying quality being measured. Relying on one single number like arrest rate to show the alleged criminality of a neighborhood or community is an especially poor and thin proxy for the real thing.
Arrest rate, as the story relates, so easily misrepresents the truth of criminality. Unfortunately, arrest rate is easy to measure and easy to manipulate, making it easy for police departments to move up in the competition for substantial funding grants simply by arresting more people. It’s like, “We’re more criminal than they are, so give us the money.”
Takeaway 3 – Numbers, despite their reputation, are not truth-telling
Why not just report high numbers, no matter the reality? Falsifying records to look good and gain rewards is an age-old practice and certainly not unique to police departments. Everyone knows examples of this kind of cheating.
But the ugly truth told by Dr. Alexander is that the police go much further. She relates how police, in an effort to raise the numbers, go out and arrest more people, even on completely false charges. And they can get away with that, suffering no consequences, she says. Because, she says, those in charge don’t care if these innocent people are marked criminal for life. That’s corruption, I say, and not only are innocent lives recorded as blemished, their entry into wage-earning society is made all the harder, derailing and condemning way too many for life. It’s actually a kind of murder. With malice aforethought.
You get what you measure? If you think you can measure a community’s “criminality” by arresting its members, without regard to their actual behavior, but just because you can, one can legitimately wonder who’s the real criminal. Measures obtained that way not only misstate truth by an intolerably large margin, the consequences of such measurement create real damage.
Takeaway 4 – If you want to be malicious, numbers work as well as words
And regrettably, we’re not even able to say that the damage created falls under the heading of “unintended consequences,” because in this story those consequences are certainly intended. The damage to all those maliciously arrested is obvious and incalculable, and as a karmic corollary, so is the damage to the police department. Major corruption.
# # #
Published originally by Steven E. Mayer, Ph.D. / Effective Communities Project
February 22, 2013 / gently revised July 8, 2019
