Even simple measures have problems…

A well-established emergency food shelf near me “distributed 542,453 pounds of food” last year, according to its website.  No doubt it calculated and announced this measure of program productivity to answer questions about impact coming from its principal donors and stakeholders.

Some strengths and weaknesses of “pounds of food distributed” as a measure:

“Pounds” is a well-accepted, non-controversial measure of weight, with a very precise meaning regulated by the U.S. National Bureau of Standards.

It’s not so useful in regions where weight comes in kilograms, but here, it’s gold.  Or iron; when you pick up two 10-pound weights at the gym, chances are excellent they each weigh the same, at least close enough for use in a fitness regimen.

Pounds of food is less precise.  Presumably, two packages marked “16 oz” of the same brand of Mac&Cheese weigh the same.  But different brands?  And how much is food vs. packaging?  Those are questions for Consumers Union.

And how do you compare a 16 oz package of Mac&Cheese with a pound of spinach?  They may both weigh the same, but they don’t carry the same meaning or value.  They have very different nutritional values, very different comfort value, very different familiarity to different cultures, very different ease of preparation, very different yum factor – all of which is lost in the non-controversial “pound of food” measure.

A pound of Mac&Cheese lasts months, even years.  It can be trucked and shipped without losing nutritional value.  A pound of spinach lasts days, requires refrigeration, and doesn’t travel far without spoiling.  A pound of Mac&Cheese may be a good measure  for relief purposes, but not as good for evaluating a healthy lifestyle.  Spinach, vice-versa.   Needed is a metric called “pounds of nutrition.”

When we’re talking “pounds of distributed food,” who gets this food, and should we care?  Does a faith-based charity operate with different expectations than a tax-supported one?  “Pounds of distributed food” doesn’t tell us what happens to the people who receive some.  Is their distress relieved by just one meal, or by more?  It doesn’t tell us if people coming to the food shelf also get attention for other needs.  What ultimately happens to them?  And what about shut-ins? These questions are avoided with a single measure like “pounds of food distributed.”

Key lessons:

542,453 pounds of food distributed in a year is a lot of food, over 10,000 pounds per week.  Clearly this is a well-organized charity doing very important work.

“Pounds of food distributed” has, as a measure, two big things going for it: it neatly sums up a key theme of a food shelf’s organizational productivity, and it’s easier to measure than some other things.

One measure of a charitable organization’s productivity, no matter how common-sensible,  isn’t nearly enough to convey all that’s important.

And even though 542,453 pounds of food sounds like a lot, the measure tells us nothing of how adequate this is given the size of the population in need.  This is not to pick on this food shelf in the slightest, it’s only to show how inadequate even a straightforward measure can be.  Shouldn’t we be curious about all these other “outcome measures”?

Calculating “pounds of food distributed” is no easy task; someone has to weigh it out and add it up for a year.  Chances are it’s not done like that, but it illustrates how costly it becomes to answer all the questions raised above.

Funders of relief efforts should consider what, really, are the important measures of success.  If there’s money enough to evaluate two or three outcomes, what should they be?  Organizations providing relief efforts should continually educate their donors what  is and is not being communicated with their measures of success.