It’s a nonprofit’s responsibility – and opportunity – to understand and help others appreciate its own  construct of effectiveness in all its grandeur, complexity, and value to the world.   As a nonprofit, you want to present evidence that you’re making progress toward your mission in language that speaks to your people and your different stakeholders.  You get to tell the story of who benefits from your work and how.  

Nonprofit program benefits

Every nonprofit manager, CEO, and board member gets asked, “How do you know if your program is effective? What benefits can your programs show?”  These questions are usually difficult to answer.

We believe a nonprofit is effective when it can show it’s delivering in the ways suggested by its mission statement.  A good mission statement clearly suggests just who or what a nonprofit organization intends to benefit.  If a nonprofit delivers on the promise of its mission statement, it can be said to be effective.

For example, one nonprofit’s mission statement might be, “To address hunger in our community, we serve nutritious meals to those who can’t afford them.” This means that poor and hungry individuals in the community are this nonprofit’s primary intended beneficiary.   Being effective means showing evidence you’re serving nutritious meals to those who can’t afford them, thereby making progress in the challenge of ending hunger in your community.   

In other words, the definition of impact is “making progress in the organization’s declared mission work – and making the case for progress with some kind of evidence.”  The nonprofit in this example doesn’t have to solve the hunger problem in its community, but it must show that it’s making progress in its goal. It can do this on its own terms – more on this below. 

Who benefits and how?

“Who benefits and how?” is the meat and potatoes question-and-answer of nonprofit program evaluation.  This question may not be on top of everyone’s list. Institutional funders may want to know if the organization has lived up to the terms of the grant.  Some want to know more about cost, and the proportion of budget spent on administration or fundraising.   Some are satisfied just to know that the  organization is surviving the pains of staying alive.  Those may be legitimate variations of the theme of effectiveness, but to us they are secondary.

To us, the central focus of nonprofit effectiveness is brought by the question, “Are you being genuinely useful to your intended beneficiaries in the ways suggested by your mission statement?  Can you show – by words, numbers, visuals, music, facts on the ground, receipts – the ways you’re making your mission come true?”

It’s an important question because deep down this is what donors and allies really want to know.  They want to know they’re contributing not just to a good cause, but to an organization that’s actually making good happen in the way suggested by its mission statement. 

I’m not saying you must show you’ve solved the community’s hunger problem, or that you have to prove anything – but without evidence that you’re making progress, donors have little to go on when they’re asked to re-up.  A nonprofit can not say with credibility that it’s effective without some evidence. With some evidence, it’s more likely it will gain support to continue delivering as well as to build its organizational capacity to deliver more, better  – in a virtual cycle of success.

Progress counts

Showing progress with evidence isn’t necessarily the easiest thing in the world, but it’s not the hardest either.  And just as there’s lots of nuance and wriggle room in the way the mission is stated, there’s also nuance and wriggle room in the way a nonprofit can show this progress.  

Some might say this is a dodge, an escape from demands of “real proof with hard data your program works and you’ve solved a big problem.”  Fortunately, our vision of effectiveness respects the ongoing cumulative work required to make progress. Every bit of progress demonstrates accumulating impact.  We want to make the demonstration of increased progress the goal of nonprofit development and evaluation.  Rather than seek the scientist’s version of proof or the entrepreneur’s version of bottom line return on investment, we prefer to seek evidence that progress is indeed being made in delivering intended benefits to intended beneficiaries. Evidence of improvement is more important to the state of the world than proof the program works. 

Beneficiaries and benefits: Who gets the Love, Money, and Power?

OK, that heading was written to feed the bots that scour the internet looking for buzzy keywords.  To bring some clarity to the concept of who benefits and how, we offer this simple breakdown of four major groups of intended beneficiaries of nonprofit activity.  We see four categories of program beneficiaries: individual, system, community, and organization.

Nonprofit program benefits at the level of individuals 

Working to benefit individuals is the core of the charitable sector’s work, at least historically. Delivering services to address the basic human needs of individuals is the traditional work of “human service” or “social service” agencies.  A “hierarchy of needs” famously described by Abraham Maslow parallels the gamut of charitable programs developed in the last 150 years – or 1500 years, depending on where you start.

Maslow’s hierarchy starts at the most basic and necessary level, the physiological needs for survival – the  need for food, water, air, warmth, clothing, rest.  

The hierarchy progresses to needs for safety, security, stability.  The US is particularly rich with nonprofit and public programs that serve individuals in these areas.

Higher in Maslow’s hierarchy are the needs for acceptance, belonging, support, friendship, affection, love.  Higher still are the self–esteem needs for skills, knowledge, confidence, accomplishment, dignity. And highest is the need for self–actualization, including purpose and meaning, fulfillment, creativity, achieving one’s full potential.  

Thousands of nonprofits and millions of self–help books, websites, and commercial services are designed to address these needs. The goal of each is to show a reduction of need and an increase in quality of life.  

• In programs with a mission to address physiological needs for survival, the intended benefits are a reduction in risk of imminent misery or death – and greater chance of comfort, health, and survival.
• In programs with a mission to address needs for safety, the intended benefits are reductions in fear, instability, joblessness, homelessness, and poverty – and increases in comfort, stability, productive work, satisfactory shelter, and some measure of financial security, and improved prospects of surviving and thriving.
• In programs with a mission to address higher–order needs, the intended benefits are reductions in isolation or feelings of worthlessness and unhappiness – and increases in purpose, dignity, and meaning.

Nonprofit program benefits can be at the system level: improving dysfunctional social systems so that more individuals and communities can better benefit

Societies since forever have developed systems that are geared to addressing the same needs that traditional charities address at the individual level – but they do it “systemically.”   Think of the educational system, health care delivery system, criminal justice system, wealth building system, electoral system, food distribution system, the clean water distribution system – just to name a few.  We’re all a part of all these systems even though we can’t see them in their entirety.

Systems operate at all levels from local to international and every level in-between, typically with different mandates and authorities at each of them.   Systems are often mysterious, simultaneously formal with rules and instructions, and informal with cultures, values, and habits.  Each is intentionally and unintentionally designed and operated by humans, and they therefore operate with flaws – often serious.  And because they’re operated by humans they can all be improved.

It is the work of many nonprofits to assist the improvement of these systems so that they work better for more people.   Activists in the system-change world often speak of “moving the needle,” referring to a metric (such as the one on your car’s dashboard showing level of fuel in your tank) allegedly showing the performance of different essential parts of the overall system.  So far, this is largely science fiction, but a useful metaphor.

With social systems as complex and uncontrolled  as they are, the challenge is to identify the mission–critical parts of the overall system, discover and implement promising upgrades, and monitor the just–right metrics to discover where the remaining kinks are. 

Working on systems ideally allows the relevant performance needles or metrics to move to ever-more positive territory.  The work of system improvement (aka reform or transformation) typically intends progress in…

• Building ever more capable organizations with supportive cultures and allies that want to make progress.
• Developing leadership, relationships built on trust, creating networks, building clout, gaining access to greater authority.
• Advancing promising solutions, from idea to testing, from upgrading to scaling up, with protocols for training, monitoring, and accountability.
• Accumulating a variety of resources – money, information, technology, blueprints, wisdom, influence – and using them smartly.

Showing progress along any of these essential pathways is progress indeed.  And at the risk of repeating myself, showing progress is the key to saving the world.  There are bright futures for nonprofits who learn to do this.

Nonprofit program benefits can be at the community level – supporting communities that support individuals and develop their potential

The needs documented by Abraham Maslow that are addressed in programs serving individuals can also be addressed in programs serving communities.  And while the category of Communities is a diverse set, the commonality in programming is that it is directed to a community as a group – delivery of a curriculum, a concert, a conference, a campaign, etc.  A community can be defined by:

• Demography, ethnicity or mutual interest or circumstance, such as youth, Italo–Americans, or classical music lovers.
• Geography, such as a neighborhood, a watershed, or a political jurisdiction like county, state, or nation.
• Audiences, such as those of arts and culture or educational institutions, targeted audiences of public health messages, or disparate audiences of those viewing your organization’s website.
• Affinity, such as those experiencing a community–wide condition or experience like disaster or disease, or the Class of 2001, everyone who’s seen Inglorious Basterds, or members of the State Association of Whatever. 

The intended benefits of community–serving programs are similar to programs serving individuals.  Ideally, progress is noted as a success rate for the group or community being targeted – rates of a good thing increasing and/or a bad thing decreasing.  Policy organizations or professional associations in many arenas, themselves typically nonprofits, have been advancing measures of a community’s “quality of life” for many years now.

Common examples could include satisfaction rate among audience members, positive change of [something good happening] in a neighborhood, improved mortality in a high-risk group, lower prevalence of reported abuse, improved rates of people going on to pursue a career they want, etc.  Nonprofit program benefits can be at the organization level

Nonprofit program benefits can be directed to other organizations 

Working to benefit organizations – nonprofit organizations in particular – is the special work of grantmaking foundations, professional associations, and associations of networks.  While these entities may well hope that individuals and communities will ultimately benefit, the primary and most immediate beneficiary is the recipient organization – hopefully in the form of increased ability to deliver in their missions, aka “organizational capacity.”

Increased organizational capacity means the organization is becoming stronger and more capable in carrying out their missions.  These gains in organizational capacity can include:

• Upgrades to administration including improved board governance, management, staffing, budgeting, resource control, etc.
• Upgrades to resource development including improved communications, fundraising, saving, information and technology, infrastructure;
• Upgrades to linkages with other key mission-appropriate organizations and networks, strengthened strategic partnerships, clarified and reputable role in the community; and
• Upgrades to the ways programs are conducted, and ways of learning from experience. 

Issues in delivering intended benefits to intended beneficiaries 

The four categories of intended beneficiaries of nonprofit programs express their hopes and dreams, as well as blood, sweat, and tears. Many things, especially life itself, can get in the way and spoil the effort.  Resources are short and staff and volunteers are overworked.  The problems created by contemporary society seem to be accelerating, and the struggle for making progress on missions seems increasingly uphill.  If progress is to be made, the whole enterprise including the discoveries of its progress and barriers must be better funded. 

If a nonprofit delivers good things but no one notices, can it get sustained funding?

Short answer: Probably not for long, unless it’s very persuasive.  This is why people with Liberal Arts or Communications degrees are in demand in the sector – to produce really persuasive grant applications and presentations to stakeholders.  Still, such people can do better work if they are working with evidence of progress.  

Unnoticed, unclaimed, unloved – and unfunded

In truth, most nonprofits have not only a primary beneficiary as presented above, but often they have secondary beneficiaries, often unstated and unrealized.  Even though nonprofits typically get paid to deliver for their primary intended beneficiary, they miss opportunities for support and advanced  development if they don’t count what they’re doing for other categories of beneficiaries. Those benefits can go unnoticed, unclaimed, uncommunicated, unloved, and unfunded.

For example, a soup kitchen may intend primarily to serve nutritious meals to individuals, but it might also participate in efforts to improve the food distribution system, partner with and assist other organizations in their efforts, or provide nutrition education to segments of the community.  It can easily happen that the nonprofit is paid for work with its primary beneficiary – individuals receiving meals – but not paid for the other work, for which it is essentially obliged to volunteer.  I’ll bet it explains a lot of the burnout we see in the sector.

There’s a potential success story to tell about each of your intended beneficiaries – primary and secondary. And there are potential upgrades to the design of your programs to deepen your impact (aka advance your progress) with all of them.  So when a nonprofit serves multiple beneficiaries it should try to develop evidence for how it benefits each of them, and to take credit for all it does.  Too often, nonprofits take credit for too little, and lose out on opportunities to upgrade and move forward with additional support.

Unrealistic demands – proof of success for outcomes or progress not in your control

It’s commonplace that funders expect a nonprofit to create impact by reaching beyond its grasp into areas where it has no control.  For example, an alternative high school will be asked to prove that its graduates go on to enter esteemed professions later in life – proof that even Harvard can’t provide.  Or a chamber orchestra will be asked to prove that audience members go on to become lifelong lovers of  chamber music. Or an organization supplying medicines to displaced persons along the border will be asked to show it is stemming the flow of refugees.  As underfunded as most nonprofits are, more realistic demands for impact and evidence should be the basis of future funding discussions.

Realistic demands – show us something

A mission statement is only aspirational.  Just saying “We save lives” or “We develop leaders for tomorrow” or “We benefit the homeless” doesn’t make it so.   You can state your intentions to save lives, develop leaders or benefit the homeless, but your stakeholders want you to show evidence that you’re working on those intentions and making progress.  This doesn’t strike us as an unreasonable demand.  Be always ready to show something – “signs of progress” or “early signs of impact,” documenting them in some way your stakeholders can appreciate.

So for example, if you’re serving meals, maybe you can also show (perhaps in addition to number of meals served), progress in trying out new sources of nutritious food, or steps that further partnering with others to extend food supply into communities that could be called food deserts, or strengthening the board of your organization so it can better help you make these connections?  Steps taken should count.

A jury of your peers can judge

Here’s an exercise to expand your sense of what’s possible in response to demands for “proof your program works.”  Pretend your nonprofit’s Executive Director is charged with “operating an ineffective program” and hauled before an “Evaluation Court” to defend themselves.  To avoid conviction, you and your ED must provide evidence that supports the view that the work of the organization is not ineffective but instead contributes to progress against its mission.  First, imagine the kinds of evidence that could be mustered in this case, and where it could come from – the files (correspondence, meeting notes, memos of agreement, program records), from testimony by experts familiar with the logic of the program, from survey results of those who’ve experienced the program, from receipts showing expenditures for that little gathering, from photos, even from detritus left behind. The challenge: What evidence could persuade a jury of the program’s peers that the program is doing a good job.  

In this exercise in make-believe, you’re encouraged to think more broadly about what effectiveness means in the context of your organization.  Effectiveness for your organization is not theoretical, it’s what you’re working on day-to-day to accomplish.  You and your colleagues actually get to define what effectiveness is, what success “looks like” given the vision of the organization as expressed in its governing documents.  

What sort of “witnesses” could offer “testimony” or “hard and soft data” that considered together “makes the case” for this program’s effectiveness? Of course you can call on not just one witness but several, each with a different vantage point or experience, drawing on different kinds of data.  Sounds a bit like real life, yes?  It’s how people make all kinds of judgments, yes?  With different bits of evidence from different sources that somehow add up to a conclusion.  It’s essentially an empirical process, based on observation of “data points” on which reasonable people can reach a reasonable conclusion.  

What we’re doing here is substituting the rules of a courtroom for the rules of a science experiment.  With this courtroom approach we can have multiple arguments, presented by witness after witness, using measure after measure.  We can have multiple voices from multiple points of view.  

Not scientific, you might say? Actually, it is

True, that exercise is not the stuff of controlled laboratory experiments.  We’re not talking about randomized clinical trials with control and experimental groups.  The conditions needed for randomized field trials with control and experimental groups simply do not exist in the real world; it is crazy to insist on them if we as a society are serious about learning how to make progress in solving social problems.  Creating conditions that approach scientific control outside a lab is financially expensive, administratively complex, ethically fraught, and in the end insufficiently reliable.  It is well beyond the realm of any organization that’s not totally standardized in its operation.  Yet they are considered the gold standard in studiously academic circles.  

But “science” is much more than that.  Our process of making ongoing judgments of growing effectiveness is empirically sound in that it demands evidence that others can agree is meaningful, and empiricism is the root of all science.   We’re talking about collecting data (“making and recording observations”) that can be reviewed by a group of reasonable people who can reach a conclusion.

Show me = Empiricism

“Show me” is the heart of empiricism, which in turn is the heart of wisdom. In olden times, as historians of the early 20th century would tell us, when testing whether a new method of planting corn, for example, was better than traditional practice, they’d try the new method next to the old one and invite all the nearby farmers to see how it’s going, maybe even tasting the fruit.  Empiricism is as old as the hills, as old as farming.  If license plates can be believed, Missouri is “The Show Me State.”

So flash forward to the case in hand: “Show us all the signs of your progress – let’s see what you’ve got.”  For example,

• You could find experts that can speak to the merits of your program design. Are there people from National who can testify you follow Good Practices?
• You could count the number of people participating in your programs.
• You could interview people who’ve participated and ask them about their experience.  They could give you an informative narrative.  They could also express things on a 5–point scale indicating how much safer or less hungry or more secure they now are.
• You could present an album with photos of significant events, important documents, or letters of appreciation.
• You could put together some cool graphics from data already collected.
• You could make comparisons for any of the above against the evidence from last year, or against the industry average, or against your standing from the beginning.

And since many who serve on a nonprofit’s board and its funders’ boards have taken at least a couple social science courses, the social science model prevails.  The social science model includes, indeed begins with empiricism: the observation and assessment of experience.

The M–word: Measurement

The word “measurement” can be intimidating.  But measurement is nothing more than making comparisons and drawing meaning from them.  That’s it.  The use of numbers is not inherent to measurement.  We “take the measure” or “gauge” a situation or person all the time.  We size them up, we take a reading, we reckon, we judge qualities, we rank, we rate, we compare, we gain an understanding.  The distinction between “hard data” and “soft data” is as silly as the testosterone that inspired it.  

As humans we measure and evaluate all the time.  We invite the neighbors over for a taste and judge for themselves.  We know happiness when we see it or hear it.  Measuring program benefits can be a human activity, even a community-building activity.  It’s familiar and meaningful, and we all do it.  “Taking stock, sizing up, looking closer” can be engaging, jointly owned, productive and useful.  Just talking with your colleagues about what you could measure that indicates progress is a big step forward for many organizations.

What kind of comparisons could you make? The state of your organization’s ability to produce program benefits today compared to last year. The number of people who experience X versus those who don’t, or who experience Y.   The number of good things you can say when participants leave compared with the number of bad things (maybe from an agreed upon list of things you hope to hear and not hear). Over time, if you’re granted this important gift, you can improve this skill, learn to better lift up what’s important, getting better at observing, noting, compiling and presenting evidence to show your organization’s progress towards mission.

Then comes the E–word, Evaluation 

Evaluation is the process of making meaning from measurements.  Contrary to what many say, data do not speak for themselves.  And without belaboring the obvious, data don’t speak at all.  Meaning has to be made from data, and that’s a job for humans.   We guarantee no two people will draw the same conclusions, insights, implications, or recommendations from even one table or graph.  But you as a nonprofit director or leader must own your data – you get to tell your stakeholders what they mean, and how they speak to the value of your efforts.

And finally, the T–word, Truth

One truth about data, no matter the source or their alleged precision, they’re not as objective as conventional wisdom suggests.  Deciding what to count, compare, photograph, make notes of etc are all human choices, and human interpretation of what’s collected is required, even though numbers-oriented people like to deny this. Numbers don’t lie, and numbers don’t tell the truth, either.

We believe that the point of evaluation is to “improve rather than prove” the value of work done by nonprofits.  The notions of “outcomes,” or “results,” or “impact” have the problem of sounding final, which can’t be true.  The world doesn’t end when you take a measure – the stream of behavior and events that produce outcomes, results or impact continue.  In the business world, the accountant’s audit is valid for the date it’s completed, but inflows and outflows of money continue.  At the very least, one could speak of “outcomes to date,” or “results so far,” or “early signs of impact.”  An end-of-year audit produces results to date as well as opportunities for improvements to be made during the next period. 

“Effectiveness” is a construct, and you can make it work for you.

 Effectiveness is a “construct.” A construct is an abstraction, not directly observable, but understood from a variety of different signs or markers taken from different points of view.  “Health” and “Wealth” and “Success” are constructs, too, meaning there are many different measures or indicators that, taken together, can give an understanding of what the construct means.

What would you want “effectiveness” to mean for your organization?  That’s an excellent question to be explored in your next board or staff retreat.  The answers depend almost completely on your mission, the culture and context of your organization, where the program focuses its energy, what you’d like your stakeholders to understand, and the imaginations of your people.  

If your program focuses on reducing infant mortality, use a measure of infant mortality that your program can influence.  If it focuses on increasing high school graduation rates, use measures of high school graduation your program can relate to.  If your program has no intention of increasing high school graduation rates, but instead of helping students imagine their own future and how to get there,  then don’t allow funders to say “You were supposed to increase high school graduation” when you weren’t.  Instead, make your presentation of “effectiveness” flow from your own mission.  Choose measures that relate to what you are actually doing, and own them.  Make “effectiveness” yours.

It’s a nonprofit’s responsibility – and opportunity – to understand and help others appreciate its own  construct of effectiveness in all its grandeur, complexity, and value to the world.   As a nonprofit, you want to present evidence that you’re growing in effectiveness, making progress toward your mission.  Use evidence and language that speaks to your people and your different stakeholders.  You get to tell the story of who benefits from your work and how.  

***

This blogpost was originally published in an earlier form to this website on June 4, 2020.  It was originally written as Chapter 1 of a book entitled How to Show Off Your Wicked Strong Sexy and Incredibly Effective Nonprofit and intended for an audience of nonprofit leaders.

How to cite this blogpost: Mayer, Steven E., Nonprofit Development and Evaluation: The Untold Story of Who Benefits from Nonprofit Programs. Minneapolis: Effective Communities Project. Downloaded from EffectiveCommunities.com [month, date, year]