From the e-book, How to Show Off Your Wicked Strong Sexy and Incredibly Effective Nonprofit / by Steven E. Mayer, Ph.D. / Chief Evaluation Strategist / Effective Communities Project / (c) 2020
Nonprofit program benefits
Every nonprofit manager, CEO, and board member gets asked, “What are the benefits of your nonprofit programs? Who or what gets these benefits? What nonprofit program benefits can your organization show? How are we supposed to know if your program is effective?” These questions are usually difficult to answer.
We believe a nonprofit is effective when it’s delivering in the ways suggested by the organization’s mission statement – and is able to show it. A good mission statement makes clear just who the organization intends to benefit. It names intended beneficiaries and intended benefits. If a nonprofit delivers on the promise of its mission statement, it can be said to be effective.
For example, one nonprofit’s mission statement might be, “To address hunger in our community, we serve nutritious meals to those who can’t afford them.” This means that poor and hungry individuals in the community are this nonprofit’s primary intended beneficiary. Being effective means making progress in the challenge of ending hunger in your community by serving nutritious meals to those who can’t afford them – and being able to show that you’re making progress. Making progress in the mission is the definition of impact. Said the other way, “having impact” means “making progress in the mission work of the organization.” The nonprofit in this example doesn’t have to solve the hunger problem in its community, but it has to show that it’s making progress in its goal.
Who benefits and how?
“Who benefits and how?” is the meat and potatoes question of nonprofit program development and evaluation. This question may not be on top of everyone’s list. Some want to know if you’ve lived up to the terms of the grant. Some want to know more about cost, and the proportion of budget spent on administration or fundraising. Some are satisfied to know that your organization is growing. Those may be legitimate variations of the theme of effectiveness, but to us they are secondary.
To us, the central focus of effectiveness is brought by the question, “Are you being genuinely useful to your intended beneficiaries in the ways suggested by your mission statement? Can you show the ways you’re making your mission a reality?”
It’s an important question because deep down this is what donors and allies really want to know. They want to know they’re contributing not just to a good cause, but to an organization that’s delivering.
We’re not saying you must show you’ve solved the community’s hunger problem, but without evidence that you’re delivering on the challenge of serving nutritious meals to poor and hungry people, the organization is banking simply on a promise and the good will of people who care. With evidence, it’s more likely you’ll gain support to build your nonprofit’s organizational capacity to make even more progress.
In short, a program is effective when it can show it’s making progress on its mission. And just as there’s lots of nuance and wriggle room in the way the mission is stated, there’s also nuance and wriggle room in the way a nonprofit can show this progress. But if this nonprofit is unable to show even a lick of progress, it can’t be considered effective.
Some might say this is a dodge, an escape from demands of “real proof you’re ending hunger.” But our vision of nonprofit effectiveness respects mission and the ongoing cumulative work required to achieve it. Every bit of progress is accumulated impact. We want to make improved effectiveness, gains in progress, increased impact — all synonyms in this book — the goal of nonprofit development and evaluation. Rather than seek the scientist’s version of proof or the investor’s version of bottom line return on investment, we prefer to seek evidence that progress is indeed being made in delivering intended benefits to intended beneficiaries.
We’ll have more to say about nonprofit missions and evaluating for effectiveness in Chapter 2 – Dirty Little Secrets of Mission Effectiveness. And more about evidence in Chapter 5 – Sexy Evaluation Strategies.
Beneficiaries and benefits: Who gets the Love, Money, and Power?
OK, that heading was written to feed the bots that scour the internet looking for buzzy keywords. To bring some clarity to the concept of who benefits and how, we offer this simple breakdown of four major groups of intended beneficiaries of nonprofit activity. We see four categories of program beneficiary: individual, system, community, and organizaton.
Nonprofit program benefits can be at the individual level
Working to benefit individuals is the core of the charitable sector’s work, at least historically. Delivering services to address the basic human needs of individuals is the traditional work of “human service” or “social service” agencies. A “hierarchy of needs” famously described by Abraham Maslow parallels the gamut of charitable programs developed in the last 150 years.
Maslow’s hierarchy starts at the most basic and necessary level, the physiological needs for survival; this includes the need for food, water, air, warmth, clothing, rest. The hierarchy progresses to needs for safety, security, stability, housing, work, some measure of financial security. The US is particularly rich with nonprofit and public programs that serve individuals in these areas.
Higher in Maslow’s hierarchy are the love and belonging needs for acceptance, belonging, friendship, affection, support, love. Higher still are the self–esteem needs for skills, knowledge, confidence, direction, accomplishment, dignity. And highest is the need for self–actualization, including the need for meaning, purpose, fulfillment, creativity, dignity, achieving one’s full potential. Thousands of nonprofits and millions of self–help books, websites, and commercial services are designed to address these needs.
Intended benefits at the individual level
The intention is that there is indeed improvement in the quality of lives of individuals in ways suggested by the mission of the program – essentially in a reduction of urgent need and an increase in prolonged vitality.
- In programs with a mission to address physiological needs for survival, the intended benefits are a reduction in risk of imminent misery or death – and greater chance of comfort, health, and survival.
- In programs with a mission to address needs for safety, the intended benefits are reductions in fear, instability, joblessness, homelessness, and poverty – and increases in comfort, stability, productive work, satisfactory shelter, and some measure of financial security, and improved prospects of surviving and thriving.
- In programs with a mission to address higher–order needs, the intended benefits are reductions in isolation or feelings of worthlessness and unhappiness – and increases in purpose, dignity, and meaning.
Nonprofit program benefits can be at the system level
Societies since forever have developed systems that are geared to addressing the same needs that traditional charities address at the individual level – but they do it “systemically.” Think of the food distribution system, the clean water distribution system, the educational system, health care delivery system, criminal justice system, wealth building system, electoral system – just to name a few. that we’re all a part of even though we can’t see them in their entirety.
Systems operate at all levels, from local to international, typically with different mandates and authorities at each level. They are formal, with policies, and informal, with habits and values. They are designed and operated by humans, and they therefore operate with flaws (some serious). They therefore can all be improved.
It is the work of many nonprofits to improve the performance of these systems so that they work better for more people. Activists in the system change world often speak of “moving the needle,” referring to a metric (such as the one on your car’s dashboard showing level of fuel in your tank) allegedly showing the performance of a mission–critical part of the overall system. With social systems as complex and out–of–centralized–control as they are, the challenge is to identify the mission–critical parts of the overall system, and shape and improve the just–right metrics once we’ve agreed on the goals we have for these systems.
Intended benefits at the system level
Working on systems ideally allows the relevant performance needles or metrics to move to ever more positive territory. The work of system improvement (aka reform or transformation) typically intends progress in…
- Building ever more capable organizations with supportive cultures and allies that want to make progress.
- Developing leadership, relationships built on trust, creating networks, building clout, gaining access to greater authority.
- Advancing promising solutions, from idea to testing, from upgrading to scaling up, with protocols for training, monitoring, and accountability.
- Accumulating a variety of resources – money, information, technology, blueprints, wisdom – and using them smartly
Nonprofit program benefits can be at the community level
The needs documented by Abraham Maslow that are addressed in programs serving individuals are also addressed in programs serving communities. And while the category of Communities is a diverse set, the similarity is that programming is directed to them as a group – a concert, a curriculum, a conference, etc. A community can be defined by:
- Demography, ethnicity or cultural group, such as youth, Italo–Americans, the elderly, or classical music lovers.
- Geography, such as a neighborhood or watershed, or any political jurisdiction like county, state, or nation.
- Audiences, such as those of arts and culture organizations, intended audiences of public health messages, or those viewing your organization’s website.
- Affinity, such as those experiencing a community–wide condition or experience, such as victims of any disease entity or man–made disaster, or the Class of 2001, everyone who’s seen Inglorious Basterds, or members of the state association of state associations.
Intended benefits at the community level
The intended benefits of community–serving programs are similar to programs serving individuals. Progress is typically noted as a mission–relevant success rate for the group or community being targeted. Common examples could include satisfaction rate among audience members, income level in a neighborhood, mortality rates, wealth rates, etc. Policy organizations in many arenas have been advancing measures of a community’s “quality of life” for many years now.
Nonprofit program benefits can be at the organization level
Working to benefit organizations – nonprofit organizations in particular – is the work of grantmaking foundations, individual donors, private investors, and networks of like–minded organizations. While these entities may well hope that individuals and communities will ultimately benefit, the primary and most immediate beneficiary is the recipient organization.
Intended benefits at the organization level
Recipient organizations can benefit by becoming stronger and more capable in carrying out their missions. These gains in “organizational capacity” can include:
- Upgrades to administration including improved board governance, management, policies and procedures, staffing, budgeting;
- Upgrades to resource development including improved communications, fundraising and endowment building, information and technology, infrastructure;
- Upgrades to community linkages including strengthened connections with other key organizations and networks, strategic partnerships, and clarified role in the community; and
- Upgrades to programming to become a stronger learning organization, with stronger connection between understanding consequences of program activity and strengthening the program to create more better benefits to beneficiaries.
For more on organizational capacity click here and on whether foundations or nonprofits should get credit for ending hunger and such, click here (later.)
These four categories present the four essential intended beneficiaries of nonprofit programs – and the essential benefits intended for each. These intentions express hopes and dreams, as well as blood, sweat, and tears. Many things, especially life itself, can get in the way and spoil the effort. Resources are short and staff and volunteers are overworked. The problems created by contemporary society seem to be accelerating, and the struggle for making progress on missions seems increasingly uphill. If progress is to be made, progress has to be funded.
If good things happen but no one notices, does it count?
If good things happen but there’s no one there to notice, do they count? If your program delivers benefits but no one can tell, can you still get credit for them? Answer: Probably not. There are several things to consider in your pursuit of showing off your wicked strong sexy and incredibly effective nonprofit — the title of this book! — by telling the story of your progress toward your mission.
Unnoticed, unclaimed, unloved
In truth, most nonprofits have multiple beneficiaries, and for each there are multiple intended benefits – often unstated and unrealized. Even though they typically get paid to deliver for their primary intended beneficiary only, most nonprofits impact other beneficiaries as well. But unless a nonprofit can track what’s happening with other categories of beneficiaries these benefits go unnoticed, unclaimed, uncommunicated, unloved, and unfunded.
For example, a soup kitchen may intend primarily to serve nutritious meals to individuals, but it might also participate in advocacy efforts to improve the food distribution system, partner with and assist other organizations in their efforts, or provide nutrition education to segments of the community. It can easily happen that the nonprofit is paid for work with its primary beneficiary – individuals receiving meals – but not paid for the other work, for which it’s forced to volunteer. I’ll bet it explains a lot of the burnout we see in the sector.
There’s a potential success story to tell about each of your intended beneficiaries. And there are potential upgrades to the design of your programs to deepen your impact with all of them. So when a nonprofit serves multiple beneficiaries it should try to take credit for all it does, and to muster evidence to back it up. Too often, nonprofits take credit for too little, and lose out on opportunities to upgrade and move forward with additional support.
It’s commonplace that funders expect a nonprofit to reach beyond its grasp into areas where it has no control. For example, an alternative school will be asked to prove that its graduates go on to enter esteemed professions later in life – proof that even Harvard can’t provide. Or a chamber orchestra will be asked to prove that audience members go onto become lifelong lovers of their music. Or an organization supplying medicines to displaced persons along the border will be asked to show it is stemming the flow of refugees. As underfunded as most nonprofits are, more realistic demands for evidence should be the basis of more productive discussions.
Just saying “We save lives” or “We develop leaders for tomorrow” or “We benefit the homeless” doesn’t make it so. You can state your intentions to save lives, develop leaders or benefit the homeless, but your stakeholders need you to show evidence that you’re working on those intentions and making progress. This doesn’t strike us as an unreasonable demand. It’s showing “signs of progress,” or “early signs of impact” that’s important.
So for example, if you’re serving meals, can you show (perhaps in addition to number of meals served), progress in trying out new sources of nutritious meals, partnering with others to extend food supply into communities that could be called food deserts, or strengthening the board of your organization so it can better help you make these connections?
A jury of your peers
An exercise to expand your sense of what’s possible to response to demands for “proof your program works.” Pretend your Executive Director is hauled before an “Evaluation Court” and charged with “operating an ineffective program.” To avoid conviction, imagine the kinds of evidence that could be mustered in defense, and ideally show the opposite – that in fact this program really is effective? In this exercise in make believe, you’re encouraged to think more broadly about effectiveness and evidence. What sort of “witnesses” could offer “testimony” that “makes your case” for this program’s effectiveness? Pretend further that this case must persuade a “jury of your peers and stakeholders.” Maybe there’s even opportunity for cross–examination. Pretend you have a year before going to trial. That should give you time to gather witnesses, prepare testimony, and develop a convincing case.
OK, it’s not the stuff of controlled laboratory experiments. We’re not talking about randomized clinical trials with control and experimental groups. But it is empirically sound, and empiricism is the root of all science and reason. All we’re doing here is substituting the rules of a courtroom for the rules of a science experiment. With this courtroom approach we can have multiple arguments, witness after witness, measure after measure. We can have multiple voices from multiple points of view. The conditions needed for randomized field trials with control and experimental groups do not typically exist in the real world. Creating such conditions is expensive, administratively complex, ethically fraught, and in the end probably not replicable. Yet they are considered the gold standard in studiously academic circles. And since many on a nonprofit’s board and its funders’ boards have taken at least a few social science courses, the social science model prevails. We encourage the social sciences to go back to their roots: empiricism.
Show me = Empiricism
“Show me” is the heart of empiricism, which in turn is the heart of the scientific tradition. In olden times, as historians would tell us, when testing whether a new method of planting corn, for example, was better than traditional practice, they’d try the new method next to the old one and invite all the nearby farmers to see how it was going. Empiricism is as old as the hills, as old as farming. If license plates can be believed, Missouri is “The Show Me State.”
So flash forward to the case in hand: “Show us all the signs of your progress – let’s see what you’ve got.” For example,
- You could find experts that can speak to the merits of your program design. Are there people from National who can testify you follow Good Practices?
- You could count the number of people participating in your programs.
- You could interview people who’ve participated, and ask them about their experience. They could give you an informative narrative. They could also express things on a 5–point scale indicating how much safer or less hungry or more secure they now are.
- You could present an album with photos of significant events, important documents, or letters of appreciation.
- You could put together some cool graphics from data already collected.
- You could make comparisons for any of the above against the evidence from last year, or against the industry average, or against your standing from the beginning.
The word “measurement” can be intimidating. But measurement is nothing more than making comparisons and drawing meaning from them. That’s it. The use of numbers is not inherent to measurement. We “take the measure” of a person or situation all the time. We size them up, we take a reading, we reckon, we judge qualities, we rank, we rate, we compare, we gain an understanding. As humans we measure and evaluate all the time. We invite the neighbors over to take a look and judge for themselves. Measuring program benefits should be a human activity. It’s familiar and meaningful, and we all do it. “Taking measures” can be an engaging activity, jointly owned, productive and useful. Just talking with your colleagues about what you could measure and how is a big step forward for many organizations.
What kind of comparisons could you make? The state of your organization’s ability to produce program benefits today compared to last year. The number of people who experience X versus those who experience Y. The number of good things you can say when participants leave compared with the number of bad things (maybe from an acceptable list of signs of success). Over time you’ll learn to better lift up what’s important, getting better at observing, noting, compiling and presenting evidence to show your organization’s progress towards mission.
Then comes the E–word, Evaluation
See the word “value” in “evaluation?” Evaluation is the process of making meaning from your measurements. Does your table of numbers, or a photo collection, or a graph reveal the value of your program?
Data do not speak for themselves. And without belaboring the obvious, data don’t speak at all. Meaning has to be made from data, and that’s a job for humans. We guarantee no two people will draw the same conclusions, insights, implications, or recommendations from even one table or graph. Own your data and tell your stakeholders what they mean, and how they speak to the value of your efforts.
One thing about data, no matter the source or their alleged precision, it’s not as objective as purported. Deciding what to count, compare, photograph, make notes of etc are all human choices, and human interpretation of the collection of data is required, even though numbers-oriented people like to deny this.
Goal: Improve, more than prove. We believe that the point of evaluation is to “improve rather than prove” the value of work done by nonprofits. The notions of “outcomes,” or “results,” or “impact” are limiting – they sound final, as if denying the continuing stream of behavior and events that happen in the real, uncontrolled and unstandardized world. At the very least, we could encourage “outcomes to date,” or “results so far,” or “early signs of impact.” Evaluations done with an Improve not Prove mentality should dig up lots of opportunities for improvement, upgrading, development – all to be incorporated into the next funding requests.
“Effectiveness” is a construct – make it work for you.
Effectiveness is a “construct.” A construct is an abstraction, not directly observable, but understood from a variety of different signs or markers taken from different points of view. “Health” and “Wealth” and “Success” are constructs, too, meaning there are many different measures that, taken together, can give an understanding of what the construct means.
What would you want “effectiveness” or success look like for your organization? That’s an excellent question worthy of a board or staff retreat. The answers depend almost completely on your mission, the culture and context of your organization, and what you’d like your stakeholders to understand. If your program focuses on reducing infant mortality, use a measure of infant mortality that your program can influence. If it focuses on increasing high school graduation rates, use measures of high school graduation your program can relate to. If your program has no intention of increase high school graduation rates, then don’t claim you can and don’t allow funders to say “You were supposed to increase high school graduation” when you weren’t. Instead, make your presentation of “effectiveness” flow from your own mission. Choose measures that relate to what you are actually doing, and own them. Make “effectiveness” yours.
It’s a nonprofit’s responsibility – and opportunity – to unpack Effectiveness in all its grandeur, complexity, and value to the world. As a nonprofit, you want to present evidence of progress toward your mission. Use language that speaks to you and to your different stakeholders. Make sure the story of who benefits gets told.
Revised June 4, 2020
(c) Effective Communities LLC