How do you evaluate a charity's impact?

One of the tensions in giving significant resources is how to evaluate the charity on the other end.

Before buying a pair of shoes, you can tell quickly whether they fit. You try them on and walk the aisle. With charity there is no equivalent. The beneficiaries may be distant, in geography or in circumstance. They may not be human at all. You may lack the subject matter expertise to judge the work even if you could see it. This is called the stakeholder gap: the person paying and the person served are rarely in the same room.

People bridge the gap in two poor ways. Some lean on tropes like the overhead ratio, a crude measure of how a charity spends money that says nothing about what the money did. Others spend millions on evaluation, or make charities jump through hoops for every dollar. Most donors avoid the question altogether.

We think the answer is in the middle. There is a simple set of universal principles any charity can be evaluated against. Not a checklist, but a guide. Then a few sector-specific questions for the nuances of each kind of work. What follows is the universal part. It is not exhaustive. It is a framework to begin with.

These scale. A large charity answers with formal evaluation frameworks and third-party audits. A small one answers with a page of honest notes and a clear head. Both count. Whether the thinking has been done is the point.

The program: does the logic hold up

1. Is there a clear theory of change?

Theory of change is a fancy phrase for "why we do what we do": the organization's assumptions about how the change it wants actually happens. Not "we help people," but a logic: these activities produce these outputs, which lead to these outcomes, for these reasons. For a small organization, one page will do. If a charity cannot say why its approach should work, that is a problem before you look at a single result.

2. Do they measure outcomes, and ideally impact, not just outputs?

Outputs are "we distributed 10,000 meals." Outcomes are "food insecurity fell among the people we served." Impact is what that made possible: a year on, say, the same people are healthier, more stable and living better lives, and the organization has some way of knowing. Most charities are far better at the first, because it is easier to count and it always looks good.

3. Is there a credible counterfactual, or an honest attempt at one?

Would this have happened anyway. Informal comparison groups or before-and-after tracking beat no comparison at all. Very few small and mid-sized charities do this well, which is fine and expected. The absence should temper how much weight you put on the impact they claim, not disqualify them.

The evidence: can you trust it

4. Do they ask the people they serve?

Beneficiary feedback is perhaps the thing most likely to separate a serious organization from a well-marketed one. Not testimonials, but whether it systematically asks whether the work is landing, and how it improves when it is not.

The organization: is it well run

5. Are the financials transparent and proportionate?

Not the tired question of how low overhead is. Whether the statements are audited, whether spending is roughly proportionate to the stated activity, and whether the numbers hold together across the CRA return, the audited financials and the annual report. Look at three years. Direction tells you more than any snapshot.

6. Does the board provide real oversight, and can the organization change course?

The hardest to see from outside. Ask how often the board meets and what it actually oversees. Ask for evidence of adaptation. A charity reporting the same static metrics year after year, whatever they say, is not running a feedback loop. The world looks different than it did five years ago, and an organization that has kept up can show you how.

7. Do they publish failures, or only wins?

A charity that never mentions anything that did not work is either extraordinarily lucky or not looking closely.

Whenever you are given a number, ask how it compares with peers or the sector norm, and where the comparison comes from. You do not need to know the benchmark. A learning organization knows where its peers are.

As a funder, you are part of the system you are evaluating. Charities do not publish failures because funders punish them for it. They do not measure outcomes because nobody funds measurement. They lose staff because one-year grants produce one-year jobs. If you want honesty, make it safe to be honest. If you want outcomes measured, be willing to pay for the measuring. Most of what looks like a charity's weakness on this page is, somewhere upstream, a funder's habit.

The thing none of the seven measure is how the work is done. Whether people are treated with dignity. Whether the quality is what you would want for your own family. Whether there is any care in the room. Philanthropy means love of humanity, and an organization can satisfy every question on this page and still miss that entirely. Many have. Ask about it, and where you can, go and look.

Questions people also ask

What is the difference between outputs and outcomes?

Outputs are what the charity did, such as meals served or wells drilled. Outcomes are what changed as a result, such as food insecurity falling or waterborne illness dropping. Impact is what that made possible over time.

Should I expect a small charity to have impact evaluation?

No. Formal evaluation is expensive and most small organizations cannot fund it. Expect honesty about that rather than evaluation you cannot afford to demand.

What is the fastest signal of a serious organization?

Whether it publishes something that did not work.

What is the stakeholder gap?

The distance between the person paying for charitable work and the person it serves. In most purchases the buyer and the user are the same person. In philanthropy they almost never are, which is why evaluating a charity takes a method.