Can You Quantify Net Good?

Editorial

Dec 19

Can you quantify the net good that a nonprofit creates?

This question is every bit as complicated as it sounds. Recently, Upaya was awarded a five star rating and listed as one of the best poverty alleviation nonprofits by ImpactMatters, a new rating organization. The ImpactMatters team reviewed publicly available information, along with granular data provided by our team about our program expenses and the income change we have documented for Upaya’s jobholders. They use this data in an attempt to quantify the how much impact organizations create with one dollar of donation.

This kind of work is sneaky hard. They readily admit that this is a first effort, and we expect their methodology to keep improving.

According to their assessment, for every $25 donated to Upaya, a jobholder household sees an income improvement of $91. That suggests a very good return on your donation investment, and we are very proud of the outcome.

Some of our colleagues are pushing back on this kind of evaluation, noting that progress on certain social issues are easier to quantify than others, and if we take this too far, we risk choking out organizations with more qualitative benefits like the arts and environmental preservation. That is a valid risk, but we need to attempt to do a better job at rigorously evaluating true impact, i.e., Are we actually making a difference in the problem we have set out to address?

For years, the industry recommendation for “smart philanthropy” was to understand an organization’s overhead rates. The overhead rate is the amount allocated to supporting functions like administration, fundraising, and marketing. Leaving aside the fact that those functions are crucial to a well-run organization, there is no industry standard for determining how to allocate costs to any of them. That means that someone like me, often the same person communicating our overhead rate to the donor, determines a system for allocating overall budget costs based on what feels right to me. So, as an evaluation method, overhead rates tell you nothing.

More importantly, the focus on maintaining “low overhead” leads to chronically under-supported organizations that do not scale their solutions because they do not have the infrastructure beneath them, something well chronicled as the nonprofit starvation cycle. Finally, and most importantly, overhead rates tell us nothing about whether the organization is actually accomplishing anything. A super lean organization may or may not be having an impact. There is no correlation.

So if this is such a worthless metric, why has it held on for so long in nonprofit guides? The short answer is that it’s easy to understand and actual impact is difficult to quantify; it’s almost impossible to quantify in a comparable framework across types of nonprofits. That’s why what ImpactMatters is attempting is really valuable.

Earlier efforts to dig into the question of actual, cost-effective change in a social issue have led to ridiculously complicated evaluation systems that will never scale. I’m looking at you, Give Well. (Of the 1.6 million nonprofits in the United States, does it even pass the smell test that there are only 16 worthy of giving money to?) That does not even address the fact that nonprofits are not commodities. Humans do not give money to them because of a quantitative efficiency test. They give money because of a passion for a cause. Secondarily, they consider how to have the greatest impact possible.

This brings me back to the approach that ImpactMatters is trying: to quantify impact per dollar of donation. It may not be perfect yet, but I have confidence in the motivations of those leading this effort. We as a field have to get better at measuring whether our approaches are working and being honest with ourselves and our donors when the result is not positive.

Donors have a role to play here too. This work is hard, and if there were easy solutions, these problems would no longer exist. Donors can support organizations that experiment, that learn from mistakes, and that invest in transparency. That is the only way that seemingly intractable problems like global poverty can be solved. Upaya is grateful to be recognized as that kind of nonprofit and deeply grateful to all of the donors who allow us to do this work.