GiveWell aims to find and fund programs that will do the most good per dollar. To do this, we carefully evaluate potential grants before making them—assessing academic evidence, building cost-effectiveness models, and talking to people in the sector who know the program well.
But our work doesn’t stop there. When a program we’ve supported nears the end of their funding, we also regularly evaluate its results to decide whether to continue our support. This typically involves gathering and analyzing extensive monitoring data. In most cases, the results are consistent with what we expected, and we renew the programs’ support. But sometimes we decide that, even if a program is doing a lot of good, it may not be having the impact we expected. In that case, we decide not to renew our support and instead direct those funds to where we think they’ll do much more good for people in need.
In this episode, GiveWell CEO and co-founder Elie Hassenfeld speaks with Senior Program Officer Erin Crossett about the research that led GiveWell not to renew support for Evidence Action’s Dispensers for Safe Water—a program that installs chlorine dispensers at rural water points so that households can treat their drinking water and reduce waterborne disease—in Malawi and Uganda.
Elie and Erin discuss:
How independent data revealed a significant gap in program reach: Early signals from a separate GiveWell-funded study and Evidence Action’s own internal review of the program in Kenya suggested chlorination rates were far lower than routine monitoring indicated. GiveWell then commissioned an independent survey in Uganda and Malawi to find out whether the same was true there. The survey found that only about a third as many people were using dispensers as previously estimated: roughly 2 million rather than 5 million.
Potential reasons for the data discrepancy: We believe that no single error drove the discrepancy. Instead, there were five or six contributing issues that together caused the differences in estimated usage. For example, chlorine was measured by matching a test result to a color wheel, which can be subjective and affected by lighting. This data was collected by Evidence Action’s own staff, which may have led them to interpret the color wheel results more favorably.
What GiveWell learned and its effect on future grantmaking: We believe that approving the initial grant in 2022 was the right decision, given what we knew at the time. We also think that we could have done better by, for example, investing earlier in independent verification. We now apply these lessons to our current grantmaking. For