GiveWell works to continually improve our research so we can make better grant decisions and direct donations where we believe they can do the most good. GiveWell’s Cross-Cutting research subteam is an important part of that effort—it investigates the impact of the programs we fund, works to resolve our largest uncertainties, and aims to improve our research. As GiveWell’s grantmaking grows, the Cross-Cutting subteam is strengthening our data collection and analysis to support that growth.
In this episode, GiveWell co-founder and CEO Elie Hassenfeld speaks with Program Director Alex Cohen, who leads the Cross-Cutting subteam, about how better data can improve our decision-making, our work to gather stronger evidence, and the ways we’ve tested AI as a tool to increase our research capacity and quality.
Elie and Alex discuss:
Triangulating multiple data sources to check impact: Directing donations to programs we believe will do the most good requires understanding the impact of the programs we support. Organizations provide us with monitoring data, but we know there are many ways that, despite the best intentions, this data can be biased or incomplete. To address this, we are investing in additional sources and types of data to improve the accuracy of our estimates. For example, in Burkina Faso we’re funding dried-blood tests to better understand whether caregivers are completing the second and third at-home doses of seasonal malaria chemoprevention (SMC), which provides preventive malaria medication. We think this will improve our understanding of how accurate self-reported data is and whether funding to boost coverage of second and third doses of the medication could cost-effectively help people.
Setting quality standards for survey data: GiveWell often collects survey data at the end of a program to better understand its impact and answer our open questions, but survey data is prone to error. For example, GiveWell researchers shadowed surveyors in Uganda and Nigeria who worked long days under time pressure, creating conditions where hard-to-access households were more likely to be skipped. Because those households may also be the ones programs are most likely to miss, this could lead to overestimating coverage. We’re working to reduce these errors by investing in more independent data checks, and we’ve developed survey guidelines that include practices like GPS tracking and verifying whether a random sample of households was surveyed.
Using AI to critique our work: GiveWell is experimenting with AI tools to increase our research capacity and improve its quality. Our cost-effectiveness models are an important part of our funding decisions, and we carefully vet them to reduce the chance of mistakes. We’ve