GiveWell’s research doesn’t end once we’ve made a grant. We evaluate a subset of completed grants, comparing what we thought would happen to what actually took place, then try to use what we learn to improve our future funding decisions. Over the past year, we’ve formalized and expanded this work, publishing comprehensive “lookbacks” for select grants.
A recent lookback on grants GiveWell made to fund insecticide-treated net distributions supported by the Against Malaria Foundation (AMF) in the Democratic Republic of Congo (DRC) illustrates the growing capacity of GiveWell’s research team. We drew on multiple independent data sources, funded qualitative interviews to gather more information, and conducted a novel empirical analysis to deepen our confidence.
In this episode, based on a conversation originally aired on GiveWell’s internal podcast for staff,* GiveWell’s co-founder and CEO Elie Hassenfeld provides additional context while GiveWell’s Chief Research and Program Officer Teryn Mattox dives deep into the details with Program Director Alex Cohen and Researcher Steven Brownstone, examining how we conducted the lookback, what we found, and how what we learned may shape our future nets grantmaking.
Elie, Teryn, Alex, and Steven discuss:
A more expansive and rigorous approach to evaluating past grants: This lookback draws on three independent quantitative sources—AMF’s monitoring data, a recent Demographic and Health Survey (DHS) conducted in the DRC, and an original survey commissioned by GiveWell—alongside qualitative research involving in-depth interviews with people involved in DRC’s net distribution system, from health zone administrators to village focus groups.
Conducting a novel mortality analysis using DHS microdata: Because net campaigns roll out on staggered schedules across DRC’s provinces, we were able to use the timing of children’s births relative to the date of local net campaigns as a natural experiment. We compared mortality risk for children based on when they were born, and thus the length of time they had protection from a net, and found that the net campaigns reduced the risk of death by around a quarter. That finding provides additional support for the mortality effect estimate we use in our cost-effectiveness models.
What qualitative research revealed: Interviewers asked people across five provinces in DRC whether households received nets and whether households were using nets—and in cases where they either didn’t receive nets or weren’t using them, why not. Although we heard some anecdotes of misuse or diversion of nets, the data suggested overall that the nets are highly valued by the communities receiving them.
How durability data could inform campaign design: Our analysis of DHS data confirmed earlier research indicating that nets in DRC degrade before