At GiveWell, we’re committed to understanding the impact of our grantmaking and improving our decisions over time. That’s why we’ve begun conducting “lookbacks”—reviews of past grants, typically two to three years after making them, that assess how well they’ve met our initial expectations and what we can learn from them.
We conduct lookbacks for two main reasons: accountability and learning. By examining both the successes and challenges of past grants and publishing those findings on our website, we aim to be transparent about the impact of donor funding. Systematically reviewing past grants also helps us identify ways to improve our decision-making. When lookbacks identify challenges, lower-than-expected impact, or key questions that we think we should have an answer to, we use these findings to adjust our approach to similar grants in the future or prioritize follow-up research. When lookbacks show higher-than-expected impact, that’s valuable, too—in those cases, we made the error of underestimating impact and might be granting too little to certain programs.
Lookbacks compare what we thought would happen before making a grant to what we think happened after at least some of the grant’s activities have been completed and we’ve conducted follow-up research. While we can’t know everything about a grant’s true impact, we can learn a lot by talking to grantees and external stakeholders, reviewing program data, and updating our research. We then create a new cost-effectiveness analysis with this updated information and compare it to our original estimates. There’s a lot we can’t know with certainty—like how many lives were saved because of a grant—but we believe these lookbacks teach us a lot about the quality of our past decisions and how to improve.
Lookbacks at two different scales
We recently completed our first two lookbacks: one reviewed all 10 of the grants we have made to New Incentives for childhood vaccination incentives in northern Nigeria (totaling $120 million), and the other reviewed a single $7.5 million grant to Helen Keller Intl for vitamin A supplementation expansion in Nigeria. We reviewed all New Incentives grants together since they implement the same program model with consistent implementation and data collection across the northern Nigerian states where they work. For Helen Keller, we focused on just one expansion grant rather than the entire portfolio we support, which would have been too complex for a single lookback. We selected these grants for our initial lookbacks because enough time had passed to observe meaningful outcomes, the findings could inform upcoming grant decisions, and the grants were large or otherwise important to our portfolio.
These initial
