Tag Archives: Cost-effectiveness analysis

How we work, #3: Our analyses involve judgment calls

This post is the third in a multi-part series, covering how GiveWell works and what we fund. Through these posts, we hope to give a better understanding of our research and decision-making.

How we work, #1: Cost-effectiveness is generally the most important factor in our recommendations
How we work, #2: We look at specific opportunities, not just general interventions

Our goal is to recommend funding to the programs we believe have the greatest impact per dollar donated. There’s no simple algorithm for this question. Answering it necessarily involves making judgment calls. Our first post in this series discussed the importance of cost-effectiveness analyses and the many factors we consider; in this post, we’ll share:

How we make subjective choices in the face of imperfect information
Some examples of judgment calls that illustrate our approach:

Combining data and intuition: Estimating the effect of water chlorination on mortality
Valuing disparate outcomes: Comparing clubfoot treatment to life-saving programs
Anticipating the likely decisions of other actors: Predicting the impact of technical assistance for syphilis screening and treatment

Making decisions with imperfect evidence
Our work relies heavily on evidence, but the available evidence never answers a question with certainty.
Academic literature and its limitations
We don’t take the results of any given study at face value.[1] Instead, we often make adjustments along the way to come to a final estimate. As part of that process, we might consider:

The methodological limitations of the available studies
The likelihood of publication bias or spurious results
Whether the study results are likely to represent the impact of the specific program we’re considering funding, which requires looking at potential differences in contexts and in the programs being implemented
How plausible the results seem when considering other relevant information, including whether there’s a known mechanism by which a program might have a certain effect
The opinions of expert advisors
Other factors not listed here

Some questions can’t easily be addressed by studies but are still important for assessing the impact of a program. Those include topics like:

Will another funder support this program if we don’t?
Will this program be successfully transitioned to the government?
How likely is it that new research will provide information that changes our minds two or three years from now?
How bad is the experience of having disease A compared to the experience of having disease B?

Considering multiple perspectives
Some donors and other experts might reasonably disagree with our

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

GiveWell’s 2023 recommendations to donors

We’re excited about the impact donors can have by supporting our All Grants Fund and our Top Charities Fund. For donors who want to support the programs we’re most confident in, we recommend the Top Charities Fund, which is allocated among our four top charities. For donors with a higher degree of trust in GiveWell and willingness to take on more risk, our top recommendation is the All Grants Fund, which goes to a wider range of opportunities and may have higher impact per dollar. Read more about the options for giving below. We estimate that donations to the programs we recommend can save a life for roughly $5,000 on average,[1] or have similarly strong impact by increasing incomes or preventing suffering.

Why your support matters
We expect to find more outstanding giving opportunities than we can fully fund unless our community of supporters substantially increases its giving. Figures like $5,000 per life saved are rough estimates; while we spend thousands of hours on our cost-effectiveness analyses, they’re still inherently uncertain. But the bottom line is that we think donors have the opportunity to do a huge amount of good by supporting the programs we recommend.
For a concrete sense of what a donation can do, let’s focus briefly on seasonal malaria chemoprevention (SMC), which involves distributing preventive medication to young children. We’ve directed funding to Malaria Consortium to implement SMC in several countries, including Burkina Faso.[2]
In Burkina Faso, community health promoters go from household to household across the country, every month during the rainy season (when malaria is most common). They give medicine to each child under the age of five, which involves mixing a medicated tablet into water and then spoon-feeding the medicine to infants and having young children drink it from a cup. They also give caregivers instructions to give additional preventive medicine over the next two days.
It costs roughly $6 to reach a child with a full season’s worth of SMC (though this figure doesn’t account for fungibility, which pushes our estimate of overall cost-effectiveness downward).[3] If a child receives a full course of SMC, we estimate that they’re about five times less likely to get malaria during the rainy season (which is when roughly 70% of cases occur).
Community distributor providing SMC medication to a child sitting on mother’s

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

How we work, #2: We look at specific opportunities, not just general interventions

This post is the second in a multi-part series, covering how GiveWell works and what we fund. The first post, on cost-effectiveness, is here. Through these posts, we hope to give a better understanding of our research and decision-making.
Looking forward, not just backward
When we consider recommending funding, we don’t just want to know whether a program has generally been cost-effective in the past—we want to know how additional funding would be used.
People sometimes think of GiveWell as recommending entire programs or organizations. This was more accurate in GiveWell’s early days, but now we tend to narrow in on specific opportunities. Rather than asking whether it is cost-effective to deliver long-lasting insecticide-treated nets in general, we ask more specific questions, such as whether it is cost-effective to fund net distributions in 2023 in the Nigerian states of Benue, Plateau, and Zamfara, given the local burden of malaria and the costs of delivering nets in those states.
Geographic factors affecting cost-effectiveness
The same program can vary widely in cost-effectiveness across locations. The burden of a disease in a particular place is often a key factor in determining overall cost-effectiveness. All else equal, it’s much more impactful to deliver vitamin A supplements in areas with high rates of vitamin A deficiency than in areas where almost everyone consumes sufficient vitamin A as part of their diet. Similarly, one of our top charities, New Incentives, has chosen to operate in northern Nigeria largely because relatively low baseline vaccination rates mean its work is especially impactful there.[1]
As another example, we estimate it costs roughly the same amount for the Against Malaria Foundation to deliver an insecticide-treated net in Chad as it does in Guinea (about $4 in both locations). But, we estimate that malaria-attributable deaths of young children in the absence of nets would be roughly 5 times higher in Guinea than in Chad (roughly 8.8 deaths per 1,000 per year versus roughly 1.7 per 1,000), which leads AMF’s program to be much more cost-effective in Guinea. Overall, we estimate that AMF’s program is around 27x cash in Guinea and around 5x cash in Chad.[2]
This map from Our World in Data gives a sense of how deaths from malaria vary worldwide.[3]

Because cost-effectiveness varies with geography, we ask questions specific to the countries or regions where a program

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

How we work, #1: Cost-effectiveness is generally the most important factor in our recommendations

This post is the first in a multi-part series, covering how GiveWell works and what we fund. We’ll add links to the later posts here as they’re published. Through these posts, we hope to give a better understanding of our research and decision-making.
Why cost-effectiveness matters
The core question we try to answer in our research is: How much good can you do by giving money to a certain program?
Consider how much good your donation could do if you give to a program that costs $50,000 to save a life versus one that costs $5,000 to save a life (which is roughly what we estimate for our top charities). Giving to the latter would have 10 times more impact. While in an ideal world both programs would receive funding, we focus on identifying the most cost-effective programs so that the limited amount of funding available can make the greatest difference.
The basics
We’ve written in detail here about our approach to cost-effectiveness analysis and its limitations. Our bottom-line estimates are always uncertain, and we don’t expect them to be literally true. At the same time, they help us compare programs to each other so that we can direct funding where we believe it will have the greatest impact.
At a very high level, assessing cost-effectiveness generally involves looking at:

The cost per person reached. For example, how much does it cost to treat one child with vitamin A supplementation for one year?
The outcomes of the program. Determining these outcomes often involves examining two factors:

The overall burden of a problem. For instance, how many kids who will be reached with vitamin A supplementation would otherwise have died?[1]
The effect the program has. For example, how much does vitamin A supplementation reduce mortality rates relative to that baseline, and are there other benefits to providing vitamin A?

We use unconditional cash transfers as a benchmark for comparing opportunities, such that a program is estimated to be “12x cash” if we believe it’s 12 times more impactful per dollar than giving money directly to people living in poverty. In other words, if we estimate that a program is 12x cash, we think donating $100 to that program does as much good as donating $1,200 to a program that delivers unconditional cash transfers.
More detail
We aim to come to an all-things-considered view that

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

The winners of the Change Our Mind Contest—and some reflections

In September, we announced the Change Our Mind Contest for critiques of our cost-effectiveness analyses. Today, we’re excited to announce the winners!
We’re very grateful that so many people engaged deeply with our work. This contest was GiveWell’s most successful effort so far to solicit external criticism from the public, and it wouldn’t have been possible without the participation of people who share our goal of allocating funding to cost-effective programs.
Overall, we received 49 entries engaging with our prompts. We were very happy with the quality of entries we received—their authors brought a great deal of thought and expertise to engaging with our cost-effectiveness analyses.
Because we were impressed by the quality of entries, we’ve decided to award two first-place prizes and eight honorable mentions. (We stated in September that we would give a minimum of one first-place, one runner-up, and one honorable mention prize.) We also awarded $20,000 to the piece of criticism that inspired this contest.
Winners are listed below, followed by our reflections on this contest and responses to the winners.
The prize-winners
Given the overall quality of the entries we received, selecting a set of winners required a lot of deliberation.
We’re still in the process of determining which critiques to incorporate into our cost-effectiveness analyses and to what extent they’ll change the bottom line; we don’t agree with all the critiques in the first-place and honorable mention entries, but each prize-winner raised issues that we believe were worth considering. In several cases, we plan to further investigate the questions raised by these entries.
Within categories, the winners are listed alphabetically by the last name of the author who submitted the entry.
First-place prizes – $20,000 each1Both of these entries were outstanding, and they represent very different approaches. Because they are similarly excellent, we are naming two winners rather than one winner and one runner-up. jQuery(‘#footnote_plugin_tooltip_14065_1_1’).tooltip({ tip: ‘#footnote_plugin_tooltip_text_14065_1_1’, tipClass: ‘footnote_tooltip’, effect: ‘fade’, predelay: 0, fadeInSpeed: 200, delay: 400, fadeOutSpeed: 200, position: ‘top right’, relative: true, offset: [10, 10], });

Noah Haber for “GiveWell’s Uncertainty Problem.” The author argues that without properly accounting for uncertainty, GiveWell is likely to allocate its portfolio of funding suboptimally, and proposes methods for addressing uncertainty.
Matthew Romer and Paul Romer Present for “An Examination of GiveWell’s Water Quality Intervention Cost-Effectiveness Analysis.” The authors suggest several changes to GiveWell’s analysis of water

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

Announcing the Change Our Mind Contest for critiques of our cost-effectiveness analyses

We’re extremely excited to be announcing the Change Our Mind Contest to encourage critiques of our cost-effectiveness analyses that could lead to substantial improvements of our overall allocation of funds. For all the details, see this page.
Cost-effectiveness is the single most important input in our decisions about what programs to recommend, and we believe it’s possible that we’re missing important considerations or making mistakes that lead us to allocate funding suboptimally. We’ve been excited to see people engaging with our cost-effectiveness analyses, and we’d like to inspire more of that engagement.
With that in mind, we’re inviting you to identify potentially important mistakes or weaknesses in our existing cost-effectiveness analyses and tell us about them!
The first-place winning entry will receive $20,000, the runner-up will receive $10,000, and the honorable mention will receive $5,000. We may offer multiple runner-up and honorable mention prizes if the quality of submissions is particularly high. All other entries that meet our criteria will receive a participation prize of $500, capped at a total of 50 participation prizes for the first 50 submissions.
In addition to the monetary prizes, excellent entries may lead to changes in how we allocate millions of dollars of funding, leading to more lives saved or improved.
Entries must be received by October 31, 2022, and the requirements are described in detail on the contest page. We will announce winners by December 15, 2022, and will publish the winning entries online.
If you have any questions, please leave a comment on this post or email change-our-mind@givewell.org.
We’re running this contest because the recommendation decisions we make are extremely important, and we want to incentivize feedback that will improve our work, thereby enabling us to do more good. We hope you’ll consider participating!
The post Announcing the Change Our Mind Contest for critiques of our cost-effectiveness analyses appeared first on The GiveWell Blog.

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>

An update on GiveWell’s funding projections

As little as six months ago, we were in the position of having more funding available than we could spend on opportunities that met our very high cost-effectiveness bar. Today, the opposite is true—we don’t expect to have enough funding to support all the cost-effective opportunities we find.
In this post we will:

provide an update on GiveWell’s projected funding position,
explain how we have been successful in identifying cost-effective opportunities, and
share our initial thoughts about what this update means for GiveWell’s forward-looking grantmaking strategy.

The state of funding
We wrote last year that we would roll over approximately $110 million in funding from 2021 to spend this year. We ultimately rolled over substantially less because we were imprecise in calculating our projected funds in and out (more details available on our mistakes page). But at a high level, it remained true that we received more money than we chose to spend on highly cost-effective funding opportunities.
We expected to be in a similar position this year, rolling over approximately $110 million. We now believe that we will be funding constrained. There are two core reasons for this:

We found a lot more cost-effective opportunities that need funding. Based on our current research pipeline, we think we’ll be able to recommend up to approximately $750 million in grants that are at least 6x as cost-effective as cash transfers.1We compare charities (and funding opportunities within them) using multiples of our estimate for the impact of directly transferring cash to beneficiaries. For example, we describe an opportunity as “6x cash” to indicate that we think it’s six times as cost-effective as giving that amount in cash directly to the beneficiary. There’s an intuitive case for asking whether a program is better than what beneficiaries would buy for themselves using cash. If not, wouldn’t it be better to just give them cash instead? Any program we consider must exceed this standard and be multiple times better than cash in order for us to recommend it. jQuery(‘#footnote_plugin_tooltip_13763_1_1’).tooltip({ tip: ‘#footnote_plugin_tooltip_text_13763_1_1’, tipClass: ‘footnote_tooltip’, effect: ‘fade’, predelay: 0, fadeInSpeed: 200, delay: 400, fadeOutSpeed: 200, position: ‘top right’, relative: true, offset: [10, 10], }); Last year, we identified about $500 million in grants, most of which were at least 8x.
We think we will receive less money than we projected due to recent

Givewell Blog | http://feeds.feedburner.com/TheGivewellBlog

Goto full post >>