top of page
png8 copy.png

OUR RESEARCH PROCESS 2021-2022

This is a summary of our general research process for finding cost-effective interventions. The process inevitably varies across different cause areas that we evaluate. Cause areas we have researched recently using this process are: international development policy (2021), animal welfare (2021), health security (2022) and large-scale global health and development (2022).
 

This process is also fluid and ever changing, we test each stage as we go and adapt accordingly.

OUR GOALS

Our primary goal is to identify the most high-impact intervention opportunities.

Our secondary goals are:

Providing support to future incubatees through high-quality research reports and implementation advice.

Seeking additional wins (supporting other organizations, keeping track of high-impact interventions, identifying potential program participants).

Ongoing improvement of the research process.

OUR DECISION CRITERIA

We determine policy interventions to be particularly high-impact based on the following 5 targets (applied in priority order):

Substantially better than our bar to beat – E.g., for global health policy we set a bar of 5x more cost-effective than AMF and other top GiveWell evaluated charities.

Evidence quality – Evidence should be high quality and robust to uncertainty.

Limiting factors –  There should be minimal ways that a new charity could fail.

Variation – The overall set of opportunities identified should vary in type.

Maximally impactful – Highest expected value.

THE PROCESS

Our research process comprises seven stages, which were preceded by strategic process design. We timecap each stage to produce two-three high-quality recommendations that will be implemented through our yearly Incubation Program.

STAGE 0: PROCESS DESIGN

Goal: Designing the decision-making process for the cause area.
Time spent: ~150 Hours
Implementation:

  • Defining crucial parameters for decision-making

  • Research into available metrics

  • Creating templates for each step and method

  • Establishing a benchmark of recommendation

  • Test run

STAGE 1: TOPIC LEARNING

Goal: Gaining background knowledge

Time spent: ~ 100 Hours

Implementation: 

  • Broad reading and research discussed with the team

  • Cross-applicable research presented to the team

STAGE 2: GENERATING IDEAS

Goal: Creating a list of ~200-350 intervention ideas in our target cause area

Time spent: ~ 50 Hours

Implementation: 

  • Generating ideas based on:

    • Contacting experts

    • Contacting key organizations

    • Broad reading and cross-applicable research

    • CE Team’s input

  • Put ideas into key categories

  • Remove any obviously low-quality ideas

STAGE 3: QUICK PRIORITIZATION

Goal: Narrowing down ideas list to 25% (~60) most promising interventions

Time spent: ~190 Hours

Implementation: 

  • Researching top considerations that might eliminate sets of ideas

  • Organizing ideas (e.g., merging similar ideas)

  • First independent rating, taking 2-12 minutes  (Researcher 1) 

  • Second independent rating, taking 2-12 minutes (Researcher 2)

  • Additional research 

  • ​​Review meeting and final decision-making

STAGE 4: SORTING IDEAS

Goal: Narrowing down ideas list to 25% (~15) most promising interventions

Time spent: ~130 Hours

Implementation: 

  • Assessing strength of evidence for the intervention (30 min. per idea)

    • Looking for evidence (e.g., RCTs, recommendations from effectiveness-focused organizations, macro/country-level data on effects, consensus among experts, existing effective implementations)

    • Assessing quality of evidence

    • Cutting 20% of least promising ideas

  • Identify implementability/execution difficulty and limiting factors (30 min. per idea)

  • Quantitative modeling (30 min. per idea)

    • CEA/BCR estimates 

  • Comparing to the benchmark interventions

STAGE 5: SHALLOW INVESTIGATIONS

Goal: Arriving at ~5-10 top ideas for the most complex research stage

Time spent: ~150 Hours
Implementation: 

  • 5-10 hour research reports on each intervention

  • Expert interviews

  • Identify and assess crucial considerations

  • Look for evidence against intervention

STAGE 6: DEEP DIVE

Goal: Selecting top 2-4 charity ideas that we recommend to be started through our 2022 Incubation Program, by producing high-quality research reports on the top 5-10 ideas.

Time spent: 700-900 Hours

Implementation: 

  • Background reading

  • Mapping a causal chain of the intervention and theories of change

  • Geographic assessment weighted factor model

  • Stakeholder mapping

  • Interviewing experts

  • Quality of evidence assessment (academic evidence, case studies, historical data, macro-level data trends, theoretical evidence, assessing the strength of evidence for each step in the causal chain)

  • Assessing the potential for a new charity (neglectedness, limiting factors, ease of implementability, scalability)

  • Cost-effectiveness analysis

  • Externalities/crucial considerations

  • Expert review

  • Final decision meeting

STAGE 7: IMPLEMENTATION REPORTS

Goal: Creating 2-3 implementation reports on the most promising interventions in order to get new charities up and running right after the program.

Time spent: ~5-30 Hours

Implementation: 

  • Additional research used to inform a successful start and implementation by Incubation Program alumni.

If you are interested in seeing a longer, internal, unpolished version of the process, please contact Morgan at morgan@charityentrepreneurship.com​

bottom of page