Studies Explore Taking Risks to Innovate and Increase Impact Through Cash Benchmarking

At USAID, we take smart risks to test new ideas and harness innovative approaches to accelerate development. However, we do not do this alone. Our partners also take on these calculated risks, and often this risk-sharing can lead to successful collaborations. In one recent example of this type of collaboration, USAID and its partners recently experimented with cash benchmarking—a new, rigorous research design methodology—to better understand the cost-effectiveness of its investments.  

There are currently five cash benchmarking studies underway, each with a different research design. The studies examine how unconditional cash grant programs (money provided directly to recipients without constraints) impact development indicators targeted by USAID programs. Each study provides an amount of cash similar to the cost of delivering programs to typical beneficiaries in order to see the impacts of that transfer on the individuals. The most recent paper by the Center for Effective Global Action at UC-Berkeley presents the midline results of a 36-month study that explores how cash grants compare directly to a youth workforce development program in Rwanda, on a cost per beneficiary basis.

USAID and several partners co-created the research design and methodology and established a randomized controlled trial (RCT) to better understand the impact on beneficiaries. Our partners identified youth in Rwanda interested in joining a workforce development program and assigned them into three groups. The first group was a control group that received no development assistance. The second group was placed in the Huguka Dukore Akazi Kanoze (HDAK) workforce development program, and the third group received unconditional cash grants of various amounts, ranging from about $317 to $750.  A final group received both HDAK and a cash grant.

This approach is fairly new and experimental within international development, and we commend all partners involved who took on this endeavor. While we await the final results of the 36-month study, we spoke to our partners, Education Development Center (EDC), the youth workforce program implementer, and GiveDirectly (GD), the cash grant implementer, on the study thus far. We wanted to gain their perspective on the risks they took, why taking smart risks is necessary, and what they hoped to gain from this study. 

Why Our Partners Take Risks

Tim Haskell, the Project Director for HDAK at EDC stated that EDC signed up for the study because “at its heart, EDC is a research organization” and they were “interested to see what [they] could learn from the study to improve outcomes for youth.” EDC is constantly evaluating their programs, not only in workforce development but also in health and education. He cautioned that “one of the risks [of participation] is that too many conclusions will be drawn from a single midline report.” He remarked that EDC knows “from experience that these [youth livelihoods] programs take a while to develop roots and produce gains. You have to train youth, get the private sector to buy-in, and then create links between those two groups.”  

However, Michael Cooke, the Research Director at GD, had a different initial attraction to this research. The team at GD was intrigued by the idea of benchmarking and “...a simple, powerful question.” Specifically, “can a program do more good for people in poverty than those people could do for themselves with the program’s costs?” Cooke stated that GD was “...excited by USAID’s pioneering use of cash benchmarking to better understand—and ultimately improve—the cost-effectiveness of its programs.”  

Emerging Findings 

While Cooke emphasized his excitement for this research, he also highlighted that logistics presented a significant challenge. Managing an RCT of a single intervention is often a complex task with multiple partners spread across multiple organizations. GD has taken part in a large number of RCTs, which are already challenging for a single implementer. He said, “...integrating two organizations’ operations into an RCT is harder still, making communications to the youth involved more complex, and increasing the risk of project delays.” He noted that this study was possible because EDC, GD, and the researchers were flexible when solving challenges that arose.

The study is currently at its half-way point, and the results are mixed. Neither intervention improved the overall employment levels of Rwandan youth at this point in time, but the HDAK workforce development program improved productive hours, assets, savings, subjective well-being, and business knowledge. Cost-equivalent cash grants improved productive hours, assets, savings, subjective well-being, plus consumption, income, and wealth. In a direct costing comparison, cash is more effective across a number of economic outcomes, while HDAK outperforms cash in the production of business knowledge (for example bookkeeping, financial literacy).

Looking forward to the study’s completion and the final 36-month results, both EDC and GD said they ultimately hope the research helps everyone better understand how to help unemployed youth achieve better development outcomes.  

Worth The Risk: What Can We Learn From The Research 

Cooke says GD is interested in the long-term impacts of cash grants and “whether the substantial gains in livelihood outcomes that the study documents at 18 months…persist at 36 months.” He said, “some studies find that cash transfer gains persist for a number of years, while others find that outcomes for the control group eventually catch up (e.g. 4 to 9 years into a study in Uganda).”  

Haskell at EDC agrees, but also thinks COVID-19 will complicate what is learned. He remarked, “A few months ago, I would have said that we expected to see the HDAK  results continue to get better over time—that with more time to apply their skills, youth would continue to see gains even against youth who received cash. Obviously, the pandemic has had a significant economic impact on youth in Rwanda, and it’s difficult to predict the future now.”  

Regarding COVID-19, Cooke stated, “Based on rapidly accumulating evidence based on the economic impacts of COVID-19 across the world, we expect youth livelihoods in Rwanda to have been meaningfully affected by the pandemic in all study arms. We may see that those who have been able to accumulate productive assets are less able to use these assets to generate income during the pandemic—this was the case in recent study of GD transfers in Kenya during COVID-19.”

Even though Rwanda implemented one of the strictest lockdowns in Africa in response to COVID-19, as the study is an RCT, it will still produce valid comparisons across study groups. The 36-month results will indicate how youth who received HDAK, a cash grant, or both fared in response to the pandemic.  

At USAID, we welcome innovative partners who are committed to improving programming through research and evidence generation. This study represents the collaboration across USAID, involving ongoing insights and feedback from colleagues in the Africa Bureau, the Bureau for Resilience and Food Security/Nutrition, Rwanda Mission, and the Bureau for Economic Growth, Education and Environment, and the Bureau for Policy, Planning & Learning. USAID will continue to invest in rigorous evidence building through its development programs.