Impact Evaluation of USAID's Community-Based Crime and Violence Prevention Approach in Central America

This multi-method, multi-country, multi-year evaluation was designed to contribute to an understanding of the effectiveness of USAID's community-based crime and violence prevention approach. It was carried out in Guatemala, El Salvador, Honduras, and Panama. This package of interventions includes activities such as planning by municipal-level committees; crime observatories and data collection; crime prevention through environmental design (such as improved street lighting, graffiti removal, cleaned up public spaces); programs for at-risk youth (such as outreach centers, workforce development, mentorships); and community policing. USAID's community- based crime prevention projects are inherently cross-sectoral; they integrate education and workforce development, economic growth and employment, public health, and governance interventions.

This study design is technically known as a "Cluster Randomized Experiment" (CRE). The research design used by LAPOP for this evaluation scores a 5 (the highest level) on the Maryland Scale of Accuracy in the Design of Evaluation, indicating that the findings yielded by the study should be considered strong evidence for a cause-and-effect relationship (or lack of relationship; Sherman 2012). The CRE design is an especially appropriate research design for USAID's community-based programs because they correspond perfectly to the group-based unit of assignment as the neighborhood grouping of individuals. To avoid the problem of contamination of control neighborhoods by applications in treatment neighborhoods, our CRE design uses non-contiguous clusters (e.g., neighborhoods or communities) that are then randomly assigned to each study condition (i.e., treated/non-treated). LAPOP used a CRE design with a stratified selection. Rather than selecting neighborhoods from one large pool in Guatemala, El Salvador, Honduras, and Panama, the sample was first stratified by country.

The samples was further stratified by selecting neighborhoods by municipality so that all of the treatment and control communities were concentrated in selected high-crime municipalities in each country. Once the stratification was complete, neighborhoods within municipalities in each country were assigned randomly to treatment and control groups. Baseline differences across neighborhoods within municipalities were then accounted for or controlled for in the statistical analysis using the control variables in the surveys. Additionally, a qualitative stakeholder survey was undertaken to gather information from key persons most knowledgeable about a given neighborhood situation, via semi-structured, open-ended questionnaires.

The main finding, on average, of this multi-year impact evaluation of the community-based interventions is that in several key respects the programs have been a success. Specifically, the outcomes in the treatment communities improved more (or declined less) than they would have if USAID's programs had not been administered:

1. The community-based crime prevention program under CARSI produces a significant reduction in the expected level of crime victimization and violence
2. The community-based crime prevention program under CARSI resulted in a significant increase in the expected level of citizens' sense of security
3. The community-based crime prevention program under CARSI is responsible for a significant decrease in the expected level of neighborhood disorder, such as loitering and gang presence
4. Satisfaction with police performance has increased significantly as a result of the community-based crime prevention program under CARSI.
5. Indirect effects of the community-based crime prevention program under CARSI values include strengthening democratic values, which have increased significantly over the expected level in the absence of the program.