Pivoting in a Pandemic: Adapting the Rapid Education and Risk Analysis in Colombia

This is the next blog post in our new Pivoting in a Pandemic series. In each blog post, we'll highlight how different programs responded to COVID-19. This post focuses on how the RERA in Colombia pivoted from in-person to fully remote data collection. 

In Colombia, COVID-19 emerged in a dynamic mix of 1.6 million Venezuelan migrants and refugees, a legacy of long-standing armed conflict and massive internal displacement, and vulnerabilities to multiple natural hazards. Planning an education sector response in this complex environment demanded a Rapid Education and Risk Analysis (RERA). The Government of Colombia needed data from the RERA to implement flexible and effective models of education for both migrants and the communities that host them, to strengthen already stressed education systems, and tailor support for education access, retention, and school community resilience. In March 2020, with funding from USAID/Colombia, DevTech began planning to conduct a RERA in urban and peri-urban school communities receiving migrant Venezuelans.

Within a month, due the COVID-19 containment measures, the RERA research team pivoted from in-person to fully remote data collection, administering quantitative surveys and qualitative interviews by cell phone across 23 school communities. The RERA team collaborated intensively with eleven education secretariats and school principals by telephone to identify and gather phone numbers of over 600 respondents. In two weeks, the RERA research team surveyed 457 people (out of 600) and interviewed 201 people (out of 457). In each school community, the RERA team attempted to speak to the school director, teachers, students, their parents, and community members.

Key Lessons in Carrying Out a Remote RERA

1. Early and clear explanations of adaptation options help agile decision-making.
   DevTech drafted a concept note in March on how to adapt the RERA to remote management, with optional scenarios and budgetary implications, to inform a quick decision by USAID/Colombia and the Ministry of Education. 
2. Intensive collaboration with local education authorities and principals is even more crucial when working remotely.
   • After formal introductions from the Ministry of Education, the RERA team iteratively communicated by email and telephone with local education secretariat officials to explain the RERA’s remote process, discuss sample requirements and school selection, as well as respondent types, contact information, and connectivity requirements. This took more time given the crisis that secretariats and schools were managing. Getting through by phone was understandably challenging, and in one case, data privacy restrictions required additional discussions. 
   • Principals in several school communities contacted all respondents personally to explain the RERA and encourage them to participate.
3. Various technologies are available and effective to support remote fieldwork.
   The RERA Team used KoboToolbox for the survey, and a mixture of Skype (both Skype-to-Skype and Skype-to-telephone) for interviews. Documents were developed, shared and discussed live using Google Drive and Google Documents.   
4. Contracting a local call center can increase call efficiency. 
   • The RERA Team contracted a local data collection firm’s own call center to administer the survey by telephone in order to reach respondents without access to data or the internet. The call center scheduled and conducted survey interviews, recorded informed consent, and scheduled all follow-up, qualitative interviews.
   • Local call center capability also optimized a culturally sensitive approach—local specialists were adept at reaching intended respondents in often busy households, speaking in local jargon, explaining the process in an understandable way, and establishing trust. 
5. Next time, we would conduct the survey and follow-up interview on the same call, to increase the response rate.
   Response rates for the RERA survey were strong, but dropped off for the follow-up interview calls to the same respondents just a week later.
6. Remote training can be effective—with sufficient time. 
   • Asynchronous and synchronous approaches should be combined for effective training. For example, allow participants time to study materials off-line and then use platforms—such as live written or verbal question-and-answer sessions—strategically and efficiently. Plenary, synchronous sessions are important for specific topics which the entire team must discuss, such as ethics. 
   • Running simulations to assess data collectors and test the protocols themselves are a necessity, especially for open-ended interviews, but sufficient time is needed for corrections and re-runs. The RERA Team could have benefitted from  more time (one to two days) to adequately test survey and interview questions before the launch.
   • If a data collection firm is hired, maximizing the firm’s internal quality control and management systems to discuss training content, prioritize questions, and consider running internal question-and-answer sessions can winnow and prioritize the list of questions for plenary. 
7. Requiring individuals to have a cell phone (or internet connectivity) to respond to a survey or interview can bias the sample against the most vulnerable or marginalized. 
   Tackling a lack of access to technology is a challenging issue beyond the scope of an assessment, especially rapid ones. Some partners doing face-to-face surveys have issued phones and solar chargers to respondents, but this approach for remote, rapid assessments may not be feasible. Local phone sharing strategies have potential, but these are also complicated by the COVID-19 reality. A common approach is to find ways to supplement their data in order to infer the situation of vulnerable populations. 
8. Remote management may reap less insights.
   • Conducting conversations on sensitive issues by telephone can affect the level of confidence in the respondent. On the one hand, speaking remotely might increase a respondent’s trust and candor. However, in contexts where child recruitment and other predatory behaviors are common, speaking on the telephone can reduce trust.
   • Typically, doing in-person school visits for a RERA offer opportunities for the team to observe the school environment and gain additional insights from being present, a common method of triangulation. This dimension is lost in a remotely managed RERA. 
9. Including children in remote data collection in an area of conflict and crisis has unique challenges.
   • In one area prone to child recruitment into armed groups, a secretariat required additional assurances of data protection, such as a signed agreement to license data use, before it would share student contact information.
   • In interviews, there were instances where parents accompanied their children on the telephone and directed them what to say. Interviewers can explain to parents the need to let their children speak freely, but this must also be carefully monitored to ensure quality control of the data
10. Working under COVID-19 social restrictions affects everyone—including the RERA Team.
    RERA is designed to be a rapid exercise, thus it is important to recognize the pressures and unique stresses that affect not only respondents and partners, but RERA Team members themselves. Understanding, flexibility and creativity all play important roles in managing for quality and results. 

Overall, we found The RERA Toolkit to be readily adaptable to a fully remote process. Though the RERA Toolkit was not piloted in a fully remote scenario, the Colombia RERA demonstrates that it is possible. The RERA process involves two important moments of consultation with stakeholders—a briefing before data collection starts, and a consultation on preliminary results as data collection closes. These consultations were held using different on-line platforms and remain vital to the legitimacy of the RERA process and to the quality of the RERA conclusions and recommendations.  

Adapting the Rapid Education and Risk Analysis in Colombia: A Companion Brief

Rapid Education and Risk Analysis: Colombia

Análisis Rápido de Educaión y Riesgos: Colombia