To build meaningful and innovative tools that meet urgent needs, collaborate with implementing partners.
by Jennifer Fei and Jessica Sadye Wolff
Due to the COVID-19 pandemic, nonprofit service providers around the world had to find new ways to continue their operations and check in with their clients remotely. Through conversations with partner organizations, IPL recognized a pressing need for tools to support their adaptation to the pandemic. The moment also presented an opportunity for research collaborations, which we began exploring with our partners in spring 2020.
At that time, the IPL co-designed, developed and launched a WhatsApp-based survey infrastructure for nonprofits to conduct outreach to their clients. Conducted in partnership with Lutheran Immigration and Refugee Service (LIRS) and Mercy Corps—and with financial support from the UK Foreign, Commonwealth & Development Office, awarded through the Innovations for Poverty Action (IPA) Peace & Recovery Program—the project aimed to facilitate “rapid response” surveys, which could be deployed quickly to a set of clients via mobile platforms like WhatsApp and SMS.
Even before March 2020, IPL had been in conversation with partner organizations about ways to use mobile phones to stay in touch with clients who may be on the move or plan to move in the future. Service providers and researchers often rely on direct outreach efforts such as phone calls or in-person interviews. For example, a service provider may want to collect outcomes data that indicate how its immigrant or refugee clients are faring during their initial integration period. These efforts can be cost-intensive in terms of time and money, and they prove very challenging when participants move internationally and change their contact information.
Building on a strong foundation from prior collaborative projects with innovative partners, IPL teamed up with Mercy Corps and LIRS to co-design a low-cost data collection method for continued communication with refugees and asylum seekers who are potentially on the move. Both organizations were eager to collaborate on developing and testing a new client outreach method. As a research partner, IPA saw value in the exploratory approach and potential for large-scale impact as a low-cost data collection tool, and it committed support for the pilot project in Colombia.
We ran pilot projects for the survey tool with refugees resettled in the United States (with LIRS) and Venezuelan refugees in Colombia (with Mercy Corps). Our goal was a robust proof of concept: results that would demonstrate this survey method’s feasibility as a low-cost method for ongoing client engagement that service providers could use at a large scale.
Supporting partner organizations during the COVID-19 pandemic
To help refugees and migrants struggling with the pandemic and national lockdowns, our partners first needed to know more about their experiences: how their daily lives were changing and what were their greatest challenges. With that information, these organizations could more efficiently allocate resources and staff time to benefit the people relying on them. Mercy Corps implemented a panel survey via WhatsApp starting in March 2020 with Venezuelan households in border regions of Colombia and followed up with quarterly surveys. This allowed for repeated engagement with the same group of individuals to assess how food security, access to medical care, and other indicators of well-being changed over the course of the pandemic.
LIRS launched a baseline survey in summer 2020 of refugees it had helped resettle in the United States over the past ten years. It then implemented regular follow-up surveys via WhatsApp and SMS to learn about the impact of the pandemic on the experiences of resettled refugees in the United States. Survey questions covered areas such as housing and food security, employment, and access to education.
For Laura Wagner, former senior program officer for integration at LIRS, the project was an opportunity to gain insights crucial for its response to COVID-19:
At LIRS we had some assumptions and many anecdotes about how COVID-19 was impacting refugees across the US but limited data or evidence we could use to advocate for funding or policy and inform how to shift programing. By partnering with IPL to send a low-cost survey to almost 10,000 refugee clients who arrived over the past 10 years, we were able to not only gather data on the impact of COVID-19 but also expand our capacity for using WhatsApp surveys in the future.
Beza Tesfaye, director of research and learning for migration and climate change at Mercy Corps, saw the benefits of timely engagement with clients in Colombia:
Mercy Corps had been working in Colombia to provide vulnerable Venezuelan migrants with access to cash to meet their immediate needs, when the COVID-19 pandemic hit in 2020. The WhatsApp surveys that we deployed with IPL enabled us to better understand the evolving needs of the population we were targeting. This new approach allowed us to quickly and cost-effectively gather critical information through remote means, without exposing participants or front-line staff to COVID-19.
For both projects, the survey questions are updated as needed to respond to current conditions or new programmatic priorities. (For more information specifically on the survey method, see this working paper.)
“The survey results illuminated a more prominent level of food insecurity than anticipated, and therefore LIRS was able to encourage affiliates to focus on access to food for refugee clients,” said Wagner. “The survey also provided evidence that the vast majority of refugees were continuing to work in person, a data point that LIRS has used to advocate for investment in career pathways programming to create opportunities for refugees to advance beyond low-wage, entry-level work.”
Co-creation to ensure meaningful development
We used a process of co-creation to design, develop, and implement the new survey method. IPL researchers worked through an interactive survey design process with our partner organizations’ program staff, jointly identifying key topics of strategic or programmatic importance and then designing and testing related survey questions and logic. Additionally, since the survey was designed to be delivered by WhatsApp or SMS, we worked together to test usability and ensure that the survey experience—including question display, phrasing, and sequence—was simple and understandable.
“LIRS brought our expertise in working with refugees, and IPL brought its expertise in survey design and data analysis to co-create the survey,” said Wagner. “LIRS does not have dedicated staff for data and research, but in partnering with IPL, LIRS was able to expand the research skills of content experts.”
Long-term adoption and sustainability
At the start of this project, we set out to develop a survey method that non-technical users could learn and maintain for future outreach and data collection efforts. One key indicator of success, which we defined at the outset, was whether or not partners would go on to use the method in other settings, without our help. This would tell us that the method was simple enough, sufficiently cost-effective, and scalable.
After a series of capacity-building sessions between IPL and LIRS to walk through the technical manual and steps required to set up and run the surveys, LIRS is now managing the survey method internally for other client outreach initiatives. Mercy Corps is also exploring opportunities to incorporate WhatsApp outreach in program evaluations and its Community Accountability and Reporting Mechanisms. IPA adapted the tool to fit its data guidelines as a research and policy-focused nonprofit, and it adopted this survey method at a global scale across its regional offices.
Carlos Bohm, a research associate for Innovations for Poverty Action’s Global Research & Data Support team, described the value of this WhatsApp method for broader research goals:
At IPA, we see WhatsApp as a very useful tool to maintain contact with, and collect information from, study participants, especially among harder-to-reach, mobile populations. This has the potential to lower study attrition and reduce data collection time and costs.
We built upon the foundational knowledge created by IPL to adopt and test the methodology in several projects in Colombia. Through these experiences, we were able to streamline and standardize the processes for easier integration into current and future projects as well as identify and address several data security challenges. For example, we developed a method for encrypting WhatsApp responses with personal information within the platform before it’s published in Google Sheets, allowing full compliance with IRB protocols. These learnings were shared with IPL, and we are collaborating on incorporating them into an updated manual.
Building a community of practice
From the beginning of this project, we intended to make all related materials open to the public, in the hopes that this method would be valuable to other organizations beyond Mercy Corps and LIRS.
Together we offered two public webinars for other researchers and organizations interested in the method, which were attended by hundreds of people from more than 30 countries. We created a WhatsApp survey hub on the IPL website, which includes a methodological working paper, the comprehensive technical manual with step-by-step guidance and video tutorials, and the link to a public Github repository.
Since the public launch of these materials one year ago, numerous other organizations and researchers have contacted IPL to learn more about the method and build on our resources with their own innovations and developments. IPA also developed additional functionalities that are now available to supplement the technical manual.
Looking ahead, we’re eager to collaborate with institutions around the world to further enhance and promote this client engagement methodology to serve organizations and their clients across diverse contexts.
