PDRI/ISSER Policy Brief 3: Lessons learned on survey design and implementation during a pandemic: The case of GridWatch

This policy brief was prepared by Innocent S. K. Agbelie, Koko Namo Lawson Zankli, and Simon Bawakyillenuo of the University of Ghana’s Institute of Statistical, Social and Economic Research (ISSER). It forms part of a series of policy briefs released by PDRI in cooperation with ISSER.

Key takeaways

• Adaptation is key to success in times of a pandemic. It became quickly evident that the old things of carrying out surveys could not be utilized in the case of the implementation of the GridWatch project. The project team had to adapt to the situation and incorporate new strategies such as virtual meetings and monitoring in the implementation design.
• Increased frequency of meetings among research collaborators at various locations is an indispensable action to bring about success amidst a constantly evolving pandemic phenomenon. The successful completion of the GridWatch project in 2021, was in part due to the mainstreaming of this action in the project implementation process.
• The component of the GridWatch project’s implementation design that got affected the most by the pandemic was the implementation cost. The initial implementation cost quickly became insufficient because of the COVID compatible new design that required extra budgetary allocation. It is therefore imperative that all projects’ budgets incorporate realistic and effective contingency lines during pandemic or non-pandemic eras.

Introduction

The novel COVID-19 coronavirus, which was first detected in Wuhan, China in November 2019 was declared a pandemic in March 2020 by the World Health Organization (WHO). To combat the fast-spreading virus, governments around the world imposed restrictions on movement, and the wearing of facemasks, regular handwashing, as well as social distancing were promoted by governments, scientists, and the media. Streets became empty, stores and markets remained mostly closed, the prices of common household items increased, schools went online, busy towns became ghost towns, and everyday living became confined to people’s own dwellings. By April 2020, half of the world was under a lockdown[1] as the world looked to science for answers.

The two most populated regions in Ghana (Greater Accra and Ashanti) were placed under lockdown in March 2020 after the first COVID-19 cases were reported by the Ghana Health Service (GHS). Only essential services such as health and security continued. Most economic activities came to a halt or changed their operations significantly, including educational services.

By this time, enumerators had been trained by the Institute of Statistical, Social, and Economic Research (ISSER) in February 2020 on the ‘Evaluating the socioeconomic impacts of differences in electricity reliability: Evidence from Accra, Ghana’ (GridWatch) project. Their plans to begin fieldwork had to be postponed. ISSER quickly liaised with the project leads at the University of California, Berkeley (UCB) to put the commencement of fieldwork on an indefinite hold. Data collection activities were eventually implemented after the easing of restrictions in 2021. This policy brief highlights the lessons learnt during the implementation of the survey amidst the COVID-19 pandemic.

The status quo approach pre-COVID

Implementing a survey in the pre-COVID era was not always without challenges. Implementing institutions often faced challenges, but different in kind to the ones that arose in the COVID era. Under normal operating procedures at ISSER, once contractual issues are completed between ISSER and its collaborators or funders, enumerators are recruited and trained for fieldwork. The training may take a day or up to a month depending on the complexity and length of the survey instrument. If the enumerators do not begin field activities within 5 days of completing their training, there is often a day or two refresher training for the enumerators. During fieldwork, the project implementation team regularly encounters challenges, including:

• Poor reception by targeted populations: This is mostly experienced in urban areas or towns that have experienced research saturation. Often, subjects decline to participate in the research, explaining that individuals and organizations have been gathering data on them for several years, yet they are not experiencing any positive change in the very things being researched.
• Enumerator pull-out: Sometimes, one or two enumerators fail to turn up for fieldwork after training. This might occur for varied reasons, including ill-health, better opportunities, or bereavement.
• Difficult terrain: Some terrain may become inaccessible to the enumeration team due to conditions such as conflict, bad weather, poor communication lines, and poor access roads. This often affected the targeting design and project implementation.

Effective physical monitoring of field teams has always been the hallmark of obtaining high quality data at the end of the survey. With the upgrade to gathering data using Computer Assisted Personal Interviewing (CAPI) systems, which allows for data synchronisation in the field, the monitoring team is also able to scrutinize the data remotely and seek clarifications from fieldworkers while they are still in the field.

Once the data collection phase comes to an end, the data is prepared for use by the collaborators and/or ISSER. For some projects that ISSER undertook, the data collected were analysed and reports were prepared and submitted to the funders. The project only comes to an end when the funders or collaborators are satisfied with the outputs provided by ISSER.

GridWatch project design and implementation in the COVID era

The baseline phase of the GridWatch project, which was led by the University of California, Berkeley, from 2019 to 2021, fieldwork activities implemented by ISSER, sought to examine the reliability of the electricity provided by the Electricity Company of Ghana (ECG) or Power Distribution Services (PDS) in this area, and to understand how lack of reliable power may affect the economic welfare of households and businesses. While the baseline survey was originally planned to be implemented in 2020, it got postponed to 2021 because of COVID restrictions. By the time the baseline survey activities resumed in February 2021, the need to restructure the implementation design and strategies to help accomplish the project goals became compelling. These changes contrasted pre-COVID protocols and strategies.

Introduction of stringent health screening

After partially lifting the COVID restrictions to allow for movements and limited gathering at public spaces, the government advised that all COVID safety protocols must be strictly adhered to. To ensure the safety of enumerators and respondents, a two-way health screening was to be carried out every day before each survey. All enumerators were required to screen themselves daily for potential COVID symptoms, record their screening results, and share them with their supervisors. An enumerator only went ahead with the survey of a particular respondent after being judged fit for the survey. The respondents were also screened via a checklist and only those who were passed fit were eligible to be interviewed. These types of stringent health screenings had previously been alien to ISSER’s survey implementation processes.

Rising survey costs triggered by COVID-19

The COVID-19 pandemic had extra cost implications for the implementation of the GridWatch baseline survey. To avoid over-crowding in public vehicles, the project re-budgeted for single-passenger transportation systems for the enumerators. They were required to use chartered taxis or ride hailing apps like Uber, Bolt, and Yango for their transportation needs. Pre-COVID, ISSER only gave transport allowances to enumerators but did not specify the mode of transport to use.

Pre-COVID, there were standard logistics ISSER used to provide during survey implementations. These include survey collection instruments (CAPI, audio recorders), stationery (notepads, pens, pencils), bags, incentives for respondents, and measuring equipment (if required). With the emergence of COVID-19, personal protective equipment (PPEs) such as face masks, face shields, sanitizers, gloves, etc., had to be procured additionally for fieldworkers and respondents. This had cost implications on the project’s budget and implementation.

In addition, there was a 36% post-training attrition of enumerators. Upon easing the COVID-19 restrictions, some of the enumerators that were initially trained before imposition of the COVID-19 restrictions could not no longer take part in the survey. This necessitated the conduction of a full-blown training for both new and previously trained enumerators.

Higher refusal and discontinuation rates than expected

Every survey implemented by ISSER in the pre-COVID era had some level of refusals and discontinuations. While this phenomenon existed during the GridWatch survey in 2021, the COVID-19 pandemic contributed to higher cases of refusal. In anticipation of such occurrences, ISSER trained the enumeration teams adequately to handle such cases. The first critical actions enumerators were coached to undertake to contain an overwhelming rate of refusals included the deployment of an effective community entry protocol, creating good rapports with the potential respondents, and assuring them of following all safety protocols.

Strategic monitoring

The project implementation team at ISSER was concerned that the uncertainties surrounding COVID-19 could influence over- or under-stated responses from the respondents. A team was therefore tasked to monitor the field activities in person and virtually, especially the progress of work, the kinds of responses being collected, and the challenges facing the enumerators. The virtual monitoring predominated in-person monitoring due to the risks and uncertainties about COVID-19, and it helped to identify issues that needed immediate attention.

Lessons learned from running surveys during the COVID-19 pandemic

Key lessons were learned regarding the design and implementing a project such as GridWatch in the face of the COVID-19 pandemic.

• Effective preparation is required: During the lockdown period, ISSER had many virtual meetings with the collaborators concerning potential changes to the implementation design, the cost implications, and the general future of the project. These meetings allowed the ISSER team to be well prepared and ready for the implementation when the restrictions were partially lifted. During the implementation phase, ISSER had two meetings every week to discuss the progress of the survey. These facilitated comprehensive discussions and sharing of information to ensure success.
• Cultivation of resilience: The successful implementation of the GridWatch project would not have been possible without committed implementation and enumeration teams in the wake of the COVID pandemic. The implementation team resumed work as soon as the University of Ghana was reopened, and the enumerators availed themselves for work once they were called.
• Safety assurance and leading by example: Implementing surveys successfully during the COVID-19 pandemic era requires the institution of measures to safeguard the respondents from contracting the disease from the interviewer and demonstrating that practically. This was effectively done in the case of the GridWatch project, a process whereby the enumerators screened themselves and the potential respondents, maintained the required social distancing during the interviews, kept their masks on and ensured the respondents had theirs on as well. The use of the hand sanitizers underscored the safety precautions observed.
• Contingency budget lines are indispensable: All projects undertaken by ISSER always have a budget line for contingencies. The exigencies of COVID-19 gave more impetus to the need to build in a contingency budget line of 10% (of the project implementation cost) for all field surveys. In the original GridWatch project, there was not a contingency line. However, with the emergence of COVID-19, extra budget expenditures amounted to about 10% the original budget.
• Effective tracking and monitoring system are required: ISSER put together a strong monitoring team that consistently and frequently tracked the progress of work, the movement of enumerators, and the quality of the data being collected at every stage of the enumeration exercise. This was instrumental in the successful implementation of the GridWatch project. With the deployment of efficient remote monitoring of field activities during pandemics, high quality data can still be gathered, even without physical monitoring.

Acknowledgement

The development of this policy brief was supported financially by the Millennium Challenge Corporation (MCC) through the project, “Evaluating the socioeconomic impacts of differences in electricity reliability: Evidence from Accra, Ghana” implemented by Professor Duncan Callaway at the University of California, Berkeley; Professor Steven Puller at Texas A&M; and Professor Susanna Berkouwer at the University of Pennsylvania.

[1] According to Euronews (https://www.euronews.com/2020/04/02/coronavirus-in-europe-spain-s-death-toll-hits-10-000-after-record-950-new-deaths-in-24-hou)  and BBC (https://www.bbc.com/news/world-52103747)