While ODK provides a robust platform for enhancing data quality, the ultimate key to success lies in the individuals using it – the fieldworkers.
ResearchPro team members are presently engaged in a project focused on gathering longitudinal data from tens of thousands of individuals in Kenya to investigate the effectiveness of the drug ivermectin in combating malaria. We are using ODK for our data collection efforts, and as discussed in this post, we have uncovered a range of strategies to enhance the accuracy and quality of data collected through leveraging strategic ODK form design techniques.
Our experience has also shown that while ODK provides a robust platform for enhancing data quality, the ultimate key to success lies in the individuals using it – the fieldworkers. They are the individuals on the ground entrusted with capturing accurate and reliable data.
In this post, we will explore various strategies to optimize data accuracy and quality through fieldworker empowerment, monitoring, and ODK-specific techniques. All the examples presented here are drawn from our authentic challenges, experiences, and valuable insights gained from the field.
Getting to Know the Fieldworkers
Understanding fieldworkers is crucial for the success of any data collection endeavor. Surprisingly, this fundamental aspect is occasionally overlooked by project and data managers. By delving into their priorities, motivations, linguistic preferences, and other key details, we not only fortify our collaboration but also enhance the quality of data gathered.
Here are some reasons why getting to know fieldworkers can help improve data quality:
Better Training: By understanding the strengths, weaknesses, and backgrounds of fieldworkers, training can be tailored to meet their specific needs. This ensures that they are well-prepared to collect data accurately and consistently.
Feedback Loop: Knowing fieldworkers well can foster open communication. They may provide valuable feedback about the data collection process, point out potential issues in the field, or suggest improvements.
Motivation and Morale: Fieldworkers who feel understood and appreciated are likely to be more motivated. High morale can lead to increased diligence and attention to detail, both of which are crucial for high-quality data collection.
Collaborative Form Development
In many research projects, including those we work on, the conventional workflow typically proceeds as follows: Scientists, driven by the need to gather data that addresses their specific research questions, take the lead in formulating survey questions. Once these survey questions are established, data managers and ODK form developers step in to digitize the survey forms. After undergoing internal quality control, these digitized forms are then handed over to the field team, who take over to test the forms and commence training fieldworkers for the subsequent data collection.
While this linear approach is commonly followed and may suffice, a more advantageous alternative involves including the field team at an earlier stage in the process to provide insights and feedback on the form design and user experience.
In the malaria study, our team digitized several ODK forms based on specifications provided by the project sponsors and scientists. After spending hours on iterative testing within project staff, the forms were relayed to the field staff for fieldworker training. During this phase, several challenges and issues surfaced, necessitating time-intensive modifications to the forms.
Our experience taught us that engaging fieldworkers earlier in the process of form development and testing can optimize timelines. It allows for harnessing the expertise and practical knowledge of field team members to create survey forms that are better suited to the unique challenges and conditions they'll encounter during data collection. Their early input not only enhances the ultimate usability and relevance of the survey forms but also promotes a sense of ownership and engagement among the field team, ultimately contributing to more efficient and accurate data collection processes.
Comprehensive Training & Pilot Testing
Mastering ODK Collect can prove challenging for those unfamiliar with such software. Therefore, it is imperative to deliver comprehensive training to fieldworkers on both ODK Collect usage and the specific forms they will encounter.
Among the primary hurdles faced by the fieldworkers in our malaria project included form navigation, particularly involving repeat groups (i.e., adding individuals to households), saving an unfinished a form, and later returning to complete specific repeats. Recognizing these difficulties enabled our on-site data managers and trainers to allocate additional time and resources to assist and empower fieldworkers in navigating these obstacles prior to real data collection.
Conducting pilot tests of ODK forms and the data collection process in the actual setting of fieldwork prior to real data collection is another indispensable step. It helps in pinpointing and addressing issues such as technical glitches and potential challenging scenarios, and can prove highly beneficial in ensuring that fieldworkers are well-prepared for the task at hand while empowering them to feel confident and important in their roles.
Monitoring & Quality Control
Even with comprehensive training, within a diverse team of fieldworkers, it's common for some individuals to lack a full grasp of ODK Collect or project-specific protocols even once they begin data collection. Some team members may also have different priorities, preferring speed over precision. Recognizing data collectors who consistently introduce errors into the dataset or exhibit a tendency to rush through questionnaire completion is essential for project managers. This identification helps pinpoint areas where additional training and support are required to enhance data quality and accuracy.
One of the benefits of using ODK for data collection is being able to capture the start and end times of a form submission.
By examining the time stamps – the moment a fieldworker starts and finishes filling out a questionnaire – we were able to calculate the duration of time fieldworkers spent completing each questionnaire and identify individuals (using the few lines of code in R below) who consistently completed a survey that should require a substantial investment of time in less than 10 minutes.
# Convert date-time strings to POSIXct objects
sv$start_time <- as.POSIXct(sv$start_time, format = "%Y-%m-%dT%H:%M:%OS", tz = "UTC")
sv$end_time <- as.POSIXct(sv$end_time, format = "%Y-%m-%dT%H:%M:%OS", tz = "UTC")
# Calculate the duration
sv$duration <- round(difftime(sv$end_time, sv$start_time, units = "mins"), 2)
## Detect short visits: duration <= 10 mins
short <- sv %>% filter(duration <= 10) %>% select(hhid, duration, wid)Identifying these fieldworkers presented us with an opportunity to notify their supervisors to initiative a discussion with them regarding the important role of proper data collection in upholding the study's integrity.
We also developed a system to identify and address other anomalous values, such as a young child being designated as the head of the household, or spotting duplicated individuals within the database. While the primary purpose of this quality control measure was to flag inconsistent data for correction, it also allowed us to identify fieldworkers who were consistently inputting multiple errors and might require additional training or support or where we needed to utilize other ODK form design techniques (eg., constraints, hints, etc.) to make it easier for fieldworkers to input accurate data.
The identification of data collectors consistently producing inaccurate data is a pivotal step in upholding the integrity of any fieldwork project. It serves as a valuable diagnostic tool, enabling project managers to evaluate their team's proficiency and implement targeted solutions to enhance data accuracy.
Achieving and maintaining high data accuracy with fieldworkers is not just a goal - it's a crucial necessity for the integrity of data-driven research. Through a combination of understanding fieldworkers, training, practice, and vigilant monitoring and quality control measures, and collaborative form design, we can empower fieldworkers to consistently collect accurate and high quality data using ODK.
Comments