Data Part 1: It’s not the technology, it’s about who uses it and how

DATA PART 1:
IT’S NOT ABOUT THE TECHNOLOGY, IT’S ABOUT WHO USES IT AND HOW 

Often times our work at InSTEDD revolves around projects that require some form of data collection. Typically we help people implement systems that facilitate work and use the relevant data to achieve a specific result. For example, we’ve worked on projects that help potential tuberculosis patients get tested earlier, HIV patients stay on track taking their medicines and tracking down sources of malaria to help eradicate it. 

Over the course of working on a variety of projects that require data reporting, we’ve learned many valuable lessons. One such lesson is that the data collection system needs to be directly addressing the issue at hand, and often times the group running the project has to be coached up front in order to design the data collection system that deals with the real problem. In order to properly design a system for collecting data that is actionable, it’s important to understand the why, the who, the when and the context in which the data will be used.

Data for action, or data for data’s sake?

One of the first questions we ask is “why is this data important?” What will change in the world as a result of having it? Can we shift the design towards the outcome we want (the action) and not the means (the data itself)? Framing the discussion with these questions helps many of our partners shift their mindset so that they are better able to design an mHealth solution that supports the desired outcome: healthier people. The world is not Newtonian so bringing to existence a piece of data does not cause an equal corresponding reaction.

You want your data to be great, on time and with little resources

Any project manager with some experience is familiar with the ‘iron triangle’. If you want to deliver a project with certain quality, you can’t change time, scope and resources independently. Changing one will imply changes in the others. You can’t dictate that suddenly the project will deliver twice as much value, without changing the time and resources available.

In data collection a similar trade off happens: there is a tension between collecting data that is:

• Timely — data is available quickly after a relevant event
• Complete — data comes in from all relevant sources
• High Quality — data tells you something happened when it did (it has good sensitivity) and that the data doesn’t tell you something happened when really it didn’t (it has good specificity)
   

However, there are some correlations worth exploiting when designing your solution as each particular problem set will have different needs and priorities. For example, infectious disease surveillance, polio eradication, reports of pregnant mothers with complications, street violence reports and post-disaster damage assessments all put different emphases on different ‘points’ of the triangle. The trade off is not a matter of the technologies themselves, but rather who uses them and how.

Additional hurdles need to be cleared when dealing with different technologies and their user experiences for collecting data. For example, filling out a 20-field form via SMS is painful. You also can’t send pictures as easily as you can do voice based messaging. Not everyone has a high-quality smartphone that lends itself to collecting highly structured data. People may not have the proper application, training or knowledge required to collect and/or share certain data. Filling in a form is can be a useful mechanism, but it’s important to be aware that the design of the form can influence the data in ways inexperienced people may not have predicted. Asking people the same question in different ways produce worse data. For example:

• What symptoms do you have? 
• Do you have any of these symptoms? 
• Check all that apply: A, B,C 
• Do you have symptom A? 
• Do you have symptom B? 
• Do you have symptom C?” 

We have learned that even a more advanced device which is capable of collecting richer data can produce worse data if the system isn’t designed properly.

Are you putting all your eggs in one basket?

The best strategy is not to rely on a silver bullet approach, or pick the option with the least obvious problems. Rather, it is wiser to create a combination of approaches that create datasets that you can use together. Such as

• Create a ‘hotline’ to get some information from the population directly via voice, or with SMS callbacks. For example, this could be a hotline set up for farmers who can report suspicious livestock deaths (which could be an early indicator of an H5N1 outbreak).
• Develop a consistent reporting schedule so that those closest to the action on a daily basis are in the habit of regularly reporting what they see. For example, community health workers can report routine and occasional events with more detail, through an SMS or voice reporting system.
• Equip a highly-trained team with knowledge and technology to use more advanced technologies to gather rich information from the field. For example, community outreach teams can use smartphones or tablets in the field while they do routine/surprise interviews or unplanned field investigations to that the information collected is sent back to headquarters in real time.
• Correlate the field information collected to verified diagnostic reports from reference laboratories validating disease diagnostics. For example, you can relate the information gathered from the community health centers to a larger more complex system to help verify if the diagnosis was correct and the treatment appropriate. 

As you see, each data set will be different. Balancing the timeliness, completeness and quality of each solution gives you an opportunity to use the data complement each other. When brought together in the right way, these different information sets create a better picture than each one could independently. The whole is much greater than the sum of its parts.

We have learned many valuable lessons from our work in the field and the above post is by no means an exhaustive list. Our hope is that by sharing some of our experiences from working directly with the ones using our tools, that we will be able to help others improve the quality of their work as well.

Stay tuned for part two of this blog next week…