Data: It’s not the technology, it’s about who uses it and how

DATA:
IT’S NOT ABOUT THE TECHNOLOGY, IT’S ABOUT WHO USES IT AND HOW

In the past, we’ve blogged about some of the many lessons we’ve learned when it comes to working on projects that require some form of data collection.  As a follow up, we’ve added more of what we’ve learned in the hopes that it will help you better deal with similar situations.  Below is a continued list of our lessons learned and how we have been able to work around the problems we’ve faced in the field.

Are you incorporating sensor information into your data sets?

The breadth of mobile coverage has opened many doors for data collection by humans – but what about sensor information? There are dozens of strategies to get more data automatically from remote places, especially as the costs and power requirements of Global Systems for Mobile (GSM) and wifi electronics plummet.

For instance, you can:

• Use built-in smart phones sensors such as GPSs, accelerometers, cameras and/or touchscreens to automatically collect information on the go.
• Extend the capability of a device with accessory gadgets and connect them via the device’s wires or bluetooth.
• Hack devices to change their sensors or their inputs

Think what you can learn if you put sensors into portable vaccine fridges, or ambulances, water pumps or even prosthetic knees!  Of course, this can be expensive. You can’t just outfit hundreds of thousands of water pumps with connected sensors to evaluate the quality and quantity of the water in the blink of an eye.  But, in terms of the completeness-timeliness-quality trade off issue (see image below), the embedded and connected sensors can give very timely, high-quality data. Going back to our example, maybe if only 1% of well-chosen sentinel water pumps are outfitted with sensors, and you rotate them over time, you can get data leading to insights that otherwise would have been lost.

What are the motivations and incentives of the people who will use the tools?

Any good design process uncovers and adapts a solution to a large number of explicit and implicit motivations, incentive systems, fears and hopes experienced by the human users. We run into many researchers for which ‘data is king’ – and as they lead a data collection project they run into the reality that other people care about other things besides just collecting that data.

One useful tool we use at InSTEDD is a simplified approach to going through payoff matrices of the type used in game theory.  We use this to evaluate the ‘payoff’ of people doing or not doing an intended or unintended action. The payoff matrices are just a guide for our dialogue to make sure we ask all the pertinent questions, such as:

• Why would you do X instead of Y? or
• Is it better for you to do A or B?

You can roughly quantify the answers and see what actions are more likely to happen. Sometimes this leads to discovering ways in which your users ‘game the system’ – which can be a slightly tense, humorous and often embarrassing moment.

There is usually a complex web of incentives that shape the situation.  Incentives related to money, power, social/professional hierarchy, geography or even a sense of righteousness are all common influencing factors. Attempting to analyze such an intertwined and complex web can be a fool’s errand.   While we may never be able to fully understand all the factors at play, it’s important to investigate as much as possible so that we can design a solution that will be of value.

 

One of the greatest things about the InSTEDD iLabs are that they are made up of local staff who already have a deep understanding of the region, culture, language and many of the obstacles the users face within this context.

 

When some type of monetary reward is brought in as an incentive, payoff matrices rapidly become distorted.  Many people approach InSTEDD with a design idea that will give the people who are using their technology a financial reward when certain data gets reported within a specific time frame.  Unfortunately, I have yet to encounter any situation where providing money as a reward for data doesn’t skew the data beyond repair. Let’s say you give the health care workers who are reporting diseases some sort of cash bonus if they refer a tuberculosis patient that tests positive in a lab to treatment.  By doing this, you’ve just created an incentive to have a health worker time the referrals to suit their own needs.  The health worker may know of some tuberculosis patients in another village who need treatment, but the health worker may not refer them to treatment until it makes financial sense for him to do so.  The sick patients can quickly become some sort of savings bank account or ATM for that individual health worker, as opposed to creating a collaborative “we’re all in this together because we are serving our community” incentive structure.

Many researchers like to think of themselves as objective and impartial as they focus on “the data”, but all too often these researchers unconsciously set incentive systems that distort the data they are helping to collect.  They naively hope that if they pay for the data, then they will see the quality and consistency they desire.  Unfortunately, many of the people who are in the field collecting data are focusing on other layers of Maslow’s hierarchy of needs and see the financial incentive as a great opportunity for them to take advantage of the system.

 

An interesting trend is the incorporation of “gamification” techniques which works by making technology more attractive and encouraging users to engage in desired behaviors by taking advantage of humans’ psychological predisposition to play games.  Some examples of these gamification techniques include virtual currencies, hierarchy status, ranks, medals, titles, commendations, points and levels, just to name a few. These virtual incentives become truly valuable in the participant’s mind and they have been shown to dramatically change behaviors.

Military groups have long exploited gamification techniques and use video games as simulations for combat.  Military groups have played off of human motivations and psychology for most of their history, and are now incorporating much of their techniques into social networks and virtual communities in order to increase the social exposure of participants.  Just like someone can become a ‘Mayor’ in FourSquare, data collection users may be motivated by the opportunity to show off their ‘Perfectionist’ medals and ‘gold credits’ for doing things well. I have yet to see, however, better research in how these gamificiation patterns work for people who spend more time closer to real basic human needs (lower in the Maslow pyramid).

Does the data matter to your users?

One design approach to improving the quality of data that is collected is to ensure that the data being submitted actually matters and is useful to the user. This may take the form of responding with aggregate reports, comparative trends with neighboring areas, or some other piece of useful information that closes a feedback loop to the user. Some of the questions we’ve learned to keep asking are:

• How does this data, or what happens to it, matter to the people doing the collection?
• What could they do differently because of their connection to the data?
• What do they aspire to do more of, or to do better?
• How can the data they receive back be used to help them achieve it?

For example, community health workers sometimes enjoy a simple elevated ‘status’ because of their work. Sometimes, getting health information about what is going on in neighboring villages can help them maintain that status as it helps them stay ‘in the know’. The information sent back need not be useful in of itself, it just needs to make the receiver happy and somehow be derived from the data submitted by them in the first place.

Using a Suite of Tools: InSTEDD Technologies

Based on years of experience working with users within their own contexts, we’ve developed a suite of tools that help people turn data into action to support health, safety and development.  We highly encourage you to browse through our free and open source “modern tool belt” to help you better collect, analyze and use data to achieve a maximum social impact.