I turn 50 this month (eep!). If you told my teenage self that I would one day decide what to buy, where to go on holiday, and where to work based on Amazon, Tripadvisor, and GlassDoor ratings, I don’t think I’d have believed you. And yet – here we are.
The growing pervasiveness of these digital ratings has pushed me to consider whether they could be leveraged by the humanitarian research community. How would this system work? Would it be a good idea?
How Might It Look?
The ideal outcome of applying a Tripadvisor model to aid is being able to compare interventions of all shapes and sizes around the world.
The “AidAdvisor” would therefore have to work at scale, collecting information from as many people as possible. To achieve that scope and give access to all community members, the system would probably bypass monitors, allowing anyone to take part whenever they wanted.
We’d need to embrace digital tools and platforms, such as WhatsApp, Facebook, or an endless list of other options.
Mobile users in countries receiving aid could directly use these platforms to provide feedback. For instance, QR codes could be placed at food distribution sites, allowing recipients to scan the code and rate their satisfaction with the distribution. These sites could even be directly embedded in Google Maps.
There would need to be a central coordinator – one that communities know and trust. Probably the UN. The involvement of the UN would assure communities that the initiative was serious and their data was safe. NGO and CSOs could link into this core UN set-up, accessing free, user-friendly software.
Of course, we’d need to ensure the inclusion of the vulnerable and illiterate. For this, we might learn from Twaweza’s ‘Sauti za Wananchi’ initiative, which distributes mobile phones and solar chargers to communities without.
Would It Be a Good Idea?
It’s hard to say at this point. There are so many challenges. One obstacle would be fairness. Some projects are just more difficult to implement than others. I’ve evaluated fantastic projects that would receive low scores simply because they were in war zones where nothing worked perfectly. Recipients in this context are unlikely to give 5-star ratings.
A platform like this could also over-simplify – or even belittle – the work of civil society. Do human rights organizations exist in the same universe as companies offering packaged vacations in Cancun?
Another issue is that the direct beneficiaries are often not communities, but a limited number of individuals managing a project. If, for example, we were reviewing a small project providing teacher training, we would want feedback from the teachers at least as much as the children. Surely a healthier way to give feedback in this situation would be directly and intimately between the teachers and funder.
For some projects, the idea may not work, and that’s okay. And for many projects, sample sizes (and, certainly, statistical precision and representation) would represent significant problems.
But the upsides are also alluring! Above all, it would give communities a stronger voice. And it could do it efficiently! For me, a one-click survey offers a more respectful, sustainable way forward than lengthy surveys.
Foregoing monitors could lead to data quality issues, but it would also reduce social desirability bias. Of course, many vulnerable people don’t have phones, but a growing number do.
Perhaps the greatest benefit would be the ‘back-end’. I can only imagine the feverish debate that the initiative would create among senior officials in DC, Brussels, and London – eyes on stalks as they see which programs were rated better and worse by those who matter most. If we add the ability to upload videos (just like you upload photos of grimy bathrooms on Tripadvisor), the whole thing reaches another level.
Would evaluators start leaning on this? Could it eventually replace Third-Party Monitoring? Would I need to look for a new career? I think we’d glean real insights at the demographic, geographic, and thematic levels.
For the cynics out there who believe there’s just too much competition baked into the system for this to fly, the automotive industry already did it. Years ago, car makers realized that progress would require knowing who was better than whom, at what, and why. So, they took a bold step and asked one company, JD Power, to manage the whole industry’s customer satisfaction data. It revolutionized the industry and made consumer feedback easy.
On balance, I think it’s time for someone to pilot this idea. My gut tells me that investing in transparency is a good bet.
“AidAdvisor” may only give us a small part of the data picture – and one we’d hopefully learn to use judiciously. But I think we have slept walked into the creation of a humanitarian data space comprised of myriad isolated data sets. Much of the real value seems buried and yet to be unleashed!
Richard Harrison has 25+ years of experience leading MEL and research projects for the UK, EU, UN, US, Canada and World Bank, working on the ground in over 30 countries spanning the Middle East, Africa, and Asia.