How WhatsApp can help you assess the impact your projects

It's one thing initiating a media development programme, but it's another making sure it meets the needs of those being targeted. Audience feedback is an important element in designing information projects and it must be inclusive.

In the Sahel, given the deteriorating security situation and associated humanitarian crisis, providing information to communities, who are isolated because of conflict and more recently the Covid-19 pandemic, is vital. However, collecting feedback from these communities using traditional methods such as focus groups or face-to-face surveys can prove impossible. Restricting feedback just to those in the accessible capitals would hardly be representative.

In a research project called Radio and Women's Empowerment in the Sahel, we implemented an innovative approach: Using WhatsApp to contact conflict-affected communities in West Africa. Subsequently, further projects adopted this method to reach populations isolated because of Covid-19.

The project, a collaboration with the Swiss-based media development NGO Fondation Hirondelle, aimed to assess radio's impact on women's empowerment by investigating broadcasts produced by Fondation Hirondelle's studios in Mali, Niger and Burkina Faso (Studio Tamani, Studio Kalangou and Studio Yafa respectively). What are best practices and areas for improvement? Which formats and themes are of interest to listeners? How has their knowledge and behaviour changed after listening to programmes?

Building up a remote feedback platform

We recruited participants throughout the countries with the help of local radio stations. All communication was remote. They were introduced to the project via introductory messages sent in local languages (oral consent was gathered at this stage too). Then, we sent questions via WhatsApp broadcast lists which enable bulk quantities of messages to be sent (up to 256). Unlike group chat messages, respondents reply directly to the sender and are not party to other people's responses thus giving them privacy to speak without the influence of group members.

The questions were sent out in batches; over the course of four days we sent four questions per day four times in 2020 on a particular theme. We used closed questions or short multiple-choice questions to obtain quantitative data and two long questions to get richer qualitative information. Correct question design was fundamental to optimise the quantity and quality of responses. The questions had to interest the respondents and be in an understandable format for them to answer.

In Niger, a women's listening group was set up as a result of the collaborative project work

Participants responded by written or voice messages which allowed those with low literacy rates to respond and thus provided qualitative data – a major advantage. Rarely did participants respond by text. We passed on participants' feedback to the radio studios as we needed to ensure that not only were isolated communities' voices heard but that they were listened to and acted upon. 

A flexible and empowering approach

The method permitted a large geographical reach allowing access to those in conflict- and pandemic-affected areas and nomadic communities. Participants appreciated the format and simply being able to be included. "We liked being involved and also seeing how we contributed to changes," they said.

According to the women participants, the asynchronous nature of WhatsApp meant that they were able to respond when convenient to them around their domestic chores. They also liked being able to answer in private in contrast to focus groups or face-to-face interactions. Many respondents already used WhatsApp and were in multiple WhatsApp groups so the app was familiar and the cost to the research team was relatively low (no transport, room hire, refreshments etc.).

One benefit is that this method can provide a snapshot of a situation. If feedback is needed instantly in response to a specific situation, questions can be quickly sent out to communities, answered and analysed for action. From a participant perspective, it is good for low literacy populations, participants are not influenced by peers, and groups of participants can be converted to listening groups for ongoing feedback.

A major outcome of this online data collection is that listeners are no longer passive receivers of information. Providing increased participatory opportunities for communities in programme design by simply asking listeners what they want, will lead to a shift away from top-down imposition of assumed information needs from broadcasters.

Working with WhatsApp is not a perfect solution for everything, and it does indeed have many limitations, including connectivity, technical skills, drop off rates, and knowing who was actually responding. However, it does its job and that is to reach conflict- and pandemic-affected communities and ensure they have a voice. It's another tool in the box. It is then up to NGOs, authorities, humanitarian agencies and others to listen to that voice.

Dr. Emma Heywood is a Senior Lecturer in Journalism, Radio and Communication at the University of Sheffield in the UK. With a particular focus on women's empowerment, her research examines the role of the media, and particularly radio, in conflict-affected areas.