Impact in a nutshell

Ever since theParis Declaration of 2005, the focus of the international development has been on “managing and implementing aid in a way that focuses on the desired results”. Monitoring and evaluation (M&E) based on predefined indicators has been the method of choice in making outcomes and impact visible. While project-specific M&E is commonplace and national indicators have experienced an upsurge (MDGs, SDGs), the organizational and institutional levels of impact measurement have by and large remained underdeveloped.

Recently, both Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) and DW Akademie developed methodologies whereby the total aggregate impact of their activities funded by the German Ministry for Economic Cooperation and Development (BMZ) might be measured. The aim was to enable the organizations to make statements about the overall effects of their development measures, beyond the level of individual funded projects. The following is an account of DW Akademie’s methodology for aggregate impact. For GIZ’s methodology, click here (in German).

The challenge of measuring aggregate organizational impact is manifold. With projects starting and ending at different points in time, what time frame do you use to measure them all? How do you define aggregate indicators transcending cultures and national contexts? What topics or aspects of media development measures are suitable for aggregate measurement?

DW Akademie adopted a Strategic Model in 2014, outlining four areas of action: Political and legal frameworks, qualification, professionalization / economic viability of the media sector, as well as participation in society. These strategic areas were used as a guiding framework for determining the aggregate indicators. In a pilot assessment ten topical indicators were derived from them, documenting the impact of the Akademie’s recurring activities for the period 2014 to 2016, such as fostering media and information literacy or supporting community media. The ten topics were chosen based on the following criteria:

• The topic should be established in the international media development community and be of strategic importance to DW Akademie;
• the topic should be of public interest;
• the topic should be relevant in several projects so as to justify reporting it at the aggregate level.

The backbone of aggregate impact measurement is DW Akademie’s results-oriented monitoring system, defining project-level indicators, the data of which can be used to cumulate the total impact achieved. Additional aggregate indicators specify how the aggregate impact is to be measured over a pre-defined time period. Only aspects that can be said to effect substantial, structural changes are eligible for aggregation, such as the expansion of media outlet offerings, the creation of new media outlets or newly approved media legislation. This is why the measurement period (in this piloting case three years) should not be a point in time, but rather an interval that provides enough room for structural results.

Every indicator deals with one topic, is based on absolute numbers (rather than percentage values), has to allow for easy measurement and should be expressed in a precise manner. SMART (specific, measurable, attainable, relevant, time-bound) criteria apply to them as much as they do to project-level indicators. Nevertheless, in some cases measurement is based on plausible estimates, for example based on the technical reach of media, for lack of hard data.

Before going into the details of DW Akademie’s aggregate impact for 2017, a word of caution should be expressed. The quantification of impact should not result in more populous regions and topics that achieve the highest reach to be treated as preferential. The Akademie’s human rights based approach stipulates that cooperation takes place where it is needed most, and this is often in sparsely populated rural areas or in the context of minority segments of the population. This focus remains untouched by efforts to quantify total impact. 

Also, the impact achieved is not something DW Akademie can claim exclusively, since our work is based on partnerships with on-site organizations that contribute decisively to the success of these projects. The following are the aggregate impact topics reported on for the period 2017:

- Media councils

The work of press councils in 4 countries has been reinforced with the support of DW Akademie. Press councils investigate complaints about journalists and the media, and monitor breaches of the press code. 7,700 journalists recognize these councils, which have investigated 121 complaints over the last year. 99 million citizens can now file a formal complaint about potential breaches of media ethics in their country. 

- Media legislation

Thanks to a new broadcasting law, 51 million Myanmar citizens have gained the right for the first time to establish their own community radio. In order to achieve this, DW Akademie advised policymakers on topics concerning freedom of expression and community media, and helped with a pilot project to set up the country’s first community radio station.

- Training for media professionals

DW Akademie has worked in 13 countries to help set up 44 practice-based, topical training programs for media professionals. 945 students have gained qualifications with partner institutions, helping to lay the foundation essential for professional and diverse reporting.

- Conflict-sensitive reporting

DW Akademie has helped 146 media organizations sensitize and diversify their reporting on conflict and the situation in their countries. 12 million people in 11 countries were thus given access to objective information on conflicts and crises. The media have an important role to play here in actively promoting peace.

- Sustainable business models

DW Akademie has helped 5,500 people working for 149 media outlets develop sustainable and innovative business models for their places of employ. The aim is to enable journalists, supplying their fellow citizens with relevant and professional content, to make a decent living in their chosen profession.

- Youth programs

Young people* need to be able to understand events in their country in order to contribute to the shaping of their futures. DW Akademie has helped promote 171 youth-oriented media, program formats and youth projects. These gave 680,000 young people in 14 countries access to information relevant to them. This enables them to have their say and speak out against ills and wrongdoings.

- Innovation and dialogue online

DW Akademie has helped 129 media organizations prepare for the challenges of digitization. 99 media partners have been able to expand their reach – through the use of new digital media formats such as social media platforms, blogs and virtual radio stations. 1,3 million people in 25 countries are thus now able to access a more diverse range of information and can participate in public debate.

- Public service media

35 million people in 3 countries have been able to profit from changes to their state media. Support and consultation, structural reforms and training programs have helped to make reporting more varied and programs more attractive for their audiences. With the support of DW Akademie, state media have been transformed into public service media.

- Media and Information Literacy

500 change agents have helped young people in 10 countries with their levels of media literacy. As a result of the training programs, 13,000 young people can now use the media at their disposal responsibly. They can analyze content more efficiently, make informed decisions and differentiate between objective news content and rumors or propaganda. They are better able to make good use of their right to freedom of expression and access to information.

- Community media

Worldwide, 190 community media outlets give disadvantaged populations a voice and support their basic freedom of expression. DW Akademie has helped train 2,000 citizen journalists working for local stations, which means that 15 million people in rural areas have access to relevant local information.

- Strengthening civil society

DW Akademie has provided consultation services for 79 non-governmental organizations working in 16 countries for the improvement of human rights and freedom of expression. This has opened up a platform for dialogue between state institutions, civil society and media stakeholders, and improved the access to information and social participation of 800,000 people.

- Media for refugees

DW Akademie has helped refugees in border regions and refugee camps, among them 67 displaced journalists and 22 citizen journalists, with targeted training programs. With the help of 23 media organizations, 3,5 million people have been reached, in a vital contribution towards mutual understanding and communication between diverse ethnic groups, refugees, local populations and those left behind at home.

All figures and information are based on project data and refer to the calendar year 2017.

Data on people reached (inhabitants, people, adolescents) are estimates based on social media data and broadcasters’ data, and adjusted with reference to mean values from previous years.

* DW Akademie concurs in its definitions of youth with that of the country in question.