MEDIA SUPPORT FOR STRENGTHENING, GOOD GOVERNANCE AND EMPOWERMENT (MESSAGE) MID-TERM EVALUATION REPORT FINAL REPORT July 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Nigeria Monitoring and Evaluation Project (NMEMSII), implemented by The Mitchell Group, Inc. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. MEDIA SUPPORT FOR STRENGTHENING, GOOD GOVERNANCE AND EMPOWERMENT (MESSAGE) FINAL REPORT CONTENTS EXECUTIVE SUMMARY I INTRODUCTION 1 BACKGROUND 2 EVALUATION – SCOPE OF WORK 9 METHODOLOGY 10 FINDINGS 15 CHALLENGES 34 LESSONS LEARNED 35 KEY INFORMANT SUGGESTIONS 36 CONCLUSIONS 37 RECOMMENDATIONS 39 ACRONYMS AKBC Akwa Ibom Broadcasting Corporation BBCMA British Broadcasting Corporation Media Action ( BBWST British Broadcasting World Service Trust CSOs Civil Society Organizations FRCN Federal Radio Corporation of Nigeria MCVS Midline Citizen Voice Survey October 2011 MESSAGE Media Support For Strengthening Advocacy, Good Governance, and Empowerment SWOT Strength, Weakness, Opportunity, and Threat TTT Thematic and Technical Training USAID United States Agency for International Development WAHAP Women Against Hunger Amidst Plenty LIST OF TABLES Table 1 MESSAGE Programming Content Analysis Table 2 Showing the broadcast pattern of Story Story in the past one year (from analysis of the media monitoring data) Table 3 Showing lesson learnt from listening to Story Story (multiple responses) Table 4 Showing the proportion of Story Story listeners who thought differently about anything as a result of listening to the drama (multiple responses) LIST OF FIGURES Figure 1 Understanding the Need for Editorial Values in Program /News Making Figure 2 Applications of Editorial Values in Program Production/News Making Figure 3 Uses of Various Elements of Audience Interaction to Produce Engaging Programs Figure 4 Audience Generated Content in Program Making Figure 5 Chart showing the change in Story Story audience base from baseline to midline Figure 6 Chart showing the distribution of Story Story listenership within the past one year Figure 7 Chart showing proportion of respondents who had thought differently or done anything differently as result of listening to Story Story Figure 8 Chart showing the proportion of respondents who have done something differently as a re￾sult of listening to Story Story (Multiple responses) MESSAGE I EXECUTIVE SUMMARY The BBC MEDIA ACTION MESSAGE project aims at enhancing the ability of the media and civil society in six focal states, these states are Anambra, Akwa Ibom and Bayelsa as one state (Akwa-Bayelsa), Bauchi, FCT (Abuja), Lagos as well as Sokoto and Kaduna as one state (Sokoto-Kaduna), to play a more effective role in encouraging responsive and accountable governance in Nigeria. The Project goal was to have a plurali￾ty of twenty-four radio stations, federal, state and private. Four stations were to be selected from the six focal areas. With little media plurality in northern Nigeria, Kaduna and Sokoto were combined to include a private radio station. In southern Nigeria, USAID wanted the Project to include Baysela. However, BBCMA securi￾ty concerns led USAID to compromise with combining Akwa Ibom and Baysela with two stations from each state. The project focused on the radio sector as it is the most accessible medium in the country, ensuring that the planned project outputs reach the widest possible audience. However, there were not enough radio stations available in some of the focal states, thus in certain states a television station was selected. Overall, the objec￾tives of the project, which were developed in consultation with USAID Nigeria, were threefold:  To increase the Nigerian media’s capacity to serve the public interest through greater professionalism  To improve the Nigerian media’s technical and thematic capacity to produce radio programs which engage audiences in informed discussion around governance issues such as oil sector transparency, effectiveness of government institutions, management of public resources, health and water man￾agement and community service delivery, education and conflict mitigation.  Build the capacity of civil society organizations to use the media. These objectives fit closely with the BBCMA’s own approach to governance work, which is built around three strands – enhancing transparency, accountability and participation. These media development activities were comprised of intensive on-station training, mentoring and distance learning projects and should have resulted in the production on each partner radio station of interactive mag￾azine-style outputs to raise the audience’s awareness of and knowledge around the key issues of national rele￾vance, looking, for example, at particular service delivery or policy issues from the perspective of citizens. The magazine-style format incorporated a range of mini-formats, brought together by the presenter. These might include audio diaries, head to head interviews, panel discussions, call-ins responding to an interview, feature or news story, mini dramas, and short documentary reports which investigate issues in some detail and can be revisited for updates throughout the project’s lifetime. Radio projects should have been highly interactive, with audience participation a central feature. In addition to media development activities, the Project produced and transmitted its ever-popular govern￾ance drama, Story Story. During pre-production, the Project convened with USAID and representatives from civil society groups with whom USAID has already been working with to agree on governance themes and messages. Each of the three series were initially transmitted on the BBC World Service, and then on the entire Project’s partner radio stations throughout Nigeria. This way, a USAID-supported Story Story was on air somewhere in Nigeria throughout the project’s lifetime, stimulating discussion about the themes contained within and building popular engagement with the governance issues which USAID is keen to address. MESSAGE II In addition to providing media training workshops and online training modules to civil society groups identi￾fied by USAID, the project’s media trainers fostered the development of long-term relationships between civil society groups in the six states and the Project’s media partners in order that each can continue to posi￾tively inform the other’s activities after the project has ended. Evaluation for this Project was severely limited due to the scope of work (SOW) constraints, providing only 12 in-country days for a qualitative in-depth interview research with key informants. Complicating the evalu￾ation were security issues which caused the team members to modify the sample of focal states due to USAID Nigeria restrictions. Over a period of 12 days, the evaluation team flew to the samples’ 3 focal states conducting 31 interviews with key informants. In addition to the interviews with key informants, the evalua￾tion team examined and reviewed MESSAGE reports and studies as part of this assessment. In light of these constraints and limitations, care should be taken in generalizing the evaluation results. The evaluation’s objective was to assess the BBCMA MESSAGE project results concerning the “Nigerian media’s capacity to cover governance and related issues.” Through thematic training, both editorial and tech￾nical, the MESSAGE project was successful in influencing the evaluation’s key informants and stations to cover governance related issues and themes. The evaluation team found that the BBCMA MESSAGE train￾ing effectively improved the professional editorial and technical skills of the key informants. Through the thematic training the key informants understood the importance and relevance of governance issues, and the impact information about the issues has on radio audiences. The result was that the MESSAGE key inform￾ants engaged in covering and developing programming on governance issues, particularly with the help of the Bursary Awards. Important in the evaluation team’s findings are that every participant interviewed had im￾plemented the principles they learned, investigating, researching, and getting the opinions of the people on issues to produced balanced news and programs for their respective stations. Management at participating stations in the sample was enthusiastic about the training their employees re￾ceived from the MESSAGE project. Every station manager observed that they came away from the training with a new understanding about the need for programming and marketing to work together on programs that serve and attract audiences and advertisers. From the evaluation interviews managers indicated that, after returning to their respective stations after the MESSAGE training, they immediately implemented what they had learned. Station managers interviewed noted that the revenue enhancement learned via MESSAGE train￾ing was being applied and they were seeing success. The evaluation also involved assessing the MESSAGE project’s entertainment drama Story Story series funded by USAID. Analysis of the Project’s research indicates the USAID funded Story Story series had influence on a considerable number of its listeners, empowering them to become more engaged in governance issues and the electoral process. The BBCMA research results showed that a majority of listeners were influenced by the USAID funded Story Story series, motivating them to think differently about governance issues, register and vote in elections, and influence others to do the same. The CSO component for the MESSAGE project presented the evaluation team with the most daunting as￾pect for assessing the MESSAGE project. The BBCMA reports noted positive impact in training CSOs on how to connect with the media through press releases and programs attractive for broadcast. The two inter- MESSAGE III views with CSOs conducted by the evaluation team presented conflicting results. One individual observed that the sessions were helpful, but very limited. The other interviewee felt the MESSAGE trainings were very useful and helped to develop media relationships. However, both noted that they had difficulties with the Media Dark Kit that was supposed to be used to support the Story Story component of the project, neither one used it. Yet, the BBCMA MESSAGE reports indicate positive results in other states. This is something the evaluation team was unable to verify directly. One interesting discussion that evolved during this evaluation concerns whether the MESSAGE project was attempting to do too much. While there is logic of the holistic approach that the MESSAGE project took in training media professionals and CSOs, the evaluation team felt that it was possibly stretching the limits and resources of those involved in the training and administration. From interviews with key informants, this project accomplished a great deal regarding training and developing media professionals, especially with re￾gards to governance issues and engaging audiences. But it is difficult to assess how successful the CSO com￾ponent was compared to the rest of the MESSAGE project, particularly with regard to having empirical evi￾dence available. Finally, the administration of the MESSAGE project by USAID and the BBCMA encountered difficulties due to factors beyond their control. The evaluation team makes the following recommendations: 1. Future programs should consider the Nigerian media environment. FCRN and state owned and op￾erated stations experience censorship. In addition, both government and private broadcast stations have limited resources for costs associated with investigative reporting and for internet access, which is needed for research and audience interactivity. Also, private stations are profit oriented and will not invest in programming unless it will produce revenue. The evaluation team learned that station managers respect the BBC editorial standards and can be influenced to air programs that follow those principles. Trainers should educate and discuss with the participants about how those stand￾ards can be used to influence station management to air programs. In addition, more management training is needed concerning the revenue benefits of quality programming, including marketing and promotion. 2. The MESSAGE project should consider the computer literacy of participants and internet access at the stations and the availability to the trainees. In addition, the MESSAGE website should be de￾signed so that participants can share programs and interact with one another and with CSOs. 3. The MESSAGE project’s interventions should target the educational system, i.e., those organizations responsible for training media practitioners. This would include journalism and media instruc￾tors/professors with an emphasis on a practical, hands-on approach. Thus, journalism’s editorial standards and best practices could be incorporated into the curriculum at such institutes. 4. The MESSAGE project should incorporate more training on social media. Participants are discov￾ering the importance and utility of engaging audiences, and desire more training in this area. 5. USAID should continue to fund the Story Story series on governance issues. The series documented success is noteworthy. The show has a following and is well established among the Nigerian popula￾tion. 6. As emphasis is laid on governance and the Civil society organizations are key, more activities should be designed to involve those organizations 7. Future SOW for evaluations needs to provide enough time in-country and multiple methodologies to adequately assess a project as extensive as MESSAGE. MESSAGE 1 INTRODUCTION This report contains the evaluation findings for the USAID funded BBC MEDIA ACTION Media Support For Strengthening Advocacy, Good Governance And Empowerment (MESSAGE) project. Commissioned by the USAID/Nigeria Mission, the evaluation was conducted from February 20, 2012 to March 3, 2012. The purpose for this evaluation was to assess the results of the MESSAGE project to date, providing evi￾dence of success, failure, obstacles and lessons learned; and to provide recommendations and strategies to improve the MESSAGE project over the remaining life of the project. This evaluation was guided by three general questions and nine specific questions as outlined in the Scope of Work (SOW), found in Annex A. The first part of the report provides the background on the challenges confronting Nigeria and a description of the BBCMA MESSAGE project. The remainder of the report discusses the evaluation goals, methodology, findings, discussion and conclusion, challenges, lessons learned and recommendations. MESSAGE 2 BACKGROUND Confidence among Nigerians in their democracy is at an all-time low. A recent survey found that only one in four respondents had confidence in their system of governance, compared to 80% in 19991 , the year in which Nigeria held its first elections after years of military rule. After thirteen years it is clear that the majority of the population do not believe that democracy has lived up to their expectations. The country’s leadership has great ambitions to make Nigeria one of the world’s 20 most economically pow￾erful nations by 2020. With such vast natural and human resources at its disposal, combined with Nigeria’s pivotal role in Africa today, the potential is clear. However, Nigeria appears to be travelling in the opposite direction, and Human Development Indicators are worsening, although GDP has risen. There is an over￾whelming disillusionment with the Government’s ability or willingness to deliver development that would improve the lives of the Nigerians and provide the opportunities needed for them to contribute to economic growth. The poverty suffered by many Nigerians is at odds with the country’s vast potential oil wealth. A recent World Bank report entitled ‘The Missing Link, Fostering Positive Citizen State Relations in Post￾Conflict Environments’ argues for the necessity of a well-developed public sphere in fostering effective de￾mocracy: ‘Participatory processes, accountable and transparent institutions, and constructive citizen-stage re￾lations require a national dialogue platform that only a functioning public sphere provides; it is here where citizens and state meet through the exchange of information and expression of opinions and where citizen￾state relations and processes are shaped.’2 The report goes on to note that a well-functioning public sphere is one where state institutions are transparent, responsive and inform the public about their activities. Where the media is self-regulating, professional and delivers information to the public, who can then form an opinion about the performance of the state. The public’s opinion is then fed back to the state through civil society groups and the media, and this contributes to the shaping of policy.3 BACKGROUND – USAID FUNDED BBCMA MESSAGE PROJECT The media are central to the relationship between citizen and state. The media can play a critical watchdog role in holding the government accountable by questioning government policy in the public interest. In addi￾tion, the media can foster debate which is necessary for the public to feel they can influence development strategies in such areas as health, education and poverty reduction at all levels of society. However, the me￾dia’s role to provide quality independent, plural and accessible journalism is substantially undermined when it is politically and economically insensitive, and does not act in the public interest by demanding accountabil￾ity.4 The watchdog function of the Nigerian media is enshrined in the constitution, and charges the media with upholding ‘the responsibility and accountability of the government to the people’. Yet the media’s ability to 1 Nigerbus Survey 2 P9, ‘The Missing Link, Fostering Positive Citizen-State Relations in Post-Conflict Environments’ 3 P 9, ‘The Missing Link, Fostering Positive Citizen-State Relations in Post-Conflict Environments’ 4 Pg 14,’ BBC MESSAGE Cooperative Agreement #620-A-00-09-00016-00 ’ MESSAGE 3 support democratic principles of participation, transparency and accountability and act as a genuine counter￾weight to government is limited in practice. Nigeria is placed at 131 of 161 in the Press Freedom Index. 5 The reasons for this low rating are manifold. The national media outlets with the greatest reach are owned and run by the State which imposes significant self-censorship by limiting the media’s ability or willingness to challenge government policies and activities. Private newspapers, TV and radio stations tend to have greater editorial freedom. But they are still regulated by the National Press Commission, and can have their licenses suspended or revoked if they step out of line or attempt to address sensitive issues. In addition, libel is a crim￾inal offence and the onus rests on the defendant to demonstrate innocence. 6 Furthermore, private ownership does not necessarily guarantee independence in editorial policy, especially since official government agencies tend to be the source of news. Private media outlets also need to generate revenue in order to be commercially viable, and selling airtime to the highest bidder, which is often the gov￾ernment, can significantly affect their choice of content. 7 Aside from the question of ownership, journalists tend to be poorly paid and susceptible to being compro￾mised. They may lack the will and/or resources to demand access to information which should be in the pub￾lic realm. Many are not well trained, which means that they may not fully understand the stories they are re￾porting, or be sufficiently skilled in applying standards of impartiality, objectivity and accuracy when writing reports. Investment in training and infrastructure is badly needed. Media practitioners need to be sufficiently well informed by having access to basic factual information which will ensure accurate coverage, stimulate and facilitate public debate. 8 Taken together, these factors result in the public not receiving the information it needs to ask questions or participate in decisions about the way the government runs the country at all levels. As a result, there is com￾pelling evidence that a large proportion of Nigerians lack confidence in their democracy, its elections and the electoral process. This pessimism derives from the country’s poor record of managing elections, which have often been bedeviled with violence and malpractice. Citizens have also grappled with discrepancies between campaign promises and elected politician’s commitment to deliver on such promises. 9 Citizens have to make a real and concerted effort if they want to be well-informed. Moreover, the media does not yet offer itself as a forum through which questions can be asked, either by journalists on behalf of audi￾ences, or by the people themselves. Ultimately, a situation has arisen whereby the State is free from the need to make it accountable. 10 According to a World Bank report, ‘the quality of the public sphere is influenced by the quality and quantity of the input – the information released by different state institutions and by civil society, as well as the listen￾ing capacity of these actors. It hinges further on the capacity and professionalism of the media to select, pro- 5 Pg 3 ‘MESSAGE Year 1 Activity Report – 1 October 2009 to 30 September 2010’ 6 Pg 14,’ BBC MESSAGE Cooperative Agreement #620-A-00-09-00016-00 ’ 7 Pg 14,’ BBC MESSAGE Cooperative Agreement #620-A-00-09-00016-00 ’ 0 8 Pg 14,’ BBC MESSAGE Cooperative Agreement #620-A-00-09-00016-00 ’ 9 Pg 14,’ BBC MESSAGE Cooperative Agreement #620-A-00-09-00016-00 ’ 10 Pg 4 ‘MESSAGE Year 1 Activity Report – 1 October 2009 to 30 September 2010’ MESSAGE 4 cess and shape available information before it reaches citizens…A positive relationship between state and media, between state and civil society and between civil society and the media allows for constructive dynam￾ics’11 . The media must come to terms with factors that impinge on their objectivity and balance, be they imposed or self-inflicted. They must be aided to deal with the limitations that are undermining the profession and pre￾venting the media from playing the veritable watchdog role. It is important to restore people’s faith in de￾mocracy, and the media is in a unique position to make people more aware of the issues involved. Media pro￾fessionals must acquire better skills in handling these issues and communicating them to audiences. The lynchpin of the MESSAGE project was to boost the capacity of the media to play a more effective role in enabling citizens to claim their rights and hold the government accountable. This evaluation assessed the role of the project to effectively enhance the ability of the media and civil society to have a more responsive and accountable government. 12 The aim of this project was to enhance the ability of the media and civil society to play a more effective role in encouraging responsive and accountable governance. The project sites were located in six focal areas con￾taining eight Nigerian states, Anambra; Akwa Ibom and Bayelsa were treated as one group (Akwa-Bayelsa); Bauchi; FCT (Abuja); Lagos; and Sokoto and Kaduna were treated as one group (Sokoto-Kaduna). With little media plurality in northern Nigeria, Kaduna and Sokotu were combined to include a private radio station. In southern Nigeria, USAID wanted the Project to include Baysela. However, BBCMA security concerns led USAID to compromise with combining Akwa Ibom and Baysela, with two stations from each state.. A list of MESSAGE Project participating stations can be found in Annex B. 11 Pg 11, ‘The Missing Link, Fostering Positive Citizen-State Relations in Post-Conflict Environments’ 12 Pg 4 ‘MESSAGE Year 1 Activity Report – 1 October 2009 to 30 September 2010’ MESSAGE 5 Sokoto Kaduna FCT Bauchi Lagos Bayelsa Akwa Ibom Anambra The project focused on the radio sector as it is the most accessible medium in the country, ensuring that the planned project outputs would reach the widest possible audience. However, there were not enough radio stations available in some of the focal areas, thus in certain states a private television station was selected. The media development activities were comprised of intensive on-station training, mentoring and distance learning. These activities should have resulted in the production of interactive magazine-style outputs by each partner radio station to raise the audience’s awareness and knowledge about key issues of national relevance. The magazine-style format incorporated a range of mini-formats, brought together by the presenter. These formats might include audio diaries, head to head interviews, panel discussions, call-ins responding to an in￾terview, feature or news story, mini dramas, and short documentary reports which investigate issues in some detail and can be revisited for updates throughout the project’s lifetime. The radio projects have been highly interactive with audience participation a central feature. In addition to media development activities, the project produced and transmitted the popular governance drama, Story Story. During pre-production, USAID and representatives from civil society groups with whom MESSAGE 6 USAID has already been working agreed on the governance themes and messages. Each of the three series have initially been transmitted on the BBC World Service, and then on all the project’s partner radio stations throughout Nigeria. The USAID-supported Story Story should have aired somewhere in Nigeria throughout the project’s lifetime, stimulating discussion and building public engagement on the governance issues that USAID is keen to address. The project recognized the importance of engaging civil society actors. In addition to providing media train￾ing workshops and online training modules to civil society groups identified by USAID, the project’s media trainers fostered the development of long-term relationships between civil society groups in the six focal areas and the project’s media partners for each to continue to positively inform the other’s activities after the pro￾ject has ended. In addition, the project produced a Media Dark Kit in which Story Story clips illustrate key governance themes, along with a discussion guide, that civil society organizations can use in their work at the community level. Groups were trained by the project to ensure the kits are used effectively. Finally, the project was informed by research activities delivered by the in-house audience research team. These activities included formative research; needs assessments with media partners to inform training activi￾ties; and pre-testing of outputs to ensure the audience has received the messages as intended. Overall, the objectives of the project, which were developed in consultation with USAID Nigeria, were three￾fold:  To increase the Nigerian media’s capacity to serve the public interest through greater professionalism  To improve the Nigerian media’s technical and thematic capacity to produce radio programs which engage audiences in informed discussion around governance issues such as oil sector transparency; effectiveness of government institutions; management of public resources; health and water man￾agement; community service delivery; education and conflict mitigation.  To build the capacity of civil society organizations to use the media. These objectives fit closely with the BBCMA’s own approach to governance work, which is built around en￾hancing transparency, accountability and participation. BACKGROUND - MESSAGE PROJECT ACTIVITIES To achieve these objectives the BBCMA worked with existing radio partners across the six USAID priority focal areas to build up their capacity to international standards in covering key governance issues by applying values of impartiality, accuracy, balance and objectivity. Thematic and Technical Training (TTT) workshops brought participants together for sessions that provided training to enhance their ability and willingness to produce interactive programming formats. Training participants to effectively utilize these formats would give voice to a range of perspectives, bringing citizens, policy makers and civil society actors together in informed discussions and provide opportunities for citizens to ask questions directly to policy makers, and for policy makers to explain policies. The training sessions covered both technical and thematic modules in order to deliver improvements in the trainees’ technical capacity, but also to improve their knowledge about govern￾ance matters. The training would not only result in increased coverage of key governance issues, but result in an improvement in the quality of that coverage. A coproduction component was delivered face-to-face in-station training. A needs assessment was conducted beforehand with the training tailored to the specific needs of each station. In addition to the TTT and copro- MESSAGE 7 duction components, media development involved trainers being assigned as mentors to each participant. The mentors maintained contact via telephone and email, advising the participants and addressing individual training needs, concerns and problems that were encountered in producing their respective programs. Anoth￾er training component involved online learning through the BBCMA I-learn website. I-learn required partici￾pants to read, take tests and successfully complete a series of online modules tailored for the MESSAGE pro￾ject. Each I-learn module guided the learner through a number of logical, progressive steps. The first step takes the learner to the I-learn webpage that contains an interactive element, i.e., asking the reader to think about an issue or to come up with an idea before moving forward to the next step. At the end of a module there is a series of questions to test trainees. Finally, trainees were asked to complete an assignment based on the in￾formation they have just learned which was submitted online and graded by a dedicated mentor. The MESSAGE project also provided the participants with the opportunity to apply and receive a Bursary Award, which provided funding for the production of programs that focused on thematic issues addressed during their training. The bursaries were made available to trainees who wanted to pursue stories that would otherwise not be covered due to the lack of resources. Trainees were invited to submit story ideas. Successful applicants were mentored and supported by BBCMA trainers. Acceptable stories were included in the maga￾zine programs being produced as a result of the training. The BBCMA worked directly with a total 24 partner radio stations, 4 per focus area, over a period of 36 months. The station managers were offered the same training as their staff. In addition, they also were invited to participate in a tailored workshop on commercial management designed to teach them about financially sustainability. Topics covered included audience research and securing advertising revenue. The partner sta￾tions also received an equipment package to ensure that they had the tools to apply what they had learned. MESSAGE training was intended to result in local production of relevant magazine programs, focusing on governance issues that confront listenersin their everyday lives. The magazine format is flexible and allows a range of mini formats to be used, rather than the talking heads format characterized in many current affairs programs. The mini formats include audio diaries, interviews, mini dramas, skits, vox pops - known in the USA as person in the street interviews -, mini- documentaries and discussions designed to engage and appeal to audiences. Trainers at each station were responsible for helping trainees to develop and maintain regular contact with local Civil Society Organizations (CSO) and Non-Governmental Organizations (NGO) that could contribute to programming and improve relations with the media in the long term. Story Story Radio Drama In addition to media development, the MESSAGE project also involved a development communications ed￾utainment element. The BBCMA used its popular and long running radio drama Story Story to raise awareness about key governance issues and to stimulate discussion (Annex C). Three series were produced in the pro￾ject’s first year and broadcast on the BBC World Service initially, then broadcast via partner radio stations around the country. The scheduling of theStory Story series allowed them to be aired somewhere in Nigeria for the duration of the three year project. Story Story is set in a typical Nigerian market place. It is designed to raise awareness of the major governance issues facing Nigeria via an entertaining and engaging format, through stories of everyday people – market traders, business leaders, farmers, health workers and families. Some characters are likeable for their integrity and sense of humor; others for their humility and lavish use of language. The powerful human-interest story- MESSAGE 8 lines are, in turn, a flexible vehicle for delivering a wide range of information tailored to impact audiences’ knowledge, attitudes and practices. For the purposes of this project, USAID was involved in sculpting the themes from the outset and signed off on scripts throughout the production process. Story Story Outreach Kit A Media Dark Kit was produced in order to maximize the reach and impact of the project. It takes clips of the drama and factual outputs that illustrate particular priority themes and combines them with a discussion guide. Partner CSOs could use the Media Dark Kits to help deepen audience knowledge, increase understand￾ing of the issues and stimulate community-based discussion around them, some of which could be fed back into programs. Project Website An interactive website was produced specifically for MESSAGE and maintained throughout the project’s lifetime. The website was to serve as a highly cost-effective means of pulling together all the outputs and activities in one place, establishing coherence and longevity. It also was intended to perform a number of other key functions: as a forum through which CSOs, journalists and policy makers could network; as a portal or repository for factually accurate and objective explainers onkey governance issues about which the project intended to increase levels of knowledge; and as a source for relevant documents, such as state budgets. The website was intended to serve as an independent and moderated, but uncensored space, where people could freely exchange views about key issues and where the BBCMA could collect feedback on the project. MESSAGE 9 EVALUATION – SCOPE OF WORK Objectives of this Evaluation Goal: To determine how best to assist the media in Nigeria to play an effective role in holding government accountable. Purpose: The purposes of the evaluation were two-fold: 1) to assess the results of the MESSAGE project to date, especially in terms of evidence of success, failure, obstacles and lessons learned; and 2) to provide rec￾ommendations and strategies to improve the MESSAGE project over the remaining life of the project. Objectives: The objectives of the evaluation were to assess the results achieved in the areas of:  Nigerian media’s capacity to cover governance and related issues;  The financial sustainability of the media partners;  The capacity of Civil Society Organizations to engage in the media; and  Audience engagement in governance issues. Issues and Questions to Be Addressed The following questions guided the team’s analysis during the evaluation. General Questions  What MESSAGE activities appear to have more or less impact? And why?  To what extent has the project improved the Nigerian media’s, i.e., partner radio and TV stations within project focal states, technical and thematic capacity to produce radio projects which engage audiences in informed discussions around governance issues, e.g., transparency; effectiveness of gov￾ernment institutions; management of public resources; health and water management; community service delivery; education and conflict mitigation; as well as election and electoral practices?  To what extent has the project built up the capacity of Civil Society Organizations to use the media? Specific Questions  Has assistance helped the media partners in diversifying methods of revenue generation?  What has been the impact of training on the areas of financial sustainability and revenue generation?  What has been the extent of resources leveraged from BBCWST and how has it impacted on the in￾tervention?  To what extent have thematic and technical training improved upon service delivery, i.e., quality of project production and content, of the media stations?  To what extent has audience engagement in governance issues increased?  What have been the results from the Bursary Awards provided to media practitioners?  Have there been any improvements in interactive and governance-related projects in the media, i.e., partner radio and TV stations within project focal states?  Has there been an improvement on the percentage of people who believe the media enables them to be more engaged, and have a voice in governance? Attribution To what extent has equipment distributed to media stations been effectively utilized in ways relevant to gov￾ernance intervention? MESSAGE 10 METHODOLOGY The methodology used for evaluating this Project was severely limited due to SOW constraints, providing only 12 in-country days for qualitative in-depth interview research with key informants. Complicating the evaluation were security issues which caused the team members to modify the sample of focal states at the direction of USAID Nigeria. In regards to the methodological approach, the SOW states: “USAID/Nigeria seeks a two-pronged, mixed methodological approach for this evaluation. In the first in￾stance, evaluators should compare project baseline and end state information as collected by BBCWST in the project performance management plan. This information should be analyzed in conjunction with available project input and context data to determine the results of activities vis-à-vis stated project objectives. In the second instance, evaluators should gather additional evidence of change through key informant inter￾views and other data sources, as available. Data collected through interviews or other sources must document results of project activities, such as informed opinions of changes due to the adoption of recommended tech￾niques or testimonies of trainees about how they have applied skills learned to their job. Testimonials stating that training or technical assistance was “useful” without supporting statements of how these led to changes in targeted institutions will not be considered evidence of change or impact.” Over a period of 12 days, the evaluation team flew to each of the samples’ 3 focal areas conducting 31 inter￾views with key informants. In addition to the interviews with key informants, the evaluation team examined and reviewed MESSAGE reports and studies as part of this assessment. In light of these constraints and limi￾tations, care should be taken in generalizing the evaluation results. The SOW that guided the evaluation team’s methodology design is found in Annex A. In addition, the evalu￾ation team’s calendar of activities is found in Annex D. Design Based on the SOW, and considering the security and time limitations, the evaluation incorporated a triangu￾lated approach to evaluating the MESSAGE outcomes. This involved utilizing a qualitative approach, as stipulated in the SOW, combined with a review of BBCMA MESSAGE materials. For the qualitative re￾search component, the evaluation team conducted onsite visits and in-depth interviews with key informants, and inspected the equipment donated by the project. The BBCMA MESSAGE materials included survey research, content analysis, pre and posttests, reports, feedback materials, and success stories. Triangulating the data from these sources enabled the evaluation team to affirm and validate the results of the assessment. Key Informant Interviews were conducted with the following individuals: USAID 1. DG Team leader 2. DG Deputy Team leader 3. DG MESSAGE COTR BBC MEDIA ACTION 1. Country Director/COP MESSAGE Project MESSAGE 11 2. Project Director 3. Research and Education Director 4. Trainer 5. Operations Director 6. Producer RADIO AND TV STATIONS 1. Chief Executive officers 2. Marketing Managers 3. Beneficiaries of MESSAGE Training (TTT, Co-production, mentoring and I-learn) a. Producers b. Journalists 4. Bursary awardees 5. Equipment beneficiaries CSOs 1. Executive Directors LISTENER CLUB GROUPs 1. Facilitators 2. Participants Sampling A purposeful sampling approach was used for this evaluation, involving the selection of focus areas, stations, key informants, and documents. The security limitations significantly restricted the states that could be se￾lected for the sample. The evaluation team was concern about attaining a balance between private and public, state and federal, ra￾dio and television, and university stations. USAID Nigeria was briefed on the methodology and sample se￾lected for this evaluation. Due to security issues, USAID Nigeria instructed the team to remove Sokuto/Kaduna from the sample and substitute it with another area. Thus, the following three focal areas and stations were selected for inclusion in the sample, and key informants were selected and onsite observa￾tions were made. Selected states and stations: 1. Akwa Ibom/Bayelsa a) Akwa Ibom broadcasting station (State station) b) University Radio Uyo 2. Anambra State a) MINAJ Television Ogosi b) Purity FM Akwa (FRCN) 3. Lagos State South South South West South East Akwa Ibom Bayelsa Lagos Anambra MESSAGE 12 a) EKO FM (Lagos State) b) Star FM (Private Radio Station) Protocols In conducting the in-depth interviews, the evaluation team developed standard interview protocols for inter￾viewing key informants at the BBCMA, the station trainees, and CSO participants. Protocols served as a guide with the evaluation team asking additional probing questions during the interviews. The protocols can be found in Annex E. Interviews During the 12 days in-country, the evaluation team flew to each of the focal areas in the sample. A total of 31 interviews with key informants were conducted during this period. BBCMA Reports and Research As noted in the SOW, the evaluation team used a qualitative method of key informants interviews and trian￾gulated responses from informants. BBCMA documents (reports, researches conducted) were also reviewed. In reviewing and examining the BBCMA reports and research, the team sought to affirm the results con￾tained in the materials by evaluating the methodologies used to gather the data and the analysis used to reach their conclusions. Further confirmation of the results contained in BBCMA reports and research would be accomplished through key informant interviews. It was important to validate the methodologies used in the BBCMA research since the evaluation team trian￾gulated key informant interviews with data contained in the studies. The methodologies used for the BBCMA MESSAGE Programming: Midline Radio Governance Output Assessment: Research Findings July 2011 (content analysis) and the BBCMA MESSAGE Midline Citizen Voice Survey October 2011 were well grounded in established social science fundamentals and practices. The BBCMA MESSAGE Programming: Midline Goverance Output Assessment: Research Findings July 2011’s content analysis questionnaire was designed in two parts. During the program review sessions, the first part was coded by a member of the research team while the second part was done by a member of the train￾ing team. No trainer coded/reviewed any program they were involved in during co production to minimize bias. Each program was coded by a member of the Research team and a member of the training team assigned to review any of the programs to ensure that the challenge of multiple coding was eliminated. Each coding was based on agreed upon criteria. Thematic aspects and less technical features such as themes addressed in pro￾gramme, voices heard, nature of voices and mode of interactivity was coded mainly by the research team and the more technical parts e.g. use of editorial guideline, production values and presentation skills were coded by the trainer(s). For the BBCMA MESSSAGE Midline Citizen Voice Survey October 2011 a multi-stage probability sampling technique was employed to select a representative sample of 2,006 adults between ages 18 – 64 years in urban and rural locations. The first stage of sampling involved a purely random selection of rural and urban locali￾ties using a table of random numbers. Stage 2 involved a random selection of enumeration areas (EAs) from the list of EAs within the selected rural and urban localities. Stage 3 involves the selection of household MESSAGE 13 through the use of day’s code, the starting point is determined by the day’s date and a regular interval main￾tained throughout. This is followed by the selection of individual respondents, a respondent is selected from a selected household through the listing of all eligible respondents in that particular household and the inter￾viewed respondent selected through the use of a Kish grid. Data Analysis Data analysis involved daily discussions among the evaluation team members in interpreting the day’s inter￾views with key informants and review of documents. As the assessment continued, each day the team re￾quested more information from the BBCMA MESSAGE project as more questions arose and as additional documents became available. At the end of the evaluation period, the team met one last time to review the daily analysis, discuss the findings, interpret the results, determine where more information was needed and finalized the conclusions and recommendations for the report. Limitations - Constraints This evaluation was severely limited in two ways. One, concerns the time allocated to the evaluation pro￾cess. The team was provided only twelve days in-country to conduct the assessment. This time limitfor con￾ducting the evaluation handicapped the team in regards to the travel to focal areas, the number of stations that could be selected for visits and interviews with key informants. Originally, the team had selected three stations per state project site in the sample, but had to settle on two stations due to time constraints. This affected not only the number of stations selected, but also hindered the team’s ability to balance public – pri￾vate, state – federal, radio – television and university stations selected for the sample. In addition, the time constraints prevented the evaluation team from adequately assessing the outcome of the Story Story program component of the MESSAGE project that aired on the radio stations in Nigeria. There was no time to either conduct focus groups or survey listeners in order to determine the effects that the pro￾grams has had on listeners to see if they acted on what they learned from the program. This significantly hampered the team’s ability to assess the outcome of this significant part of the MESSAGE project, relying instead on the interviews with the listener group facilitators and participants and BBC Media Action survey research and success stories. The time restrictions also significantly reduced the evaluation team’s ability to meet with and interview CSOs that participated in the MESSAGE project. The other limitation concerns the security issues surrounding the Nigerian states involved in the sample. There had been violence in the last year following the 2011 elections. Prior to the MESSAGE evaluation, Nigeria was experiencing domestic unrest due to several incidents of church bombings by a group identifying themselves as the ‘Boko Haram.’ In addition, in the previous year there had been kidnappings of foreigners in the country. Also, there had been street demonstrations in January protesting the elimination of the Nigerian government’s subsidy for gasoline. Due to these events, there were security issues for the evaluation team. During the design and planning for the evaluation, the team had wanted to balance the sample to include at least one of the northern Nigerian states in the sample, Sokoto. Unfortunately, during the in-brief outlining the details to USAID, the team was informed that it could not include any of the northern Nigerian states in the sample. Thus, the evaluation team was limited to visiting only the southern Nigerian states involved in the MESSAGE project. Finally, there is the limitation concerning generalization of the evaluation team’s research. The SOW speci￾fied a qualitative methodology utilizing interviews with key informants for assessing the MESSAGE project. MESSAGE 14 The team met with and interviewed a total of 31 key informants for this evaluation. Every effort was made to include in the sample of key informants representatives from federal, state, and private radio and television stations. Despite these efforts, the evaluation team was not able to visit as many stations and meet everyone it had originally planned on due to time constraints. The evaluation team found overwhelming support for its findings from interviews with key informants and in reviewing BBCMA reports and research. Yet, care must be taken in generalizing the results of this evaluation. MESSAGE 15 FINDINGS From February 20, 2012 to March 3, 2012, the evaluation team conducted an assessment concerning the out￾comes of the USAID funded BBCMA MESSAGE project. This section presents those findings which are organized by the project’s three fundamental objectives. The objectives for the MESSAGE project, developed in consultation with USAID Nigeria, were threefold: 1. To increase the Nigerian media’s capacity to serve the public interest through greater professionalism 2. To improve the Nigerian media’s technical and thematic capacity to produce radio programs which engage audiences in informed discussion around governance issues such as oil sector transparency, effectiveness of government institutions, management of public resources, health and water man￾agement and community service delivery, education and conflict mitigation. 3. To build up the capacity of civil society organizations to use the media. The evaluation team assessed the results of the MESSAGE project to date, especially in terms of evidence of success, failure, obstacles, challenges and lessons learned. The following are the findings of the evaluation team’s assessment. Outcomes: Objectives 1 and 2 1. To increase the Nigerian media’s capacity to serve the public interest through greater professional￾ism. 2. To improve the Nigerian media’s technical and thematic capacity to produce radio programs which engage audiences in informed discussions around governance issues such as oil sector transparency; effectiveness of government institutions; management of public resources; health and water man￾agement and community service delivery; education; and conflict mitigation. From the evaluation team’s key informant interviews with key informants, BBCMA reports and research, it is clear that objectives #1 and #2 were overwhelmingly achieved among the sample stations and respondents. Professional development and capacity building can be seen in the assessment results and the evaluation in￾terviews. The BBCMA ‘Assessment of Trainee’s Knowledge and Skills’ (October 2010) report observed in all six focal areas that: Across the board, reaction towards the thematic training workshops was positive and the participants described the training as interesting, educative, interactive, and enlightening. They also claimed that the training exposed them to new ideas, provided them with knowledge on the attributes of good journalism and will generally help media practitioners develop skills and knowledge required to carry out their duties in their respective fields. (p 20). Pre and post-tests of the participants indicated a very strong shift in knowledge, understanding and applica￾tion of fundamental editorial values. For example, participant knowledge of editorial values was low before co-production. By the end of training program the test scores were high. Nearly all participants (92%) felt their knowledge and understanding of the need for editorial values in program/news making had significantly increased. MESSAGE 16 Figure 1: Understanding the Need for Editorial Values in Program/News Making With regards to application of editorial values, findings also reveal a considerable shift in trainees’ knowledge/skill after the training (34% - 85%). Figure 2: Application of Editorial Values in Program Production/News Making After two years of the MESSAGE project’s implementation, the evaluation team found overwhelming sup￾port for these results with the in-depth interviews conducted at the sample radio and television stations. All participants in the evaluation were quick to note the knowledge, value and importance of the BBC editorial standards, and the technical skills they learned through the MESSAGE project. In addition, the interviewees Understanding the need for Editorial values in programme/News making (Base:74) 22% 45% 34% 1% 6% 92% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low Average High Pre Post Application of editorial values in programme production/ news making. (Base:74) 16% 50% 34% 4% 11% 85% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Low Average High Pre Post MESSAGE 17 talked about how they were applying what they had learned to the programs they produce. MESSAGE Training/BBC Editorial Standards Respondents noted that the training empowered and equipped them with new skills and new techniques in reporting and production. In the each of the interviews, participants discussed how important the MESSAGE project’s training involving TTT, co-production, mentoring, I-learn and Bursary Awards were to them. For many of the interviewees this was their first exposure to professional training in broadcast journal￾ism. Equally important to MESSAGE participants was the training based on BBC editorial and production stand￾ards. During training sessions, trainers and participants discussed ways to deal with issues concerning the ap￾plication of the BBC editorial standards. All the interviewees emphasized the importance of applying BBC editorial standards and the media’s role as a watchdog on governance. Participants noted that the MESSAGE training taught them how to apply editorial values and make programs more interactive, balanced and inter￾esting by engaging real everyday people as contributors. The interviewees noted that they are now more bal￾anced, fair and careful in the language used in the programs they produce. Some participants noted that be￾fore the MESSAGE training they were quick to make judgements about issues and events and did no back￾ground research for their programs. Now they take time to critically examine all aspects of a governance is￾sue, conduct research, confirming information, getting eyewitness accounts, and providing balanced coverage while being careful about the language used. From the MESSAGE training many of the interviewees under￾stood that by applying the BBC editorial standards they would gain the trust of the people; the station’s audi￾ence. Interviewees observed that they feel their programs now have more impact and sound more credible. Re￾spondents at state owned broadcast stations noted that sometimes they have been able to overcome censor￾ship issues by applying BBC international editorial standards. Through the training they have been able to convince management that the BBC standards were followed in producing a fair and balance program, and that they should air it. One interviewee observed that the MESSAGE training focuses on issues and balance without directly challenging the government. One example cited by a federal radio station manager concerns a program produced on the problems with electricity in her area. The community now is getting electricity. In another expose’ her radio station focused on children not going to school during the rainy season. In both cases the government reacted positively, and the focus on issues is something she would like to see continue. However, not all participants were successful. One feature program producer remarked that she was frustrat￾ed in using the BBC editorial standards in a state radio station environment. The evaluation team found using the BBC editorial standards to be problematic where censorship exists at the state radio stations. For exam￾ple, this program producer planned to use a story concerning the selling of mosquito nets, which were sup￾posed to be distributed for free. She was ordered to “kill” the story by station management, illustrating the challenges trainees have in incorporating what was learned in the MESSAGE workshops. Despite these is￾sues, this particular MESSAGE participant desired to have more training, particularly more on investigative reporting using an approach that would not cause her to lose her job. A senior radio producer noted three things he learned from the MESSAGE sessions. One observation was that research is key to producing programs. It is important to get valid background information in order to MESSAGE 18 talk about issues and help listeners to understand all sides. Second was scripting, which was an eye opener for this participant. He observed that scripting was important in order to be mindful of the environmental ambi￾ence needed to make the radio program credible. Finally, he understood the importance of packaging when it comes to producing a radio program. This interviewee learned that a five minute scripted program can have a greater impact than an unscripted 30 or 60 minute program. He found that before he had a lot of irrelevant material in his radio shows. With the MESSAGE training he learned that audiences will not spend the time listening to long radio programs, particularly if they are filled with extraneous information. With shorter pro￾grams, he found that he could have more impact by focusing on every element of the production. One respondent learned the importance of applying the editorial guidelines and “how to persevere and keep going” to get the story. For example, the participant noted that the real reason for a conflict may not be ap￾parent. She observed that it is important to find the facts and the reasons behind the conflict, while not esca￾lating the crisis. From the MESSAGE training, this participant learned that the station has a social responsi￾bility to do work for Nigerians as agents for change. The interviewees found the practical instruction in MESSAGE training very useful because they put it into practice. They learned practical skills in writing, producing and scripting. The co-productions were largely practical because the participants learned the fundamentals in writing spots, packaging reports, using the edit￾ing software and equipment. Through the MESSAGE training TV producers learned about storyboarding, writing to pictures, and keeping programs short and precise. Equipment Participating radio stations received equipment that the participants found very useful, particularly the Ma￾rantz digital recorder. The Marantz was important for participants since it provided them the ability to go on location to gather information and peoples’ opinions. As producers and journalists, respondents learned how to get straight to the point and not cloud the issues all the equipment were branded with the USAID logo which confirmed the source. Mentoring Various respondents noted that their MESSAGE mentor was available to assist with any issues. A few partic￾ipants observed that the most attractive part of the MESSAGE training was the mentoring. Some individuals indicated that they often talk with their mentor, sometimes biweekly, to discuss story ideas and approaches. Interactivity/Engagement An important aspect of the MESSAGE training involved teaching participants about audience engagement. For example, a respondent noted the difference the training has on his regular radio show. Before he received a couple of phone calls and one or two texts from listeners, but now he is getting many more calls and texts and is utilizing social media to connect with listeners. Through Facebook and Twitter, he gets feedback and the audience gets a sense of participation. Social media was part of the MESSAGE training he attended that focused on engaging audiences. Another example of the impact the MESSAGE training had on audience engagement involved another par￾ticipant’s radio program called ‘Keeping the Promise.’ After MESSAGE training, the interviewee converted the show into an audience call-in program that provided “many voices to make it real.” One participant stat￾ed that, “Interactivity is now a constant characteristic of my programme and has elicited wider participation.” MESSAGE 19 From the MESSAGE trainings, another respondent learned to use more user generated content in his on-air talk show. He learned how to handle telephone calls, how to treat callers, and how to express himself on-the￾air, being conscious of the language he uses. The challenge he has is that telephone signals are poor in the area, and the station has no internet connection. He uses his mobile phone to conduct research on the issues. One significant example cited by a participant from a TV station is that the governor came to the station, went on-the-air and took live calls from viewers concerning issues. Thus, the TV station served to bridge the gap between the government and its citizens. The interviewees reaffirmed the effectiveness of audience interaction in the pre and post-tests that BBCMA conducted during the MESSAGE training. The Participants’ knowledge about the use of various elements of audience interaction to produce engaging programs was somewhat low before the training as only (31%) felt that their knowledge was high. However, this improved significantly after the co-production (85%).13 Figure 3: Use of Various Elements of Audience Interaction to Produce Engaging Programs Through the training, participants learned about using social media and crowd sourcing to gather information, opinions and engage audiences. One respondent stated that “crowd sourcing makes sure information is true and accurate in using multiple sources to validate.” Before the training, only a third of the trainees felt their knowledge of audience generated content, such as crowd sourcing, was high. This increased considerably (70%) at the end of co-production.14 13 MESSAGE Assessment of Trainees Knowledge and Skills: Research Findings October 2010. pg. 24. 14 MESSAGE Assessment of Trainees Knowledge and Skills: Research Findings October 2010. pg. 24- 25. MESSAGE 20 Figure 4: Audience Generated Content in Program Making Since the MESSAGE training, MINAJ TV has created a new vox pop program called ‘People Talk.’ For this show the producers and journalists go out to the people to get their reactions to various issues. They also improved another program called Current Issues. It is a studio based show where specialists are brought in to discuss various subjects. Before the MESSAGE training, the producers would only bring in one expert. Now they bring in two specialists to provide a more balanced discussion, often leading to disagreements which make the program more interesting. I-learn The I-learn element of the MESSAGE training had mixed results. Some participants had trouble completing the I-learn part of the training due to internet access issues. A couple of interviewees had no experience with using computers and could not access the I-learn component. One respondent noted that he uses his phone and a cybercafé to access the internet. However, other interviewees completed the I-learn online sessions and learned the importance of preparing properly by researching an issue, thereby reinforcing the BBC editorial values. Bursary Awards An important part of the MESSAGE training involved the Bursary Awards. Many respondents noted that their stations did not have the resources to cover transportation and research expenses as such the interven￾tion came in at the time of need. One participant applied for and received a Bursary Award to produce a pro￾gram focusing on the transportation situation in Uyo where the government has banned motorcycles. An￾other individual produced a radio show called Water Scarcity in Uyo. The program focused on the portable water problem in Uyo. Households were buying water every day, people were drinking water that was impure and many were going to the hospital as a result. The Marantz audio recorder the station received via the MESSAGE project helped considerably in interviewing people on the street about the problem, and this im￾proved the quality of the production. Other radio programs produced through a Bursary Award included a program titled ‘Youth and Thuggary’ which was about discrimination against women in decision making. Management Participants Another element of the MESSAGE project involved training radio and television station management. Man￾agement participants noted they learned that marketing, programming and news must work together to pro￾duce programs and generate revenue. During the MESSAGE training individuals learned to develop and structure programs giving marketing due consideration. Participants revealed that an important outcome was MESSAGE 21 that now they conduct research on what type of programs audiences desires, something they had not consid￾ered before. They now focus on programs that are attractive to audiences and advertisers, and will help to sustain the television station. With this training, one participant observed that they developed live program￾ming during the election whereby all political parties were provided opportunities to present their platforms. As a result, the station has “made a lot of money.” One management participant attributed the MESSAGE training to bringing in more advertisements for the station. In addition, she remarked that the radio station has a small staff, low wages, and little resources. So the MESSAGE training for both management and staff was valuable. The management training proved valu￾able for the Federal Radio Corporation of Nigeria (FRCN) radio stations that now have a mandate to focus on revenue generation. A respondent from FRCN learned that it is important to market and promote the programs rather than just the station. In addition, she realized how important it was to add value to the con￾tent they produce in order to attract more listeners and advertisers. What the evaluation team found interesting were managers’ discussion concerning the importance of the BBC editorial values to establish the integrity of the station and generate revenue. This included giving “a fair hearing to every party” in the station’s programming. Another benefit that managers received from the train￾ing was meeting and networking with the management staff from other stations participating in the MESSAGE project. They were able to exchange ideas and develop relationships during the training sessions. Unfortunately the team could not verify value with increment in station revenue. University Station and Passing it On During the evaluation the team also visited a university community station and interviewed students who par￾ticipated in the MESSAGE training. They were very enthusiastic about how valuable the training was to them and to the station. One interviewee noted that the BBCMA MESSAGE training provided valuable pro￾fessional and practical experience while the university provided only theories. He felt inspired by the MESSAGE training and has a real passion for radio. What sets this MESSAGE participant apart is that he has graduated from the university, yet still works as a volunteer at the radio station training students in skills that he learned in the MESSAGE project. It is a story the evaluation team heard from many other MESSAGE participants, i.e., upon returning to their respective stations, they trained and passed on to their colleagues the skills they had learned from the project. Unfortunately, the university’s professors who need training did not participate in the MESSAGE training. Challenges/Obstacles to Technical and Thematic Training One of the challenges in the MESSAGE training concerned personal computers. During the TTT trainings the participants are organized into groups of 7 or 8 and have one computer to use. Many interviewees ob￾served that this made it difficult for everyone to learn. In addition, computer literacy was a problem for some MESSAGE participants. Another challenge that participants noted was the lack of internet access at some of the radio and TV sta￾tions. This made it difficult for respondents to access the I-learn website and to conduct research. Some par￾ticipants went to internet cafés, if they were available. Some used their mobile phones. A side issue to this was computer literacy. Two interviewees were computer illiterate which limited their ability to access the in￾ternet. An additional challenge that the MESSAGE project encountered was that some radio station management would not sign a Memorandum of Understanding (MOU) due to the terms and conditions in the MOU. The MESSAGE 22 CEO at one station stated, “We did not sign the MOU with BBCMA because of contested issues pertaining to air time that the partner was trying to impose.” In an interview with the BBCMA MESSAGE Project Man￾ager, Fidelis Ekom, he acknowledged problems with radio and television stations not signing and returning the MOUs due to issues that management had with the terms and conditions. To deal with this issue, the Project Director arranged for the presentation of commemorative plaques to management, recognizing the partnership that exists between the radio or television station and the USAID funded BBCMA MESSAGE project. MESSAGE Training Impact on Nigerian Broadcast Programming In addition to the evaluation team’s interviews with key informants, a good indication of the impact that the training had on station producers and journalists is found in the BBCMA MESSAGE Programming: Midline Radio Governance Output Assessment: Research Findings July 2011. This research was a content analysis comparing the programs produced before the training with those created afterwards. The study examined the representation of views and number of voices found in the programs, and an analysis was done to assess the diversity of viewpoints heard on each program. The research found a significant in￾crease in the number of programs with voices from average citizens represented, comparing the baseline of 17 (59%) to midline of 22 (81%). (see Table 1 below). There was an average of 8 citizens’ voices heard per program. Most of the programs at midline (12) that featured voices of average citizens are from public sta￾tions, while 10 were from private stations. This shows an increase from 9 and 8 programs recorded in public and private stations respectively at baseline. Table 1: MESSAGE Programming Content Analysis Category of People # of programs with views represented/heard – Baseline (Base = 29) # of programs with views represented/heard Midline (Base = 27) An average citizen 17 22 State government personnel 9 5 Local government personnel 3 - National Government personnel 3 5 Politician (political parties) 3 2 CSO/NGO/CBO/FBO representatives 3 10 Medical doctor 2 - Legal practitioner 2 1 Individual Experts 2 4 Union Leader 1 - Children 1 - MESSAGE 23 Traditional leaders 1 1 Religious Leaders 1 - INEC officials/reps - 5 People running for office - 3 Public analyst - 2 Questions and comments from the audience for most of the programs were adequately addressed, mainly by the guests on the program. There was also a significant increase in the number of CSO/NGO/CBO/FBO representatives within the programs at midline. A total of 37% (10) of the programs featured CSO/NGO/CBO/FBO representatives’ voices at midline compared to the 10% (3) recorded at baseline. These programs are from eight different stations, with equal number of 4 from both private and public sta￾tions, compared to three stations (2 public and 1 private) recorded at baseline. STORY STORY - Outcome Except for meeting and interviewing three individuals from the listener groups, the evaluation team was una￾ble to assess the impact that the Story Story series had on the Nigerian people. USAID funds were used to fund Story Story series 17, 18 and 19, with each series composed of 12 episodes. This communication compo￾nent of the MESSAGE project involved an entertainment element whereby the BBCMA used its popular and long running radio drama Story Story to raise awareness and stimulate discussion around key governance is￾sues. Two series of 12 episodes each were produced in the first year of the project and initially broadcast on the BBC World Service, then via partner radio stations around the country. The USAID funded series of Story Story was scheduled to aired somewhere in Nigeria for the duration of the three year project. 15 The BBCMA MESSAGE Midline Citizen Voice Survey October 2011(MCVS) report provided indications as to what that effect the series had on the Nigerian electorate. A detailed description of the research objectives and methodology used for the MCVS survey can be found in Annex F. A description of the Story Story’s pro￾duction and themes can be found in Annex C. Prior to the midline study, the USAID funded Story Story series aired on 87 stations, including 22 MESSAGE partner radio stations in the six program focal areas, broadcasting episodes between once or twice weekly, and devoting an estimated 1,372 hours to the airing of the episodes of series 17, which were also broadcast on the BBC World Service. In effect, millions of listeners followed the broadcasts and repeats.16 During the year a total of 24 Story Story episodes, series 17 and 18, were produced and aired on a total of 87 stations, including 22 partner radio stations in the MESSAGE project from the six program states. They were 15 MESSAGE Year 1 Activity Report (01 October 2009 to 30 September 2010) pg. 11-12 16 MESSAGE Year 1 Activity Report (01 October 2009 to 30 September 2010) pg 11-12 MESSAGE 24 also aired on the BBC World Service. These episodes explored themes about elections, women participation and governance issues concerning social service delivery.17 Information concerning the final Story Story series 19 was unavailable. In addition, information requested about how many times each episode aired and repeat￾ed was unavailable, although BBCMA reports indicated that the USAID funded episodes were repeated and aired during the MESSAGE project period. Story Story Reach The MCVS survey found that Story Story reached approximately 6 million people in the MESSAGE focal states, a significant increase from the baseline study. Story Story’s reach among media consumers increased sig￾nificantly from baseline to midline. The overall reach of Story Story increased due to the substantial growth in media consumers who had not previously listened to Story Story. In the baseline, out of a total of 1,503 re￾spondents interviewed, 17% were Story Story listeners. This increased significantly to 26% among 2,006 re￾spondents interviewed at midline as shown in the figure below. This figure projected onto the target audience population, i.e. 18-64 years, in the 8 survey states revealed that Story Story reached an estimated 6 million peo￾ple out of the 21 million within these states.18 On the other hand, if projected onto the national population, this reveals an estimated reach of approximately 20 million (26% of population within the ages 18-64 years) of the total national population have listened to USAID funded Story Story. 19 Figure 5: Chart showing the change in Story Story audience base from baseline to midline The midline also found that 88% of the respondents who had never listened to Story Story claimed to have listened to the drama in the 12 months aligned with the USAID funded series broadcast. Based on this study, the USAID funded Story Story series that were produced as part of the MESSAGE project reached 88% of the 17 MESSAGE Year 1 Activity Report (01 October 2009 to 30 September 2010) pg. 11-12 18 BBCMA MESSAGE Midline Citizen Voice Survey October 2011. Pgs 3, 20-21 19 The National Population index estimated the proportion of total population within the ages 15-64 years to be 55.5% of the total population 17% 26% 0% 5% 10% 15% 20% 25% 30% Baseline (1503) Midline (2006) The distribution of Story Story Audience base (Base: All respondent) MESSAGE 25 Story Story audience base. This formed 26% of radio listeners and 23% of all study respondents in the focal areas. Therefore, approximately 5 million people have listened to the USAID funded Story Story series during the 12 months of broadcasts across the 6 focal areas.20 Listenership during those 12 months was high across all states, albeit it was the highest in Sokoto where all respondents who have never listened to Story Story, listened within the last year. Figure 6: Chart showing the distribution of Story Story listenership within the past one year Demographically, the listenership pattern to Story Story revealed that more males (57%) listened in the past year compared to females (43%). In terms of age, listenership was higher among ages of 26 – 33 years and lowest among the older population (50 years+). The listenership pattern across the focal areas was approxi￾mately distributed evenly except for Lagos (17%) and Bauchi (15%). More people in the rural areas listened to the drama in the past 12 months compared to the urban dwellers.21 Story Story and Partner Stations Broadcasts During the 12 months after the MCVS research was conducted, Story Story had aired a total of 1,676 times on 32 MESSAGE partner stations across the six focal areas, with 354 repeat broadcasts on 19 stations. The 4 partner stations that had repeat broadcasts are Anambra, FCT and Kaduna in the past year, while none were repeated in Bayelsa state. Repeat broadcasts were highest in Anambra state and lowest at the Lagos stations, despite Lagos having the highest number of partner stations that broadcast Story Story.22 20 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pgs. 3, 21. 21 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg. 21. 22 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg 23 - 24 88% 100% 96% 94% 89% 89% 89% 79% 77% 0% 20% 40% 60% 80% 100% 120% Total Sokoto Anambra Kaduna FCT (Abuja) Bauchi Bayelsa Lagos Akwa - Ibom Listened to Story Story in the past 12 months (Base: All respondents who have ever heard Story Story [n=520]) MESSAGE 26 Table 2: Showing the broadcast pattern of Story Story in the past one year (from analysis of the media monitoring data) Nearly all Story Story listeners (91%) across the survey states listen to the drama on local radio stations, while less than one-tenth (9%) listened on the BBC World Service. The majority of those who listened to the drama on the BBC World Service are from Kaduna state and the FCT.23 Story Story – Self Reported Impact From the MCVS research, Story Story listeners reported that the drama made them think and act differently about their role in the recent national elections. Through a series of open ended questions, respondents were asked what they had learned, if they had thought about anything differently, and had done something differ￾ently due to listening to Story Story. More than half of the Story Story listeners (53%) claimed listening to the drama made them think differently about something. Many said it changed the way they thought about the rights and responsibilities of voters in the recent elections. This represents approximately 2.5 million people in the 6 focal areas.24 Approximately four out of every five Story Story listeners (81%) responded that they had learned something from Story Story. The main lessons learned from Story Story pertained mostly to governance related issues in line with the focus of the drama. Specifically, the main lessons acquired related mostly to corruption, elec￾tions, voting and girl-child education. Among the respondents, 14% indicated they had learned from the dra￾ma that corruption can lead to a lack of accountability, while 11% learned that voters should know the dos and don’ts of elections, and that it is important to present credible candidates. The table below notes the les￾sons learned by respondents listening to the USAID funded Story Story series. 23 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg. 24 24 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg 26 Project Focal states No of Part￾ner stations Total broadcast Repeat broadcast PS broad￾cast twice per week Anambra 5 298 77 4 Akwa Ibom 2 129 43 2 Bauchi 2 126 41 2 Bayelsa 2 88 0 0 FCT 6 286 48 4 Lagos 9 394 8 1 Kaduna 4 228 76 4 Sokoto 2 133 61 2 MESSAGE 27 Table 3: Showing lessons learned from listening to Story Story (multiple responses) Story Story listeners also were asked whether they had thought differently or done anything differently as a re￾sult of listening to the drama. A little more than half (53%) of Story Story listeners claimed that the drama made them think differently, while one third (32%) held that the drama influenced them to do something differently. What have you learned from listening to "Story Story-Voices from the market"? (Base= 455) % who men￾tioned each Politics and Electoral process Voters should know the dos and don’ts on election day to safe guard their votes 11% It is important to present credible candidate 11% It is the responsibility of all to register for elections 10% Under age voting is against the law 10% It is important for political parties to discuss internally in a democratic way 8% The incursion of money into politics ensures that the citizens have no moral right to ask for good gov￾ernance 7% When people don’t register, they leave room for inflated voter register 7% God fatherism gives little room for a robust political platform 6% It is important to mobilize for elections 5% One must vote correctly if your vote must count 4% Zoning should be done in a democratic manner 4% Hold debates to know your candidates and what they plan to do if elected 4% One must try and safeguard one’s vote 4% It is important for me to be interested in elections 4% We should desist from choosing candidates based on ethnic considerations 4% People should know the track record of their candidates 4% When a popular and credible candidate is chosen, the party stands a great chance 3% The electoral body should endeavour to step up and set up the rules 3% Party crisis should be in order to avoid cross carpeting 2% When voter register is displayed, it is important for it to be verified in order for claims to be made 2% Women should be part of the political process 2% Corruption Corruption ensures that a highly wrong system comes up, where no one is accountable to anyone 14% Fighting corruption and enforcing best practices 8% Girl Child Education It is important that girls are educated like their male counterparts 9% Conflict Resolution Leaders should endeavour to always restrain their followers to avoid the loss of lives and property 6% MESSAGE 28 Figure 7: Chart showing proportion of respondents who had thought differently or done something differently as result of listen￾ing to Story Story For the 53% of Story Story listeners who held that the drama made them think differently, most said it made them think differently about issues concerning elections, voting, corruption and girl child education. Specifi￾cally, 16% of Story Story listeners claimed that the drama made them think differently about the fact that “it is the responsibility of all to register for elections” while 13% also held that they have thought differently about the fact that "it is important to present credible candidates”. These were consistent with the issues addressed in Story Story. 25 The following table notes the issues that respondents are now thinking differently. 25 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg 27 0% 20% 40% 60% 80% 100% Think differently Done anything differently Has listening to "Story Story-Voices from the market" made you to think differently about anything ? Have you done antything as a result of listening to "Story Story_Voices from the market"? (Base: All SS listeners in the past 12 months [n=455])) Think differently Done anything differently MESSAGE 29 Table 4: Showing the proportion of Story Story listeners who has thought differently about anything as a result of listening to the drama (multiple responses) Among the 32% of listeners who stated that they have done something differently as a result of listening to Story Story, 40% voted for the candidate of their choice as opposed to either being compelled or induced with gifts by politicians, friends or family to vote against their wishes. Another 26% of Story Story listeners regis￾tered to vote. In addition, 20% of the listeners held that the drama influenced them not to sell their vote; 15% waited for their vote to be counted on election day; 11% mobilized other people to go and cast their vote; and 9% checked back at the registration center to ensure their name was in the voter’s register after reg￾istering. Importantly, a majority were able to change their attitude and others’ attitudes positively in the area What has it made you to think differently about? (Base=241) % who men￾tioned each Politics and Electoral process It is the responsibility of all to register for elections 16% It is important to present credible candidate 13% Voters should know the dos and don’ts on election day to safe guard their votes 12% It is important to mobilize for elections 8% Under age voting is against the law 7% We should desist from choosing candidates based on ethnic considerations 7% When people don t register, they leave room for inflated voter register 6% One must try and safeguard ones vote 5% The incursion of money into politics ensures that the citizens have no moral right to ask for good gov￾ernance 5% Women should be part of the political process 5% One must vote correctly if your vote must count 5% It is important for political parties to discuss internally in a democratic way 4% Zoning should be done in a democratic manner 4% When a popular and credible candidate is chosen, the party stands a great chance 4% People should know the track record of their candidates 4% Godfatherism gives little room for a robust political platform 3% Hold debates to know your candidates and what they plan to do if elected 3% Party crisis should be in order to avoid cross carpeting 2% When voter register is displayed, it is important for it to be verified in order for claims to be made 2% The electoral body should endeavour to step up and set up the rules 1% Corruption Corruption ensures that a highly wrong system comes up , where no one is accountable to anyone 11% Fighting corruption and enforcing best practices 10% Girl Child Education It is important that girls are educated like their male counterparts 8% Conflict Resolution Leaders should endeavour to always restrain their followers to avoid the loss of lives and property 8% MESSAGE 30 of elections and voting. The following chart notes how listeners have done something differently as a result of listening to the Story Story series.26 Figure 8: Chart showing the proportion of respondents who have done something differently as a result of listening to Story Sto￾ry. (Multiple responses) Story Story Listener Groups The MESSAGE project formed radio listening groups to strengthen the reach and impact of the output of Story Story, as well as to sample opinion and views about the drama. It provided the MESSAGE project with an opportunity to assess whether or not the drama was in line with what was directly obtainable in the com￾munities. However, the BBCMA Story Story Listener Groups were only formed and conducted in Abuja, the Nigerian capital. While feedback from these groups was valuable to the BBCMA for assessment and story development, they don’t necessarily represent the rest of views of the Nigerian population. Listener group objectives and methodologies can be found in Annex G. Due to time constraints, availability just in Abuja, and limited number of participants, the evaluation team was able to meet with only four participants in the Story Story listener groups that the BBCMA MESSAGE project had formed in Abuja. Three of the respondents were group facilitators, while the other was a participant. In Abuja, there were four listener groups with between 9 and 13 members; three groups were male and one fe￾male. The groups would meet once a week after listening to Story Story. The facilitators would use a “Facilita￾tors Guide” to conduct the meetings (Annex H). The facilitators noted that group participants would discuss the latest Story Story episode, what they heard and learned from the drama, particularly what message the participants received from listening to the drama. In 26 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg. 29 MESSAGE 31 addition, the facilitators found that many of the participants had a passion for Story Story. Group participants that did not have a passion for the drama before the meetings, developed one afterwards. Besides determin￾ing the messages received via Story Story, the facilitators obtained script ideas from group members. One key informant facilitator suggested that some provision should be made to cover participant transportation ex￾penses. Another key informant’s suggestion was that participants be given small Story Story promotional items as a token for participating in the listener groups. BBCMA MESSAGE Project noted one particular case study associated with the Story Story listener groups. The case study concerned Mallam Hashiru Abdullahi, a 23 year old man, who works and lives in Abuja. He has been a listener of Story Story for one year, and is currently a member of the Story Story listening group in area 1, Abuja. He described Story Story as a drama with a market /garage setting, which is educative and enter￾taining. Abdullahi claimed that it was the educative aspect of Story Story that influenced him. According to Hashiru, Story Story is similar to his own environment because of the mixture of different ethnic groups in￾volve in the drama.27 Specifically, Abdullahi observed that he registered to vote during the April 2011 polls as a result of listening to Story Story. He stated: “Story Story changed my life during the past election, before I said I will not register nor vote for anyone because even if I vote for my candidate he might not win. So when I listened to the programme Story Story what they said was we should all come out, vote and guard our vote by waiting after the election. Our vote will be counted in front of us and we will hear the result before we leave. When I heard this it encouraged me to come out and vote. I have been listening to Story Story for about a year now, before I will not come out and vote, but after I listened to Story Story they educated me on voters register, how to vote and guard your vote after election, it’s interesting. If I did not listen to Story Story and was not interested in the programme, I will not come out and vote and nobody will see me in the Daily Trust newspaper. Listening to Story Story has encouraged me to vote. When I was voting, Daily Trust kept my picture in their newspaper and many people called me, like my friends in Sokoto and Lagos and everybody in the country saw me, even the president. Story Story has really done a lot for me”28 CIVIL SOCIETY ORGANIZATIONS (CSOs) - Outcome Another aspect of the MESSAGE project concerned recognizing the importance of engaging civil society organizations in the project. As noted before, Objective 3 of the MESSAGE endeavor was to “build the ca￾pacity of civil society organizations to use the media.” In addition to the media training workshops and online training modules, the MESSAGE project worked with CSO groups identified by USAID(see Annex I for CSO participant list). The project’s media trainers were responsible for fostering the development of long-term relationships between civil society groups in the six focal areas and MESSAGE’s media partners for each to work together on issues and programs after the project has ended. Furthermore, the MESSAGE project produced a Media Dark Kit in which Story Story clips illustrated key governance themes, packaged with a discussion guide for use by the CSOs in their work at the community level. Such groups were to be trained by the MESSAGE project to ensure the kits were used effectively. 27 ‘Story Story’ Impact Assessment: A Case Study Of Story Story Listening Group Member. 28 ‘Story Story’ Impact Assessment: A Case Study Of Story Story Listening Group Member pgs. 3-4. MESSAGE 32 During the 12 day project evaluation, the team was only able to meet with two CSOs located in Akwa Ibom. Outcome results based on only two interviews is extremely limited and cannot be generalized. Thus, because little data could be gathered by the evaluation team for this part of the evaluation, the team relied more on BBCMA research, pre-posts test materials and success stories. One CSO interviewee noted that the training was too short and lasted for only one day. The training session he attended concerned the present political situation, and what can be done to reach people without the in￾fluence of the states or federal government. The workshop involved learning about political conditions, and included instruction on how not to inflame situations. The respondent commented that he was fascinated by the Media Dark Kit. He understood it, but found it difficult to use. This participant thought that too little time was devoted to the training on how to use the kit. The day this interviewee attended there were about 100 people participating in the training session. They were divided into groups of 10 to 15 with presentations focused on the issues. Messages were developed for the media people to use. As this participant described it, the media participated, including participants from the local television station and state and federal radio stations. After the session, the respondent noted that the CSOs were supposed to develop a program, approach the radio stations and request to have it broadcast. However, he wanted to use the Media Dark Kit first before developing a program, but had difficulty utilizing the kit. Consequently, he has not done anything with respect to improving his relationship with the media. He would like to see more time devoted to the workshops instead of just one day. The interviewee felt that more time devoted to training on program development would be very beneficial, especially if a full day could be devoted to that purpose, a day to produce the program and a day to present it. This CSO participant has experience with other programs, such as the EU’s Non State Actors program where they provide instruction, work to develop a proposal and program, and provide funding and mentors. He observed that the MESSAGE project only provided information. The evaluation team then met with another MESSAGE CSO participant. His experience with the MESSAGE project involved a training session in establishing and working with the media. The participants in this respondent’s session were taught strategies on working with the media, such as developing press releases that make it easier to get CSO information accepted for broadcast. From the two training sessions, the re￾spondent now feels he has a stronger relationship with the media. Through MESSAGE, he remarked that the training opened his eyes to working with the media. Since taking the MESSAGE training the respondent has published two or three programs, and it is now easier because of the relationship he developed with the me￾dia. Both MESSAGE workshops this respondent participated in were only one day sessions. He was introduced to the Media Dark Kit, but has not used it because he did not understand the concept. From the MESSAGE trainings, this participant expected to learn about how to get his CSO message out via broadcast, particularly on the BBC. In addition, he anticipated developing relationships with the local media and acquiring the ca￾pacity to develop messages for the media. The interviewee feels his expectations have been met and achieved. He has successfully had AKBC radio broadcast the CSOs messages on governance related issues and also published information in local newspapers. MESSAGE 33 The respondent suggested future periodic workshops and refresher meetings be held. He also felt there should be more focus on developing messages concerning success stories that would promote what the CSO does, both locally in Nigeria and to the outside world. In a BBCMA Media Dark Kit Success Story titled ‘How Akowonjo Community Women Braced up for Elec￾tions 2011,’ the account relates how one individual from a CSO used the kit’s excerpts from Story Story for voter education in the local area. The Secretariat of Akowonjo Local Council Development Area (LCDA) in Egbeda, Alimosho Local Government was the venue for a discussion by women in the community ahead of the recent general elections. The discussion was held under the auspices of Women Against Violent Elec￾tions (WAVES), an initiative of a local peoples’ organization known as Women Against Hunger Amidst Plen￾ty (WAHAP). 29 In this CSO participant’s Egbeda community, women are vulnerable to material inducements during elec￾tions. To address this problem, WAHAP began voter education in the local markets. Before undertaking the voter education program, the CSO participant had taken part in an introduction session on the use of the MESSAGE Media Dark Kit. Excerpts from an episode of Story Story were played back to the attendees, which identified key talking points on the theme of women’s participation in politics/elections.30 The talking points for the discussions were: factors that lead to apathy on the part of women in politics and electioneer￾ing; poverty; women’s multiple roles; lack of societal support for women’s political aspirations; etc. In re￾sponse to these challenges, attendees offered a variety of solutions. BBCMA MESSAGE Project - Administration The evaluation team found that USAID and the BBCMA encountered challenges administering the MESSAGE project due to factors beyond their control. During the MESSAGE project implementation the BBCMA experienced staffing changes, particularly for the Program Manager’s position. Also, USAID un￾derwent staffing turnover and was currently understaffed and vulnerable. USAID observed that the BBCMA MESSAGE IP was not available during the project. At the same time the BBCMA MESSAGE IP stated that there was little oversight. In addition, this was the first BBCMA project receiving USAID support and BBCMA was unfamiliar with USAID’s reporting requirements. The evaluation team’s analysis is that there were staffing issues at both organizations, and the fact this was the first time BBCMA had implemented a USAID funded project significantly contributed to the administrative challenges. Yet, despite these challeng￾es the MESSAGE overall project was successful according to key informant interviews and BBCMA re￾search. 29 How Ako wonjo Community Women Braced up for Elections 2011 30 How Akowonjo Community Women Braced up for Elections 2011 MESSAGE 34 CHALLENGES From the key informant interviews with trainees, station management, CSO participants, BBCMA administra￾tors and USAID officials, the evaluation team learned about several challenges confronting the MESSAGE project. They were: 1. Media practitioners lacked independence. Station resources are limited, whether or not it is a gov￾ernment or private broadcast station. Thus, many producers and journalists practice armchair jour￾nalism by relying on the phone to interview government officials, community leaders, etc. who make themselves available. This also means that media workers are reliant on station owner￾ship/management permission to expend resources to cover issues. It goes back to the old adage that he who pays the piper dictates the tune. 2. Private stations are motivated by profit. They will only pursue activities that will generate revenue, while keeping expenses low. Producing quality programs and news stories can be expensive. 3. Media professionals are not able to apply editorial values because of censorship by government or private stations interests. 4. The turnover of activity managers in the BBCMA MESSAGE Project affected implementation. 5. In house residential training interfered with personal and official schedules. Trainees often had prob￾lems attending local TTT training sessions due to work and family responsibilities. 6. Most stations do not have internet connectivity. Many of the MESSAGE trainees interviewed had no access to an internet connection at the stations or at home, making it difficult for them to use the project website or I-learn. 7. Computer literacy was a problem. Some interviewees indicated that their limited computer skills in￾hibited learning the editing software, their access to the online project website and I-learn. 8. USAID turnover and short staff affected the administration and oversight of the BBCMA MESSAGE Project. 9. Equipment issues also affected the TTT training sessions. Every trainee interviewed indicated that the TTT workshops were limited since the trainees were organized into groups of 7 to 10 with only one laptop to use. 10. While trainees interviewed reported that they learned about engaging audiences, some noted that more was needed on social media. 11. University students interviewed noted that the MESSAGE training was valuable in providing practi￾cal, hands-on experience. Their institutions were theoretically focused. 12. More time was needed in training and working with the CSOs. MESSAGE 35 LESSONS LEARNED 1. The multifaceted approach to professional training received by the radio and television station partic￾ipants had a synergetic result that contributed to the successful outcomes reported by BBCMA in its reports and by the evaluation team. a. The combination of the TTT, Coproduction, Mentoring, I-learn and Bursary Awards served to reinforce the professional editorial and technical production standards leading trainees in￾terviewed to internalize the training they received. This approach also served to overcome the weaknesses that may exist in any one area of training with strengths in the other compo￾nents. b. Trainees interviewed step down what they learned to colleagues at their respective stations. 2. The BBC is highly respected and this made station ownership/management and trainees more recep￾tive to the BBC’s internationally aclaimed and respected editorial standards. Even management in￾terviewed at federal and state stations admired the BBC standards and appreciated the training their employees received. Due to this appreciation for the BBC editorial standards, producers and journal￾ists interviewed at federal and state operated stations sometimes were able to convince management to broadcast programs that might otherwise not be aired. 3. While the multifaceted approach contributed significantly to the positive outcomes of the MESSAGE project, there were aspects of the training that were weak due to various technicalities with the components. a. One major flaw concerns the I-learn component of the training. Several trainees interviewed noted that access to the internet was a major problem. Some respondents were able to access the internet at their respective station facilities. However, at the FRCN radio facility the sta￾tion did not have internet service. Others went to a cybercafé to access the I-learn training on the internet. Another problem associated with the I-learn component concerned com￾puter literacy. b. The TTT component’s weakness was the training group approach. The problem centered on having a group of 7 or 8 trainees work on just one computer. This made it difficult to learn how to use the software that trainers were introducing in the sessions. MESSAGE 36 KEY INFORMANT SUGGESTIONS Key Informants suggested that the MESSAGE training should train and reach more people in the Nigerian broadcast system. Respondents noted that it would help make all broadcasters in Nigeria more trustworthy, if stations adopted editorial standards. Some interviewees offered a similar suggestion, i.e., they would like to see more journalists participate in order for everyone “to be on the same page,” particularly with respect to applying the BBC editorial standards. Another suggestion that almost all the key informants offered concerns the structure of the TTT training. The trainers divide the participants into groups of 7 or 8 individuals with each group having one laptop com￾puter to share and use. This made it difficult for some to learn the material. It was suggested that more com￾puters be allocated for the TTT training. Also, concerning the TTT training, some respondents noted that the close proximity of the workshops to their stations and homes caused problems. Often work and family obligations interfered with their training. They suggested having the TTTs located away from their residential locations. Other key informants indicated they would like to see future training deal more with social media. During the interviews some respondents spoke about the social media training they received and used, while other inter￾viewees noted that this aspect of the training was glossed over by the trainers. Management key informants suggested more training in marketing and promotion was needed, and how to make money from a station’s website. Some respondents would like to see the website have a component where trainees could share pro￾grams they produce with each other. The FCRN manager stated that the training was important for the professional development of her staff, be￾cause none of them had received training from the Nigerian government. She would like to see the MESSAGE training for management extended to the regional level where it is definitely needed. MESSAGE 37 CONCLUSIONS The evaluation’s objective was to assess the BBCMA MESSAGE project results concerning the “Nigerian media’s capacity to cover governance and related issues.” Through thematic training, both editorial and tech￾nical, the MESSAGE project was successful in influencing trainees interviewed and sample stations to cover governance related issues and themes. The evaluation team found that the BBCMA MESSAGE training ef￾fectively improved the professional editorial and technical skills of the trainees met. Through the thematic training the key informants understood the importance and relevance of governance issues, and the impact information about the issues has on radio audiences. The result was that the MESSAGE trainees interviewed engaged in covering and developing programming on governance issues, particularly with the help of the Bur￾sary Awards. While there were challenges when it came to the training, key informants were enthusiastic about the new principles and techniques they learned. Important in the evaluation team’s findings are that every participant interviewed had implemented the principles they learned, investigating, researching, and getting the opinions of the people on issues to produced balanced news and programs for their respective stations. Because the trainees’ interviewed were excited about their experience, they were eager to pass on their newly acquired knowledge and skills to colleagues at their respective stations. The limiting factors for these trainees were station’s resources and, for government stations, censorship. Thus, the Bursary Awards were critical for the key informants to fully implement what they learned in the MESSAGE workshops. In addition, the train￾ing discussions that the participants interviewed had concerning censorship, and the sessions that focused on issues helped some federal and state respondents to often overcome censorship issues. Management interviewed at participating stations was enthusiastic about the training their employees received from the MESSAGE project. Every station manager met observed that they came away from the training with a new understanding about the need for programming and marketing to work together on programs that serve and attract audiences and advertisers. From the evaluation interviews managers indicated that, after returning to their respective stations from the MESSAGE training, they immediately implemented what they had learned. While the evaluation team was unable to get specific numbers, station managers interviewed not￾ed that the revenue enhancement learned via MESSAGE training was being applied and they were seeing success. The evaluation also involved assessing the MESSAGE project’s edutainment drama Story Story series funded by USAID. As the findings indicate, the USAID funded Story Story series had influence on a consid￾erable number of its listeners, empowering them to become more engaged in governance issues and the elec￾toral process. The BBCMA research results showed that a majority of listeners were influenced by the USAID funded Story Story series, motivating them to think differently about governance issues, register and vote in elections, and influence others to do the same. An additional evaluation objective concerned “the capacity of CSOs to engage in the media.” MESSAGE 38 The CSO component for the MESSAGE project presented the evaluation team with the most daunting as￾pect for assessing the MESSAGE project. The BBCMA reports noted positive impact in training CSOs on how to connect with the media through press releases and programs attractive for broadcast. The two inter￾views with CSOs conducted by the evaluation team presented conflicting results. One individual observed that the sessions were helpful, but very limited. The other interviewee felt the MESSAGE trainings were very useful and helped to develop media relationships. However, both noted that they had difficulties with the Media Dark Kit that was supposed to be used to support the Story Story component of the project, neither one used it. Yet, the BBCMA MESSAGE reports indicate positive results in other states. Unfortunately, this is something the evaluation team was unable to verify directly. One interesting discussion that evolved during this evaluation concerns whether the MESSAGE project was attempting to do too much. While there is logic in the holistic approach that the MESSAGE project took in training media professionals and CSOs, the evaluation team felt that it was possibly stretching the limits and resources of those involved in the training and administration. From the interviews with key informants and BBCMC research, this project accomplished a great deal regarding training and developing media profession￾als, especially with regards to governance issues and engaging audiences. But it is difficult to assess how suc￾cessful the CSO component was compared to the rest of the MESSAGE project, particularly with regard to having independent empirical evidence available. Another issue was that the administration of the MESSAGE project by USAID and the BBCMA encoun￾tered difficulties due to factors beyond their control. Finally, the evaluation team’s effectiveness was significantly restricted due to the limited in-country time allot￾ted for assessing the MESSAGE project, the methodologies stipulated by the SOW and security issues. Twelve in-country days were allocated for the evaluation team to travel to sample focal areas and interview key informants. The team found this to be woefully inadequate with the evaluation results limited in their ability to be generalized to the project. This was particularly evident in regards to assessing the CSO compo￾nent of the project. MESSAGE 39 RECOMMENDATIONS 1. Future programs should consider the Nigerian media environment. FCRN and state owned and op￾erated stations experience censorship. In addition, both government and private broadcast stations have limited resources for costs associated with investigative reporting and for internet access, which is needed for research and audience interactivity. Also, private stations are profit oriented and will not invest in programming unless it will produce revenue. The evaluation team learned that station managers interviewed respect the BBC editorial standards and can be influenced to air programs that follow those principles. Trainers should educate and discuss with the participants about how BBC standards can be used to influence station management to air programs. In addition, more manage￾ment training is needed concerning the revenue benefits of quality programming, including marketing and promotion. 2. The MESSAGE project should consider the computer literacy of participants and trainees’ internet access available at their stations. In addition, the MESSAGE website should be designed so that par￾ticipants can share programs and interact with one another and with CSOs. 3. The MESSAGE project’s interventions should target the educational system, i.e., those organizations responsible for training media practitioners. This would include journalism and media instruc￾tors/professors with an emphasis on a practical, hands-on approach. Thus, journalism’s editorial standards and best practices could be incorporated into the curriculum at such institutes. 4. The MESSAGE project should incorporate more training on social media. Participants interviewed are discovering the importance and utility of engaging audiences and desire more training in this area. 5. USAID should continue to fund the Story Story series on governance issues. The series documented success is noteworthy. The show has a following and is well established among the Nigerian popula￾tion. 6. Finally, future evaluations need to allocate more time to conduct a thorough assessment that can produce results generalizable to the project. __________________ 32 BBCMA MESSAGE Midline Citizen Voice Survey October 2011 pg 26 MESSAGE 40 ANNEX A. SCOPE OF WORK – EVALUATION Evaluation of USAID MEDIA SUPPORT FOR STRENGTHENING ADVOCACY, GOOD GOVERNANCE AND EMPOWERMENT (MESSAGE) PROJECT Cooperative Agreement No.620-A-00-09-00016-00 Implemented by BBC World Service Trust (Estimated start date: Second week of November 2011) I. Objectives of this Evaluation Goal: To determine how best to assist the media in Nigeria to play an effective role in holding government accountable Purpose: The purpose of the evaluation are two-fold: 1) to assess the results of the MESSAGE project to date, especially in terms of evidence of success, failure, obstacles and lessons learned; and 2) to provide rec￾ommendations and strategies to improve the MESSAGE project over the remaining life of the project. Objectives: The objectives of the evaluation are to assess the results achieved in the areas of: 1) Nigerian media’s capacity to cover governance and related issues 2) Financial sustainability of its media partners 3) Capacity of Civil Society Organizations to engage in the media 4) Audience engagement in governance issues II. Issues and Questions to Be Addressed The questions should guide the team’s analysis, but do not need to be addressed individually or sequentially in the report. With the concurrence of the USAID/Nigeria DG Office, the team may decide to add, refine or delete questions, or modify the proposed structure, if they believe it would highlight key findings or otherwise add value to the analysis. III. Project Results to Date General Questions  What MESSAGE activities appear to have more or less impact? And why?  To what extent has the project improved the Nigerian media’s (i.e. partner radio and TV stations within project focal states) technical and thematic capacity to produce radio projects which engage audiences in informed discussions around governance issues, e.g. transparency, effectiveness of gov￾ernment institutions, management of public resources, health and water management and community service delivery, education and conflict mitigation as well as election and electoral practices?  To what extent has the project build the capacity of civil society organizations to use the media? IV. Specific Questions ANNEX, . 2  Has assistance helped the media partners in diversifying methods of revenue generation?  What has been the impact of training on the areas of financial sustainability and of revenue genera￾tion?  What has been the extent of resources leveraged from BBC WST and how has it impacted on the in￾tervention?  To what extent have thematic and technical trainings improved upon service delivery (i.e. quality of project production and content) of the media stations?  To what extent has the increased audience engagement in governance issues?  What have been the results from Bursary Awards provided to media practitioners?  Have there been any improvements in interactive and governance-related project in the media (i.e. partner radio and TV stations within project focal states)?  Has there been an improvement on the percentage of people who believe the media enables them to be more engaged, and have a voice in governance? Attribution  To what extent has equipment distributed to media stations been effectively utilized in ways relevant to governance intervention? V. Methodological Approach USAID/Nigeria seeks a two-pronged, mixed methodological approach for this evaluation. In the first in￾stance, evaluators should compare project baseline and end state information as collected by BBCWST in the project performance management plan. This information should be analyzed in conjunction with available project input and context data to determine the results of activities vis-à-vis stated project objectives. In the second instance, evaluators should gather additional evidence of change through key informant inter￾views and other data sources, as available. Data collected through interviews or other sources must document results of project activities, such as informed opinions of changes due to the adoption of recommended tech￾niques or testimonies of trainees about how they have applied skills learned to their job. Testimonials stating that training or technical assistance was “useful” without supporting statements of how these led to changes in targeted institutions will not be considered evidence of change or impact. Getting to the Right Answer Evaluation Ques￾tion Type of Analy￾sis/Evidence Needed Method of Data Col￾lection and Source Sampling or Selec￾tion Approach Data Analysis Method (s) Method Data Source What MESSAGE activities appear to have more or less impact? And why? Review project strategies, work plan, reports, tes￾timonies and pub￾Qualitative Interview notes Work plan reviews and testimonials Interviews with IPs Third source of inde￾pendent information (Public). Purposive BBC WST and audience Content Analysis Constant Compari￾son/Grounded Theory Narrative Summary Anal- ANNEX, . 3 Evaluation Ques￾tion Type of Analy￾sis/Evidence Needed Method of Data Col￾lection and Source Sampling or Selec￾tion Approach Data Analysis Method (s) Method Data Source lications. ysis Triangulation of infor￾mation gathered from different sources. To what extent has the project im￾proved the Nigeri￾an media’s? Qualitative Interview notes and triangulation template Interviews with key informants. Media Stations State Ministry of In￾formation and proprie￾tors Random selection of informants States and media stations Logical Analysis/Matrix Analysis To what extent has the project built the capacity of civil society organiza￾tions to use the media? Qualitative and quantitative. Questionnaire and interview tem￾plates Survey questionnaires and Interview with CSO partners, Random selection of CSOs across the Country Constant Compari￾son/Grounded Theory Narrative Summary Anal￾ysis Triangulation of infor￾mation gathered from different sources. Has assistance helped the media partners in diversi￾fying methods of revenue genera￾tion? Qualitative and Quantitative Information on revenue comparing before and after Key informants Purposive selection of informants (Media Stations) Logical Analysis/Matrix Analysis What has been the impact of training on the areas of financial sustaina￾bility and of reve￾nue generation? Qualitative Interviewer notes and testimonies Review program objectives and Key Informants Purposive selection of informants (Media Stations) Logical Analysis/Matrix Analysis ANNEX, . 4 Evaluation Ques￾tion Type of Analy￾sis/Evidence Needed Method of Data Col￾lection and Source Sampling or Selec￾tion Approach Data Analysis Method (s) Method Data Source deliverables com￾paring before and after What has been the extent of resources leveraged from BBC WST and how has it impact￾ed on the interven￾tion? Qualitative and quantitative Evidence of re￾source leveraged over a period of time and activities funded with such resources Interview with IP and review of resources and sources Purposive and ran￾dom selection of informants Content analysis To what extent has thematic and tech￾nical training im￾proved service delivery (i.e. quality of project produc￾tion and content) of the media sta￾tions? Qualitative and quantitative. Number of people trained and im￾proved services by the media station Interviews with select￾ed media stations Random representa￾tive selection of me￾dia stations Content Analysis Constant Compari￾son/Grounded Theory Narrative Summary Anal￾ysis Triangulation of infor￾mation gathered from different sources To what extent has the increased audi￾ence engagement in governance issues? Qualitative Interviewer notes and triangulation template Key informants Purposive selection of informants Logical Analysis/Matrix Analysis What have been the results from Bursary Awards provided to media practitioners? Qualitative Innovations and project undertaken by the awardee Key informants Purposive selection of informants Comparison/Grounded Theory Narrative Summary Anal￾ysis Triangulation of infor￾mation gathered from ANNEX, . 5 Evaluation Ques￾tion Type of Analy￾sis/Evidence Needed Method of Data Col￾lection and Source Sampling or Selec￾tion Approach Data Analysis Method (s) Method Data Source different sources Have there been any improvements in interactive and governance-related project in the me￾dia (i.e. partner radio and TV sta￾tions within pro￾ject focal states)? Qualitative and quantitative Interviewer notes and questionnaire administered Key informants Random representa￾tive selection of me￾dia stations. State Ministries of Infor￾mation could be con￾tacted to prove activ￾ities of State media stations Qualitative and quantita￾tive data analysis. Has there been an improvement on the percentage of people who believe the media enables them to be more engaged, and have a voice in govern￾ance? Quantitative and quantitative. Interviewer notes and questionnaire administered Key informants Purposive selection of informants Descriptive statistics To what extent has equipment distrib￾uted to media sta￾tions been effec￾tively utilized in ways relevant to governance inter￾vention? Qualitative and Quantitative Number distribut￾ed and in use. Notes from as￾sessment of equipment Key informants Purposive selection of media stations Logical Analysis/Matrix Analysis VI. Recommendations Given the above analysis, recommendations should address any programmatic adjustments that should be made to maximize the effectiveness of USAID’s assistance to MESSAGE, as well as any necessary changes to improve effectiveness of MESSAGE, to:  Assist media outlets that receive USG-supported training to promote financial sustainability  To train journalists and media managers, and  Improve citizen knowledge about governance issues through the project’s media assistance activities VII. Required Tasks and Work Plan ANNEX, . 6 VIII. Team Composition, Activities and Timing Team Composition The team will include:  One local consultant with expertise in the Nigerian media and citizen participa￾tion/civil society participa￾tion in governance  One international media research or evaluation ex￾pert- Team Leader.  In Addition, the team will be complemented by Mark Koening from USAID Washington HQs. And  NMEMS II and its part￾ners (MiraMonitor Con￾sulting and Management Strategies for Africa) to as￾sist with the data collection platform, data entry and data analysis. IX. Activities and Timing The USAID/Nigeria Mission requests that the team arrive in Abuja at the same time for the initial briefings and discussions with USAID’s DG Team Leader and other Mission officers. Subsequently, the team may commence field trips and meetings. During the initial meetings in Abuja, the evaluation Team Leader will present in writing and orally the team’s proposed work plan for the entire period of the team’s presence in Nigeria as well as thereafter with respect to the submission of the draft and final reports, i.e., draft due no more than five working days after comple￾tion of field work and final due no later than four work days after receipt of the Mission’s final comments on the draft report. The work plan will also include a schedule for periodic USAID meetings/progress reports and possible submissions of specific work products, as determined by the two parties. While subject to change with the acceptance by both parties, it is envisioned that all team members will be in Nigeria the entire duration of the evaluation’s in-country component. The team leader will be provided a total of 4 additional days back in the U.S. to ensure the completion and transmission of the final report as well as the closure of any outstanding matters. The team leader will be responsible for managing the team members, organizing its work, and the draft of the report. NMEMS II is responsible for quality control and delivery of the required report as agreed to by USAID. X. Deliverables Deliverables will include the out-briefing and supporting documents and the final report. A detailed written outline and oral de-briefing of the conclusions, supporting findings, and recommendations Tasks (All team members unless otherwise noted) Team Leader (TL) CCN Travel- Return Trip 2 Document Review, attend Team Planning Meeting, Washington meetings, Evaluation Work Plan, Tool & Instruments Development & Prepare first draft of report’s outline for DG team leader’s approval 6 6 In-country Abuja Interview & Fieldwork (initial briefings, meetings, field visits, & debrief￾ings) 12 12 Data Analysis, Debrief, Draft Report & Submit to NMEMS COP/TMG 5 4 Final Report 4 2 TOTAL 29 24 ANNEX, . 7 of the evaluation team will be presented to the USAID/Nigeria DG Team Leader prior to the departure of the team. Briefings for other USG and/or Nigerian officials will be provided as determined by USAID. The final report, in MS Word, will not exceed 40 pages (excluding executive summary and annexes). Single line spacing and 11 point font are acceptable. The report should include: A. Executive Summary (not to exceed 10 pages, which can be used as an independent briefing paper) B. Introduction C. Methodology D. Conclusions and Supporting Findings A. Project results B. Management C. Fiscal E. Recommendations for Ongoing and Future Activities A. Assistance to Core Institutions B. Project management C. Fiscal project management F. Annexes: This section may include, inter alia, an annex of data sources utilized, key informants in￾terviewed, and listing Useful Tools and Products that could be repackaged and produced for wider distribution to other USAID-funded implementing partners in Nigeria or USAID-funded projects in other countries. Following receipt of USAID/Nigeria comments, the team will address questions and incorporate any re￾quested additions or changes in the draft document. The final report will be submitted 10 work days after the team leader’s receipt of USAID’s final written comments on the draft report. The Mission will receive ten paper copies of the final report and an electronic version. ANNEX, . 8 ANNEX B. LIST OF PROJECT MESSAGE PARTNER STATIONS State S/No Stations FCT (Abuja) 1 FRCN Network Headquarters 2 Hot 98.3 FM 3 Vision FM 4 Love FM Anambra 5 Anambra Broadcasting Service (ABS Radio), Awka 6 Anambra Broadcasting Service (ABS), Onitsha 7 FRCN Purity FM 8 MINAJ TV Akwa Ibom 9 AKBC Radio 10 Radio UniUyo Bayelsa 11 Radio Bayelsa (Bayelsa Broadcasting Corporation) 12 RayPower FM, Elebele Bauchi 13 BRC AM / FM 14 Bauchi TV 15 FRCN Globe FM 16 NTA Bauchi Lagos 17 Eko FM 18 Raypower FM 19 Star FM 20 FRCN Choice FM Sokoto 21 Rima AM / FM 22 Rima TV 23 NTA Sokoto 24 Nagarta 747, Kaduna ANNEX, . 9 ANNEX C. MESSAGE Year 1 Activity Report (01 October 2009 to 30 September 2010) STORY STORY PRODUCTION DETAILS 6.3 Production During its first year, elections and the electoral process constituted the focus of production of Story Story. The episodes of Story Story addressed the following governance-related themes: 1. Peaceful demonstration as a way of seeking redress 2. Resolution of pre-election disputes through dialogue 3. The role of godfathers in the sustenance of democracy; the need to enshrine ideology as a guiding principle in politics 4. The effects of materialism on politics 5. The dangers of girl child labor 6. The role of the media in the sustenance of democracy 7. Political debates and campaigns in democracy; the media and the politics of bribery 8. Political education and enlightenment as tools for the sustenance of democracy 9. Violence and intimidation in politics 10. Political education and enlightenment as tools for the sustenance of democracy 11. The effects of materialism on politics 12. Good conduct of elections Planning for the resumption of production of our popular and long running radio drama Story Story started in November 2009. Three series were planned for Year One of the program, and would be broadcast on the BBC World Service from June 2010 initially, then via partner radio stations around the country a month later. In the 2nd quarter, the plan for the quarter was for the production of 12 episodes of series 17 of Story Story. Scripts were generated and Production took place during the first week of April. The thematic issues ad￾dressed in the scripts were developed from the creative writing process. Related activities included a content research conducted in Sokoto and Bayelsa states through collaboration between the production and audience research teams. It also involved interactions with media practitioners, politicians, traditional rulers, youth leaders, local government chairmen and INEC officials aimed at getting their perspectives on governance issues. In the 3rd quarter after recording Post-Production took place at the BBC WST studio. The writing of Series 18 then followed between 9 and 29 May. The writing took place over three weeks at the BBC WST Guest house in Life Camp, Abuja between 9 and 29 May. In order to achieve the relevant MESSAGE Intermediate Result, namely, “Increased citizens’ en￾gagement in governance issues,” participants explored themes around the electoral process, such as voter reg￾istration, the need to elect credible leaders, and what citizens need to do before and during the voting and took into cognisance the linkages between thematic areas. Overall, four (4) different episodes of series 17 were pre-tested using focus groups. ANNEX, . 10 In the 4th quarter, the focus of production was to get 12 episodes of series 18 of Story Story ready for airing on the BBC World service on 2 October. Activities centered on recording of the episodes, an important stage of the process to ensure broadcast-ready material, following the very arduous script writing and edit exercises.  87 stations, including 22 MESSAGE partner radio stations in the six program states, broadcasting between one and twice weekly episodes, devoted an estimated 1,372 hours to the airing of the epi￾sodes of series 17, which were also broadcast on the BBC World Service. In effect, millions of listen￾ers followed the broadcasts and repeats. In summary, during the year, 24 episodes of Story Story in two series (17-18) were produced and aired on a total of 87 stations, including 22 partner radio stations on the MESSAGE project in the six project states and on the BBC World Service, These episodes explored themes around lections, women participation and gov￾ernance issues in social service delivery. ANNEX, . 11 ANNEX D. EVALUATION TEAM - CALENDAR Evaluation Activity/Sites and Interviews Activity/Site Date Media Station/ Or￾ganization Activity/ Interviewees Evaluation Team February 20, 2012  Develop Evaluation Plan and Interview Protocols. USAID - In Brief BBC Media Action Pro￾gram Office, Abuja February 21, 2012 1 st Meeting: Tuesday, February 21, 2012 USAID BBCMA  Brief Ekanem Bassey USAID on Evaluation Plan  Mr. Fidelis Ekom, Pro￾gram Manager  Anu Mohammed, Re￾search Officer BBC Media Action Pro￾gram Office, Abuja 2 nd Meeting: Wednesday, February 22, 2012 BBCMA  Ms. Linda Nwoke, COP Akwa Ibom State Friday, February 24, 2012 AKBC, Akwa Ibom State (State owned) Uyo University Radio  Inyene  Edemanwan Udoh  Emeh, Marketing Direc￾tor  DG Radio Station  Uko Edet, Graduated 2012 BBCMA Trainer Inter￾view (held at Akwa Ibom)  Fatima Shehu, BBC MA Trainer Saturday, February 25, 2012 CSOs; Youth Dev. Corps Nig. African Welfare and Dev. Center  Essien  Ekanem Inyang, Exec. Chairman Lagos State Monday, Feb. 27, 2012 Eko FM (State owned Station)  Muyiwa Ojekunle, Snr Producer  Oluwabunmi Oloke, Fea￾ture Prg Producer  Ayo Shotonwa, Dir. Of Marketing ANNEX, . 12  Oloke Bunmi  Omolgi Olubode Star FM Lagos (private owned)  Ms. Jumoke Adesua, Head Marketing  Faosiyah Gbadeyinka  Seun Awosika Anambra State Tuesday, February 28, 2012 MINAJ TV Station, Oposi, Anambra State (Private owned station)  Mr. Efeanyi, Head of Sta￾tion  Mr. Onyenchi Oke￾chukwu, Head of Market￾ing  Mr. Paul Lawrence, (in charge of Station’s Ac￾counts) Wednesday, February 29, 2012 Purity Radio Station (FRCN) (Federal owned Station)  Mr. Chinedu Imanu, Pre￾senter/Producer  Mrs. Chinwe Ononye, General Manager  Ms. Rosemary Anyika, Reporter/Editor  Joyce Oramu, Prin. Mar￾keting Officer ABUJA Thursday, March 1, 2012 Listener Group Facili￾tators and Participant ABUJA USAID - Debrief Friday, March 2, 2012 USAID  Team Leader Minnie Wright  MESSAGE AOTR Ekanem Bassey ABUJA Saturday, March 3, 2012 MESSAGE OFFICE  Evaluation team meets to discuss final results and assessment ANNEX, . 13 ANNEX E. STATION PROTOCOL Give us their overview of their participation in the MESSAGE project What did they like? What did they dislike? What would they want changed? Suggestions for change Trainings and types Agendas Financial sustainability for management Co-production TTT - Technical and Thematic Training Mentoring I-learn The outcomes and impact of the training (indicators) What have you done differently due to the trainings? MOUs/Agreements/Contracts with BBC Media Action Need to see the contracts -- samples of agreements What would they like to see differently? Bursary Awards/Agreements -- with management Need to know procedures How management perceives the Bursary How were the programs aired? Are there recordings of the program? Were stations involved in station selection? EQUIPMENT Verify equipment acquired List of equipment How has it enhanced your work? Was it what they need? In regards what to do differently -----Journalists and program makers who receive training and mentoring at the partner radio stations will demonstrate increased impartiality, accuracy, fairness and technical competency in their coverage of key governance issues. Their progress towards these objectives will be measured via a combination of interviews, trainer reports and content analysis exercises which will demonstrate improve￾ments as a result of the training. ANNEX, . 14 TRAINEES - PROTOCOL Tell us about your experience with the MESSAGE project Access I-learn? What have they learn? How easy is to use? Recommendations or what would like to see? Has it impacted their work? How What did you like? What you did not like? What have you learned to do things differently? What would you suggest for changes to the MESSAGE project? Would you participate if the program is extended? How were your programs aired? How did you become aware of the Bursary Awards? Did you apply for a Bursary Award? Did you receive a Bursary Award? If so, what program did you produce? If not, how do they feel about it? In what regards as to what do differently ----Journalists and program makers who receive training and men￾toring at the partner radio stations will demonstrate increased impartiality, accuracy, fairness and technical competency in their coverage of key governance issues. Their progress towards these objectives will be measured via a combination of interviews, trainer reports and content analysis exercises which will demon￾strate improvements as a result of the training. (CVs) What is your background? Number of trainings held over a period of time How trainings are evaluated Relationship between TTT coproduction mentoring and I-learn How much time do you spend in the field? How is your work assigned? Assigned states? Or as needed among project states? Curriculum/Workshop Agenda Did you participate in developing the DART tool kits (multimedia)? What are the outcomes? What is expected outcome? Challenges ANNEX, . 15 Lessons learned What do differently? Suggestions ANNEX, . 16 ANNEX F. BBCMA MESSSAGE Midline Citizen Voice Survey October 2011 RESEARCH METHODOLOGY Research Objectives: This midline study has two main aims. The first main aim is to monitor shifts or changes in audience percep￾tion of governance issues such as: • Election and electoral processes • Media’s role in informing people • Media’s role in giving people a voice • Engagement with the media The second aim of this midline study is to assess the performance of project outputs in reaching audiences. That is, to measure: • Listening to the Trust’s drama programme Story Story • The reach of the project partner stations • Listening some of the programmes project partner stations are producing as part of the ca￾pacity building engagement with the Trust In order to assess trends / changes since the baseline, the midline study was designed to generate data to be compared with 2010 Baseline Citizen Voice Survey and simultaneously provide Methodology: A midline population based sample survey was conducted in eight (8) project focal states (i.e. Anambra, Akwa Ibom / Bayelsa, Bauchi, Lagos, Sokoto / Kaduna and the FCT) which fall within the 6 geopolitical zones as depicted below: North East North Central North West South South South West South East Bauchi FCT Sokoto Kaduna Akwa Ibom Bayelsa Lagos Anambra Sampling Frame The midline study sample frame was adults aged 18 – 64 years in urban and rural locations from all eight states. The Nigeria demographic statistics estimated the urban/rural split to be around 40:60. The amount of urban and rural clusters selected in each state was reflective of the urban/rural split within each state. The data pre￾sented from the midline is un-weighted. ANNEX, . 17 The sampling frame for the midline differs from that of the baseline in two respects. First, in the baseline study, only individuals identified by screening questions as media consumers (radio and TV) were included in the study. In the midline study, in order to obtain a population-based measure of the project’s reach, respondents included both media consumers and not. Second, in the baseline, seven states were surveyed namely: Anambra, Akwa-Ibom, Bauchi, Bayelsa, FCT, Lagos and Sokoto. For the midline survey, Kaduna was added to the states surveyed making a total of eight locations. These variations between baseline and endline samples were taken into account during analysis to ensure comparison of like with like. In this report, therefore, the sections and figures that presents "trend analysis" between baseline and midline data only include media consumers and exclude all respondents from Kaduna. Sampling A multi-stage probability sampling technique was employed to select a representative sample of 2006 adults be￾tween ages 18 – 64 years in urban and rural locations. The first stage of sampling involved a purely random se￾lection of rural and urban localities using a table of random numbers. Stage 2 involved a random selection of enumeration areas (EAs) from the list of EAs within the selected rural and urban localities. Stage 3 involves the selection of household through the use of day’s code, the starting point is determined by the day’s date and a regular interval maintained throughout. This is followed by the selection of individual respondents, a respond￾ent is selected from a selected household through the listing of all eligible respondents in that particular house￾hold and the interviewed respondent selected through the use of a Kish grid. Questionnaire The questionnaire was originally prepared based on generally accepted governance related themes in English and translated into 4 languages (Pidgin English, Yoruba, Hausa and Igbo). It contained the following broad themes: • Perception of governance • Political engagement and democracy • Elections • Corruption • Conflict • Perception of media’s role in governance • Assessment of partner stations and programme • Story Story assessment; and • Media habits Pilot Study Prior to the fieldwork, a pilot study was conducted in urban Nasarawa State to test the instrument, survey methodology and data management. This was conducted by members of R&L team and freelance research consultants, who administered 12 questionnaires. ANNEX, . 18 Fieldwork Through a competitive bidding process, an independent research agency (AGF Logistic Services) was con￾tracted, to undertake the fieldwork. The agency recruited the supervisors and interviewers that were used for the fieldwork. The training of all field workers was done by members of BBC WST Research and Learning Unit. Supervisory visits and quality control were undertaken by selected members of R&L team. It was useful to translate the questionnaire into local languages to aid discussion, therefore, the questionnaire was translated into Yoruba, Igbo, Hausa and Pidgin languages for use during fieldwork (questionnaires are available upon request). The translated copies were used by the interviewers as master copies. The same ap￾proach was adopted at the baseline in 2010. There was one interview team per state, each team consisting of 5-6 interviewers and a field supervisor. Table showing the breakdown of language of interview per survey location Language of interview Survey location where the language was used Pidgin FCT, Anambra, Bayelsa, Akwa Ibom and Lagos Hausa FCT, Bauchi, Sokoto, and Kaduna Yoruba Lagos Igbo Anambra The table below shows the breakdown of proportion interviews by each of the language used with almost half (45%) of the interviews conducted in English. Of the other local languages used, Hausa language is the most frequently (32%) used language of interview, this may be as a result of the fact that the language is an indige￾nous language in four out of the eight states where the survey was conducted as can be seen in the table above. Only 5% of the interviews were conducted in Igbo language. Language of interview Proportion of respondents interviewed per lan￾guage [Sample size=2006] English 45% Pidgin 10% Hausa 32% Yoruba 8% Igbo 5% ANNEX, . 19 Training The training of survey personnel was at two levels: central training and state level training. The three day central training held in Lagos involved selected members of the R&L team, all supervisors from the research agency and interviewers that worked in Lagos. The training was on sample selection (including household selection) and all aspects of fieldwork. A considerable amount of time was devoted to the review and practice of the ques￾tionnaire. Field workers for other states were recruited and trained by the supervisors trained at the central level training. Field Data Retrieval Data retrieval was done on a daily basis. The interviewer collected the information from the respondents, checked the questionnaire and submitted his/her quota for the day to the supervisor who edited the question￾naire on a daily basis and returned them to the same agency. When possible data errors (e.g. inconsistent infor￾mation, omission of part of questionnaire etc.) were traced to their original source, through re-visits, mistakes and omissions were corrected. A quality control officer ensured that all instructions were followed, responses were consistent and that questions were fully answered. A questionnaire was not considered accepted until a quality control officer had certified it. Data Entry, Validation and Analysis Data entry software with in-built check command was used for data entry. The data were subsequently import￾ed into SPSS (version 18.0). The data submitted by the agency was checked for cleanliness, validity and con￾sistency and certified suitable for analysis. Data analysis and reporting were conducted by the BBC WST R& L team. In evaluation of difference in mean between two independent samples (i.e. Baseline and Midline), student-t test for difference in mean was used while chi-square test of independence was used to test the significance among variables in the midline analysis. Trends, Midline and Impact Analysis In order to compare like with like, different bases (or denominators) were used for trend, midline and impact analysis. Trend analysis The trend analysis in this report includes only the media consumers and respondents from the seven states of Anambra, Akwa-Ibom, Bauchi, Bayelsa, FCT, Lagos and Sokoto. Midline respondents from Kaduna and those who are non-media consumers are not included in the trend analysis. Midline analysis When looking at just midline data, all midline respondents are included. This includes respondents from all eight states, including Kaduna, as media consumers and non-media consumers. Impact analysis When looking at impact analysis, comparing sub-groups of midline respondents to other sub-groups, re￾spondents from all eight states, including Kaduna, are included. For some questions all respondents are in￾cluded; in others only media consumers or other sub-groups are included. These bases are noted in the tables and charts. ANNEX, . 20 Baseline Midline Media consumption TV a a Radio a a Internet x a None x a State Anambra a a Akwa Ibom a a Bauchi a a Bayelsa a a FCT (Abuja) a a Lagos a a Sokoto a a Kaduna x a a included in sample x NOT included in sample Highlights those respondents included in trend analysis. Highlights those respondents NOT included in trend analysis. Significance test The significant variables are represented with bold bars in charts and bold variables (e.g. where there are sig￾nificant differences between locations, the variable “location” is represented in bold) in tables throughout the report. Therefore, wherever bar or variable is presented in bold format, this implies that there exists a signifi￾cant difference in the proportion presented. It should be noted that it is also stated wherever there exists a significant difference otherwise difference in proportion are mere descriptive. Ethics The structured research ethics were observed from baseline through midline during the fieldwork, the re￾searchers ensured the following: - Consent was obtained from community leaders/heads especially in the rural locations and head of household in urban prior to the commencement of interviews in the different community and household. - Respondents clearly understood the purpose of the study and gave the consent to be interviewed without being pressured therefore participation was voluntary. - Interviews were conducted in private locations i.e. no third parties were allowed. - Respondents were not pressured into answering questions; they were also not pressured into giving specific answers. ANNEX, . 21 - Respondents were also informed their right to withdraw from the interview whenever they wish to do so. - Respondents were not induced with gift or incentives before giving their consent to participate in the research. - Respondents were assured of confidentiality of information given and not to divulge the personal da￾ta or information without their consent. ANNEX, . 22 ANNEX G. STORY STORY LISTENING GROUPS33 Objectives The key objectives of the “Story-Story” listening group(s) in the FCT are to:  Strengthen awareness of the output as well as gather regular feedback to feed into the production of future series.  Better understand listeners’ engagement with Story-Story and any impact it has on the audience within communities  Understand how listeners perceive the media's role in the forthcoming elections and governance process more generally, particularly the use of drama programs.  Serve as an Audience Consultative Forum for program makers as well as to provide a space for lis￾teners to share their thoughts and generate ideas on issues that concern them in the communities, that can feed into script writing for Story Story  Inform development of future MESSAGE ‘Media Dark’ activities through learning from the listen￾ing groups Methodology Groups: A total of 3 groups within the FCT are monitored on a weekly basis. This includes one (1) male and two (2) female groups in 3 different locations namely Karshi, Area one and Gwagwalada. Each group consists of 9 to 10 persons who speak and understand English / Pidgin consistent with the lan￾guage of the drama. Group profile: Age: All group members fall within the ages of 18 to 64 years old consistent with the target group of the drama Story Story. Religion: Each group comprises Christians and Muslims alike. Radio listenership: All group members are regular radio listeners who listen at least three to four times in a week. They are also active listeners of Story Story who listen weekly. Mode of group operation: Prior to their weekly meetings, each group member listens to Story Story individually and meet with other group members for about 30 – 60 minutes to discuss issues raised in the drama as well as other relevant issues spurred by the drama / episode they listen. ANNEX, . 23 Data Collection process: In each group, a discussion guide designed in English is used to facilitate discussion guided by the group facil￾itator. The group facilitator is then responsible for collating / summarizing feedback from the group mem￾bers into a facilitator’s feedback form which is then passed to the BBC WST Research team for analysis. A monthly report is then generated from the feedback forms collated in the month. ANNEX, . 24 ANNEX H. MESSAGE – Listener Groups Facilitator Guide Introduction: The objective of this discussion is to get group reaction on the episode you listen to: this will help the Trust to develop further strategies to make the drama betterSection 1: (5 min) Participants Introduction/profile: o Welcome participants o Introduce self/observer(s) if any o Explain purpose of group session Facilitator: It is important to let them know, that: o Everybody’s reaction is quite important. o They should not all speak at the same time. After all of the above have being well explained, the facilitator now moves on to self-introduction by partici￾pants to enable a comfortable atmosphere. Facilitator says: Can you please tell us your name, occupation and your hobbies one after the other. After the first meeting, facilitator asks this section if he/she notices any new member in the group: Can you please tell us: o Your name? o What you do? o Where you stay? Section 2: Exercises (5 min) Facilitator: Instruct them to do any exercise that is customary by group members then ask: o Who can tell the group what we discussed last week? Section 3: Feedback from Episode (25 min) Facilitator, please ask the following questions and use this to guide the discussion; Facilitator say: Thank you every one. Now I would like us to discuss the last episode of Story Story that you listened to. Ensure everybody responds 1) Can you briefly describe the last episode you listened to? Facilitator listened and record all men￾tioned 2) On which station did you listen? 3) What day and time did you listen 4) What topic(s)/ issues were treated in this week’s episode? Probe fully (asked any other) 5) What did you like about the episode? Record all mentioned and asked why? 6) Is there anything you don’t like in the episode? If Yes, what? And why? 7) What do you think the main message of this week’s episode is? probe fully ANNEX, . 25 8) Was there anything you heard in the episode that you do not believe? If yes what can be done to make it more realistic? 9) Did you find any part of this episode confusing? If yes which aspect of this episode? 10) Did you learn anything from this episode that you just listened to? If yes what did you learn? 11) What is the major lesson that you would share with others? 12) Which of the characters do you like most in this week’s episode? Why? 13) Which of the character do you least like in this episode? Why? 14) Do you have any suggestions to help make the drama better than it is now? Thank you. ANNEX, . 26 ANNEX I. LIST OF CSOs INVOLVED IN MESSAGE S/N Name Organisation’s Name Position 1. Elder Ogazi Emeka Transparency & Economic Devpt. Initiatives/ PWP Executive Director 2. Afuye Yakubu Wole SOyinka Centre for Investigative journalism IT Support Officer 3. Yusuf Yakubu Arrigasiyyu Muslim League for Account￾ability Executive Director 4. Omotunde Ellen Thompson Centre for Advocacy for Per￾sons with Disabilities Executive Director 5. Bola Abimbola Youngstars foundation Admin Officer & Director of Programme 6. Pastor Praise Okon AKBC, Uyo Director General 7. Plangret Best Youth Initiative for Adovacy, Growth & Advancement Media and Communications Officer 8. Hon. Dandeson Nwankpa Hart Joint National Association of Persons with Disabilities National Secretary 9. Martins Oloja The Guardian Editor 10. Antonio Cruise Hot FM Abuja COO 11. Habiba A. Ahmed Rahana Women Dev. Pro￾gramme Programme Director 12. Balbasahi Ibrahim USAID-NEIFOMWAN Pro￾ject Focal person 13. Elizabeth Carr BATV Reporter 14. Orji Ogbonna Orji NEITI Director 15. Aishatu Ibrahim Kulishi FOMWAN/NEI Focal officer 16. Hussaini Aliyu Ministry of Women Affairs Bauchi OVC Desk Officer 17. Nanvyap, Yacit Noel National Association for Peaceful Elections in Nigeria Project Co-ordinator 18. Sulaiman Garba Fawoydi Org ICT Officer 19. Abuma Ibrahim NEI SOkotot CSACEFA Desk Ofiicer 20. Helen John NEI/CSACEFA Programme Officer ANNEX, . 27 21. Ibrahim A.Shuni Coalition of NGOs Sokoto State Chairman 22. Ekanem Inyang Applicants Welfare and Nat. Dev. Centre Executive Chairman 23. Isaac Williams Ibeno Youth council Secretary 24. Itipbe Ikono Akwa Ibom Youth Support Network PRO 25. Ann Uduka Community Partners for De￾velopment Program Manager 26. Ekanem Owoidoho Uforo Small Business Owners Asso. Project Officer 27. Prof. G.O Essien JP YDC of Nkim Uyo Director 28. Rosalyn Efretnei FAIWA National President 29. Idem Udoevong Sustainable Development Network Initiative Exec. Director 30. Inyene Archibong (STEP) Support Training Ent Program Program Officer 31. Akuku A.J. Udom INT.L. Society for Safety Health & Environment President / Director Prog 32. Sampson T. Agba EXDF, Iko Town Exec. Director 33. John Cyril MPINO NNITO IBENO Vice President 34. Omolekun, E Abantu for Development Project officer 35. Ihejirika Samie SEMA Executive director 36. Emmanuel S. Tagwai JDPKaritas Programme Officer 37. Hannatu Wyorkson Daar Communication Head of Station 38. Priscillia Akau Human rights Monitor Deputy Programmes Manager 39. Sember Diwah Fanana Gender and Human value proactive (HUVAP) Programme officer 40. Rebecca Philip Connecting Gender and dev. Admin officer 41. Adejor Abel League of Democratic women Senior Programme Officer 42. Mrs. Sarah Sunday Centre for Development Ad￾vocacy Admin officer ANNEX, . 28 43. Fidelis Ojonugwa Otene Community Youth Forum Asst. Coordinator 44. Samson Auta Ime Kaduna Youth/North west coordina￾tor 45. O.O Olowokure O.O Olowokure Chairman 46. CSP A.A Musa NPF DPO ANNEX, . 29 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov