December 2021 This document was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. USAID/SOUTH SUDAN FINAL PERFORMANCE EVALUATION OF i-STREAM SOUTH SUDAN: STRENGTHENING FREE AND INDEPENDENT MEDIA IN SOUTH SUDAN December 2021 USAID/SOUTH SUDAN FINAL PERFORMANCE EVALUATION OF i-STREAM SOUTH SUDAN: STRENGTHENING FREE AND INDEPENDENT MEDIA IN SOUTH SUDAN December 2021 This publication was produced at the request of the United States Agency for International Development and developed under Contract/Task Order Number: RFTOP USAID No 72066821R00015. It was prepared independently by: Vivien Marles, Team Leader Anthony Ochan Materno, South Sudan Media Specialist Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 301-652-4334 www.MEandAHQ.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT This evaluation assessed Strengthening Free and Independent Media in South Sudan’s (i-STREAM) progress strengthening autonomous, journalistic professionalism and facilitating the development of a financially self￾sustained, independent media in South Sudan; and recommends steps to improve the sustainability, size, and resources of United States Agency for International Development (USAID)-supported radio operations in South Sudan. The evaluation was conducted using a mixed-methods approach, which included document reviews, key informant interviews, focus group discussions, and site visits. The evaluation team (ET) found the program is widely considered to have been overly ambitious in its design, scope, and timeframe given South Sudan’s context. However, it had a strong performance and was able to be flexible and adaptive. Eye Radio has become the most trusted and respected radio station in the country, and the supported community radio stations provided a lifeline to communities in the most traumatic of times. The ET recommends the program move from the start-up phase (2013 to 2021) into a new phase (Phase 2 – Consolidation) with a focus on improving the cost effectiveness and income generating capabilities of the supported radio networks, while retaining their separate broadcasting, content and audience mandates, by expanding the sharing of central resources and developing single centers of excellence (which in turn could be potential income earners) such as: • Digital content development • Staff training • Administration functions (human resources, finance, procurement) • Engineering and maintenance • News gathering and content development • Sales and marketing CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... i 1.0 EVALUATION PURPOSE AND QUESTIONS........................................................................................... 1 1.1 Evaluation Purpose .............................................................................................................................. 1 1.2 Evaluation Audience............................................................................................................................ 1 1.3 Evaluation Question............................................................................................................................ 1 2.0 BACKGROUND ................................................................................................................................................ 1 3.0 EVALUATION METHODS AND LIMITATIONS...................................................................................... 2 3.1 Evaluation Methodology..................................................................................................................... 2 3.2 Data Collection.................................................................................................................................... 3 3.3.1 Data Collection ................................................................................................................. 3 3.3.2 Data Analysis and Reporting .......................................................................................... 4 3.3.3 Evaluation Limitations and Mitigations......................................................................... 4 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ............................................................... 5 4.1 Findings................................................................................................................................................... 5 4.1.1 The Changing Media Paradigm....................................................................................... 5 4.1.2 Design and Implementation of the i-STREAM Program..........................................5 4.1.3 Progress Toward the Development of an Institutionally Well-Managed and Financially Self-Sustained Independent Media............................................................. 8 4.1.4 Enhance Media Education and Journalism Training – Sub-Grant 3 .....................15 4.2 Conclusions.........................................................................................................................................19 4.2.1 Eye Radio ..........................................................................................................................19 4.2.2 The Radio Community ..................................................................................................19 4.2.3 Community Engagement Network.............................................................................20 4.3 Recommendations.............................................................................................................................21 4.3.1 Eye Radio, The Radio Community, and Community Engagement Network Strategic Recommendations.........................................................................................21 4.3.2 Recommendations for Capacity Building...................................................................22 4.3.3 Recommendations for Advocacy for Capacity Building.........................................23 4.3.4 Way Forward – Phase 2: Consolidation....................................................................23 ANNEXES .........................................................................................................................................................................25 Annex 1: Statement of Work ........................................................................................................................26 Annex 2: Work Plan ........................................................................................................................................29 Annex 3: High-level KII Stakeholders Interviewed...................................................................................57 Annex 4: Summary of Community Leader Informants............................................................................58 Annex 5: FGD Participant Composition.....................................................................................................59 Annex 6: Station Staff Interviews..................................................................................................................61 Annex 7: Data Collection Tools...................................................................................................................62 ACRONYMS AIRS Abyei Information and Radio Service AMDISS Association of Media Development in South Sudan BBC British Broadcasting Corporation BBTT Boda Boda Talk Talk C4D Communication for Development CEN Community Engagement Network CPA Comprehensive Peace Agreement CRN Catholic Radio Network CRS Community Radio Station EDC Educational Development Center EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion FY Fiscal Year GBV Gender-Based Violence HIS Humanitarian Information Service HR Human Resources IDP Internally Displaced Person i-STREAM Strengthening Free and Independent Media in South Sudan IOM International Office of Migration IP Implementing Partner IT Information Technology KII Key Informant Interview km Kilometers M&E Monitoring and Evaluation MDI Media Development Institute ME&A ME&A, Inc. NGO Non-Governmental Organization NSS National Security Service PMP Performance Management Plan POC Protection of Civilians RRF Rapid Response Fund SATT Seat at the Table SMT Senior Management Team SPLM Sudan People’s Liberation Movement SRS Sudan Radio Services TRC The Radio Community UJOSS Union of Journalists of South Sudan UN United Nations UNMISS United Nations Mission in South Sudan UOJ University of Juba USAID United States Agency for International Development USG United States Government VOA Voice of America i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of this evaluation was to assess Strengthening Free and Independent Media in South Sudan’s (i-STREAM) progress strengthening autonomous, journalistic professionalism and facilitating the development of an institutionally well-managed, financially self-sustained, independent media in South Sudan, and make recommendations to inform the future and shape of the United States Agency for International Development South Sudan Mission’s (USAID/South Sudan) continuing support for the sector, including the steps to improve the sustainability and primary activities of USAID-supported radio operations and capacity-building activities in the country. BACKGROUND South Sudan’s independence in 2011 was initially greeted with optimism for peace and greater prosperity. However, almost immediately post-independence, a new wave of civil war, which started in 2013, plunged the country into a political and economic crisis and fragmentation. An early casualty of the conflict was the media sector, which saw the closure of many press titles and radio stations and persecution and expulsion of journalists. The environment for media outlets and practitioners became very hostile, toxic, and personally dangerous. There are many documented incidents of threats and atrocities against practicing journalists including two female journalists who were reportedly beaten by an army general on October 31, 2019 while covering a military event at the military headquarters in Juba. The general was not happy with their presence in the event in which the president was officially opening the defense headquarters building famously referred to as “the Eagle House” and another in July when government forces stormed the Terrain Hotel in Juba raping expatriates including foreign journalists and killing their male south Sudanese colleague. The evaluation team (ET) also heard harrowing stories of atrocities experienced by members of staff at Internews during the early years of the project implementation. It was against this background and in this unpromising environment for the development of a healthy media sector that the i-STREAM program was launched in early 2013 (just as the first civil war broke out) with the key strategic aims of “strengthening professional capacities, industry association building and solidarity across South Sudan’s developing media sector.” The program encompassed a number of interlocking components, including the production and distribution of quality content, the financial sustainability of locally owned and locally led media outlets, a fair and predictable legal environment, and the existence of strong support institutions. i-STREAM received $60 million from USAID over an initial five-year period and was implemented by Internews, whose approach involved promoting partnerships and collaboration. EVALUATION DESIGN AND METHODOLOGY The main evaluation question (EQ) was: “To what extent has the objective regarding developing an institutionally well-managed and financially self-sustained, independent radio sector in South Sudan been achieved under i-STREAM?” To answer the key EQ, the ET adopted a mixed-methods approach using a combination of secondary and primary research techniques, including: 1) project documents and literature review; 2) in-depth interviews with stakeholders and key informants; 3) a modest telephone survey of training beneficiaries; 4) partner station assessments based on interviews with management and staff, and 5) focus group discussions (FGDs) with radio listeners. ii MAIN FINDINGS Eye Radio – Sub-Grant 1 For 15 out of the 18 key stakeholders interviewed, i-STREAM was synonymous with Eye Radio, regarded as the flagship of independent media and the radio sector in South Sudan. It is widely spoken of as the pioneer of independent media in the country, with industry respect for its high-quality journalism and production values, its near country-wide coverage and audience reach, and its commitment to covering controversial topics. However, no one interviewed believed Eye Radio is a financially viable media outlet without continued USAID/donor support. Since its transition to 100 percent local ownership and management in 2014, Eye Radio’s achievements have been considerable, and it is now the most listened to and respected media brand in the country with strong public-service broadcasting values, state-of-the-art infrastructure, and high-quality technical abilities. Its management is well regarded and, according to independent assessments, has progressed from being classified as a young organization in 2013 to that of a mature organization today. However, the biggest challenge facing Eye Radio is sustaining, or even making a significant contribution to, covering its high operating costs in the unpromising market economy of South Sudan, where opportunities and the infrastructure for commercial income generation are extremely limited and Eye Radio’s commercial acumen is weak. The Radio Community (TRC)—Sub-Grant 2 TRC’s network of community-based stations are well-established and well-respected, integral to the communities they serve, and vital to the fabric of community life. Its headquarters staff in Juba support the network across all technical, management, and operational functions and require slimming down to reduce operational and staff overheads. The individual community stations themselves are modest affairs, housed in small facilities with a single broadcast studio. Most are powered by solar panels and support around four staff. There is little opportunity for cost saving at the community level or for meaningful revenue generation. TRC remains overly dependent on donor funding and suffers from the weakness of the private sector in the country and the almost total absence of commercial revenues. Further, TRC is not well-placed to generate its own income as private sector financing for community radio in South Sudan is almost non￾existent and the communities themselves do not have the resources to offer any sustainable funding. Against this, however, the ET found that a small number of commercial media outlets, such as Classic FM and Capital FM, are succeeding financially by exploiting the limited advertising market (Telco’s) and existing business opportunities such as thematic-based programs supported by non-governmental organizations (NGOs). These stations are lean and focus mainly on appealing to youth audiences, in and around Juba, through music, chat-shows, and phone-ins and consciously avoid covering controversial topics. Community Engagement Network (CEN)—Sub-Grant 2 The management team at CEN’s Juba headquarters is relatively small and young, with strong community engagement skills; and it is the least donor-dependent of the USAID-supported stations. CEN is in the process of developing human resources (HR) and infrastructure capacity in the form of equipment and facilities and financial management systems. It is actively seeking and establishing partnerships as part of its financially self-sustaining strategy and, with low operational costs and a strong network of sponsorship and funding partners, has the potential of becoming self-sustaining. Media Education and Journalism Training—Sub-Grant 3 At an industry level, i-STREAM was instrumental in raising entry-level journalism standards, but not in strengthening media management and business skills (which are considered to be poor) or advanced-level journalism capacity such as investigative journalism, election coverage, and digital media. iii The embedded capacity-building model within the partner stations was highly effective in improving capacity, systems, and in-house journalism and technical standards. It was a resource-intensive method but with proven success. Support given to the Association of Media Development in South Sudan (AMDISS), Media Development Institute (MDI), Union of Journalists in South Sudan (UJOSS), the National Editors Forum, Association for Media Women in South Sudan, and the University of Juba (UOJ) resulted in an overall uplift in journalism capacity, but the lack of prospects for journalists is a disincentive to enter the profession. With the right support and incentives, AMDISS and MDI are thought to be capable of financial independence. Advocacy Strengthening With the support of i-STREAM, both AMDISS and MDI have become respected organizations and succeeded in establishing themselves as respected media watchdogs. However, i-STREAM was criticized for its low visibility in this area and limited coordination efforts which, as the most prominent actor in the field, was regretted. Professional associations and media organizations remain weak and subject to threats and violations. CONCLUSIONS Eye Radio, The Radio Community, and Community Engagement Network 1. All three suffer from over-dependence on donor funding (less so for CEN) and a lack of genuine belief USAID funding will come to an end. They have been slow to develop and implement commercial strategies. 2. Their operating costs have been extremely high, and operational efficiencies need to be brought in as a matter of urgency (beyond those of cutting staff and salaries). 3. Eye Radio and TRC are run as traditional public-service radio media outlets. They need to become more nimble, more creative, and more prepared for the digital future. They need to embrace change and diversity and review their board and senior managements’ roles and responsibilities. 4. Monitoring and evaluation (M&E) and much deeper audience insights are needed to support marketing and sales effort and refresh content and schedules. 5. Economic necessities require more innovation across the board, from repurposing content to utilizing stringers, citizen journalists, and specialist reporters more effectively to news and information gathering and dissemination (Catholic Radio Network [CRN] offers a point of comparison). 6. CEN is still in the start-up phase and can avoid mistakes, such as a top-heavy headquarters carrying unsustainable operating and salary costs. It is more agile than Eye Radio and TRC and fosters innovative approaches to meet its remit. It is forging strong partnerships to become more self￾supporting. Journalism Training and Advocacy 1. Both of these activities will require continued support, especially in the training of media managers and advanced journalism skills. 2. AMDISS and MDI may not require financial support long term if they can get assistance in upgrading their facilities and implementing a realistic commercial strategy. However, both will require technical support to develop new curricula and train a cadre of high-quality trainers across all aspects of media—not just in journalism. iv 3. There is a strong need for better coordination and stronger engagement with media institutions and other organizations working in the field and a greater emphasis on monitoring and dissemination. RECOMMENDATIONS Eye Radio, TRC, and CEN Recommendation 1: Improve the cost-effectiveness of the three networks by expanding the sharing of central resources (digital content, business development, engineering and maintenance, administration functions) while retaining their individual broadcasting mandates and target audiences. These could be established as specialist centers of excellence and become income generating activities in their own right— sub-contracting engineering expertise and broadcast media training, for example. Recommendation 2: Develop an initial five-year funding mechanism tied to specific activities and targets based on actionable business plans (innovation/market leadership/viability) with funding reduced each year as income generation ramps up, that includes better communicate expectations and monitor progress. Recommendation 3: Consider transitioning from NGO status to commercial status. Allow the stations to make profits to plow back into their operating overheads and determine their own cost/income ratio strategies. Recommendation 4: Assess readiness for direct funding. Recommendation 5: Implement culture and leadership change by employing/training board members and managers who can drive institutional change, “right-size,” consolidate, generate income, and transition to digital. Recommendation 6: Change operating modalities to bring staffing levels and salaries into line with comparable stations. Recommendation 7: Centralize and professionalize a “supported networks” business development, sales, and marketing function to more effectively exploit current and future income generation opportunities. Recommendation 8: Embrace the content and monetization opportunities of digital and social media. Move away from the traditional radio model. Recommendation 9: Develop new methods of training and upskilling board and Senior Management Team (SMT) in readiness for fully autonomous operations (accountability, self-sufficiency). Move away from the expensive embedded expert model. Recommendation 10: Update methods of in-house training and mentorship (virtual). Recommendation 11: Abyei Information and Radio Service (AIRS) requires minimal, light-touch support to keep it on track and support skills building activities. Consider a one-off direct investment for capital developments to support the drive for financial independence. Capacity and Advocacy Building Recommendation 12: Coordinate and collaborate more with other donors involved in media training to avoid duplication and better focus resources to needed areas in media development. Recommendation 13: Provide technical support for training in thematic areas—including covering elections in conflict zones. Recommendation 14: Improve internship opportunities for entry-level journalists and content producers through a wider network of international and local partners to enable graduates to put their skills to work. v Recommendation 15: Modernize the curricula and approaches to training to meet the changing trend and needs in the media industry with an emphasis on practical skills in business, digital and media law and advocacy, and a learning-through-doing approach in addition to standard classroom training. Recommendation 16: Move away from the embedded expert model of training to reduce costs and dependency on international experts. Develop a centralized model of multi-media training which may include remote access to international experts. The generous salaries and benefits enjoyed by international experts creates resentment and heightens local staff salary expectations. Recommendation 17: Continue technical assistance to MDI to develop more contemporary and relevant curricula incorporating training in management, commercial training, investigative reporting, elections coverage, etc. Recommendation 18: Continue technical support for MDI to develop more advanced journalism diplomas/courses/media management capacity with more effort directed toward training women and those outside Juba. Recommendation 19: Develop a revenue generation strategy and implementation plan for AMDISS and MDI such that by 2024–2025 they are self-reliant. Recommendation 20: Consider a one-off investment to AMDISS and MDI for capital investments, specifically for revenue-generating activities/facilities. Recommendation 21: Step up activities promoting freedom of speech and protecting independent journalists and media houses through supporting educational programs, building stronger professional networks and organizations, developing a cadre of expert media lawyers, and disseminating information more widely. Recommendation 22: Request funding for the foreseeable future and collaborate closely with other donors/funders. Recommendation 23: Reduce the over-reliance of AMDISS and MDI on donor funding by implementing aggressive financial targets and timelines. Recommendation 24: Improve visibility to attract more partners and wider industry support. Recommendation 25: Embed M&E and develop an effective dissemination strategy. 1 1.0 EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE The United States Agency for International Development’s South Sudan Mission (USAID/South Sudan) contracted with ME&A, Inc. (ME&A) to conduct the final performance evaluation of the Strengthening Free and Independent Media in South Sudan (i-STREAM) program, implemented by Internews in close collaboration with three consortium members—British Broadcasting Corporation (BBC) Media Action, Albany Associates, and Free Press Unlimited—and multiple local partners. The initial five-year program (2013–2018) received $60 million and was designed to establish and strengthen free and independent media in the newly independent, nascent South Sudan. In 2018, the program received a three-year no￾cost extension, and in 2021 the partners were informed the program would end in June 2022, with no more funding available thereafter. The total investment between 2013 and 2022 will be $84 million. The purpose of this evaluation was to assess i-STREAM’s progress toward strengthening autonomous, journalistic professionalism and facilitating the development of an institutionally well-managed, financially self-sustained, independent media in South Sudan, and make recommendations to inform the future and shape of USAID/South Sudan’s continuing support for the sector, including the steps to improve the sustainability and primary activities of USAID-supported radio operations and capacity-building activities in the country. 1.2 EVALUATION AUDIENCE The main audience for this evaluation is USAID/South Sudan, Internews, and its Implementing Partners (IPs). The evaluation was designed to provide an informed and evidence-based high-level overview of the program’s progress and achievements with pragmatic recommendations for the way forward. 1.3 EVALUATION QUESTION The main evaluation question (EQ) was: “To what extent has the objective regarding developing an institutionally well-managed and financially self-sustained, independent radio sector in South Sudan been achieved under i-STREAM?” 2.0 BACKGROUND The 2005 Comprehensive Peace Agreement (CPA) between Southern Sudan’s Sudan People’s Liberation Movement (SPLM) and the Sudan government led to a proliferation of mass media in South Sudan. However, almost immediately post-independence, a new wave of civil war in 2013 plunged the country into a political and economic crisis and fragmentation. This, as well as the continued fighting among various factions, hindered access to many parts of South Sudan, causing a sharp drop in mass media circulation and severely constraining public access to news and information. At the same time, the South Sudanese authorities, mainly the National Security Service (NSS), started adopting the same violent tactics employed by previous Khartoum regimes—including arrests, torture, and punitive laws—to censor journalists and suppress freedom of expression. Today, the NSS has extensive, unchecked powers to conduct surveillance; monitor communications; intimidate, detain, and murder journalists; infiltrate non-governmental organizations (NGOs); and shut down and impose punitive fines on private media houses. 2 Given this toxic and hostile environment, many South Sudanese journalists have fled the country or been forced to quit journalism. This has led to a dearth of opportunities for the transfer of knowledge to younger journalists, the majority of whom do not have proper training or qualifications. Equally, foreign reporters have difficulties reporting in South Sudan, with many denied entry or forced out of the country by authorities, as was the case with Associated Press Reporter Justin Lynch, who was deported in December 2016 for being very critical of the government in his reports. In addition, journalists in South Sudan complain the NSS and Media Authority have made it extremely difficult to obtain permits to travel outside the capital of Juba. There are currently only three foreign reporters in the country. South Sudan is recognized as one of the harshest environments for media and journalists. The country ranks 139 out of 180 in the 2021 World Press Freedom Index, its 2020 Global Freedom score is two out of 100, and it ranks 160 out of 163 in the 2021 Global Peace Index, with only Syria, Yemen, and Afghanistan scoring lower. It was against this background and in this unpromising environment for the development of a healthy media sector that i-STREAM was launched in early 2014 (just as the first civil war broke out) with the key strategic aims of “strengthening professional capacities, industry association building and solidarity across South Sudan’s developing media sector.” The program encompassed a number of interlocking components, including the production and distribution of quality content, the financial sustainability of locally-owned and locally-led media outlets, a fair and predictable legal environment, and the existence of strong support institutions. 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation was conducted by a team consisting of an international media expert and a local expert. The evaluation team (ET) was supported by a local firm, Consilient Ltd., which assisted with interviews outside Juba as well as telephone surveys. To conduct the evaluation, the ET adopted a mixed-methods approach, which consisted of a combination of secondary and primary research techniques, including: 1) a project document and literature review; 2) in-depth interviews with stakeholders and key informants; 3) a modest telephone survey of training beneficiaries; 4) partner station assessments based on interviews with management and staff; and 5) focus group discussions (FGDs) with radio listeners. To answer the main EQ, “To what extent has the objective regarding developing an institutionally well￾managed and financially self-sustained, independent radio sector in South Sudan been achieved under i￾STREAM,” the ET designed the evaluation at three levels: 1. Media Landscape and Ecosystem: What is the state of media in South Sudan (2011–2021) and what has been the role and impact of i-STREAM in strengthening and promoting an enabling environment for free and independent media? 2. Media Practitioners: What has been the impact of i-STREAM on building and strengthening the capacity of media professionals—journalists, media managers, and media practitioners? 3. i-STREAM-supported Radio Stations: A qualitative assessment of the USAID-supported radio stations focusing on their operations and management, outputs, audience responsiveness and financial viability. 3 3.2 DATA COLLECTION 3.3.1 Data Collection Level 1—Media Landscape and Ecosystem: What is the state of broadcast media in South Sudan (2011–2021)—with an emphasis on the radio sector? Designed to assess the progress i-STREAM made in strengthening an enabling environment for free and independent broadcast media in South Sudan, this component of the study comprised in-depth interviews with knowledgeable actors, including the Association of Media Development in South Sudan (AMDISS), Media Development Institute (MDI), Union of Journalists of South Sudan (UJOSS), National Media Authority, University of Juba (UOJ), human-rights organizations, practicing journalists, and other media actors. Level 2—Media Practitioners: What has been the impact of i-STREAM on the capacity of media professionals? To evaluate i-STREAM’s effectiveness in building and strengthening the capacity and professional skills of media practitioners, the ET originally proposed to conduct an online survey among those who attended training courses/sessions and received mentorship opportunities as part of i-STREAM’s professional capacity-building activities. However, the ET received a list of only 77 names with contact telephone numbers. Local partner Consilient set up a small call center in Juba for telephonic data collection. This very small survey was augmented by interviews with managers and staff in seven supported radio stations and their headquarters as well as key stakeholder interviews. Level 3— i-STREAM-supported Radio Stations: What is the performance of the USAID-supported radio stations in terms of management capacity, resource and management practices, and financial viability? Data and insights to inform these key questions were drawn from a total of 34 key stakeholder interviews; 12 station manager interviews; 26 station staff interviews; 28 FGDs, 30 community leader interviews, and 18 training beneficiary telephone interviews. The evaluation included: • A series of 18 high-level key informant interviews (KIIs) based on a semi-structured interview protocol were conducted in-person in Juba (one interview, with a government organization, was attended by three individuals—the ET decided to count this as one interview for the purposes of reporting). • A series of 16 KIIs based on a semi-structured interview protocol with current and former USAID and Internews staff were conducted in person and virtually. • On-site interviews with senior management staff at Eye Radio (six), The Radio Community (TRC) (three), and Community Engagement Network (CEN) (three). • Twenty-six (26) interviews with station staff. • Twenty-eight (28) FGDs with 263 participants in Juba and six TRC/CEN locations (four groups in each location) with groups of male, female, younger, and older listeners. Groups comprised pre￾recruited individuals speaking a mix of vernacular languages. • A series of five in-person interviews with community leaders in each of the six TRC/CEN station locations (30). • A telephone survey of professional training beneficiaries based on an initial sample of 77, yielding 18 complete responses. Interview protocols and sample details are in Annex 7. 4 Among the project documentation provided for review were: • Annual Reports (Fiscal Year [FY] 2013–2104 to FY 2019–2020); • Quarterly Reports (2014–2019); • Performance Management Plan (PMP); • Cooperative Agreement (2013); • Annual Work Plans; • 2016 Information Access in Abyei: Final Narrative Report and Impact Assessment; • Research and impact assessments (Internews/Forcier National Audience Surveys); • Listener feedback; and • Success stories. 3.3.2 Data Analysis and Reporting Upon completion of data collection, all KIIs and FGDs were transcribed and summarized to facilitate a thematic narrative analysis. Numerical data were analyzed using Excel. The ET used triangulation when analyzing and interpreting the findings to identify any inconsistencies and ensure reliability, reducing the “response bias” in which respondents tend to tell evaluators what they think they want to hear. 3.3.3 Evaluation Limitations and Mitigations There were several limitations inherent to the design of this evaluation. 1. Selection bias: As some informants declined or were not available for interviews, there is likely to some self-selection bias in the achieved sample. For example, no government ministers were approached for interview and the United Nations (UN) agencies approached for an interview failed to respond. These interview attempts are noted in the master list of KIIs in Annex 3. 2. Recall bias: Since some interview questions dealt with issues that took place in the past, recall bias cannot be excluded. 3. Social acceptability: There is a known tendency among respondents to emphasize socially desirable answers and alter their responses to approximate what they perceive to be acceptable. The ET attempted to minimize these limitations by: • Interviewing as representative a group of stakeholders and partners as possible across all key groups and liaising with USAID and Internews to ensure as wide a range of views as possible were canvassed. • Guaranteeing all key informants’ anonymity to facilitate more open and honest discussion and more forthright responses. • Triangulating evidence from different qualitative and quantitative data sources, which serves to increase the credibility of findings. • Collaborating closely with USAID/South Sudan to facilitate introductions and mobilization of partners, beneficiaries, and stakeholders. Mobilization helped to secure participation. • Conducting KIIs and FGDs in venues where participants were safe and comfortable. • Managing group dynamics by organizing homogenous FGDs in terms of sex and age and using experienced language-familiar moderators and facilitators. 5 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 4.1 FINDINGS 4.1.1 The Changing Media Paradigm Audience research surveys show radio remains the dominant medium in South Sudan and is likely to remain so for the foreseeable future. Internews/Forcier National Audience Surveys conducted in 2013 and 2015 report around two-thirds of citizens have access to radio and it remains the primary source for news and information. The data from these surveys were born out in the listener FGDs, which found that across the country, radio was not only the most available and consumed medium, but that it was also the most trusted and dependable source. “People’s lives are so dependent on radio especially in rural areas as it is the only source of information, and it became a lifeline for communities plagued by conflicts as they depend on it to make life-saving decisions amid conflicts and natural disasters such as floods.” (community leader) However, qualitative research findings and anecdotal evidence from young people in Juba suggest the consumption of traditional media in the form of newspapers and radio may be gradually replaced by social media and messaging apps available on their mobile phones. A trend which is evident across the East Africa region and while access to smartphones is currently stronger in urban areas, the ET believes that this trend is likely to accelerate and extend across the country as mobile phones become cheaper to buy and internet access increases. Irrespective of the high levels of illiteracy across the country (especially among women), Facebook and WhatsApp are increasing in use and popularity. “As well as radio, social media is an important source of information. We mainly used Facebook and WhatsApp for this.” (young woman, Juba) “I never read a newspaper, sometimes I listen to VOA [Voice of America], I like to use Facebook, Twitter, and WhatsApp to find out what is going on.” (young man, Juba) 4.1.2 Design and Implementation of the i-STREAM Program According to the Cooperative Agreement 2013 (Appendix B: Program Description): “The birth of the world’s newest state on July 9, 2011, was met with overriding optimism. After 22 years of civil war, independence represented a chance for South Sudan to provide its citizens with the security, freedom, and voice they had previously been denied. Two years on, the difficulties of building a new state now dominate its narrative. With immense challenges in infrastructure, resources, and capacity, and unresolved issues with Sudan, observers are increasingly concerned by trends restricting free expression and democratic space.” It was against this background that i-STREAM’s objectives were formulated, methodology designed, and implementation developed. The ET asked current and former USAID and Internews staff to reflect on the design and program implementation and, among those who felt able to comment (eight former staff members), all noted the program’s overall design had been extremely ambitious in design, scope, and timeframe (five years) for a country with no effective infrastructure, riven with violence, a very basic journalistic capacity, and no functioning economy. “The scale was huge, it was all-encompassing, setting up an entire independent media sector from scratch in an extremely under-developed and fragile context.” (USAID staff) “A media project of this scale and scope has very rarely, if ever, been done before.” (Internews staff) 6 With an initial investment of $60 million over a period of five years (2013–2018), the design was a classic, holistic media-strengthening program with a number of interconnected activities across multiple separate large-scale grants and smaller sub-grants. However, 65 percent of the grants were directed toward Eye Radio and Community Radio Stations (CRSs). The main body of work involved: • Investing in the infrastructure, reach, and capacity of independent radio stations by transitioning a number of existing USAID-supported radio stations into locally owned, managed, and run entities combined with extensive technical support and intensive in-house capacity strengthening and staff training. This component was the largest and focused on transitioning Sudan Radio Services (SRS), run by the Education Development Center (EDC) in Nairobi, Kenya since 2005, into what was to become Eye Radio and a network of community-based independent radio stations managed by TRC and later in the program, CEN. This performance evaluation focused on the institutional management and financial viability of these three radio networks. • Directing technical and financial resources toward the deepening of professional skills and media ethics through support for training institutions, including UOJ for academic education and MDI for entry-level vocational training and supporting multiple training programs on specific topics, including subject-specific thematic writing, reporting, and editing. • Strengthening the enabling environment through financial and technical support to AMDISS and the strengthening of the network of media associations and organizations. • Supporting, through smaller sub-grants and partnerships, a diversity of smaller media-related projects, including the Juba Film Festival and a number of projects with a focus on youth, women and girls, such as Advance Youth Radio and GoGirls ICT Initiative. The i-STREAM program was regarded by those with experience and knowledge of its design and objectives as a very well-designed program for the nascent and fragile-but-functioning democracy as appeared to exist in South Sudan in 2012. However, as implementation got underway and the security situation in the country deteriorated, the program felt overly ambitious with an “impossible” five-year timeframe. “It was the right decision in 2012–2013 but too ambitious under the circumstances and the timing was unrealistic.” (Internews staff) “It was very prescriptive and designed for an ‘open space,’ within a few weeks of it starting the country was in flames and all investment stopped.” (Internews staff) A key component of the program’s design was its built-in mechanism to allow for immediate response to situations as they arose on the ground through the Rapid Response Fund (RRF). The flexibility and speed with which the RRF could be deployed enabled Internews to immediately pivot into the humanitarian space created as a result of the 2013–2014 conflict, and establish an invaluable information service to displaced communities through its Boda Boda Talk Talk (BBTT) radio. However, a number of design and implementation weaknesses were identified as lessons to be avoided should there be a second phase of the program. 1. Stop-start flow of funding: In 2015, funding increased from $60 million to $75 million for the re￾building of infrastructure destroyed during the 2013–2014 war. This was almost immediately followed by a shortfall of $10 million in 2016, which resulted in cutbacks and made planning in the medium-term challenging. In 2021, the announcement was made that funding would discontinue in mid-2022. This “stop-start” approach led to a sense of disbelief among the main beneficiaries that funding will actually come to an end. There is a largely unspoken, but widely held, belief that USAID has invested so heavily into the program and the independent radio stations created are such an important part of the media landscape that financial support is unlikely to be withdrawn completely and create a “white elephant.” 7 2. Frequent adjustments in the cost-sharing arrangements and targets: The original cost-share target in 2014 was $9 million, reduced to $6 million in 2014–2015 and further reduced to $5 million in 2016. In 2018, the cost-share was increased to $6.5 million when a three-year no-cost extension was implemented. While these adjustments were made in recognition of the difficult market conditions for independent income generation, a serious consequence was the partners did not pay adequate attention to generating their own income streams. They became somewhat complacent and took the view that more funding would always be made available. 3. Short-term thinking: Most of those respondents able to give an opinion felt the original timeframe of five years was inadequate to achieve the desired results and the three-year extension awarded in 2018 represented short-term thinking and an uncertain future for the supported radio stations. One of the consequences of this approach was reported (by the IP) as hampering the program’s ability to plan, prepare, and take the long view, which affected the program’s ability to build enduring relationships and lead collaboration efforts with other organizations and donors in the field. However, despite the limitations of the design and unpromising operating environment, it was the considered view of all of those who contributed to this evaluation, that i-STREAM made substantial progress in strengthening a free and independent media in South Sudan. “Without i-STREAM there would be no independent media in South Sudan today.” (key stakeholder, academic) “i-STREAM is synonymous with Eye Radio – which is the cornerstone of the independent media and radio sector in South Sudan.” (key stakeholder, media practitioner) Internews’s management of the program was highly praised by USAID respondents, who described it as “extremely professional, knowledgeable, and resilient.” One respondent even referred to them as “a wonderful partner.” The collaboration between USAID and Internews was reported as “very strong” in the early years of the program’s implementation, which facilitated the swift and appropriate response to the fast-changing situation on the ground throughout 2014 and 2015. In the USAID and Internews KIIs, the ET asked if the key objectives of i-STREAM had changed. There was a consensus that, while the core objectives had not changed, the emphasis did change and evolve over time. Initially, i-STREAM emphasized transitioning and building the capacities of the stations to provide public access to independent information and balanced coverage of news and current affairs and build the capacities of journalists to work in a more enabling media environment. A key element of the program was to stay on air and attract wider audiences—and indeed this significant achievement was realized. While the goal of financial sustainability and viability was present throughout, it was only seriously addressed in the second phase of the funding (2018–2021) when there was a clearer understanding that the program would not be funded indefinitely, and continued operations would depend on income generation and business development. In 2019, Internews augmented its head office team with a senior sustainability advisor with the goal of working with media outlets to build strategies and capabilities in income generation. It is the informed opinion of the ET that this initiative came too late in the program cycle and the strategies, which focused mainly on seeking funds from other donors through responding and writing proposals, were too narrow. Further, the ET believes the presence, at the design stage, of a baseline and/or mid-term evaluation with outcome and impact indicators in support of the program’s output indicators would have been beneficial, both in tracking progress throughout the life of the program and better informing some of the key issues over the life of the program such that adjustments could have been made where necessary. 8 4.1.3 Progress Toward the Development of an Institutionally Well-Managed and Financially Self-Sustained Independent Media Developing an institutionally well-managed and financially self-sustained independent media was the primary objective of the i-STREAM program and made up of three components of activity, with each being a distinct project in its own right: 1. Eye Radio (independent national radio network); 2. TRC and CEN (network of independent community radio stations managed from Juba headquarters); and 3. Media education and journalism training programs. Eye Radio – Sub-Grant 1 Progress Toward South Sudan’s Leading Media Outlet The ET’s assessed Eye Radio’s achievements, challenges, and the progress it has made toward becoming an institutionally well-managed and financially viable media house through 18 high-level in-depth interviews with key stakeholders and three on-site visits to the station, where in-depth interviews with six senior managers and four staff members were conducted. The team also garnered audience feedback through a series of 28 FGDs. For the majority of stakeholders interviewed for this evaluation (15 out of 18), i-STREAM was synonymous with Eye Radio. Stakeholders widely considered Eye Radio the flagship of independent media and the radio sector in South Sudan, and acknowledged that it achieved this preeminent position as a result of USAID funding and extensive technical and infrastructure support from Internews. Eye Radio was described (by all of the key stakeholders, with the exception of the government interviewees) as “the pioneer of independent media in the country.” According to senior media figures, it raised the bar for high-quality, hard-hitting journalism and has been instrumental in creating the framework for editorially independent media. But no one interviewed—from key stakeholders to current and former program staff and the station’s staff themselves—believed Eye Radio was currently sustainable or able to be independently financially viable in the foreseeable future. “Eye Radio provided a choice when SSBC (government-funded radio station) was the only voice. It was balanced and editorially independent and had its own opinion. It showed the public that there were alternatives…when the funding stops though there will be serious challenges, it can’t raise its own money.” (key stakeholder, media practitioner) Despite listeners across all interview locations and in all FGDs saying they regarded Eye Radio as an essential and trusted source of news and information and the most used station for national news and information, this funding challenge and Eye Radio’s continuing dependence on USAID support is the most critical issue facing the independent media sector in the country. Listeners appreciate that Eye Radio has provided them with alternative perspectives and a diversity of voices. They rely on the station for important and balanced news and coverage of key national events and issues, and while there are other stations providing a similar service (notably Miraya FM), Eye Radio is the most trusted and respected. “We listen to Eye Radio, Bakhita FM, SSBC, and Miraya, but Eye Radio is the most trustworthy and accurate. It can operate more independently without fear of reprisals, and so what we hear from them is less filtered.” (Juba, males) “The best stations for national news are Eye Radio and Miraya FM, but they are mainly for people living in the urban areas as they are broadcast in English and Arabic and only to a lesser extent in local languages, which limits accessibility.” (Bentiu, males) 9 Among radio audiences, as discussed in FGDs, Miraya FM and Eye Radio are seen to be somewhat similar in content and style of coverage, and both are considered to be trustworthy and accurate. However, listeners and key informants alike think Miraya FM will cease to exist when the UN Mission in South Sudan (UNMISS) comes to an end, leaving Eye Radio as the sole independent national radio station in the country. Eye Radio’s key milestones and achievements have been extensively reported in the quarterly and annual reports and, as such, are not documented in detail here. The ET’s review of the annual reports (FYs 2013– 2104 to 2019–2020) and annual work plans and reading of the indicator matrices conclude the operational targets as measured and reported through the output indicator matrix and results framework have been regularly achieved and considerable progress in the strengthening of media institutions has been made. However, there has been a consistent, year-by-year shortfall meeting financial targets. Among the most significant milestones was the ability of Eye Radio to continue broadcasting during the most extreme and traumatic of circumstances and its ability to withstand the intimidation and threats from government and the Media Authority. When the country was still suffering the aftermath of the 2013–2014 war, Eye Radio successfully transitioned to a locally owned, managed, and run NGO in October 2014, and the purpose￾built facility in Juba became fully operational. It began to broadcast 24 hours a day, seven days a week at the end of 2014, and in 2015 it extended its footprint to eight of the ten state capitals—effectively making it the first independent national and regional radio station in the country. According to all of the stakeholders and program staff interviewed, Eye Radio’s most significant and enduring achievement is that it “shaped the independent media sector” and pioneered independent broadcasting. It has grown in stature and disrupted the dominant narrative by providing a genuine, trusted alternative voice to government-supported media and established the practice and principles of public￾service broadcasting. Further, it set the gold-standard for independent broadcasting and, through its hard￾hitting and brave reporting, emboldened other media outlets to report on potentially controversial issues. Through i-STREAM’s financial and technical support, Eye Radio has become a high-profile and highly respected media brand with first-class public-service broadcasting values: high-quality journalism, integrity, and willingness to challenge and be bold. The high regard with which Eye Radio is held among stakeholders and other media professionals was borne out through FGDs with radio listeners. Over the past six years, Eye Radio established itself as the leading radio broadcaster and the public’s most preferred source, not only for news and information, but also for its coverage of politics, health, security and conflict, education, and culture. Progress Toward Being Institutionally Well-Managed and Financially Viable In the early years, Internews supported Eye Radio with infrastructure, technical equipment, the introduction of management policies and systems, and intensive, embedded training and mentorship with the objective of creating a strong, autonomous organization. Through its analysis and interviews, the ET found evidence of well-functioning management systems, policies, and procedures. Management spoke of the policies and procedures implemented and (the few) staff interviewed confirmed that they adhered to policies such as transparent recruitment and promotion practices, regular salary payments, formalized leave arrangements, opportunities for training, and participation in meetings and company decision-making. The staff expressed great loyalty toward the station and each other; they have created strong bonds and are proud of their achievements. Working at Eye Radio was not only financially advantageous for staff, but also gave them very marketable skills and has the prestige of being a prominent and high-status organization with a sound public-service ethos and high-quality journalism standards. The five senior managers who participated in the ET’s self-assessment exercise assessed the station highly (four out of four) on the improvements it has made in “management knowledge and practices,” “access to training opportunities,” and “engagement with local communities.” Their confidence was confirmed by the independent Seat at the Table (SATT) and Organizational Training Capacity (OTC) scores and evaluations, which have been regularly conducted over the past eight years and shown continuous improvements such 10 that Eye Radio has progressed from being characterized as a “young” organization in 2013 to a “mature organization.” Scores across all the organizational categories have improved over time, with the most improvement in the Financial Viability/Resource Mobilization category. A small number of stakeholders (three) commented specifically on improvements they felt were necessary to strengthen and expand the expertise of the board and management in order to drive the station forward into the next stage of development and better equip it for the challenges ahead—those of adapting to the new media/digital and social media world, continuing to push back the boundaries of hard-hitting and investigative journalism, driving down the costs of content creation and distribution, and, above all, the imperative to be commercially astute and creative. The improvements they would like to see involved greater board accountability and embracing of the commercial imperatives, the ability to hold the management to greater accountability, and the ability of the board and senior management to work collaboratively to produce, implement, and monitor a long-term pragmatic, more commercially-focused strategy for the station. Compared with the considerable progress Eye Radio made to become a mature and respected organization with well-recognized organizational and management capabilities, its progress toward being a financially viable and self-sustaining organization has been negligible. It has extremely high and market￾distorting staffing levels and salaries and has been described as “an expensive luxury car.” The annual operating costs of Eye Radio were $1.8 million per year—no other comparable radio station anywhere on the continent has such high operating costs. The ET sourced a number of comparative examples from Uganda, Zambia, and South Africa, where stations broadly comparable with Eye Radio run at around $700,000 per annum. The cost of an in-field agricultural reporter in Uganda is around $300 per program, where an annual investment of $26,000 per year would finance the production and broadcast of three to six hours of interactive farming-related programming per week. In South Africa, where salaries are higher than in many other sub-Saharan countries, a radio show host can expect to receive $8,000 per year, a station finance manager $12,000, a senior radio journalist $9,000, and a news reporter $4,500. Sustaining, or even making a significant contribution to covering the high operating costs of Eye Radio in the unpromising market economy of South Sudan is considered “quite simply impossible.” In recognition of this fact, and with considerable reductions being made by USAID, in the time the ET was on-site at Eye Radio, significant operational cost reductions were being implemented, involving staff down-sizing and salary and benefits reviews. The impact of these cost reductions will need to be monitored and assessed to ensure the quality of output does not suffer as experienced staff leave the station in favor of less expensive and less senior staff. The Radio Community Sub-Grant 2 The ET’s evaluation of TRC was conducted through high-level, in-depth interviews with key stakeholders, on-site visits to the TRC headquarters based at the AFEX River Camp in Juba and, on-site visits to four community radio stations. In each location, local staff were interviewed and community response to the station assessed through a series of community-leader interviews and listener FGDs. Researchers spent around two weeks in the communities served by the four TRC stations and were able to form a comprehensive picture of the role of the station within each community and the relationship between the stations and members of the communities they serve. According to interviews with community leaders and community members, TRC community-based stations are well-established and well-respected, integral to the communities they serve, and vital to the fabric of community life. In all of the communities they serve, they are spoken of as the only vernacular radio stations providing communities with trusted news, information, and entertainment with local voices and in local languages. From its headquarters in Juba, TRC is beginning to generate a more active digital media presence to capture new and different audiences and extend its reach beyond the four communities to the diaspora with the objective of diversifying its current, very limited revenue streams. 11 The Juba-based headquarters supports the network across all technical, management, and operational functions. The individual community stations are modest affairs, housed in small facilities with a single broadcast studio. Most are powered by solar panels and support around four staff—a station manager and a number of reporters/community journalists. Stations’ operating costs are heavily impacted by fuel, technical and equipment maintenance, and security. They have all installed solar panels (to varying degrees of success) to reduce power costs. The main finding from the communities visited as part of this study is these community stations have become an integral part of the communities they serve and offer reassurance and a lifeline in times of crisis. “We know everything is OK when the radio station comes on.” (Listener) Additionally, they offer a voice to members of the community by broadcasting concerns, raising issues, and “getting things done,” such as the provision of water through bore holes and the cessation of a local payment by informing community members of their rights. The TRC stations visited as part of this review were Akol Yam 91 FM (Aweil), Mayardit FM (Turalei in Warrap State), Mingkaman 100 FM (Lake State), and Singaita 88.3 FM (Kapoeta). In all cases, these stations were the only community stations available in their communities, and all reported the stations made considerable contributions to the communities’ development through disseminating local information, as a mechanism for educating the community on important topics (health, gender-related topics, and education), and, perhaps most importantly, fostering good relationships between and within communities through peace and understanding. Progress Toward Being Institutionally Well-Managed and Financially Viable Internews supported TRC headquarters and its network of community stations with infrastructure, technical equipment, and the development of new draft management policies and systems and worked intensively to develop information technology (IT), technical, and engineering policies and procedures and other means of managing the network. i-STREAM supported the development of human resources (HR) and financial policies such as fraud, staff recruitment, financial management, sick leave policies, and annual leave policies in line with South Sudanese labor law. The evidence points to the implementation of good management policies and practices, but not the income to support the current level of operations. Through its analysis and interviews, the ET found evidence of TRC headquarters management working to sustain operations with increasingly limited resources while reducing costs and cutting staff. The ET was impressed with TRC’s ability to run its operations from modest temporary accommodations at AFEX River Camp, but felt it was overly dependent on donor funding and not well-equipped to generate its own income. All of those interviewed, both within TRC and outside stakeholders, expressed grave pessimism for the future of TRC and its stations in the absence of donor funding. The ET found little merit in closing any individual TRC community station. Interviews with staff members at the community stations showed staff under the stress of balancing high operating costs (power, fuel, maintenance, and security) with limited resources, and the difficulty of recruiting and retaining staff in remote areas of the country with the high expectations of the community for 24-hour-a-day news, information, and programming. TRC senior management was also pessimistic about the ability to raise sufficient funds from other sources such as other donors, agencies, NGOs, and community-based revenues to sustain its operations. At every station, staff members identified the lack of opportunities and skills in revenue generation as their greatest challenge, as exemplified in this statement, “Funding is our biggest challenge. We are currently facing large staff reductions which have resulted in a reduction of programming quality and staff workload.” Below are brief summaries of each TRC station visited as part of this evaluation. The stations have more commonalities than differences. All are modestly staffed within low-cost accommodations and have zero opportunity of raising sufficient funds locally or from the community to keep them on air but are hugely valued by the communities they serve and engage and interact closely with community members. The stations bring new and alternative voices to local communities while representing the voices of the 12 communities they serve. There is evidence of knowledge, attitude, and behavior changes as a result of the Communication for Development (C4D) programming broadcast by these stations, including health improvements (mosquito nets), reframing the importance of education (girls going to school and doing well in class), and issues around peace building and conflict reduction. Akol Yam (Aweil) Akol Yam 91.0 FM broadcasts news, information, and entertainment programs from Aweil, Aweil State. Previously known as Nhomlauu FM, the station relocated to Aweil and rebranded in 2016. Akol Yam broadcasts in the Dinka language and its wide-ranging content includes programs on education, health, agriculture, and peace-building. Akol Yam is the only community radio station broadcasting to this isolated and marginalized community; and it plays a vital role in bringing alternative voices and perspectives to its listeners, offering a voice and empowering the community and disseminating information in local languages. A station staff member described Akol Yam as: “an important mechanism for educating the community on important topics (primarily health-related topics) and…if it closes down just for one day the community will start calling us…because they totally depend on the station as their main source of information.” The station’s staff consider the station’s establishment and its ongoing operations, including installation of the solar grid to maintain a stable source of power, as their greatest achievement. However, they felt the station had not yet managed to develop a sustainable business model and staff lacked training and experience in revenue-generation activities. Ongoing challenges include the lack of resources to, for example, maintain equipment and recruit journalists in such a remote location. For both community leaders and members, Akol Yam was their preferred and most trusted source of news and information, and above all, they feared the station might close, thereby depriving them of their information lifeline. The community faces serious problems including flooding, poverty, gender-based violence (GBV), malaria, domestic violence, and polygamy, and they increasingly rely on Akol Yam’s programs to give them advice and education around these issues. In answer to the question about how critical Akol Yam was to the survival of the community, the scores out of ten (extremely critical) among all community leaders and those who attended the FGDs were ten out of ten. Mayardit FM (Turalei) Mayardit FM is based in Turalei in Warrap State. The station broadcasts 16 hours a day on 90.7 FM, and reaches around 500,000 people in Warrap State, Northern and Southern parts of Unity, Abyei, and some parts of Northern Bahr el Ghazal State. Mayardit broadcasts in Dinka, Juba Arabic, and English between 6:00 a.m. and 10:30 p.m. daily. The station’s local staff, similar to all the other stations, view their work as providing one of the few reliable sources of information for the communities they serve. They considered programs on health-related issues to be of greatest value to the community. The majority of community leaders and members the ET spoke with confirmed radio was their major source of news and information. In this community, Miraya FM was a trusted source for Arabic and English broadcasts while Mayardit FM was the preferred and most trusted source for local news and information. Community engagement is strong; three of the five community leaders interviewed said they interacted with the station in a substantive way—the religious leader has been invited to talk and contribute to programming and the youth leader has been on air discussing youth issues. Many community members visited the station to talk about their sports team, for example, or to make announcements. Mayardit is clearly at the heart of the community it serves and has innovative programming ideas, such as dedicating one day’s programming every week to programs relating to different cultures. Community members were alarmed at the prospect of the station closing and thought lobbying for continued funding was the best way to keep the station open. Interestingly though, none of the community leaders raised or mentioned any other ways for the station to generate revenue. 13 Mingkaman FM (Lake State) Mingkaman 100 FM broadcasts news and entertainment programs 17 hours a day from Mingkaman, Eastern Lakes State, and Bor, Jonglei State. Mingkaman’s programs vary in content and focus, including education, health, agriculture, music, sports, and entertainment. Formats range from news and feature magazine programs, to talk shows and special coverage. There are a number of programs and features with a specific focus on women—health, business, and education—with coverage of around 100 kilometers (km). The station’s local staff were relatively new and had only been at the station for around six months; their understanding of the i-STREAM program was therefore extremely limited. However, the team leader benefited from training on sensitive reporting in conflict zones and program development. They considered the station’s major achievement its increased reach and wider scope of programming (Mingkaman started as a humanitarian service). The challenges faced by the station include its building, the lack of an adequate number of solar panels, and difficulty attracting staff. Most of those listening to Mingkaman tuned in morning, afternoon, and evening for music programs as well as news and information. Broadcasting in vernacular languages is especially important for this community, which struggles with Juba Arabic and English. Interestingly, in this community where non￾vernacular languages are not well understood in radio programming, several listeners said they accessed news and information—such as presidential decrees and security issues—via Facebook. It is worth noting Facebook is an increasingly popular source for news and information among marginalized communities. In terms of listening and trust, Miraya FM was the most preferred, followed by Eye Radio for coverage of national issues, with Mingkaman third, but considered the best for local issues. The impact of programming on such stations as Mingkaman cannot be understated. Many personal stories related in the FGDs illustrate the power of community radio, broadcast in local languages by local people. Two participants said they allowed their daughters to go to school because they heard about the importance of girls’ education on Mingkaman FM, another said he refused to “give his daughter” to a man because she was still underage, and a female respondent successfully used the radio to announce a lost ration card. Singaita 88.3 FM (Kapoeta) Singaita FM was established in 2016 as the first state-wide radio station in Kapoeta. It runs on solar power and broadcasts local information in local languages for 17 hours a day with a coverage area of 53 km. The core of Singaita’s programs is peace building and development. The station’s local staff interacted with Internews and were familiar with i-STREAM. The station received broadcast equipment, training on reporting techniques, and vehicles allowing them to report from across the community. Radio remains the most accessed and used media across Kapoeta for news and information, and Singaita FM was highly regarded by members of the community it serves for its reporting on community issues, security, and conflict as well as keeping people informed on government activities. Coverage on area gold mining issues was also positively commented upon. Community members felt the station represented their interests and the diversity of the community well; they would like coverage to be extended further into rural areas to give people living there access to news and information currently not available to them. “Singaita is the only source of news and information, if the radio station was lost it would create a large gap in people’s ability to learn about local issues and stay informed of what is happening in their community” (Community Leader) Community Engagement Network – Sub-Grant 2 CEN was registered by the Ministry of Justice and Constitutional Affairs under the NGO and Company Act and became a recognized NGO in South Sudan in 2019. It is a network of media organizations and 14 arts and cultural groups, organizations, and associations with the remit of promoting a socially transformed and cohesive society. The ET’s evaluation of CEN’s institutional management capabilities and ability to support itself financially was conducted through stakeholder interviews, interviews with three members of the senior management team, and on-site visits to Nile FM in Malakal and Kondial FM in Bentiu. CEN transitioned into a fully owned South Sudanese NGO in early 2020 and, as such, it was too soon for the ET to evaluate its management capacity and financial viability. CEN is currently managing and supporting the operations of JamJang FM in Ajuong Thok, Radio Salaam FM in Maban, Nile FM in Malakal, and Kondial FM in Bentiu. One of the tangible outputs of i-STREAM’s support has been enabling CEN to better communicate with the communities it serves through the “Communicate with Communities” approach and to facilitate the collection, storage, and analysis of listener feedback to inform future programming. CEN believes the support its monitoring and evaluation (M&E) team received enabled them to improve their reporting and understanding on Theories of Change and develop strategies for a deeper understanding and connection with their audiences and communities. Most of the stakeholders interviewed said they were not sufficiently familiar with CEN to comment upon its activities, achievements, and progress. They did, however, know the network was supported by Internews and had a remit of providing humanitarian information services. Its most known and talked about initiative (mentioned by three stakeholders) was its recently launched South Sudan Cultural Heritage project working with conflict-affected communities and host communities, which was praised as an interesting initiative both in content and approach and one which could stimulate revenue opportunities. The management team at CEN headquarters in Juba is relatively small and young, with strong community engagement skills and reasonably commercially savvy evidenced by their partnerships with Food and Agriculture Organization of the United Nations (FAO), International Office of Migration (IOM), Transform Media UK, and BBC Media Action. The ET observed a young, proactive, enthusiastic team of committed professionals with a positive outlook on the future. While there was general pessimism about the survival of the network in the absence of donor funding and support, this network seemed less reliant—in practice and philosophically—on USAID funding than either Eye Radio or TRC and was making revenue generation a priority. CEN is in the process (with i-STREAM support) of developing HR and infrastructure capacity in the form of equipment and facilities and financial management systems and procedures, and building skills in broadcast capacity, conflict reporting, and developing content within communities. It is hoped the organization remains lean and nimble and does not become too bureaucratic and top-heavy. CEN is actively seeking and establishing partnerships as part of its financially self-sustaining strategy. Currently, it has partnerships in place with BBC Media Action and IOM. CEN devoted considerable time and effort to understanding more about and communicating with the audiences it serves and, with the support of i-STREAM, has organized focus groups and community sensitization programs. This strategy of close community engagement might serve it well in the long term as it reaches out to communities for financial support. The CEN stations visited as part of this review were: Nile FM (Malakal) and Kondial FM (Bentiu). Nile FM (Malakal) “Born out of crisis:” Nile FM is a humanitarian radio station located inside the Malakal Protection of Civilians (POC) site. The station produces programs and disseminates information in Arabic 14 hours a day and covers a range of 45 km. Nile broadcasts news and information about the humanitarian response on thematic topics such as education, peace, health, and marginalized groups in addition to broadcasting music and entertainment. Its staff have interacted with i-STREAM predominantly through training and the team leader spoke positively about the training they received outside of journalism skills, specifically managing dialogue and interacting with the community. As a very community-focused station with a good understanding of how to work with and understand the community, they have met with school directors 15 and community elders and were able to establish an elder-support committee. This will give the station a good chance to survive with the support of the community. The staff are a committed and enthusiastic group of young community broadcasters and are justifiably proud of the programs they broadcast on health, peace-building, and child protection. However, they felt the station had not yet managed to develop a sustainable business model to enable it to continue operating. Additionally, Nile lacks a local office in Malakal along with all its associated issues of maintaining and replacing equipment. All of those who participated in the interviews in Malakal said Nile FM was their only source of news and information on local issues—it was their most important source of information and the one they used every day. In this area, stations from Sudan were also listened to (Omburman and Tamaz), with older people tuned into international stations (BBC/Al Jazeera) and younger people are increasingly using Facebook. All were enthusiastic consumers of Nile FM, and the service is of special importance to those living in Internally Displaced Persons (IDP) camps in Malakal. Some expressed interest in national news, but are mainly interested in local news, information, events, and notices and Nile FM is the sole and trusted source. Kondial FM With its origins in Humanitarian Information Service (HIS), Kondial FM broadcasts to IDPs in the Bentiu POC site with humanitarian information on water, sanitation and hygiene, peace building, education, and sports. It is the only local radio station that reaches the 115,000 IDPs in the POC site. The station’s local staff in Bentiu reported a rich experience with i-STREAM through their participation in a wide variety of activities, including developing and disseminating broadcasts, media production, writing and editing, and exposure to international journalists. The staff are proud of their coverage of issues and the changes these have brought about regarding open defecation and cholera. The station organized volleyball and football matches for youth in the IDP camps. Its issues, including limited coverage area (only 25 km around Bentiu) and decreased funding, coupled with very limited revenue-generating opportunities mean staff and the number and diversity of broadcasts have been reduced. Community members agreed Kondial FM was their main source for local news and information, and they respected and trusted its outputs. 4.1.4 Enhance Media Education and Journalism Training – Sub-Grant 3 A complimentary core objective, in parallel with and support of building radio stations, was to “build and strengthen the capacity of media professionals,” through intensive in-house training with partner stations and strengthening academic training at UOJ, vocational training at MDI, and specialist, thematic reporting. Through stakeholder and in-station interviews and on-site visits to AMDISS, MDI, and UOJ, the ET learned i-STREAM made an important contribution to building journalism capacity from scratch from a very low base. As part of the high-level interview process, participants were asked to score the “professionalism of journalism in the country” and “training and skills-building infrastructure” on a ten-point scale, where a score of ten represented “extremely healthy” and zero represented “extremely unhealthy.” The scores are less interesting in their own right than in a comparative context. The average scores based on 13 participants are as follows and show a slightly above average score (five) for professionalism. Table 1: Interview Scores on Journalism Topics Average score out of 10 Professionalism of journalism 5.6 Training and skills-building infrastructure 4.8 Plurality of news sources 4.8 Range and diversity of media outlets 4.7 16 Topics Average score out of 10 Independence of media outlets 4.2 Public access to a range of news and information sources 4.1 Freedom of expression 3.9 Financial sustainability of media outlets 3.9 Based on 13 high-level stakeholder scores. Further, when asked to score the contribution of i-STREAM to a number of activities (as shown below), the highest score was for “raising the standards of media management and journalism.” Table 2: Interview Scores on i-STREAM Contribution of i-STREAM Average score out of 10 Raising standards of media management and journalism 7.2 Producing local content 6.6 Building the capacity and sustainability of the independent radio sector 5.7 Supporting and advocating for reforms in the media framework 5.6 Supporting and advocating for reforms of media legislation 5.6 While there was general agreement among stakeholders (with one or two exceptions) that i-STREAM contributed positively to building the entry-level journalism skill set, there was some criticism the program has not strengthened media management skills, which are considered to be poor for advanced-level journalism capacity (investigative journalism, for example). Much of the in-house and classroom-based training was centered in Juba, thereby disadvantaging those working outside the capital the opportunity to attend, and many more men than women received training. Partner Station Capacity Building The ET found i-STREAM made considerable progress in improving the capacity of the media practitioners working for partner stations, and interview respondents attributed this progress to the embedded model of capacity-building where an external expert sits in the radio station and conducts intensive on-the-job training. All 13 senior management staff of the partner network radio stations agreed the embedded model of in-house capacity-building for stations employees enhanced the capacity of those staff trained and, as a result, improved their overall professionalism and technical abilities to fulfill their roles. Staff especially like and respond well to the day-to-day interaction with and practical exposure to subject-matter experts and being taught through practical examples and continuous feedback. The experts were spoken of in glowing terms and credited with building the capacity of these fledgling networks in a relatively short time period. A number of station managers were also flown out of the country to visit international media stations and acquaint themselves with best practices of running their own stations. “I and the manager of the other stations were sent to media houses in the US, South Africa to look at the community radio stations there, all this we got from Internews.” (Eye Radio) As a result of the embedded in-house training and exposure visits, the senior managers reported they were well-equipped and inspired to make changes and redesign their stations activities, systems, and processes, which led to improvements in the running of the stations and new and innovative programming. “At Eye Radio, we were able to introduce two additional programs which are our most successful programs, that is the Dawn and Sundown programs, as a direct result of the capacity-building received from Internews.” (Eye Radio) The in-station capacity-building also included extensive technical training on the sophisticated studio equipment imported from the United States. The imported equipment differed significantly from that 17 widely used in South Sudan and across the region and could not be used without extensive training and mentoring. “Most staff came from other radio stations where the setup is different, so the training helped us to understand how the equipment can be used.” (Eye Radio) At Eye Radio, the ET saw two state-of-the-art on-air studios all fully furnished and equipped, and two additional studios used for recording jingles and the production of content for external clients. In terms of editorial and journalism management and skills building, good practice in terms of daily editorial meetings and fact-checking stories and reports have been instituted and staff coached on how to deal with misinformation and how to fact check stories. Capacity and practice in this area is an important reputation and trust-building mechanism. “We now have an editorial board; every controversial story must be brought before the editorial board so they can approve it.” (Eye Radio) Further, the in-house capacity-building support enabled partner stations to implement and adhere to strict editorial policies that have safeguarded them against excesses of the government. “Interviewing more than two sources for a news story and being able to demonstrate, when called upon, that we have followed our strict editorial guidelines.” (Eye Radio) The ET found that, under the watchful eyes of Internews and as a result of the intensive capacity-building activities, all partner stations have functioning systems and processes for finance, HR management, editorial guidelines, technical systems, and business offices. Senior managers interviewed for this evaluation said they collaborated with Internews on the design and implementation of these systems, and the training resulted in them being fully operational and advantageous to the effective running of the station. “Internews put the infrastructure in place that you see around, they also trained the manpower to run it as well as train people in all technical areas including journalism, finance, and business and admin.” (Eye Radio) In total, 36 Eye Radio journalists, including 14 women, received capacity-building training from i-STREAM over the lifespan of the project. Internews trained 47 TRC field staff, including 15 women, and 16 CEN staff, including four women, over the lifespan of the project. The training effort for community correspondents has been focused on specific thematic areas, including humanitarian reporting, health, education, agriculture, gender, and peace (conflict-sensitive reporting). These expert-run training programs enhanced the capacities of community radio staff to provide lifesaving information during times of flood, conflict, and other natural occurrences in ways that have improved the lives of the people. The intended online survey among training beneficiaries, designed to explore relevance, response, and application of the training, was abandoned because of the lack of beneficiary contact details available. Instead, a very modest telephone survey was conducted among the 77 contacts provided, which yielded a low response of just 18 completed interviews (23 percent). The data from this survey showed 12 of the 18 respondents were male, 11 were aged 35 and over, and 13 had at least an undergraduate degree. Most, 14 of 16, attended training courses on writing and producing news stories, media ethics and media law, program production, humanitarian reporting, news gathering, and gender reporting. Eight said they attended training on digital production techniques, and only three attended training on income generation approaches. When asked to rate the “overall improvement in their skills and knowledge as a result of attending the training,” where ten represented “a considerable improvement in skills and knowledge” and zero represented “no improvement at all,” the average score was 6.8. The ET fully acknowledge this small, unrepresentative sample does not provide real evidence of the impact of attending training courses on attendees’ skill levels, the perceived relevance of the training, or the ability to apply the training in the course of their work. In addition, it confirms asking training beneficiaries to 18 self-rate the value of training personally and to their organizations is an unsatisfactory method of assessing the value and benefit of training and skills building. As reported earlier, the ET found the partner networks were created and built to operate in a very traditional radio broadcasting style of writing, storytelling, and producing. Across the three networks, senior managers and field staff tended to operate with an analog mindset and with very little grasp of the new and digital media revolution. The lack of digital skills, coupled with the lack of skills of the existing sales and marketing personnel, were the most obvious capacity deficiencies in all three networks. The approach most used by network business managers, writing and responding to proposals for program sponsorship from donors and NGOs, represented, in the opinion of the ET, a lack of creativity and proactivity and underpinned the view that radio can only be funded by donors and NGOs. Industry-Wide Capacity Building The ET found i-STREAM made some limited contributions to the overall capacity development of the professionalism of journalism in South Sudan. Apart from training support to staff of the partner networks (Eye Radio, CEN, and TRC), i-STREAM also supported capacity-building for journalists through external national partners including AMDISS and its sister institution MDI, UJOSS, the National Editors Forum, Association for Media Women in South Sudan, and UOJ. MDI runs a basic course certificate training for early career journalists, which runs nine months and has trained 108 journalists over four cohorts, 22 of whom were female. The majority (16 out of the 20) of high-level respondents were cautiously positive about the training offered by MDI and attributed it to equipping entry-level journalists with some basic skills. However, to make a step-change in the overall quality of media management and reporting, there is a need to train media managers in management and business skills and equip journalists with higher-level professional skills (investigative journalism, feature writing, and specialized reporting) which is not being met through the current MDI programs. The ET noted MDI faces several challenges, mainly its inability to attract sufficient high-quality caliber trainees, its inability to source sufficient revenues to run diploma or degree-level training, and a lack of curriculum to cover extensive journalism courses and the trainers to effect it. The training also lacks components of media business management and media digital skills training important for today’s changing media sector. The training has nevertheless led to some improvement reported in the basic quality of journalism and professionalism of the media houses where its graduates work. i-STREAM provided institutional support to UOJ by helping to develop its curriculum and building two radio studios for training purposes. The ET found collaboration between UOJ and i-STREAM came to an end in 2016, and comments shared with the ET during this study highlighted tension between UOJ and i￾STREAM due to unmet expectations, unsatisfactory communication, and the lack of completion of the two training studios, which remain unused to this day. Training was also extended to journalists working for Internews subgrantees, which are mainly media organizations supported on specific thematic areas of reporting such as GBV and conflict-sensitive reporting. These organizations included GoGirls ICT Initiative, Advanced Youth Radio, Samaritan FM, Radio Reconciliation, Catholic Radio Network (CRN), Radio Voice of Freedom, and Abyei Information and Radio Service (AIRS). Project documentation shows i-STREAM provided training to 1,336 journalists between 2014 and 2019 across all of the networks and subgrantees, including training and mentoring of community correspondents by embedded journalism trainers at humanitarian information services sites and community radio networks and private radio stations based in remote communities where humanitarian situations exist across the country. Progress in Advocacy Capacity Building Toward Strengthening Advocacy Activities i-STREAM contributed to media advocacy efforts by supporting the lobbying and monitoring efforts of AMDISS and UJOSS. These efforts facilitated the tracing of journalists detained by the national security as 19 well as advocating for their release. Through AMDISS, i-STREAM supported networking events between security forces and journalists and strengthened media laws. With the support of i-STREAM, both AMDISS and MDI have become respected organizations and succeeded in establishing themselves as watchdogs. Some stakeholders were critical that i-STREAM, with its resources and capabilities, was not more visible in the advocacy space and felt its limited visibility and coordination efforts resulted in limited impact. Professional associations and media organizations remain weak and subject to threats and violations. 4.2 CONCLUSIONS 4.2.1 Eye Radio • Eye Radio suffers from over-dependence on donor funding (and an NGO orientation) and a lack of genuine belief that the funding will come to an end within the next nine months. Apart from reducing staff and salaries, and an increasing use of interns, there was little other evidence of any contingency plans to prepare for this eventuality. It appeared to the ET that the resultant management team could remain top heavy, and there was no serious attention given to building up an experienced media commercial team to quickly devise and implement an immediate and practical commercial strategy. • Eye Radio’s operating costs were extremely high compared with other, similar independent radio stations in the region, and further strategies will be needed such as reducing the cost of news/story gathering, content production, and distribution. • Eye Radio is managed and run as a rather old-fashioned, traditional public service radio station. It is highly process and systems driven and is not sufficiently nimble to embrace both the new financial realities and the new media environment. Additionally, it is slow to embrace change and lacks diversity and a youthful, innovative cadre of staff. • Eye Radio’s organizational structures (board and senior management team) need to be more knowledgeable about and engaged with the new media paradigm and to lead and embrace change. • Eye Radio’s management and systems need to be more nimble (less bureaucratic) to facilitate their response to the changing media environment and to be more effective at contingency planning. • M&E and much deeper audience insights need to be strengthened to support marketing and sales efforts and produce and broadcast programming for specific audience segments at the times and on the days when they are listening. 4.2.2 The Radio Community The Juba-hub is considered to be responsibly managed with policies and practices in place. During the course of this evaluation, management were in the process of reducing central overheads and grappling with the issues of reducing headquarters staff, combining management roles, and reducing salaries while retaining trained and skilled staff. At its peak, TRC had 60 staff and was finding it difficult to adjust to the new reality of a smaller, multi-tasking establishment. In addition to managing with a significantly reduced head count, a key issue for TRC going forward is improving the efficiency and quality of content aggregation, curation, and distribution from its hub. The four community stations visited as part of this evaluation were all highly valued by the communities they serve. A major challenge for these stations is reducing operational costs through a more effective use of community resources, including more extensive use of volunteers and engaging with communities on a wide range of revenue-generating initiatives. However, in interviews with community leaders there was a clear lack of understanding that the community would have to contribute to the funding of these stations. The ET concludes: 20 • TRC suffers from an over-dependence on donor funding and recognizes that without donor funding it will be unlikely to survive. Few, if any, contingency plans have been made to prepare for this eventuality, and management and staff are pessimistic about future revenue opportunities. It would make very little economic sense to reduce the number of TRC-supported community radio stations. The decision would be to continue to offer financial and technical support against aggressive operational and financial targets, or close completely. • Similar to Eye Radio, TRC’s operating costs are high compared with other, similar independent radio stations in the region and need to be reduced in an imaginative way to sustain service to communities. • TRC’s organizational structures (board and senior management team) need to be reviewed and their capacity to change-manage needs to be strengthened. There were one or two observations that the board members and management needed to more independent from one another and that acting members of staff should not be eligible to sit on the board. • More innovation and collaboration are needed to utilize local and citizen journalists more effectively to bring content from the communities into Juba, curate it, and disseminate it back into the regions (the approach used by CRN provides a good point of comparison). • At the community level, the use of community volunteers should be explored and used more. • M&E should be strengthened to support marketing and sales efforts. TRC stations are lifelines for the communities they serve, but without continuing United States Government (USG) financial and technical support, none of TRC’s community radio stations will be viable. TRC needs to be more innovative in minimizing its cost of operations to go beyond reducing headcount and salaries and implement alternative strategies such as pooling national news and information rethinking its approach to covering community-based topics 4.2.3 Community Engagement Network CEN is a young, dynamic, community-responsive network. Its managed and supported radio stations and activities are highly prized by the local people and communities they serve. CEN made strenuous efforts to foster close community relationships and listen to and act on community needs and interests. If CEN can right-size its operations and manage and plan its costs, it may be able to become financially viable through its approach to partnerships, which it is in the process of developing. CEN’s management embraces the need to be financially independent and is actively programming and working on content initiatives to attract sponsorship. One of the advantages of CEN’s remit and approach is the involvement of local communities in their programming and activities, which may present opportunities for modest revenue generation. Based on the evidence collected, the ET concludes: • CEN stations are critical in the lives of the communities they serve, providing invaluable information throughout difficult times (floods, conflict) and through their development-focused programming. • CEN has a less donor-dependent mentality than the other two networks. • CEN has a youthful, dynamic management team open to innovation and new ideas. • CEN has the potential to develop a strong network of partners, based on its experience to date. • CEN is still in its start-up phase and can learn lessons and avoid mistakes. • With appropriate levels of short-term support, CEN could become financially self-sustained, albeit at a modest level of operation and aspiration. 21 4.3 RECOMMENDATIONS 4.3.1 Eye Radio, The Radio Community, and Community Engagement Network Strategic Recommendations These following strategic recommendations are based on evaluation findings, the ET’s analysis and interpretation of results, and the challenges faced by the three networks, namely the need for management boards, senior managers, and staff to go through an intense period of change-management to move into Phase 2 of their development and be “fit for the future” by bringing costs into line with income, understanding and embracing the digital media space, and continually improving standards of journalism and media management. The ET recommends Phase 2 be built around the principle of “consolidation” to reduce overheads and costs, avoid duplication, centralize and professionalize commercial activities, and strengthen the cadre of media professionals working in the independent media sector. In arriving at these recommendations, the ET reviewed other modalities in the independent national and community-based radio sector and, of all those reviewed, is impressed by the way in which the CRN is run in South Sudan. Recommendation 1: Consolidate the headquarters operations of Eye Radio, TRC, and CEN to improve their overall effectiveness and reduce duplication of centralized resources. Reduce management/operational overheads and consolidate expertise and knowledge. • Leverage purchasing power to reduce technical and maintenance costs. • Create a single center of excellence for digital production, e.g., “first to market.” • Create a single commercial and sales function with targets for each network under the direction of an experienced sales/commercial director with a proven track record of commercial media and establish a strong, central airtime sales system. • Extend content and coverage by expanding the broadcast network and digital footprint. • Leverage leadership status and become one strong voice for independent media, quality journalism, and skills development. • Explore more cost-effective, innovative content generation and sharing through networks of citizen journalists/community volunteers. Recommendation 2: Develop an initial, five-year funding mechanism tied to specific activities and targets based on actionable business plans (innovation/market leadership/viability) with funding reduced each year as income generation ramps up. Recommendation 3: Consider transitioning from NGO status to commercial status. Allow the stations to make profits to plow back into their operating overheads and determine their own cost/income ratio strategies. Advertising revenues amount to a fraction of operating costs (around 10 percent) and the Media Authority’s interpretation of the current media regulation is that NGO status radio stations are not-for-profit and have an upper limit on the amount of revenue they can earn. This restrictive environment will continue to hamper the financial sustainability of USG-supported stations Recommendation 4: Assess readiness for direct funding. Recommendation 5: Implement culture and leadership change by employing/training board members and managers who can drive institutional change, “right-size,” consolidate, generate income, and transition to digital. Recommendation 6: Change operating modalities to bring staffing levels and salaries in line with comparable stations. • Broadly comparable stations to Eye Radio run at around $700,000 per year with senior managers earning $10,000–$12,000 a year. 22 • Comparably, CRS runs at roughly $30,000 per year with station managers salaries at $3,000. • Headquarters staff should include five to six senior managers. • Station staff should include managers and three to four reporters/stringers (not full-time employees). Recommendation 7: Build or acquire a central commercial and airtime sales team to focus on inventory management, marketing and return on investment, audience behavior, and content driver and explore revenue-generating opportunities from existing facilities. Recommendation 8: Embrace the content and monetization opportunities of digital and social media. Move away from the traditional radio model. • Look outward and learn from innovations in the region. • Embrace citizen journalism. Recommendation 9: Develop new methods of training and upskilling board and Senior Management Team (SMT) in readiness for fully autonomous operations (accountability, self-sufficiency). Move away from the expensive embedded expert model. Recommendation 10: Update methods of in-house training and mentorship (virtual) with both local and international experts developing and running training modules. The ET would like to underline the importance of using international experts as this stage of the development of the independent media sector. Recommendation 11: AIRS requires minimal, light-touch support to keep it on track and support skills building activities. Consider a one-off direct investment for capital developments to support the drive for financial independence. 4.3.2 Recommendations for Capacity Building Recommendation 12: Coordinate and collaborate more with other donors involved in media training to avoid duplication and better focus resources to needed areas in media development. There are many donors and international organizations supporting media development in the country through training and skills building initiatives, including the Norwegian People’s Aid/Norwegian Embassy, Journalists for Human Rights/Canadian Embassy, United Nations Educational, Scientific and Cultural Organization (UNESCO), and United Nations Development Programme (UNDP), among others. The ET recommends USAID, as the biggest player, use its position to coordinate the work of the other donors and reduce its own financial commitment. Recommendation 13: Provide technical support for training in thematic areas—including covering elections in conflict zones. Recommendation 14: Improve internship opportunities for entry-level journalists and content producers through a wider network of international and local partners to enable graduates to put their skills to work. Recommendation 15: Modernize the curricula and approaches to training to meet the changing trend and needs in the media industry with an emphasis on practical skills in business, digital and media law and advocacy, and a learning-through-doing approach in addition to standard classroom training. Technology could be used to greater effect in training. For example, the use of virtual newsrooms and radio stations. Recommendation 16: Move away from the embedded expert model of training. Recommendation 17: Continue technical assistance to MDI to develop more contemporary and relevant curricula incorporating training in management, commercial training, investigative reporting, elections coverage, etc. 23 Recommendation 18: Continue technical support for MDI to develop more advanced journalism diplomas/courses/media management capacity with more effort directed toward training women and those outside Juba “Training and building the capacity of local journalists is key, if there is a project which need to be sustained is the Media Development Institute, because it produces batches every year… for the MDI, when there is no war people can be asked to pay some fee, and the premises can be used for workshops, so this can bring in money.” Recommendation 19: Develop a revenue generation strategy and implementation plan for AMDISS and MDI such that by 2024–2025 they are self-reliant. Recommendation 20: Consider a one-off investment to AMDISS and MDI for capital investments, specifically for revenue-generating activities/facilities. 4.3.3 Recommendations for Advocacy for Capacity Building Recommendation 21: Step up activities promoting freedom of speech and protecting independent journalists and media houses through supporting educational programs, building stronger professional networks and organizations, developing a cadre of expert media lawyers, and disseminating information more widely. Recommendation 22: Secure funding for the foreseeable future and collaborate closely with other donors/funders. Recommendation 23: Reduce the over-reliance of AMDISS and MDI on donor funding by implementing aggressive financial targets and timelines. Recommendation 24: Improve visibility to attract more partners and wider industry support. Recommendation 25: Embed M&E and develop an effective dissemination strategy. 4.3.4 Way Forward – Phase 2: Consolidation On the basis of this evaluation, the future of independent media in South Sudan is in jeopardy. Without doubt, significant progress has been made. USAID-supported stations are well-functioning media outlets, producing and broadcasting respected, high-quality, relevant and trusted news and information to sizeable audiences across the country. Stakeholders all agree the skills and capacities of journalists have improved and the overall quality of journalism being practiced in 2020 is an improvement over that practiced before the inception of i-STREAM. However, the independent media sector remains donor-dependent and is likely so to do for the foreseeable future. This leaves USAID with a critical decision, continue to support the sector for the long haul or allow the sector to collapse. There is a strong belief (and hope), given the scale of the investment to date and progress made, that support will be forthcoming in years to come. It is impossible to predict how many more years of support will be required, but it is possible there may be more conflict in the run-up to elections. There is little sign the economic conditions for commercial-revenue generation is improving or will do so in the next few years. In light of this, the ET made the following recommendations for a way forward from mid-2022 for an initial period of five years and has characterized this second phase of development as a transition from start-up to consolidation with a focus on cost-reduction through combining and centralizing functions underpinned by more collaborative and imaginative ways of producing, curating, sharing, and distributing content. Rather than each network developing its own digital presence and commercial and income generation functions, combine them into a strong central unit. The ET also recommends continued technical support for the education of journalists and other media actors by modernizing curricula and training modalities and improving high-level skills in media 24 management and business. Consideration should be given to setting up a center for more advanced and specialist journalism training to meet the need for more senior-level training. Finally, the ET recommends USAID, as the largest and most visible donor in the field, lead an initiative to broaden the donor base and build strong collaborative partnerships to improve the threatened and deteriorating legal and regulatory environment by going beyond lobbying and strengthening the legal frameworks to include internet access and freedom of expression online. 1. Design a new, highly-focused, activity and aggressively financially targeted base program for an initial five-year period. Learning the lessons from the scope of i-STREAM, concentrate effort on: • Focusing the supported independent media outlets on a consolidated approach to shared functions and operations. • Supporting media to go digital and embrace social media. • Focusing on income generating opportunities and systems. • Continuing to support in-house professional skills development. 2. Continue technical support for in-house training for managers, content producers, and sales and marketing staff during the transition. Move away from the embedded-expert model to a more cost-effective and accountable model. 3. Expand the funding base. Reach out to technology industries and other donors/foundations. 4. Lead the donor/funder coordination efforts to maximize available funds and reduce duplication (no one organization is necessarily the best at everything). 5. Go beyond lobbying and work on implementation and dissemination. 6. Consider establishing a specialty, more advanced journalism center focusing on investigative journalism, election reporting, and the safety of journalists. 7. Reduce financial support for AMDISS/MDI and consider a one-off capital investment to allow them to develop and exploit their facilities. 8. Continue to work with AMDISS and other actors, collaborate and engage with a broader group of stakeholders and development actors to strengthen advocacy, campaigning, and capacity to work toward a healthier media environment. Improve monitoring and dissemination. 9. Improve information gathering and M&E systems to augment from simple monitoring (counts) to actionable evaluation with outcome and impact targets for all projects. 25 ANNEXES 26 ANNEX 1: STATEMENT OF WORK C.1 SUMMARY OF THE PROGRAM Title Strengthening Free and Independent Media in South Sudan (iStream) Award Type and# Cooperative Agreement - AID-668-A-13-00005 Implementer Internews Award value $84,000,000 Life of Award September 30, 2013 – December 17, 2021 AOR Victor Lako C.2 BACKGROUND Beginning in 2013, USAID has supported Internews to implement the $84 million iStream activity. The overall goals and objectives have remained consistent during the life of the activity, despite the dynamic context in South Sudan throughout this time period, including the return to conflict in 2013 and 2016. From its inception, the goal of the activity has been to strengthen free, independent, and sustainable media in South Sudan, with an emphasis on the independent radio sector. The activity has three program objectives, of which the first is: Objective 1: Strengthen autonomous, journalistic professionalism and facilitate the development of an institutionally well-managed and financially self-sustained, independent media in South Sudan. Under the iStream award, USAID has supported Eye Media, The Radio Community, and most recently, the Community Engagement Network, representing various radio stations. Eye Radio has the largest presence. Based in Juba, with repeaters in six of the10 states, Eye Radio has become a trusted source of news and information in South Sudan. The Radio Community and the Community Engagement Network both represent a consortium of community radio stations throughout the country, providing local populations with relevant, credible information in local languages. While there has been a focus on promoting sustainability throughout the life of the activity, the media institutions continue to be overwhelmingly dependent on donor financial support. Radio stations are able to cover, on average, 10 percent of annual operating expenses through non-donor revenues. Independent radio and media play a critical role in providing citizens with access to information, which is especially critical in the fragile South Sudanese context. USAID is planning to extend the period of performance of i-Stream through September 30, 2022, but with a significant scale-down of financial support to both Internews and the media institutions it supports. During this time, Internews and its media partners must make difficult decisions regarding how to maintain operations with more limited resources, ensuring that South Sudanese citizens continue to have access to credible, independent information via the radio. Findings and recommendations from this evaluation will help USAID and Internews to identify a path forward so that by the end of the award, the supported media institutions are less dependent on USAID funds. and more sustainable. 27 C.3 PURPOSE OF THE EVALUATION To assess the extent to which i-Stream has facilitated the development of an institutionally well- managed and financially self-sustained, independent media in South Sudan and to provide recommendations for how USAID/South Sudan support can further enhance the sustainability of existing media institutions in South Sudan. The primary audience for this report will be USAID/South Sudan and Internews. C.4 EVALUATION QUESTIONS, METHODOLOGY AND DATA SOURCES Evaluation Questions: Evaluation question that Contractor must address is as follows: 1: To what extent has the objective regarding developing an institutionally well-managed and financially self-sustained, independent radio sector in South Sudan been achieved under iStream? Recommendations: Additionally, to inform current and future programming, the Contractor must provide recommendations on the following: 1. What steps might be taken to improve sustainability in the radio operations currently supported by USAID (including but not limited to those that lead to reductions of operating costs, increases of revenue, and improvements of operating efficiencies). 2. How might USAID-supported media houses be more appropriately sized and resourced for the South Sudanese context? Evaluation Methodology: In developing an appropriate methodology, the Contractor must consider potential limitationsand incorporate best-practice mitigation measures. The evaluation must be supported by relevant data and information gathered from meetings with beneficiaries and all the relevant stakeholders. The evaluation team must clearly demonstrate links between the data collectedand the subsequent findings, conclusions, and recommendations drawn. Where applicable, responses to the questions above and resulting data must be disaggregated by sex and gender. As recommendations and findings will be applied to current and future USAID/South Sudanmedia development awards, the Contractor must not simply provide an accounting of performance against targets but provide independent analysis. Available Data Sources: Suggested data sources include but are not limited to: (a) Secondary data/background documents, (b) activity plans, outputs, and reports, (c) key informant interviews, (d) focus group discussions and (e) survey(s) of project stakeholders and beneficiaries. Contractor must place emphasis on the collection of reliable empirical data and/or objectively verifiable evidence, as opposed to anecdotal evidence. Where surveys or interviews are used, appropriate sampling and questioning techniques must be utilized to ensure representative results; where references are made to data generated by iStream and their partners, it must be complemented by references to independent data sources and any significant data differences must be explained. The Contractor will have access to all performance data, surveys, and other deliverables that have been collected and reported by the implementing partner as part of their Performance Monitoring and Evaluation plan for iStream. The Contractor must also review the following documents: • Relevant project reports, assessments, annual and life of project work plans. 28 • Full data set in their formats available and collected by the implementing partner. In addition, the evaluation team must consult with key informants and stakeholders, including: • Meetings with USAID/South Sudan • Meeting with Internews home office key staff and iStream current and previous program staff. • Meetings with key current/former iStream media subgrantees. C.5 GENERAL CONTRACTOR REQUIREMENTS - STATEMENT OF WORK The Contractor must conduct a methodological evaluation that must be fully compliant with USAID’s Evaluation Policy (updated October 2016) and Automated Directives System Chapter 201 and utilize tested and scientific evaluation methodologies. C.6 TEAM COMPOSITION The Contractor must utilize an Evaluation Team in accordance with Section F.4. C.7 SCHEDULE AND LOGISTICS International travel is expected as a part of this task order. The schedule (with precise due dates for deliverables) will be defined in the work plan before the start of the in-country field work. Prior to commencing the field work, all team members will review background program documents to gain a firm understanding of the situation in South Sudan and the activity. See H.3. The Contractor is responsible for ensuring the safety and well-being of Contractor's staff and must apply accepted practices in accordance with the local laws and health authorities' directives, including but not limited to related to COVID-19 quarantine requirements. 29 ANNEX 2: WORK PLAN WORK PLAN, METHODOLOGY AND EVALUATION DESIGN For: USAID/South Sudan Performance Evaluation of the Strengthening Free and Independent Media in South Sudan Activity RFTOP USAID No: 72066821R00015 Submitted by: ME&A, Inc. 4350 East West Highway, Suite 210 Bethesda, MD 20814-4413 Contact Person: Teresita Perez, COO and Executive Vice President Tel: (301) 652-4334 ◆ Fax: (301) 652-3733 ◆ Email: tperez@engl.com 30 ACRONYMS AMEP Activity Monitoring and Evaluation Plan COVID-19 Coronavirus Disease 2019 CRS Community Radio Station EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion GIS Geographic Information Systems HO Home Office HR Human Resources IP Implementing Partner i-STREAM Strengthening Free and Independent Media in South Sudan Activity KII Key Informant Interview MD Managing Director ME&A ME&A, Inc. NGO Non-Governmental Organization PMEP Performance Management and Evaluation Plan SPA Station Performance Assessment TL Team Leader USAID United States Agency for International Development 31 1. INTRODUCTION The United States Agency for International Development Mission in South Sudan (USAID/South Sudan) has contracted ME&A, Inc. (ME&A) to conduct the endline performance evaluation of their Strengthening Free and Independent Media in South Sudan (i-STREAM) Activity. The i-STREAM Activity, implemented by Internews, has a period of implementation from October 2013 to the end of December 2021, with a total estimated cost of $84 million. The overall goal of i-STREAM is to establish and strengthen a free, sustainable media in South Sudan, with an emphasis on the independent radio sector by supporting local partner radio stations and focusing on improving the professional preparation of media practitioners (editors, journalists, media managers); promoting the economic self-sustainability of media houses; facilitating an enabling environment for a free media; and supporting institutions for a freer media. The purpose of the endline performance evaluation is to assess the extent to which i-STREAM has achieved the core objective of developing an institutionally well-managed, independent, and self-sustained media in South Sudan. Findings from the evaluation will inform current programming and help USAID and its implementing partner (IP), Internews, to identify what actions need to be taken so the media institutions supported by i-STREAM can be transitioned to be more independent of USAID funds by the end of the program. The evaluation will make recommendations about steps to improve sustainability in the radio operations currently supported by USAID, and the appropriate size and resources needed for USAID￾supported media houses given the South Sudanese context. The evaluation will answer the main evaluation question (EQ) “To what extent has the objective regarding developing an institutionally well-managed and financially self-sustained, independent radio sector in South Sudan been achieved under i-STREAM.” The evaluation will assess the performance and contribution of i-STREAM at three levels: 1. Media Landscape and Ecosystem 2. Media Practitioners 3. i-STREAM-supported Community Radio Stations (CRS) To provide answers to these questions and support the recommendations with evidence, the evaluation team (ET) will use a combination of secondary and primary research techniques, including: a) USAID and Internews program documentation and a literature review; b) in-depth interviews with stakeholders, high￾level key informants, and local community leaders to yield as wide a range of views and perspectives as is feasible; c) a web-based survey among media practitioner beneficiaries of Internews training and mentorship; d) partner station performance assessments (SPAs) interviews with station management and staff; e) third-party audience research data (this could include looking at listeners’ feedback call logs, if available); and f) audience-member focus group discussions (FGDs). Additionally, we will use geographic information systems (GIS) to map the coverage areas of all the broadcast media across the country so that we can see where there is an over and under-concentration of broadcast media outlets and be sensitive to the needs of different communities in our recommendations. 2. EVALUATION TEAM AND EVALUATION MANAGEMENT The evaluation will be conducted by team of experts with knowledge and experience of media in South Sudan. The Team Leader (TL), Ms. Vivien Marles, a media expert with extensive experience in Africa, 32 including in South Sudan, will be supported by a local media expert, Mr. Anthony Materno, who has served as a journalist and a researcher for a number of international organizations in South Sudan. Ms. Marles will be responsible for the overall evaluation working in close collaboration with the ME&A headquarters team. She will be present in Juba, South Sudan, between September 5 and October 2, 2021. The ET will be supported by a local research firm, Consilient Ltd., (formerly, Forcier Consulting), with offices in South Sudan and across the region and a locally recruited and trained team of experienced qualitative and quantitative enumerators. Consilient will be responsible for the translation of the data collection tools (where appropriate) and data collection outside Juba (community leader key informant interviews [KIIs,] FGDs, and SPAs), and support with logistics in South Sudan. The organization and management (including quality control) of field-based data collection activities outside of Juba will be led by Consilient, whose project manager, Sean Robert, will provide management and technical oversight of all assigned data collection activities. He will be supported by Stanley Nyombe Gore who will be responsible for coordinating all data collection activities outside Juba. In addition to the Consilient project manager and field coordinator, Consilient will assign six field researchers to the evaluation, each of whom will work with a locally recruited enumerator to conduct the local data collection activities. ME&A Home Office (HO) will be responsible for the administration and analysis of the online survey among those media practitioners who have been trained and mentored by Internews. In addition, ME&A will provide overall management and technical backstopping of the planning, security approach, and technical deliverables. Together with the TL, the ME&A HO project manager will be responsible for the quality assurance of all technical deliverables required by USAID. 3. SUMMARY OF EVALUATION TIMELINES AND KEY TASKS The ET proposes the following schedule to complete all the tasks for a high-quality evaluation. This timeline may be subject to modification as a result of, as yet unforeseen, coronavirus disease 2019 (COVID-19) restrictions and/or specific security issues on the ground in South Sudan. We have attached a detailed field plan with day-by-day activities for travel to and data collection outside Juba (Annex 4). However, we will need to be adaptable and flexible to accommodate any changes that may need to be made to our plans. Consilient’s project manager and the TL will be co-sited in Juba and be in regular daily contact to make any necessary adaptations to the timelines, deal with any security issues, and implement the data quality control procedures. The dates and key evaluation tasks are summarized in this table: Dates Key Tasks Activities Aug. 16– Aug. 26 Task 1: Project kick￾off, preparation, and planning Initial call with USAID; document review; preparation of security plans, work plan, field plan, and draft evaluation tools; USAID introductions to Internews, stakeholders, and seven radio station managing directors (MDs) Aug. 27 Task 2: Virtual in￾brief presentation to USAID and relevant stakeholders Presentation of draft work plan. Between August 27 and September 1, the draft work plan will be revised based on the comments made at the virtual in-brief presentation. Sept. 2 Task 3: Final work plan and pre-final instruments submitted Submission of final work plan and draft evaluation tools for USAID approval 33 Dates Key Tasks Activities Sept. 4–8 Task 4: Field preparation and mobilization to the field TL travels to Juba, HEAT training, training and briefing of enumerators field teams, pre-test KIIs and finalize data collection tools and analysis templates Sept. 9– Oct. 9 Task 5: Data collection Data collection in Juba, Turalei, Mingkman, Aweil, Benitu, Kapoeta, and Malakal Oct. 11 Task 6: Out-brief to USAID Virtual out-brief presentation Oct. 12– Oct. 28 Task 7: Report preparation Data analysis and writing the draft report Oct. 29 Task 8: Submission of the draft report Nov. 11 Task 9: USAID comments on first draft report Nov. 12-20 Task 10: Finalization of the Evaluation Report Redrafting the draft evaluation report based on USAID’s comments Nov. 21 Task 11: Submission of the Final Evaluation Report 4. EVALUATION METHODOLOGIES To conduct this evaluation, the ET proposes to use a mixed-methods approach with complementary qualitative and quantitative data collection methods. The qualitative and quantitative research techniques will be augmented by the use of GIS to map the coverage areas of i-STREAM and community radio networks within the scope of the project. We will answer the main EQ and assess the i-STREAM activity at three levels: 1. Media Landscape and Ecosystem: What is the state of media in South Sudan (2011–2021) and what has been the role and impact of i-STREAM in strengthening and promoting an enabling environment for free and independent media? 2. Media Practitioners: What has been the impact of i-STREAM on building and strengthening the capacity of media professionals—journalists, media managers, and media practitioners? 3. i-STREAM-supported CRS: A Station Performance Assessment of the USAID-supported radio stations based on the key performance indicators of: a) governance, b) programming and audience responsiveness, c) resource and management practices, and d) financial viability. To inform the above questions, we will use the following primary and secondary research techniques, including: 1. Review of project documents and published literature (reports, working papers) to examine the performance of the project against its targets and to assess the state of media in South Sudan and the contribution made by i-STREAM. 34 2. In-depth interviews with key informants including project staff, stakeholders, partners, subgrantees, community leaders, and senior media representatives to yield as wide a range of views and perspectives as is feasible on the state of media in South Sudan and the impact and contribution of i-STREAM’s activities. 3. Online survey (with telephone survey options) among media practitioner beneficiaries to assess the impact of i-STREAM on building and strengthening the capacity of media practitioners. 4. Seven partner SPAs based on a moderated, numeric scoring system of the key performance indicators. This will comprise semi-structured interviews with Owner, MD, Station Manager, Human Resources (HR), Finance, and Editorial Programming and a maximum of five structured interviews with staff members. These in-station assessments will be complemented by third-party audience research data analysis, including an analysis of listeners’ feedback call logs, where available. 5. FGDs and KIIs with audience and community members in the partner station locations. 6. GIS mapping of broadcast media coverage areas. 4.1 Document and Literature Review The ET will carry out a thorough desk review of documents related to i-STREAM provided to the team by USAID, and any other documents that may be provided by Internews, partner non-governmental organizations (NGOs), or other project stakeholders. The key project-related documents will include, but are not limited to, the following: 1 Program Description from the Cooperative Agreement; 2)Quarterly and Annual Reports, including annexes; 3) Performance Management and Evaluation Plan (PMP); 4) 2016 Final Report and Impact Assessment (1 CRS) 5) approved i-STREAM Work Plans; 6) Internews Audience Surveys (2013/2015) 6) CEN and EM detailed budgets and pipeline 7) any other project deliverables (e.g., expert reports, publications, studies, research reports, etc.) produced by partner organizations; and 8) Management Systems International’s Media Sustainability Index reports; and others In addition, the ET will source respected articles, studies, and data on the state of media, journalism, and access to independent information in South Sudan, as well as information on the political, social, and economic contexts in which the project has been operating. Quantitative data from the i-STREAM project will be used to examine project performance since inception in 2013. The quantitative data from PMP, Work Plans, and Quarterly and Annual Reports will be used to compare the number of activities and beneficiaries i-STREAM planned to hold/reach to the reported results. The collective outcome of this task will be a thorough comparative assessment of i-STREAM’s successes in achieving planned benchmarks during the period of implementation and the identification of areas where the project has faced challenges or has had greater successes and exceeded planned quantitative targets. Verification of performance will focus on a review of i-STREAM’s records, which the ET will assume accurately reflect the achievement (or lack thereof) of numeric indicators or benchmarks. Quantitative findings from document review will be cross-referenced with findings from the KIIs which will explores the importance, approach, and results of these interventions with i-STREAM’s partners, beneficiaries, and stakeholders. The ET will examine findings about challenges and successes through qualitative research techniques to understand why i-STREAM had these successes or challenges and their implications for the project and USAID’s continuing support for the project’s partners The secondary data and analysis will be triangulated with the primary quantitative and qualitative data and the collective outcome of the documentation review will be a thorough comparative assessment of i￾STREAM’s successes in achieving planned benchmarks during the implementation period and the identification of areas where the project has faced challenges and successes in meeting its planned targets. 35 There will be a specific focus on the impact the project has had on the health of the media landscape and the media environment in South Sudan. We will summarize the document and literature review in an Excel template and it will form part of our deliverables. 4.2 Interviews with Stakeholders Key Question: What is the state of broadcast media in South Sudan (2011–2021) and what is the contribution, if any, of the i-STREAM activities? Interviews with stakeholders and high-level informants will be semi-structured in nature and designed to allow the interviewees to present and explain points freely. We will conduct these interviews in person, wherever possible, with the option for remote interviewing (most likely by telephone) as preferred or required. A series of stakeholder and high-level informant draft interview (KII) guides focusing on questions, topics, and issues which are relevant to the individual(s) being interviewed with the ability to draw information across interviewees where relevant will be drafted and pre-tested during the briefing and training commencing week of September 6. The KIIs will be conducted by the ET and by ME&A’s partner, Consilient Ltd. (with ET guidance and supervision) with: 1. USAID South Sudan project staff; 2. i-STREAM project staff at Internews; and 3. High-level, respected media practitioners, academics, and experts in Juba. We anticipate that the high-level interviews with senior media figures will be predominately Juba-based. The list of individuals to be included will be informed by USAID/South Sudan, Internews and our local media expert. All those identified for interview will be sent a formal letter of introduction prior to the telephone scheduling of the interview appointments. These interviews will be conducted in person or remotely by telephone by the TL and the local media expert with support from Consilient. We will work closely with USAID/South Sudan and Internews to assemble the appropriate list of the stakeholders, partners, community members, and high-level individuals for interview. Those selected will have first-hand or very detailed knowledge of i-STREAM and the media landscape and ecosystem in South Sudan. We will need the timely support of USAID and Internews to obtain contact details and letters of introduction. The ET will consult with USAID/South Sudan and Internews in the event that a sample selection is required, ensuring that those selected are best placed to answer the questions based on their knowledge and experience with i-STREAM. A formal letter of introduction will be sent to all those selected for interview by the USAID Mission in South Sudan or Internews, ensuring that all of the correct protocols are observed. The letter will explain the purpose of the evaluation, the purpose of the interview, and the interview process. Once the letters are sent, we will implement the following steps: 1. A team member will follow up the letter to schedule a mutually convenient time and place for the interview. 2. The interview will last for no longer than one hour and each interviewee will sign a consent form before the start of the interview. 3. The interview will be conducted in person by a one of the ET members, unless a remote/telephone interview is preferred. 4. The interview will be recorded for analysis purposes only. High-level interviews in Juba will be conducted by the ET and scheduled throughout the data collection period. In our experience, this type of interview can be difficult to schedule and appointments are often 36 missed or changed. We will maintain detailed records of all appointments scheduled, completed, rescheduled, or missed. We will keep the Mission informed of any issues relating to securing interviews with key individuals and endeavor to seek replacement interviews as appropriate. During the week of September 6, the ET will brief and train all data collectors and conduct a modest number of pilot interviews to ensure the questions will be conducted and observed by all team members in pairs. A full day debrief and re-briefing will be conducted by the ET before data collection teams are dispatched to the six locations outside Juba. When scheduling and conducting the interviews, we will adhere to strict “Do No Harm” principles and will be mindful and considerate of participants’ cultural beliefs and status. We will also be sensitive at all times as to how these factors may influence their perceptions of the effectiveness and impact of the i￾STREAM program. All KIIs will be recorded, transcribed, and entered into a template, which will summarize the key findings by topic and theme. Those conducting the interviews will compare notes at the end of each day to determine the quality and completeness of the information collected and ensure that all topics are fully addressed, if not by all interviewees, then by a good cross-section. The interview notes will form the basis of the analysis and will be made available to USAID/South Sudan. Some of the organizations with which we will conduct high-level interviews in Juba are presented below. The preliminary list, with names and contact details, is in Annex 3. Organization Type of Respondent # Ministry of Communications Director General 1 National Media Authority Director 1 Human Rights Watch Local representative 1 Amnesty International Local representative 1 Media Development Institute Executive Director Principle and Board Member 2 School of Journalism (Juba) Dean and Deputy Dean 2 Committee to Protect Journalists Local representative 1 International and local academics and experts Oxfam/Community Empowerment for Progress Organization (CEPO)/Southern Sudan AIDS Commission (SSAC)/CPDS 3-4 Editors of newspapers Juba Monitor 1 Association of Women’s Media in South Sudan/Chair of SS Broadcast Corporation Director 1 Union of Journalists South Sudan Head/ Deputy 1 Journalists for Human Rights Program Officer 1 Target Media (advertisers) 2 Media analysts/Monitoring and Evaluation Support Project (MSEP) 2 4.3 Online Survey to Evaluate i-STREAM’s Capacity-Building Activities To evaluate i-STREAM’s effectiveness in building and strengthening the capacity and professional skills of media practitioners, we will conduct an online survey among those who have attended training courses/sessions and received mentorship opportunities as part of i-STREAM’s professional capacity￾building program. 37 The survey instrument, which will be fully structured, will be scripted into SurveyMonkey, and administered in English. All of the training conducted by Internews took place in English and, thus, the beneficiaries will be proficient in the language. Beneficiaries will include practicing journalists, those who support journalists, media managers, and media house staff. The list of survey recipients will be sourced from Internews’ database. The ET will send a link to a web-based survey containing no more than 30 fully structured questions to all those who have been trained/mentored by Internews. We understand that getting people in South Sudan to fill out a web-survey is difficult. Therefore, we will ask Internews to introduce our team and let the survey recipients know about the survey. We will send up to three reminders to complete and return the survey. We will constantly monitor the response rate and, where there are shortfalls, we will pass the beneficiary details to Consilient to supplement the survey with telephone interviews. At the time of drafting this work plan, we do not have a full list of beneficiaries from Internews; therefore, it is not possible to estimate the level of online response or the number of telephone interviews that will need to be conducted. However, we will work towards achieving an overall response rate of 30-40 percent. The online survey will not evaluate the training programs and workshops per se, but rather focus more on the effects and limitations/impediments to the implementation and application of the training content with concrete examples. Key question areas will focus on: • Relevance of the training and capacity-building content to their work and the contexts within which they work; • Opportunities and barriers to applying the “learnings” in their everyday work and the adoption of new practices and techniques (e.g., fact checking) with specific examples; • Cascading the learnings to colleagues; and • Assessment of the training content and methods, and suggestions and recommendations for future training and capacity-building activities. Data generated by the survey will be thoroughly checked and cleaned by ME&A HO and be uploaded into SPSS for analysis purposes. The data analysis will be quantitative in nature and the output will be in the form of cross tabulations and charts, disaggregated where possible by gender, age, location, and job￾function. The level and units of analysis will be determined by the achieved sample size and profile. 4.4 Station Performance Assessments Key question: What is the performance of the USAID-supported CRS in terms of governance, programming and audience responsiveness, resource and management practices, and financial viability? Data that will inform this key EQ will be drawn from a combination of primary data collection and observation approaches: 1. In-person, in-depth interviews with key personnel at each station based on a SPA tool, which will be designed to cover the key performance indicators (governance, programming, audience responsiveness, resource and management practices, and financial viability). The greatest challenge the ET will face will be the transparent disclosure of financial and other potentially sensitive commercial information. We are considering a number of options to overcome this challenge and seek the assistance of USAID and Internews to: • Contact the owners/MDs/Senior Managers of the seven stations in writing and in person to introduce them to the evaluation and seek their cooperation and the participation of their key staff in programming, finance, HR, station management, news, and other departments, as appropriate. 38 • Consider the uploading of sensitive financial information on income, revenue, costs, salary scales and financial forecasts to a secure portal accessible only by the ET, with signed non-disclosure agreements (NDAs). 2. Modest number of in-station staff interviews using a fully structured questionnaire focusing on working practices, capacity and equipment 3. Analysis of third-party audience measurement data to inform the reach and profile of the CRS. 4. FGDs with audience members and analysis of “listener logs.” 5. KIIs with a maximum of five of community leaders, including local government officials and leaders of women’s and youth groups. 6. Primary data from the field team and, as available, i-Stream, CRS and other network data, will be integrated in a GIS. ME&A will provide two geospatial deliverables: • Using the best data available, we will map the areal coverage and listenership of selected i-STREAM and community radio networks together with population information and media outlet ownership. This map will provide insight into where there is over- and under-concentration of broadcast media outlets (state and independent). It will also serve as a tool to promote sensitivity to the needs of different communities. • An interactive geospatial dashboard that will allow the Mission to generate scenarios for rolling back tower funding. This can be accomplished by interacting with and applying weights to data gathered from: a) steps 1) through 5) and b) data sets from i-Stream and other station networks. The dashboard will also allow visualization of data such as language, station purpose, e.g. refugee assistance, entertainment, etc. SPAs will be conducted in person, wherever possible, by a team of two experienced and trained assessors. We envisage conducting a total of seven SPAs, in the following stations: • Eye Radio (Juba) • Mayardit FM (Turalei) • Mingkman FM (Mingkman) • Akol Yam (Aweil) • Kondial (Bentiu) • Singaita FM (Kapoeta) • Nile FM (Malakal) Key Informant Interviews We will conduct up to five high-level interviews in each CRS with the key personnel (e.g., Station MD, Head of Finance, and Head of Editorial/News). These senior staff are all Juba-based where the in-person interviews will be conducted. In the location of each station up to five local stakeholder KIIs (e.g., local elder, women’s group leader, and youth leader); and four FGDs among audience members will be conducted. Station Performance Assessment Our Performance Assessment Tool will allow for transparent assessment and comparisons between the supported CRS to provide USAID with evidence upon which to base decisions about future funding. The tool will be designed to provide objective scores of each CRS on the key indicators, which will be collected by our qualitative research experts through interviews with management and staff. Indicative areas that SPA will cover include, but are not limited to: 39 1. Governance in terms of ownership, codes of conduct, board meeting frequency and management, annual work plan, appointment of board and management teams, vision and mission strategy, and policies of inclusion (maximum score 25 out of 100). 2. Programming and audience responsiveness to cover audit of the output (news, information, entertainment, humanitarian programming, programming for women and youth, etc.), program review process, news and editorial policies, capacity of journalists and editors, audience reach and the extent to which the information needs of the community are being served (maximum score 25 out of 100). 3. Resource and Management Practices: Station management documentation, HR policies, management structures, plans and regular meetings, resource plan and reviews, accounting procedures, and reporting to management and board (maximum score 25 out of 100). 4. Financial Viability: Sources of income and expenditure, potential sources of income, sustainability of income streams, scenario planning in the event of lost income, and contingency planning (maximum score 25 out of 100). Third-Party Audience Research Surveys The evaluation data and scores will be augmented by third-party audience reach and profile data tracked over time (2013–2021 provided by Internews. The third-party audience research surveys will be analyzed for trends over time in: • Audience reach; • Competitive positioning; • Audience profile; • Loyalty; • Listening by daypart; and • Program performance. Focus Group Discussions We will convene four FGDs among groups of audience members who will be directed to listen to the station and its competitors before attending the groups. The focus group topic guide will cover the following key topics: • Comparisons between the stations in terms of their news and information coverage and priority content areas; • Opinions of the trustworthiness/reliability of the coverage; • Opinions of the range of topics covered (including humanitarian issues, women’s issues, and other relevant local issues); • Opinions of the style of presentation and format; and • Relevance of program and news topics. Focus group respondents will be pre-screened and recruited using a screening questionnaire, which will include questions about radio stations listened to. This will ensure that their views, opinions and comments about the stations and their content are based on personal experience. We will recruit 8-10 audience members per group based on the following criteria: • Frequency of listening to the station: We will recruit two groups of regular listeners and two groups of occasional listeners • Gender: Within each listening category, we will convene one group of men and one of women. 40 • Age: All participants will be between 20 and 40 years old. The groups will be held in a safe, secure, and neutral location. They will be moderated in the local language by a fully trained Consilient moderator who will record the conversations and summarize the notes in an Excel template for analysis purposes. Where possible, we will match the gender of the moderator with the gender profile of the group participants. Geospatial Information Systems Mapping ME&A will provide the two following deliverables: a. Subject to the above-mentioned privacy and security concerns, data collected from the IP as well as other secondary sources will be cross-referenced with other service provider coverage areas. These may be primary or secondary-sourced coverage data. As described in 4.4, data will be integrated into a GIS and analyzed to generate a physical map of the i-STREAM and CRS coverage area that incorporates a range of community data, tower performance, and tower coverage data. b. As specified above, the GIS will allow design of a network configuration that is sensitive to the needs of different communities. Therefore, we will also develop an interactive dashboard, rather than a fixed map, that is able to reflect the richness of the community data. This will be integrated with available tower performance and tower coverage data. The dashboard will allow USAID to apply weightings to the provided data and make spatially-informed decisions about optimal network configurations. The dashboard will serve as a decision support tool for the Mission and is meant to supplement the static report ME&A will deliver. • Data Requirements: We request that the Mission facilitate access to the i-STREAM records, including locational data, broadcast strength and specific tower data, ad revenue, locational data, and listenership data where available. The United Nations (UN) Mission in Juba has conducted extensive studies on radio coverage in South Sudan. We will reach out to them for coverage data, but it would likely be efficient if the Mission could facilitate this. Note, concerning licensing and ownership of data, all products generated, software licenses, and data will become property of USAID on completion of the project. 5. EVALUATION LIMITATIONS AND MITIGATIONS There are several limitations inherent to the design of this evaluation. 1. Selection bias: As some key informants may decline to be interviewed, there is a possibility of selection bias, i.e., those respondents who choose to be interviewed might differ from those who do not in terms of their attitudes and perceptions, affiliation with government/non-government structures, and socio-demographic characteristics and experience. 2. Recall bias: Since some interview questions will deal with issues that took place in the past, recall bias cannot be excluded. As i-STREAM activities were launched in 2013, some respondents may find it difficult to accurately compare organizational arrangements/access to services from seven or eight years prior to now. 3. Halo bias: There is a known tendency among respondents to underreport socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents will be prepared to reveal their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or people upon whom they depend for provision of services. To mitigate this limitation, 41 we will: provide the respondents with confidentiality and anonymity guarantees, where possible; conduct the interviews in the settings where respondents feel comfortable; and establish rapport between the interviewer and the respondent. 4. Response bias: Another possible limitation may include low response rates in one-on-one interviews. If this is the case, the results will represent the opinions of the most active and open respondents whose experience with the programs is likely to differ from that of those who decline to be interviewed. To address the above limitations, the ET will: 1. Seek to minimize response bias by interviewing as representative stakeholders and beneficiaries as possible across all key stakeholder groups given resource, time, and other constraints. 2. Guarantee all key informants’ anonymity to facilitate more open and honest discussion, and more forthright responses. 3. Triangulate evidence from different qualitative and quantitative data sources, which serves to increase the credibility of findings via validation by multiple data sources. 4. Collaborate closely with USAID/South Sudan to facilitate introductions and mobilization of partners, beneficiaries, and stakeholders. Mobilization helps ensure that partners, beneficiaries, and stakeholders are willing to participate and are well informed of the purposes of the evaluation. 5. Hold KIIs and FGDs in venues where participants feel safe and are comfortable. 6. Manage group dynamics by, among other things, organizing FGDs that are as homogenous as possible in sex, age, and social status. 7. Focus on the most robust findings—findings that appear with relatively more frequency across multiple stakeholders—for qualitative data analysis. 8. Conduct systemic data analysis of the quantitative data from the audience research, and any secondary statistical data using well-established statistical methods. 6. QUALITY ASSURANCE APPROACH We will apply rigorous quality control methods and procedures for each of the evaluation’s tasks and deliverables. They include: • Conducting a team planning meeting to assign clear roles and responsibilities of each ET member followed by a brainstorming session on the design, document reviews, data collection, selection of interviewees, standardization of interview guides, and application of interviewing protocols to safeguard data and information accuracy and consistency. The ET, together with the HO task order manager will work jointly to develop data collection methods and instruments and a clear plan on how to: 1) fill data gaps in case some of the interviewees cannot be reached; 2) address data errors that might arise from the data collection methods; 3) mitigate assessment limitations; and 4) review deliverables against standards to ensure that the final report meets USAID’s requirements. • Providing the data collection team with training on what is expected, how to ensure quality, how to protect interviewees’ privacy and confidentiality, and how to consider interviewees’ culture, ethnicity, religion, and sex. 42 • Developing a set of protocols to ensure the compliance of all evaluation and assessment activities with the ethical requirements governing human subject research. People in South Sudan have been traumatized by the extended conflicts and violence in the country. Realizing this and also that men and women deal differently with trauma, we will tailor our questions accordingly. • Requiring that the team remains in constant communication with the Contracting Officer’s Representative (COR) and ME&A HO to ensure that all issues that might arise during the evaluation are resolved quickly and positively, without causing disruption. • Creating a dedicated Google Drive in Excel to perform virtual data quality checks daily and provide timely and ongoing feedback. Field teams will be responsible for executing travel plans and reporting data collected to the Consilient project manager. Teams will also provide check-in calls to monitor and report on the current security status of their location to Consilient’s fieldwork coordinator, who will be responsible for field security management. The fieldwork coordinator will be responsible for employing a wide network of personal, professional, and governmental contacts regarding the security and accessibility of each selected area and producing synthesized risk analysis reports on a weekly basis. The fieldwork coordinator will also be responsible for verifying the details of any incident, reporting incidents to ME&A and USAID as per the security plan. 43 ANNEXES 44 Annex 1: Schedule of Work August 2021 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 8 9 10 11 Contract Awarded 12 Doc review/ Analysis design/ Prepare Work Plan 13 Doc review/ Analysis design/Prepare Work Plan 14 15 16 Doc review/ Analysis design/Prepare Work Plan 17 Doc review/ Analysis design/Prepare Work Plan 18 Doc review/ Analysis design/Prepare Work Plan 19 Kick-off conference call with USAID 20 Doc review/ Analysis design/Prepare Work Plan 21 22 23 Doc review/ Analysis design/Prepare Work Plan 24 Doc review/ Analysis design/Prepare Work Plan 25 Doc review/ Analysis design/Prepare Work Plan 26 Doc review/ Analysis design/ Prepare Work Plan 27 Virtual In-Brief Presentation of draft Work Plan with USAID 28 29 30 Finalize Work Plan and data tools 31 Finalize Work Plan and data tools 45 September 2021 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Finalize Work Plan and data tools 2 Submit final workplan 3 Workplan approved by USAID 4 TL Travels to South Sudan 5 6 Train fieldwork team Virtual KIIs (Vivien) F2F (Anthony) 7 Train fieldwork team Virtual KIIs (Vivien) F2F (Anthony) 8 Train fieldwork team Virtual KIIs (Vivien) F2F (Anthony) 9 Field team leaves for field work Virtual KIIs (Vivien) F2F Anthony 10 Virtual KIIs (Vivien) F2F Anthony 11 Data Collection 12 13 Data collection 14 Data collection 15 Data collection 16 Data collection 17 Data collection 18 Data collection 19 20 Data collection 21 Data collection 22 Data collection 23 Data collection 24 Data collection 25 Data collection 26 27 Data collection 28 Data collection 29 Data collection 30 Data collection 46 October 2021 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Data collection 2 TL departs Data collection 3 4 Data collection 5 Data collection and preparation of Out-Brief Presentation 6 Data collection and preparation of Out￾Brief Presentation 7 Data collection and preparation of Out-Brief Presentation 8 Data collection and preparation of Out-Brief Presentation 9 Data collection and preparation of Out-Brief Presentation 10 11 Out-Brief with USAID 12 Data analysis and report writing 13 Data analysis and report writing 14 Data analysis and report writing 15 Data analysis and report writing 16 17 18 Data analysis and report writing 19 Data analysis and report writing 20 Data analysis and report writing 21 Data analysis and report writing 22 Data analysis and report writing 23 24 25 Data analysis and report writing 26 Data analysis and report writing 27 Data analysis and report writing 28 Data analysis and report writing 29 Submit draft Evaluation Report 30 47 November 2021 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 31 1 2 3 4 5 6 7 8 8 9 10 11 Receive comments from USAID 12 13 14 Incorporate USAID comments 15 Incorporate USAID comments 16 Incorporate USAID comments 17 Incorporate USAID comments 18 Incorporate USAID comments 19 20 21 Submit final Evaluation Report 22 23 24 25 26 27 28 29 30 48 Annex 2: Evaluation Design Matrix i-STREAM Evaluation Design Matrix EQ with Illustrative Research Questions Data Sources Methodologies Data Analysis OBJECTIVE: To strengthen autonomous, journalistic professionalism and facilitate the development of an institutionally well-managed and financially self-sustained independent media in South Sudan EQ 1: To what extent has the objective regarding developing an institutionally well-managed and financially sustained, independent radio sector in South Sudan been achieved under i-STREAM? Key Indicators and Research Questions: ● What is the current status of the media ecosystem in South Sudan and what has been the contribution of i-STREAM in terms of the enactment of media and regulatory legislation, strengthening of media institutions, and improving the enabling environment for a free and independent media? ● Do the laws guarantee editorial independence and are they applied equally to state and independent media and how, if at all, has i-STREAM contributed to this? ● What is the status of crimes against journalists and media outlets and what, if any, has been the contribution/ intervention of i-STREAM in countering the situation? ● How has i-STREAM improved the management, professional, and technical capacity of its partner stations? ● How do journalists source, check, and produce news and information and what are the standards of their professional ethics? ● Activity documentation (narrative report, monitoring and evaluation [M&E] plan, results framework, etc.) ● USAID ● Internews ● Government officials ● Journalists & media specialists ● Media organizations ● Other projects & donors ● Secondary data ● Audiences ● Desk review ● KIIs ● Review of secondary performance data ● Review of secondary contextual data ● Station performance Assessments (7 SPAs) comprising KIIs, observation, FGDs, and audience analysis ● Web-based survey of beneficiaries ● Analysis of i-STREAM activity planning and implementation ● Analysis of the “state of media” from literature and interviews ● Station assessments ● KIIs ● Audience data analysis ● FGDs ● SPSS based analysis of web-based beneficiary survey 49 ● What is the state of technical facilities for gathering, producing, and distributing news and information ? ● What is the state of news? ● Has i-STREAM contributed to citizens’ access to a diverse range of political views? ● To what extent is the partner station output providing quality, independent, and relevant information to audiences? ● How many listeners are there and who listens? ● What are the revenue and income opportunities for the partner stations and how can these be optimized? ● How viable are the partner stations as independent entities? ● How did i -STREAM activities work to strengthen the diversity and professionalism of voices in the media? Are there effects in terms of increasing citizens’ access to civically relevant information? ● How did i -STREAM activities work to improve the legal and policy environment for media? Are there effects in terms of increasing citizens’ access to civically relevant information? 50 Annex 3: Juba-Based High-Level Interviews – Prospective Interviewee List Sn Organization Name Email Address/Phone Number 1 High Level I The following will require letters of introductions 2 Media Authority Sapana Abuyi +211920524001 3 Media Authority Elija Alier +211922277711 4 BBC World Services Nichola Mandil Ukeil +211922994000 5 Food and Agriculture Organization Tanya Birkbeck Tanya.Birkbeck@fao.org 6 International Organization for Migration Agany Isaiah Majak achuol@iom.int 7 United Nations Interim Security Force for Abyei Mwila Kamwela kamwela@un.org 8 University of Juba - Administration Prof. Mairi John Blackings mairijb@yahoo.co.uk 9 Juba Monitor Newspaper Anna Nimriano +211911538294 10 Target Media Advertisement Denis Logonyo +211926152738 11 Union of Journalists of South Sudan Patrick Oyet +211925440019 12 Okay Africa Foundation (Activist) Wani Michael +211915533833 13 University of Juba – School of Journalism Dominic Kango +211917116351 14 Parliament/University of Juba – School of Journalism Hon. Ichaa Joice +211921722078 15 University of Juba – School of Journalism/Eye Radio/ SSBC Prof. Kara Chaplain Yokoju yokojukara@gmail.com 16 Human Rights Observatory Godfrey Victor Bulla +211926550005 High Level II No letter of introductions required 17 AMDISS/MDI/Media Authority Ayaa Irene ayaa.irene@amdiss.org 18 Catholic Radio Network/SSBC Mary Ajith crn.director@gmail.com 19 AMDISS/MDI Michael Duku Aggrey mduku@amdiss.org 20 USAID Peter Khaemba pkhaemba@usaid.gov 21 USAID Warran Bully wabully@usaid.gov 22 USAID Alexis McGuines amcginness@usaid.gov 23 USAID Victor Lako vlako@usaid.gov 24 Internews - COP Ferhanaz Kermali fkermalli@internews.org 25 Internews – Sustainability Director Omnia Shawkat omnia.shawkat@gmail.com 26 Internews - MEL ??? ??? 27 Internews - Finance Linet Omwange linet.omwange@INTERNEWS.OR G Partners/Stakeholders No letters of introduction required. 28 Eye Media Stephen Omiri stephen.omiri@eyeradio.org 29 Eye Media Koang Pal Chang Koang.Chang@eyeradio.org 30 Eye Media James Kwaje James.Kwaje@eyeradio.org 31 Eye Media Saphana Khemis +211923206254 32 Eye Media Jacob Aluong +211912762529 33 Eye Media Daniel Denis +211922473307 51 34 Abyei Information and Radio Service (AIRS) Karbino Dut kdut@airs-ss.org Juba-Based Station Management Stakeholders 35 Community Engagement Network (CEN) Robert Wuda Elikana Robert.Wuda@cen-ss.org 36 CEN Julian Levi Julian.Levi@cen-ss.org 37 CEN Anna Yawa Anna.Yawa@cen-ss.org 38 CEN Rehema Siama Rehema.siama@cen-ss.org Field-Based Station Stakeholders 39 CEN Kondial FM Puot Panyuan Panyaun.puot@cen-ss.org 40 CEN Kondial FM Chondok Stephen Stephen.chondok@cen-ss.org 41 CEN Kondial FM Tabitha Gatdor Nen +211922554224 42 CEN Nile FM Chol Dau Nyok Chol.Dau@cen-ss.org 43 CEN Nile FM Stephen Anter Bol Stepehen.Anter@cen-ss.org 44 CEN Nile FM Lorince Nyawela +211914556246 Juba-Based Station Management Stakeholders 45 The Radio Community (TRC) Chris Marol (CEO) chrismarol@radiocommunity.org 46 TRC Stephen Lemmy slemmy@theradiocommunity.org 47 TRC Nelson Lokiri Abusai nabusai@radiocommunity.org Field-Based Station Stakeholders 48 TRC Adrupio Agnes aagnes@theradiocommunity.org 49 TRC Angelina Achol aachol@theradiocommunity.org 50 TRC Dominic Santo Atem satem@theradiocommunity.org 51 TRC Aguer Atem aatem@theradiocommunity.org 52 TRC Lobeye Everlyne elobeye@theradiocommunity.org Organizations Receiving Training from i-STREAM 53 Voice of Reconciliation Manyang David Mayar Manyang.davidmayar@gmail.com 54 Foundation for Youth Initiative Albino Gaw Dar albino@fyisouthsudan.org 55 Samaritan FM Denis Morris Mimbugbe denis.samafm@gmail.com 56 defyhatenow Nelson Kwaje nelson@defyhatenow.org 57 GoGirls ICT Initiative Yine Yenki Nyika yineyenki@gogirlsict.org 58 Pochalla FM Othow Worokoti worokoti@yahoo.com 59 Voice of Freedom Ayela Anthony paolgayaa@gmail.com 60 Active Youth Agency (AYA) Repent Woroh Odrande info@ayasouthsudan.org / labadia@ayasouthsudan.org 61 Kapital Movie Industry Corporation Stephen Lagu lagu@kapitalmovie.net 52 Annex 4: Day by Day Ex-Juba Field Plan Date Activities Juba Turalei Mingkman Aweil Benitu Kapoeta Malakal Notes Training Phase Sept. 6 Training Day 1: Project Info, Data Collection Plan, Security and Research Ethics, Tool Review Sept. 7 Training Day 2: “Pilot” data collection in Juba Sept. 8 Training Day 3: Review pilot data, make final adjustments of tools, finalize fieldwork preparations Travel and Permissions Phase There will be some variability in the availability of flights, and there is a good chance the evaluation will actually start a little slower than this schedule anticipates. As it gets closer to the day, we will have the actual departure dates for ET members and can adjust, as needed. Sept. 9 Flight to Wau Drive to Mingkman Fly from Juba to Aweil Fly from Juba to Rubkona Fly from Juba to Kapoeta Fly from Juba to Malakal Sept. 10 Drive to Turalei Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Drive to Benitu Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Sept. 11 Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Securing permission, Recruit Facilitator Sept. 12 Off Day Radio Station Assessments Phase This will be dependent on the key radio station personnel being available Sept. 13 Finalizing permissions and making arrangements and introduction to radio station Finalizing permissions and making arrangements and introduction to radio station Finalizing permissions and making arrangements and introduction to radio station Finalizing permissions and making arrangements and introduction to radio station Finalizing permissions and making arrangements and introduction to radio station Finalizing permissions and making arrangements and introduction to radio station 53 Date Activities Juba Turalei Mingkman Aweil Benitu Kapoeta Malakal Sept. 14 Radio Station Staff interviews Radio Station Staff interviews Radio Station Staff interviews Radio Station Staff interviews Radio Station Staff interviews Radio Station Staff interviews Sept. 15 Finalizing SPAs and ensuring data submission Finalizing SPAs and ensuring data submission Finalizing SPAs and ensuring data submission Finalizing SPAs and ensuring data submission Finalizing SPAs and ensuring data submission Finalizing SPAs and ensuring data submission Focus Group Discussion Phase While they will be organizing and collecting FGDs over the course of four days, all researchers will be required to submit all notes no later than 24 hours after completing an interview. Sept. 16 Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Sept. 17 Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Sept. 18 Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Sept. 19 Off Day Sept. 20 Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Organizing and conducting FGDs Sept. 21 Finalizing FGDs and ensuring all FGD notes are submitted Finalizing FGDs and ensuring all FGD notes are submitted Finalizing FGDs and ensuring all FGD notes are submitted Finalizing FGDs and ensuring all FGD notes are submitted Finalizing FGDs and ensuring all FGD notes are submitted Finalizing FGDs and ensuring all FGD notes are submitted Sept. 22 Halt Day – When possible, building in halt days is important to ensure that the quality control teams are able to be caught up on reviewing data and providing feedback. This also means that we are building in time for any teams that have had delays or network issues to have a dedicated day to catch up to the team, or to make sure they are uploading data. 54 Date Activities Juba Turalei Mingkman Aweil Benitu Kapoeta Malakal Researchers will have to submit notes for interviews within 24 hours of them being conducted. If they needed to do two in a day, then researchers will not continue conducting interviews until the notes have been sent. Sept. 23 Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Sept. 24 Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Sept. 25 Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Sept. 26 Off Day Sept. 27 Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Sept. 28 Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Organizing and conducting KIIs Sept. 29 Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Sept. 30 Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Finalizing KIIs and ensuring that all notes have been submitted Oct. 1 Halt Day – Researchers will not leave the location until we are able to ensure that all the notes received are approved and we are sure that no recontacts or replacement interveiws are needed. 55 Date Activities Juba Turalei Mingkman Aweil Benitu Kapoeta Malakal These are built in days that will help to accommodate any delays or fieldwork issues. It should be noted that the researchers will not know these days are built in. Researchers will be told that they need to be finished by October 1. Oct. 2 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 3 Off Day Oct. 4 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 5 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 6 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 7 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 8 Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Travel day (if approved) or recontacting/ redoing interviews Oct. 9 Travel day (if approved) or recontacting/ Travel day (if approved) or recontacting/ Travel day (if approved) or recontacting/ Travel day (if approved) or recontacting/ Travel day (if approved) or recontacting/ Travel day (if approved) or recontacting/ 56 Date Activities Juba Turalei Mingkman Aweil Benitu Kapoeta Malakal redoing interviews redoing interviews redoing interviews redoing interviews redoing interviews redoing interviews Some overall notes 1. Given our plan of one interview and the corresponding notes for that single interview each day, there is time built in for organizing subsequent interviews and taking care of other logistical issues. 2. This plan assumes regular feedback from the team. We have built in some time, but this plan falls apart if there are long delays in our feedback and then researchers need to address a backlog of issues. Since we should only be getting approximately six sets of notes per day, this plan assumes that the ET members are getting feedback sometime the following day about the notes they submitted the day prior. 3. There is likely to be at least one location where data collection runs into October because the ET were not able to catch a flight for more than a week. 57 ANNEX 3: HIGH-LEVEL KII STAKEHOLDERS INTERVIEWED Organization Position Name Gender Media Authority Managing Director 1. Elija Alier 2. Saphana Abuyi 3. John Dau M AMDISS/MDI/Media Authority Principal Ayaa Irene F Catholic Radio Network/SSBC Director Mary Ajith F AMDISS/MDI Executive Director Michael Duku Aggrey M Amnesty International Researcher Nicki Franken F University of Juba – School of Journalism Dean Dr. Rebecca Lorins F Juba Monitor Newspaper Chief Editor Anna Nimriano F Target Media Advertisement Managing Director June Flexi M Union of Journalists of South Sudan Chairperson Patrick Oyet M OXFAM Head of Communications Dominic Kango M University of Juba/SSBC Deputy Dean Prof. Kara Chaplain M Human Rights Observatory Lawyer 1. Godfrey Victor Bulla 2. Bullen Yatta M BBC World South Sudan Correspondent Nichola Mandil Ukeil M Association Media Women in South Sudan (AMWISS) Presenter Helen Ikose (Mama Helen) F US Embassy Education Office Jane Namade F US Embassy Press Officer Henry Jada M Defy Hate Now Director Marina Modi F Go Girls Director Yine Yenki F Foundation for Youth Initiative Director Albino Gau M Advanced Youth Radio Director Mawa Nemaya M 58 ANNEX 4: SUMMARY OF COMMUNITY LEADER INFORMANTS Community Leader Key Informants (Total=30) 23 males/ 7 females Aweil Bentiu Kapoeta Malakal Mingkaman Turalei Sub-Chief (M) Block Leader (M) Youth Leader in the Union of Youth Association in Kapoeta (M) Director of the Upper Nile Youth Association (M) Head Chief (M) Community Association Chairman (M) Youth￾Chairman (M) Youth Chairman (M) Chief/Sheik (M) Malakal Town camp coordinator (M) Community Elder (M) Executive Chief (M) Chairlady of the Women’s Resources Centre (F) Sector Two Leader (F) Deacon of the Parish of Our Sorrowful Lady (M) Deputy Chief (M) Youth Chairman of the Youth Association in Awerial County (M) Acting Parish Priest (M) Bishop – True Church of God (M) Community Chairman of Sector 4 IDP Camp (M) Head of Toposa Community (M) Catholic Bishop for the Greater Upper Nile (M) Chairlady of the Awerial Women’s Association (F) Chairlady of the County Women’s Association (F) Community Elder (M) Chairlady Community High Committee (F) Head of the Union of Women Association in Kapoeta (F) Representative for the Upper Nile Woman’s Association (F) Bishop of the Episcopal Church, Diocese of Awerial County (M) Chairperson of the Youth Association (M) 59 ANNEX 5: FGD PARTICIPANT COMPOSITION Characteristics of FGD Participants: (Total= 263) 133 Males/ 130 Females Location Males – 18-30 Males 31-50 Females 18 – 30 Females 31-50 Aweil Total: 35 Males: 18 Females: 17 Participants: 10 Mean Age: 25.3 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 8 Mean Age: 44.0 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 9 Mean Age: 21.6 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 8 Mean Age: 41.3 Language Spoken by Participants: Classical Arabic, Dinka, English Bentiu Total: 36 Males: 18 Females: 18 Participants: 10 Mean Age: 26.2 Language(s) Spoken by Participants: Nuer Participants: 8 Mean Age: 43.1 Language(s) Spoken by Participants: Nuer Participants: 8 Mean Age: 22.4 Language(s) Spoken by Participants: Nuer Participants: 10 Mean Age: 40.1 Language(s) Spoken by Participants: Nuer Kapoeta Total: 33 Males: 17 Females: 16 Participants: 8 Mean Age: 25.5 Language(s) Spoken by Participants: Juba Arabic, Taposa, Dindinga Participants: 9 Mean Age: 39.3 Language(s) Spoken by Participants: Juba Arabic, Taposa, Dindinga Participants: 8 Mean Age: 23.4 Language(s) Spoken by Participants: Juba Arabic, Taposa, Dindinga Participants: 8 Mean Age: 37.6 Language Spoken Language(s) Spoken by Participants: Juba Arabic, Taposa, Dindinga Malakal Total: 40 Males: 20 Females: 20 Participants: 10 Mean Age: 27.0 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 42.1 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 25.1 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 35.3 Language Spoken by Participants: Classical Arabic, Dinka, English Mingkaman Total: 40 Males: 20 Females: 20 Participants: 10 Mean Age: 26.2 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 41.7 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 25.0 Language(s) Spoken by Participants: Classical Arabic, Dinka, English Participants: 10 Mean Age: 38.0 Language Spoken by Participants: Classical Arabic, Dinka, English Turalei Total: 39 Participants: 10 Mean Age: 24.1 Participants: 10 Mean Age: 36.2 Participants: 9 Mean Age: 21.9 Participants: 10 Mean Age: 33.6 60 Characteristics of FGD Participants: (Total= 263) 133 Males/ 130 Females Location Males – 18-30 Males 31-50 Females 18 – 30 Females 31-50 Males: 20 Females: 19 Language(s) Spoken by Participants: DInka Language(s) Spoken by Participants: Dinka Language(s) Spoken by Participants: Dinka Language Spoken by Participants: Dinka Juba Total: 40 Males: 20 Females 20 Participants: 10 Mean Age: 44.1 Language(s) Spoken by Participants: Juba Arabic, English Participants: 10 Mean Age: 24.1 Language(s) Spoken by Participants: Juba Arabic, English Participants: 10 Mean Age: 20.5 Language(s) Spoken by Participants: Juba Arabic, English Participants: 10 Mean Age: 34.7 Language(s) Spoken by Participants: Juba Arabic, English 61 ANNEX 6: STATION STAFF INTERVIEWS Station Staff Interviews: Total 26 Aweil – Akol Yam FM Team Leader and News Program Editor Broadcast Journalist Broadcast Journalist/Producer Broadcast Journalist Bentiu – Kondial FM Team Leader Community Correspondent Community Correspondent Community Correspondent Kapoeta Saigaita FM Team Leader/ News and Programme Editor Broadcast Journalist Broadcast Journalist Trainee Broadcast Journalist Trainee Juba – Eye Radio Senior news producer Commercial/Sales Exec Digital media Intern Malakal – Nile FM Team Leader Community Correspondent Community Correspondent Community Correspondent Mingkaman – Mingkaman FM Team Leader/ News and Program Editor Broadcast Journalist Broadcast Journalist Turalei – Mayardit FM Team Leader/Station Manager Broadcast Journalist Broadcast Journalist Broadcast Journalist 62 ANNEX 7: DATA COLLECTION TOOLS 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110