MID-TERM PERFORMANCE EVALUATION OF THE ACCESS TO INFORMATION (A2I) PROJECT IN KAZAKHSTAN AND TAJIKISTAN MID-TERM PERFORMANCE EVALUATION OF THE ACCESS TO INFORMATION (A2I) PROJECT IN KAZAKHSTAN AND TAJIKISTAN This publication was produced at the request of the United States Agency for International Development Developed under Contract/Task Order Number: LEAD 720011518D00003 / 72011518F00003. It was prepared independently by: Lawrence Robertson, Team Leader Gulmira Shalabekova, Technical Specialist Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 301-652-4334 www.MEandAHQ.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The purpose of the mid-term performance evaluation of the Access to Information (A2I) project, funded by USAID/Central Asia and implemented by Internews Network (Internews) in Kazakhstan and Tajikistan, was to: 1) examine the progress of A2I’s activities; 2) evaluate the prospect for achieving life-of-project results; and 3) identify if any changes need to take place to ensure achievement of those results. To conduct the evaluation, the evaluation team (ET) used a mixed methods approach, which included document review, key informant interviews, e-mail and telephone surveys, and focus group discussions. The ET visited 21 media outlets in Kazakhstan and nine outlets in Tajikistan that benefitted from A2I’s support. The ET found that the project has largely met its four objectives, with few gaps in the project; its design and structure are largely appropriate; implementation performance by USAID and Internews was strong; and that implementation of A2I as a regional program had advantages, but also some disadvantages. The ET also found that some indicators used by the project are not useful for understanding progress. The ET recommends that the project continue to support journalists and other media actors to report objective information; work with partner outlets on ways to better share different types of content across different platforms as well as on ways to monetize this sharing for competitiveness; increase its support to media outlets so they meet the challenge of the digital transition; and revise its indicators to more completely tell the story of A2I’s contributions. CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... i Evaluation Purpose............................................................................................................................ i Project Background........................................................................................................................... i Evaluation Design, Methods, And Limitations.................................................................................. i Main Findings, Conclusions, And Recommendations...................................................................... ii 1.0 EVALUATION BACKGROUND, PURPOSE, AND QUESTIONS ........................................................ 1 1.1 Evaluation Purpose.............................................................................................................. 1 1.2 Evaluation Questions.......................................................................................................... 1 2.0 PROJECT BACKGROUND ............................................................................................................................. 2 3.0 EVALUATION METHODS AND LIMITATIONS....................................................................................... 3 3.1 Evaluation Methodology...................................................................................................... 3 3.2 Evaluation Limitations......................................................................................................... 5 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS................................................................ 7 4.1 EQ1: To what degree has Internews met the four objectives as stated in the cooperative agreement?...................................................................................................... 7 4.2 EQ2: What gaps in addressing the program objectives still exist? And to the extent possible, describe why/how gaps exist. ............................................................................21 4.3 EQ3: Going forward, how, and in which areas, should USAID-funded media activities be conducted to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan?.................................................23 4.4 EQ4: Given the media environment in Central Asia and goal of the program, was the design and structure of the activity appropriate to successfully achieving the goal?.......26 4.5 EQ5: To what extent were the strengths and weaknesses related to performance of the implementer and the USAID team (particularly their management effectiveness and efficiency)?..................................................................................................................29 4.6 EQ6: To what extent were the strengths and weaknesses related to the program being regional versus discrete activities for each country?..............................................32 ANNEXES ......................................................................................................................................................................... 36 Annex 1: Statement of Work ........................................................................................................37 Annex 2: Evaluation Instruments...................................................................................................41 Annex 3: List of Key Informants Interviewed by Organization.....................................................47 Annex 4: Findings, Conclusions, Recommendations Matrix ........................................................... 1 ACRONYMS Acronym Description A2I Access to Information AIB Association of Independent Broadcasters (Tajikistan) AMEP Activity Monitoring and Evaluation Plan AOR Agreement Officer Representative CAMP Central Asia Media Program CEO Chief Executive Officer COP Chief of Party CSO Civil Society Organization DRG Democracy, Human Rights, and Governance EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion FY Fiscal Year IMO Independent Media Outlet Internews Internews Network KII Key Informant Interview LGBT Lesbian, Gay, Bisexual, and Transgender ME&A ME&A, Inc. NGO Non-Governmental Organization OSCE Organization for Security and Cooperation in Europe PAS Public Affairs Service PMEP Performance Management and Evaluation Plan Q&A Question and Answer SOW Statement of Work TV Television U.S. United States UCA University of Central Asia USAID United States Agency for International Development USG United States Government i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of the mid-term performance evaluation of the Access to Information (A2I) project, funded by the United States Agency for International Development Central Asia (USAID/Central Asia) and implemented by Internews Network (Internews), is to: 1) examine the progress of A2I’s activities to date; 2) evaluate the prospect for achieving life-of-project results; and 3) identify if any changes need to take place to ensure achievement of those results. The evaluation also sought to determine: 1) what gaps exist in addressing the project’s objectives; 2) in which areas future USAID-funded media activities should focus to support journalists and other media actors in both countries to report objective information; and 3) whether the design and structure of the project was appropriate to successfully achieve its goals. The primary audience for the A2I evaluation includes the Democracy, Rights, and Governance Office of USAID/Central Asia and the Country Offices in Kazakhstan and Tajikistan. PROJECT BACKGROUND A2I began in October 2014 as a three-year program working in Kazakhstan, Tajikistan, and Kyrgyzstan. The goal of A2I is to improve governance outcomes by increasing citizen access to civically relevant information. A2I’s four complementary objectives are: • Objective 1: Increased regional cooperation to improve the competitiveness of non-state media outlets through content generation and sharing (revised to Increased access to information through regional cooperation, learning, and content sharing); • Objective 2: Increased resilience of independent media to survive the impending, treaty￾mandated shift to digital broadcasting across Central Asia in 2015 (revised to Increased resilience and competitiveness of independent media in the face of the changing environment); Objective 3: Strengthened diversity and professionalism of voices in traditional and “new” media (revised to Strengthened media and digital literacy, diversity, and professionalism of voices in traditional and “new” media); and • Objective 4: Improved legal and policy environment for media (revised to Improved legal environment for media and legal support to journalists). Internews received a two-year cost extension in September 2017, which modified the objectives of the project, extended A2I for two more years, and added resources to reach a total estimated cost of $4,999,390. USAID did not make the Kyrgyz portion of this activity a part of this evaluation or part of the modification of the award. EVALUATION DESIGN, METHODS, AND LIMITATIONS The evaluation team (ET) used a mixed-methods approach with complementary qualitative and quantitative data collection methods, which included: desk review of documents, key informant interviews (KIIs), short e-mail and telephone surveys, and focus group discussions (FGDs). These methods were used to gather quantitative and qualitative information from USAID, Internews staff, A2I partners and beneficiaries, and stakeholders in Kazakhstan and Tajikistan. The ET conducted fieldwork in Kazakhstan (Almaty, Shymkent, Kostanay, and Karaganda) and Tajikistan (Dushanbe, Tursunzade, and Khujand). The team visited 21 media outlets in Kazakhstan and nine outlets in Tajikistan that benefitted from A2I’s support. At the direction of the ET, the University of Central Asia (UCA) conducted focus groups with A2I beneficiaries in six cities (three in each country). The ET’s e-mail survey collected data from 14 beneficiaries in Kazakhstan (the ET had targeted 20 beneficiaries in its sampling method). In Tajikistan, the ET used a telephone survey instead due to the limited availability and use of the Internet in the country. There, the ET collected data from 18 beneficiaries, as planned. ii The ET triangulated and analyzed evidence from different qualitative and quantitative methods and from different data sources to reach conclusions and make recommendations. The ET focused the analysis on findings that: a) recurred with relatively greater frequency; b) were common across data gathered through different methods; and c) came from multiple partners, beneficiaries, and stakeholders. The ET expected, faced, and managed conventional limitations to performance evaluations, including: selection bias, recall bias, halo bias, a lack of baselines, difficulties assessing sustainability, and challenges assessing whether the project has met its objectives (as objectives were largely process-based). MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Evaluation Question (EQ) 1: To what degree has Internews Network (Internews) met the four objectives as stated in the Cooperative Agreement? A2I is meeting its targets for Objective 1. Internews’ assistance towards the objective has important benefits for independent media outlets (IMOs) and has also led to increased access to information for the population. The IMOs’ partners vary in their preferences for finished and unfinished content. Recommendations: USAID, through Internews, should continue to support regional cooperation in content generation and sharing to support competitiveness of IMOs. Internews should continue to work with partners to share content across different platforms and monetize sharing. Internews should also support content sharing of both finished and unfinished content to accommodate the preferences of different partner outlets. Performance data collected by the project is not adequate to demonstrate whether A2I is meeting its targets under Objective 2. A2I’s support has been important in increasing the resilience of IMOs and contributed to understanding their challenges and needs in shifting to digital broadcasting. The digital transition is driven by governments and threatens the survival of independent television (TV) stations. A2I partner IMOs need support to manage and survive the shift, which threatens the ability of IMOs to provide citizens access to civically-relevant information. Recommendations: Internews should revise its indicators to more completely tell the story of A2I’s contributions to support its partners in resilience, competitiveness, and the digital transition. USAID should work with Internews to develop approaches to support the digital transition. As part of potentially supporting the shift to a digital system in Tajikistan through a private, alternative multiplex in the north of the country, Internews should work closely with interested outlets through the media development organization Khoma, the Association of Independent Broadcasters (AIB), and the state regulator to develop a legal framework that can ensure such a system is securely placed in the country’s laws. Through Khoma, Internews should work closely with independent TV outlets in the rest of the country within the legal and regulatory framework to find ways for these outlets to make the digital transition. Performance data does not provide a comprehensive assessment of meeting Objective 3. Data that counts the number of individual beneficiaries with direct involvement in A2I exceeded targets but does not measure how activities have increased diversity or professionalism. Internews has successfully supported independent news outlets in Kazakhstan and Tajikistan in their professional development and promoted diversity in a variety of ways. A2I encourages national and cultural diversity, and gender equality, through shared media content and use of diverse multimedia tools as well as professional development. Recommendations: Internews should work with USAID to develop indicators that measure strengthened diversity and professionalism in more useful, comprehensive ways. USAID, through Internews, should continue to support: 1) the professional development of media professionals, particularly in management; 2) gender equality through media campaigns and creation of gender role narratives that contribute to women’s empowerment; and 3) coverage and inclusion of ethnic and other minorities and lesbian, gay, bisexual, and transgender (LGBT) populations in activities and in the media. iii A2I is meeting its targets for Objective 4. A2I’s support for improving the legal and policy environment was appreciated in Kazakhstan, particularly by outlets that received support. Legal support provided by Khoma was valued in Tajikistan. An improved legal and regulatory environment is a critical priority for IMOs. Partner outlets benefit from A2I’s information provision, consultancies, and residencies. Legal literacy is an area where A2I could provide more support. A broader approach and more partnerships could be considered as part of USAID’s and Internews’ support for legal reform. Recommendations: USAID should continue to support legal assistance through Internews/Khoma as needed by IMOs and journalists who effectively provide these services. USAID, through its partners, should continue outreach and advocacy towards an improved legal and regulatory environment for independent media and journalists. Internews and its partners should continue to support media outlets or journalists facing undue persecution through targeted residencies and legal counseling. Internews and its partners should also continue to provide more general information through webinars and seminars. Internews should consider expanding legal literacy training for media and developing additional delivery methods. Internews should, jointly with other legal advocacy groups, continue to work towards bringing about changes to legislation to move closer to international standards. Internews should consider setting up and supporting fora with legal advocacy organizations to examine specific legal challenges media outlets or journalists face in each country. EQ2: What gaps in addressing the program objectives still exist? And to the extent possible, describe why/how gaps exist. A2I’s partners and beneficiaries would benefit from a more comprehensive understanding of USAID’s and Internews’ programming. USAID and Internews need a consultative process to address key areas in media assistance that they and some partners view as gaps (see Section 4.2 for detail). Internews could then help fill priority gaps. Recommendations: USAID should make more information about media assistance programming public. Internews should provide more comprehensive information about media assistance programming, including the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. USAID and Internews should consider developing consultative processes to focus some program resources on areas that are considered gaps. These processes should be conducted in the context of the Central Asia Media Program (CAMP) project. USAID and Internews should consider bringing some partners into the discussion of how to address key areas in media assistance. EQ3: Going forward, how, and in which areas, should USAID-funded media activities be conducted to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan? A2I’s support for the goal and four objectives of A2I is important in supporting the provision of objective information in the two countries. A2I’s reach to diverse beneficiaries (individuals and organizations) limits the magnitude of the support for particular beneficiaries. The digital transition is a critical priority for independent TV stations that most outlets are not prepared for, which threatens their survival. A2I’s support to date for the digital transition has been limited and modest. Recommendations: USAID should continue to support media assistance in Kazakhstan and Tajikistan towards this goal and these four objectives. Internews should continue with A2I in its fifth and final year, as well as develop and implement additional programming in media assistance towards this goal and the four objectives. USAID should work with Internews to develop approaches to support the digital transition for independent TV stations within the framework of its USAID programs. Internews should provide additional assistance to independent TV outlets with the critical challenges they face with digital transition. Internews should support TV outlets to be more effective in how they work though media associations on this key area of common concern (as well as in other areas). iv EQ4: Given the media environment in Central Asia and goal of the program, was the design and structure of the activity appropriate to successfully achieving the goal? The ET found that the design and structure of A2I was broadly appropriate to achieving the goal of increased access to civically relevant information. The design and structure of media assistance projects like A2I could be explained more comprehensively to beneficiaries, potential beneficiaries, and stakeholders. A2I’s training targets a wide range of people, not only IMOs and professionals. IMOs have many needs that are not met in a comprehensive way with this design. Moving away from addressing the digital transition with the revision of A2I risks reducing attention to this critical challenge. Recommendations: USAID and Internews should continue implementing A2I with this design and structure in Fiscal Year (FY) 2019. Internews should provide more comprehensive information on the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. A2I should continue to provide support for media professionals and others. USAID and Internews should consider focusing media assistance on professionals and IMOs to concentrate results. USAID should consider directing media assistance towards targeted needs of IMOs. Internews should consider focusing on IMOs and develop widely-known application and selection processes for additional residencies and A2I staff support to selected IMOs. USAID should consider additional diplomatic efforts with bilateral and international partners and consider additional programming to support IMOs in the digital transition. EQ5: To what extent were the strengths and weaknesses related to performance of the implementer and the USAID team (particularly their management effectiveness and efficiency)? Findings show that the USAID Agreement Officer Representative (AOR) and Country Manager engage with Internews A2I staff and manage the grant with the appropriate level of involvement. Dialogue and partnership between USAID and Internews are managed effectively. Many partners, beneficiaries, and stakeholders have suggestions, ideas, and proposals for additional activities, directions, and support; these suggestions are less about strengths and weaknesses of management and more about opportunities and alternatives. There may be additional efforts that are warranted to support a stronger legal and regulatory environment for independent media, including through the transition to digital technologies. Recommendations: USAID should continue to work through grants with Internews to provide media assistance. USAID managers should participate in and observe as many Internews media assistance activities as possible—with the exception of national-level media association and government dialogues. Internews should develop fora for project and potential beneficiaries to discuss strategy and program approaches for projects like A2I as well as how Internews proposes to manage for effectiveness. Internews should endeavor to work more with and through partnerships with national organizations, particularly in working to improve the legal and regulatory environment. USAID should work through developmental and diplomatic channels in support of Internews’ and national efforts towards a more open legal and regulatory environment for independent media in Tajikistan and Kazakhstan. EQ6: To what extent were the strengths and weaknesses related to the program being regional versus discrete activities for each country? Regional programming through A2I has had important benefits; in some areas, the specific features of Kazakhstan and Tajikistan make national programming more appropriate. Recommendations: USAID should continue to support Internews initiatives that connect media beneficiaries across Central Asia and should consider supporting additional initiatives to share regional news and content across Tajikistan. Internews should consider developing programs and a mechanism to support cross-Tajikistan sharing and cooperation of local news and information by independent outlets in the country. Internews should continue to conduct country-specific events and assistance through A2I on topics and information that target the interests and needs of specific countries, particularly assistance delivered in Tajik and Kazakh. 1 1.0 EVALUATION BACKGROUND, PURPOSE, AND QUESTIONS 1.1 EVALUATION PURPOSE The United States Agency for International Development Central Asia (USAID/Central Asia) contracted with ME&A, Inc. (ME&A) to conduct the mid-term performance evaluation of the Access to Information (A2I) project in Kazakhstan and Tajikistan. A2I is a three-year program implemented by the Internews Network (Internews) through a Cooperative Agreement which began in October 2014. Internews received a two-year cost extension in September 2017, which modified the objectives of the project, extended the term of A2I until September 30, 2019, and added resources for a total estimated cost of $4,999,390. The goal of A2I is to improve governance outcomes by increasing citizen access to civically relevant information. The project is regional, with regional and country-specific activities focused on Kazakhstan and Tajikistan. Activities in the Kyrgyz Republic were part of the project in its first three years; however, USAID did not make the Kyrgyz portion of A2I a part of this evaluation or part of the modification of the award. The purpose of the mid-term performance evaluation of A2I was to: 1) examine the progress of A2I’s activities to date; 2) evaluate the prospect for achieving life-of-project results; and 3) identify if any changes need to take place to ensure achievement of those results. In addition, per the request of the Scope of Work (SOW), the evaluation examined: 1) what gaps exist in addressing the project’s objectives; 2) in which areas future USAID-funded media activities should focus to support journalists and other media actors in both countries to report objective information; and 3) whether the design and structure of the project was appropriate to successfully achieve its goals. The evaluation was conducted between October-December 2018 by an evaluation team (ET) that consisted of an international expert, Dr. Lawrence Robertson (team leader), and a local media and communications specialist, Ms. Gulmira Chalabekova. The ET was ably assisted with translation and logistics for data collection by Ms. Alfira Senkina and Ms. Maksoudova Kouysinoy in Kazakhstan and Tajikistan, respectively. The results of the evaluation are expected to be used by USAID/Central Asia for improving ongoing interventions, if needed. The primary audience for the evaluation is USAID/Central Asia, particularly the Democracy and Governance Office and the Country Offices in Kazakhstan and Tajikistan. Secondary audiences include Internews, media actors and civil society organization (CSO) stakeholders in these countries, and other international development partners of Kazakhstan and Tajikistan. 1.2 EVALUATION QUESTIONS The evaluation answers an extensive set of specific evaluation questions (EQs) set out by USAID/Central Asia. The six EQs posed by USAID/Central Asia were: 1. To what degree has Internews met the four objectives as stated in the Cooperative Agreement? 2. What gaps in addressing the program objectives still exist? And, to the extent possible, describe why/how gaps exist. 2 3. Going forward, how, and in which areas, should USAID-funded media activities be conducted to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan? 4. Given the media environment in Central Asia and goal of the program, was the design and structure of the activity appropriate to successfully achieving the goal? 5. To what extent were the strengths and weaknesses related to performance of the implementer and the USAID team (particularly their management effectiveness and efficiency)? 6. To what extent were the strengths and weaknesses related to the program being regional versus discrete activities for each country? These six EQs have been used to organize the entire evaluation—from determining the methodologies to use to collect the data to answer these questions to structuring the evaluation report around findings, conclusions, and recommendations. 2.0 PROJECT BACKGROUND Kazakhstan and Tajikistan have different socio-economic conditions and have developed as societies in different ways since gaining independence at the end of 1991. USAID has supported the development and independence of the media in Central Asia since beginning work in the region in 1993. The first two decades of support have helped to develop independent non-state media outlets, increase the professionalism of these outlets, and support them towards becoming sustainable. However, the overall environment for professional, sustainable independent non-state media in the region remains a challenging one. USAID developed the A2I project in 2013. The project was awarded to Internews based on Internews’ proposal in response to USAID’s request for proposals. The project aimed to strengthen the long-term viability of targeted media outlets in Central Asia, facilitate regional cooperation and conduct a number of activities designed to promote the modernization of newsrooms throughout the region, improve the financial viability of media, and increase access to information that will keep government institutions accountable to citizens. A2I activities took place in Kazakhstan, Kyrgyzstan, and Tajikistan in the first three years of the project. The four objectives of A2I were: • Objective 1: Increased regional cooperation to improve the competitiveness of non-state media outlets through content generation and sharing; • Objective 2: Increased resilience of independent media to survive the impending, treaty-mandated shift to digital broadcasting across Central Asia in 2015; • Objective 3: Strengthened diversity and professionalism of voices in traditional and “new” media; and • Objective 4: Improved legal and policy environment for media. USAID modified the Cooperative Agreement in 2017 with an extension due to the rapidly changing media environment. The objectives of the project were slightly modified during the extension period, from October 1, 2017 to September 30, 2019. The new objectives are: • Objective 1: Increased access to information through regional cooperation, learning, and content sharing; • Objective 2: Increased resilience and competitiveness of independent media in the face of the changing environment; 3 • Objective 3: Strengthened media and digital literacy, diversity, and professionalism of voices in traditional and “new” media; and • Objective 4: Improved legal environment for media and legal support to journalists. With the cost extension and modification, A2I’s activities focused exclusively on Kazakhstan and Tajikistan. USAID and Internews analyzed the challenges of the region and the opportunities for USAID assistance to make a difference based on the previous extensive work in the region on the media. Media in the region face a range of problems that include: newsrooms using outmoded technology; media with a weak online presence and minimal outreach to audiences; limited available advertising revenue (in particular in Tajikistan); and lack of an established feedback loop in which media keep government accountable to citizens. Internews and USAID recognized that there was little cross-border cooperation, leaving citizens reliant on Russian media or state media for news about neighboring countries. Thus, A21 and its activities focused on regional cooperation, newsroom modernization, media financial viability, and government accountability. Neither governments and the media, nor the public were seen as ready for the digital transition, which, in 2012, was planned to be made in 2015. Numerous technical, policy, and political problems were identified. Internews and USAID noted that expert opinions in the region felt that the full switch to digitalization was likely to be postponed. While this indicated that the issue was falling on governments’ priority lists, it provided A2I and its partners more time to work to improve the legal environment as well as to support media with technical and financial preparations. The A2I’s focus on raising professional standards and providing more learning opportunities for journalists and media, including in national languages, was also seen as important. Independent local media find it challenging to compete with the high-quality production of state and Russian channels. Internews and USAID shared the approach that, in both Kazakhstan and Tajikistan, higher professionalism could produce stronger independent content, particularly local content providing citizen-relevant information, where there was substantial local demand that could support and sustain independent, non-state regional media outlets. Better trained professionals were recognized as critical to produce this content; more resilient independent media outlets (IMOs) were seen as critical to providing access to independent and better￾quality information to the people of both countries. 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY The ET used a mixed-methods approach with complementary qualitative and quantitative data collection methods to collect information from USAID and Internews as well as A2I partners, beneficiaries, and stakeholders. The ET designed data collection and analysis around answering the EQs and sub-questions. The ET used a number of methods, including desk review of documents, key informant interviews (KIIs), and focus group discussions (FGDs) to gather quantitative and qualitative information from USAID, Internews staff, A2I partners and beneficiaries, and stakeholders in Kazakhstan and Tajikistan. USAID did not include Kyrgyzstan in the evaluation SOW, although it was included in the A2I program for the first three years of implementation. The ET conducted KIIs in person in both countries. In addition, it gathered data through an e-mail survey of selected beneficiaries in Kazakhstan and shorter interviews over the telephone in Tajikistan. The team conducted fieldwork in Kazakhstan from October 22-26 and November 5-8, 2018, in Almaty, Shymkent, 4 Kostanay, and Karaganda. In Tajikistan, the ET conducted fieldwork in Dushanbe, Tursunzade, and Khujand from October 29-November 2, 2018. At the direction of the ET, the University of Central Asia (UCA) conducted focus groups with A2I beneficiaries in six cities (three in each country): Dushanbe, Khujand, and Bokhtar (formerly Kurgan￾Tyube) in Tajikistan; and Almaty, Karaganda, and Uralsk, in Kazakhstan. Quantitative Research Approaches and Analysis The ET used quantitative data from A2I project reporting from October 2014 to September 2018 to examine A2I’s successes in achieving planned benchmarks as well as to identify areas in which the project faced challenges, had greater successes, and/or exceeded planned quantitative targets towards its four objectives. The ET has used the indicators developed in the project’s original Performance Management and Evaluation Plan (PMEP) from November 2014 as the clearest presentation of expected results at the outset of the project. The ET mainly used reporting from the Year 4 A2I Annual Report for these measurements. UCA’s focus groups used questions to collect quantitative data from beneficiaries through Likert scale measurements to answer EQ4 on whether the design and structure of the A2I activity was appropriate to successfully achieving the goal of the program. Qualitative Research and Analysis The objectives of the Cooperative Agreement are predominantly qualitative in nature; the EQs asked by USAID/Central Asia emphasized making a qualitative assessment of A2I’s project results to date and determining whether there are gaps in addressing program objectives. EQs about whether there are current opportunities to support journalists and other media actors in reporting objective information, views on the appropriateness of the design of the activity, and strengths and weaknesses of program management and implementation are also largely qualitative. Thus, the evaluation largely used qualitative techniques to collect valid and reliable data to answer these EQs. The ET focused mainly on collecting and analyzing valid and reliable qualitative data from USAID, Internews, A2I partners, and A2I beneficiaries and stakeholders. Qualitative data collection went beyond an assessment of whether A2I is meeting its quantifiable targets and explored the extent to which the project is meeting the needs and aspirations of partners, beneficiaries, and stakeholders, and achieving the desired impact. Qualitative data collection consisted of in-person KIIs conducted by the ET, shorter interviews through telephone calls and online survey methods, and FGDs conducted by ME&A’s partner UCA at the direction of the ET with USAID, project staff, and partners, stakeholders, and beneficiaries. The ET visited 21 media outlets in Kazakhstan and nine outlets in Tajikistan that benefitted from A2I support. The ET conducted KIIs with media managers and key staff at these outlets. The ET’s e-mail survey collected data from 14 beneficiaries in Kazakhstan (the ET had targeted 20 beneficiaries in its sampling method). In Tajikistan, the ET used a telephone survey instead, due to the limited availability and use of the Internet in the country. There, the ET collected data from 18 beneficiaries, as targeted by the ET’s sampling of individual beneficiaries from A2I-funded or implemented trainings, webinars, exchanges, conferences, and consultations. Sampling of beneficiaries for online surveys/telephone calls was based on the comprehensive beneficiary lists provided by A2I management. The ET sampled randomly from the different categories to select a number of beneficiaries for e-mails/calls, replacing the sampled people if they were already on the KII or FGD participant lists. The ET made at least three attempts to gather data from the selected person if the number or e-mail was valid, replacing the selected beneficiary if not valid or non-responsive after three tries. Table 1 below reports the level of effort required to reach this sample size in Tajikistan or almost reach this sample size in Kazakhstan (where even this mechanism yielded 14 rather than 20 respondents). 5 Table 1: Sampling of A2I Individual Beneficiaries Category Kazakhstan Tajikistan Total # of Participants # Selected # Reached # E￾Mailed to Reach Selected Number Total # of Participants # Selected # Called to Reach Selected Number Online Media School 21 2 2 6 - - - MEDIATOURS Mater Class 381 5 3 15 - - - Webinars 550+ 8 4 24 - - - Data Journalism 13 2 2 6 - - - Training of Trainers in Kazakh 5 1 1 3 - - - Legal advice 29 2 2 6 - - - Media and Social Innovation Lab - - - - 29 2 3 Cross-border exchanges - - - - 8 1 1 Conferences - - - - 60 3 4 Media master classes - - - - 76 5 7 Legal counseling - - - - 27 2 5 Trainings - - - - 89 5 10 Totals - 20 - 60 - 18 30 The UCA focus group teams used names selected by the ET from comprehensive lists of beneficiaries provided by Internews to invite people to participate in focus groups. When UCA found that phone numbers or e-mail addresses were no longer accurate or active, it requested that the ET select more potential participants. This process was repeated several times to yield small groups of project beneficiaries suitable for data collection through focus group methods. The ET triangulated evidence from different qualitative and quantitative methods and from different data sources to increase the credibility of findings. The ET also focused on findings that: a) recurred with relatively greater frequency; b) are common from across data gathered through different methods; and c) come from multiple partners, beneficiaries, and stakeholders. The ET used these conclusions to make recommendations to USAID and Internews. 3.2 EVALUATION LIMITATIONS Limitations that are common in the design of most evaluations were anticipated and managed in this evaluation. Limitations encountered and mitigated by the ET included: • Selection bias: Because some individuals from the sample of all A2I beneficiaries and stakeholders selected by the ET for interviews, telephone calls/email surveys, or focus groups were not reachable or declined to be interviewed, selection bias is possible as respondents who are not found or refused to participate might systematically differ from those who are reached and agree to participate in the evaluation’s fieldwork. To minimize selection bias, the ET collaborated with USAID/Central Asia, Internews, and A2I partners to facilitate introductions and mobilization of partners, beneficiaries, and stakeholders for interviews and FGDs, which was important in the challenging context for the independent media in Kazakhstan and Tajikistan. A2I’s beneficiaries 6 from established media outlets and key stakeholders were straightforward to reach and interview. However, outreach to particular individual beneficiaries of A2I activities for participation in the telephone or email survey and FGDs was more challenging. The ET used random selection of these beneficiaries from comprehensive lists provided by A2I staff and repeated calls and e-mails to minimize the substitution of beneficiaries within this selection. • Recall bias: Because the A2I project has supported partners and beneficiaries in the region since October 2014, it was more challenging for interviewees and FGD participants to recall A2I activities and benefits from early in the period of implementation relative to more recent Internews activities. To minimize recall bias, the ET used specific prompts to ask some interviewees about earlier activities that they may have or did participate in, and specific past Internews support for them and their organizations. • Halo bias: Respondents may underreport socially undesirable answers and alter responses to what they perceive as the social norm. The extent to which respondents are prepared to reveal their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or partners upon whom they depend for the provision of valued services. The ET followed up on some neutral, open-ended interview questions that were met with overwhelmingly positive responses and asked whether there were also things that went less well or were negative in their engagement with A2I. Explicit questions were asked about gaps in the program, per the EQs. • Lack of baselines: Due to the absence of clear baselines on objectives prior to the start of A2I, the mid-term evaluation was not able to generate clear sufficient evidence to measure the extent of change during A2I implementation in a rigorous manner. To mitigate this, the ET asked informants to reflect on the importance of A2I activities and support for changes in their outlets and independent media in their countries. • Conflation of A2I activities with other donor and Internews projects: A2I reports noted that many activities were co-funded by other donors and brought together A2I beneficiaries with other Internews beneficiaries, which posed challenges for attribution. This proved particularly relevant in Tajikistan where the Open Society Institute also implemented a project under the name of Access to Information. The ET was able to manage this general challenge, as well as the more specific issue in Tajikistan, by asking explicitly whether beneficiaries and stakeholders recognized that support was from Internews through A2I. The ET also has not made specific assertions about the project’s contributions to observed changes in the media in these countries as a result of this limitation and the absence of clear baselines noted above. • Difficulty assessing sustainability: The evaluation was conducted while many A2I interventions had another year of implementation under the modified award, which made it difficult for the ET to determine whether reported or observed outcomes are likely to continue once A2I support ends. The ET managed this challenge by asking USAID, A2I staff, and project partners, beneficiaries, and stakeholders forward-looking questions about whether they expect that some activities may be sustainable in the future and why they think sustainability is more or less likely. • Difficulty assessing whether A2I has met its objectives: A2I objectives were largely framed as processes by Internews in its program proposal. Thus, USAID’s grant to Internews has a set of stronger processes as its objectives. In addition, specific questions about whether A2I has met its objectives are only appropriate for informants that had specific, detailed knowledge about what A2I objectives were. Interviews made it clear that only USAID and Internews managers have this kind and level of knowledge about A2I. The evaluation has also used performance data from A2I reporting to answer EQ1. In addition, because the areas of work for these four objectives were relevant and important to the media outlets that A2I works with in both countries, the evaluation 7 asked their managers the relevant questions for them in terms of this EQ: what have been the results of Internews’ work through A2I with them and their outlets in the four different areas where A2I had its objectives. This data is also reported as part of answering EQ1. 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 4.1 EQ1: TO WHAT DEGREE HAS INTERNEWS MET THE FOUR OBJECTIVES AS STATED IN THE COOPERATIVE AGREEMENT? A2I generated a PMEP for USAID approval in November 2014. The evaluation used the expected results A2I developed in the PMEP as one way to evaluate progress towards meeting the four objectives below. The PMEP developed measures and ways of measuring the growth of the audience of online media in a systematic fashion. Yet, systematic methods for measuring increased citizens’ access to civically-relevant information by monitoring the audience of supported radio and television (TV) outlets for annual measurement were not developed or implemented. Targets do not appear to have been set in November 2014, but they are included in the A2I’s Year 4 Annual Report. The ET used these targets and measurements as part of addressing EQ1 through the four objectives. These targets, however, were often not disaggregated among the three original countries over the first three years of the project. Internews developed a revised PMEP, renamed the Activity Monitoring and Evaluation Plan (AMEP), in November 2015 and then a new AMEP with the modification and extension of A2I in 2017. Not all targets, however, were set in the extension or reported in the Year 4 Annual Report, which makes it difficult to evaluate whether objectives have been met. The evaluation also used the measures developed in this AMEP to assess progress towards meeting objectives. Since USAID and Internews staff knew and worked towards the objectives of the A2I program, KIIs with USAID and Internews were used to gather data to answer these EQs. A2I’s partners did not have enough understanding of the objectives to enable them to assess the degree to which Internews has met A2I objectives. KIIs with A2I partner station managers, directors, and founders asked about the contributions of A2I towards their work in these areas, but often could not address this EQ explicitly. This lack of a comprehensive understanding of the A2I project and what it was designed to accomplish also left outlets without a full understanding what Internews could potentially do to assist them. FGD participants also generally did not know the objectives of A2I. However, some FGD participants were able to speak more generally to A2I’s results when asked about the contribution of the program towards the program goal or its four objectives. E-mail and telephone survey participants were not asked to respond to this EQ as, generally, they were not informed enough about the project objectives or achievements towards them. 4.1.1 Objective 1 Increased regional cooperation to improve the competitiveness of non-state media outlets through content generation and sharing [Fiscal Year (FY) 2018 Increased access to information through regional cooperation, learning, and content sharing] Internews set five expected results for this objective in the AMEP: 1) increased cooperation of IMOs and professionals on the regional level; 2) at least five new cross-border news and/or information-sharing partnerships created; 3) at least 30 media professionals annually benefit from regional exchanges or study tours; 4) indigenous cadre of skilled media professionals is available to advance regional media issues; and 8 5) increase content sharing among non-state TV stations of the three participating countries, with the first milestone in Year 1 to be set at 100 shared news items and other programs, and in Years 2 and 3 to raise this amount by 20 percent in each year. FINDINGS USAID, Internews, and IMOs that worked with A2I were focused on their competitiveness—or even their survival—in a crowded media market in both countries. USAID and Internews’ staff interviews noted important achievements in content sharing by A2I-supported outlets in both countries, which they saw as improving competitiveness and leading to increased access to information. KIIs with managers at A2I partner media outlets in both countries confirmed that content sharing improved their competitiveness and led to increased access to information. Internews quarterly and annual reports noted that government regulation affected the level and type of content sharing. For example, changes in the media law in Kazakhstan for 2018 that reduced the use of shared content by partners by inhibiting the use of video and photos as part of international news sharing through Internews.1 In both countries, Internews and media managers noted that they did not have systematic monitoring mechanisms to measure their audiences and audience preferences. Internews reported planning to do more media monitoring under the new Central Asia Media Program (CAMP) award. Performance data from the project indicated that A2I met the Year 4 target of 24 outlets participating in Kazakhstan and exceeded the target of 22 participating outlets in Tajikistan by one (23 outlets participated). In terms of the amount of content made available for sharing through the A2I’s supported platform, Internews’ performance data indicates that Year 4 totals modestly exceed A2I’s targets. The A2I Year 4 Annual Report shows that 12 partner outlets in Kazakhstan received an average of 30 percent of their news content through shared content, and 10 partner outlets in Tajikistan received as much as 26 percent of their news content from the exchange.2 Most content came from two A2I-developed programs: News of the Regions, a 5-10 minute bulletin produced in Kazakhstan in Russian and Kazakh, and Imruz (Today) produced in Tajikistan in Tajik. Both were initially produced jointly by Internews with partners. In Year 4 of A2I, they were handed over to these local media partners, Kazmediaindustry Public Foundation in Kazakhstan and Mavji Ozod in Tajikistan. In addition, A2I reports on the number of people that participate in regional networking and training events as additional indicators of interest in regional content and learning among independent media professionals. These numbers exceed targets for both countries. Kazakhstan Interviews with USAID in Kazakhstan emphasized the value of long-standing regional cooperation supported by Internews, including under A2I, which USAID staff thought was successful. A2I was designed to change the character of cooperation by having partner outlets creating their own quality content and sharing this production as the key to making them more competitive. USAID emphasized the quality of an A2I-supported show—Heroes Among Us—and its usefulness for demonstrating the value of storytelling and quality content to outlets and educating the population about inspiring, positive role models for civic activism. Internews staff in Kazakhstan noted that A2I was “achieving its goal” in helping stations follow the language law regulations that demanded local-language content. They also said that shared content helped independent stations better compete with private media that belongs to oligarchs or rich businesses and, to a certain extent, compete with state-funded media. Regional audiences were seen as preferring “more and more” to see local news. The popularity of shared Kazakhstan content—through News of the Regions, once produced by Internews but now produced by Internews’ partner Kazmediaindustry with A2I’s 1 “Amendments and Alterations to Certain Legislative Acts of the Republic of Kazakhstan on Information and Communication” were made in December 2017 for 2018. See https://www.internews.org/updates/internews-helps-kazakh-journalists-navigate￾new-amendments-media-law 2 A2I Year 4 Annual Report, p. 3. 9 support—among independent stations in the country was seen as evidence of this preference for local news content. In Kazakhstan, managers of eight regional outlets out of the 21 interviewed highlighted the value of the unique content that is being produced and shared through A2I as critical to their survival and suggested expansion of sharing. This was especially noted by regional stations. “The idea to exchange content is extremely valuable. Similar to regional media outlets that suffer from not having access to experts in Astana or Almaty, we in Almaty and Astana suffer from not having information from regions. The program should be expanded as it makes independent media more competitive.” Media outlets emphasized the importance of content exchange as it enables them to cover news from other parts of the country, not just their region: “We did not have correspondents in regions so we started using content sharing...Now it works as we exchange information with other channels regularly.” Station managers felt this sharing increased their audience but recognized that they did not benefit from the formal measurement systems that could provide hard data on their audience and their viewing or listening preferences. One noted “With respect to News of the Region, yes it increases viewership numbers” but then noted “Since no measurements are done, we don’t know whether it is of interest to anyone.” Four regional TV managers interviewed said they engage their audiences both on air and online as the shared content can be viewed not only on TV, which is still how most Kazakhstanis get their news, but also uploaded to YouTube and posted on social media. They are looking for interactive ways of sharing information. One media manager noted “We run Heroes of the Day three times a week in Kazakh which suits us fine, but the best product for us is News of the Region, which we run three times a day in many different ways— as part of news or on its own.” In interviews, three media managers said they are able also to monetize content through advertising placements. One TV interviewee asked for help with developing a strategy for using A2I content across multiple platforms. Seven regional interviewees highlighted a need for producing quality Kazakh language content. Three TV managers out of the 21 interviewed that were not involved in the A2I content exchange noted that they had developed ideas and produced their own content based on products shared on the content exchange. Inspired by the A2I content exchange, KIIs at a newspaper in Kostanai noted that they share content with Internet portals in Uralsk. Another national news portal stated that they now share content internationally with a Bishkek partner. The A2I content sharing went beyond that, with not just media outlets but also non-governmental organizations (NGOs) getting involved. In 2018, the NGO Argo posted 10 Heroes of the Day on its site and also uploaded them to YouTube.3 Argo noted that number of hits for that period exceeded those of Internews. As a result, it was not just people in Kazakhstan and Tajikistan that viewed the content, but also people in Uzbekistan and Turkmenistan. FGD participants in Kazakhstan were asked to rate A2I’s support on a scale. Participants in all three FGDs in Kazakhstan rated regional cooperation the highest of all areas queried. The score averaging all three FGDs in regional cooperation was 4.2; Almaty FGD participants rated the area highest at 4.8, and Uralsk participants were the most critical at 3.3 (Uralsk participants were notably more critical in all areas). Tajikistan Interviews with USAID and the Public Affairs Service (PAS) in Tajikistan agreed that the “most important” factor for media outlets was the content, and that sharing content from one region of the country to another was of interest for every part of Tajikistan as the problems people face are similar. USAID and PAS interviews noted that Internews’ partners were also the ones that partnered with USAID or other 3 http://cso-central.asia/category/geroi-ryadom/ 10 PAS projects that produced content; these partners were competitively selected for other content production based on their ability to produce quality content. Internews’ Tajikistan staff interviewed emphasized the value of A2I-provided free, high-quality content from the content exchange in supporting the competitiveness of media outlets across the country. Quality independent news programs are seen as “the biggest success of our partners” by Internews, which emphasized the contribution of shared content to this success. Internews staff saw project implementation as strongest towards this objective compared to the other objectives of A2I. Internews staff also noted that content sharing has now been extended to creating a new network of independent stations for joint advertising, with two positive successes in gathering advertising funds and sharing them across five stations in spring 2018. Program staff noted that Tajik TV outlets preferred finished content to raw footage as they had limited capacity to produce their own finished news. Internews staff noted the importance of the project in negotiating together with Association of Independent Broadcasters (AIB) for free and reduced Internet rates that enable independent stations to access content sharing. In Tajikistan, managers of all of the nine outlets interviewed by the ET noted that they used Internews’ content exchange platform because the platform offered quality news content that was of interest to their audience. As one manager from a Tajikistan station noted, “News is the most important for the rating of television; we need actual and interesting news so people want to watch the news—they demand things that relate to them.” Six of the nine outlets noted that they preferred to use finished content from the exchange due to their limited news production capacity, capacity that could then be used for other production that focused on local news and information. CONCLUSIONS A2I is meeting its targets for Objective 1. Internews’ assistance towards the objective of improving the competitiveness of non-state media outlets through content generation and sharing (FY 2014-2017) and increasing access to information through regional cooperation, learning, and content sharing (FY 2018) has important benefits for independent regional media outlets. It also leads to increased access to information to the population in both countries. A2I partners and potential partners do not have a comprehensive understanding of the project and its potential avenues to support them through regional cooperation and content sharing. The many outlets that partner with A2I noted that they appreciated all of the different kinds of support through A2I, but usually were not able to discuss this support in the context of A2I objectives—only their own objectives at their outlets. The limited information may limit the partnership and joint achievement of larger goals in this objective and sector. IMO partners of A2I vary in their preferences for finished and unfinished content, based on different capacity and constraints in the two countries. RECOMMENDATIONS USAID, through Internews, should continue to support regional cooperation in content generation and sharing to support the competitiveness of independent news outlets in the two countries. Internews should continue to support regional cooperation, learning, and content sharing to improve the competitiveness of non-state media outlets in both countries. Internews should develop comprehensive information products on projects, make this information publicly available, and brief partners and potential partners on approaches and potential support. Internews should continue to work with partner outlets on ways to better share different types of content across different platforms as well as how to monetize this sharing for competitiveness. 11 Internews should support content sharing of both finished and unfinished content as partner outlets have different preferences. 4.1.2 Objective 2 Increased resilience of independent media to survive the impending, treaty-mandated shift to digital broadcasting across Central Asia in 2015 (FY 2018 Increased resilience and competitiveness of independent media in the face of the changing environment) Internews set the following four expected results in the AMEP: 1) non-state TV stations ensured legal access to digital frequencies in A2I countries; 2) the majority of independent partner TV stations survive the transition to digital broadcasting and are included in free-to-air digital multiplexes; 3) expanded digital media skills and online presence to engage wider audiences and facilitate better newsgathering; and 4) locally produced content distribution is diversified into a multi-dimensional platform and more accessible to all demographic communities. FINDINGS USAID and Internews’ A2I staff interviews noted ways that Internews had supported discussion, dialogue, and learning on the shift to digital broadcasting in both countries. The environment and regulation in both countries on the digital transition changed substantially from what was expected when the A2I project was developed in 2013 and the award signed in August 2014. The changed environment slowed the push to transition to digital in both countries, particularly for independent television outlets, and changed the character of these efforts. Kyrgyzstan undertook the digital transition successfully in the project’s first three years when Kyrgyzstan was included in the project. Internews reports that A2I’s role in that transition was considerably different and more substantial compared to A2I’s work under this objective in Kazakhstan and Tajikistan, where the transitions have been continually delayed. The evaluation has not covered this work in Kyrgyzstan as it is outside the SOW of the evaluation. Performance data from the project on Objective 2 was not useful for understanding the results of the A2I project on outlet survival with the transition to digital broadcasting. Performance data on the number of non-state media outlets that engaged with A2I or were assisted by A2I exceeded the targets set by Internews but did not measure how activities have led to increased resilience or competitiveness. Because of the changes in the environment, the project did not endeavor to meet the target of “non-state TV stations ensured legal access to digital frequencies” or “the majority of independent partner TV stations survive transition to digital broadcasting.” The utility of these indicators for assessing results was particularly affected by decisions by the state regulators in Kazakhstan and Tajikistan that did not make participation in digital multiplexes free or low cost. Other targets measured outputs, such as the number of public events by partners under the project and the number of participants in webinars. Output targets measure participation rather than how participation supports the shift to digital (or resilience or competitiveness). With a few exceptions, these output targets were exceeded by A2I and its partners. Year 4 indicators and targets that instead focused on resilience and competitiveness through the growth of unique online viewers and the percentage change in advertising revenues online for selected outputs were measured for two Tajikistani outlets. These measures showed high variance due to factors beyond A2I’s control (with illegal government blocking of websites at times over the year). Kazakhstan USAID key informants considered A2I’s legal work as central in the progress that Kazakhstan had made in the transition—but then the Russian postponement of the digital transition led Kazakhstan to postpone their transition as well, “leaving everything unclear.” 12 Internews A2I staff in Kazakhstan noted that work on the digital transition had been challenging due to Kazakhstan postponing the timing of the switch to digital several times and issues with working with Internews Kazakhstan on the digital switch. This led to A2I working on the transition through an experienced, respected consultant instead. Internews noted that government policymakers dominate decision-making and take limited account of IMOs. Internews characterized their work as “just reaching the starting point” and stated that Internews’ support will continue to help stations with the switch under this Objective. Internews noted that spending was more effective in other objectives of A2I. The change in the objective for FY 2018 to focus on overall resilience rather than digital transition was seen by Internews as a change to an area where Internews could more immediately contribute to supporting outlets, such as by helping seven independent channels that came together as a self-organizing group work as a pool in the advertising market. This enabled the group to compete for large advertisers more effectively. Internews considered this example of A2I-funded research on the audience of seven outlets as valuable and important in helping the outlets accurately know their audiences and use this knowledge to support resilience through advertising based on this research. Internews also reported that digital broadcasting is a big objective under their new USAID award for CAMP. In Kazakhstan, nearly all staff of regional TV channels interviewed by the ET described digital transition as the biggest challenge to their survival and said they are largely unprepared for this transition. The main issues were seen as the high costs of making the transition. Costs were in equipment, training, and particularly, costs for using government multiplexes to broadcast digital signals. Two regional channel managers said the high cost of the digital switch might force them off air and to go online. Another regional channel reported that they were now in talks with potential buyers based on their concerns about future viability. A TV manager in a focus group claimed that switching to digital would come at a cost to their independence: with digital, he asserted, there is “a legal detail which some might not know...in the event of violations your TV can be switched off without a court decision.” December 2018 was the start of digital terrestrial broadcasting in Shymkent. A2I partner outlets there asked for help with lowering the multiplex tariffs. One stated that it was now paying around $1,500 a month for access and another said they pay $300 a month for broadcasting in the Turkestan regional center. Some stations were not confident they would be able to continue to pay this high tariff. Other stations asked for Internews, through A2I, to help recover the money they had paid the state for analog frequencies and for the infrastructure they built at the time. Three media managers emphasized that Internews, through A2I, should get involved in raising public awareness about digital transition as not many people know what this entails—that consumers need to purchase new TVs or set-top boxes. Interviews at outlets in Karaganda and Kostanai noted major issues for them were the high cost of delivering the signal and methods for calculating signal distribution tariffs through the state multiplex. Some managers proposed Internews support—through A2I efforts to get rebates or refunds from the state for prior payments that they had made for the rights to use analog frequencies—in order to meet these new digital costs. Four media managers out of the 21 interviewed asked for help with securing advertising contracts. A2I’s work to date was recognized in this area. As a result of the consortium approach supported by Internews under A2I, one TV outlet noted that it reached an advertising contract with a soft drink manufacturer that led to 63 percent of its budget now being generated by advertising, including digital advertising, leaving only the remaining 37 percent of revenues coming from state orders (Goszakaz). Dependence on, or a high percentage of revenues from, state orders is widely understood to pose problems for the independence of regional outlets from the state. Two TV managers called on Internews to help create a regional sales agency noting that, at this time, advertising orders go through far too many levels and a private firm, Sales House Advertising Agency, takes up 50 percent of the funds, with independent TV outlets losing their potential revenues and potential for profitability as a result. 13 Tajikistan USAID and PAS interviews in Tajikistan noted that A2I’s objectives had changed with the modification to drop the focus on the digital transition as it was considered too ambitious for the project with the government’s resistance towards the transition and the policy change in dates of the transition to digital from 2018 to 2021. The revision was made in recognition of the domination of the government in decision￾making on the digital transition and the limited influence of IMOs and other stakeholders, including the media development organization, Khoma, on the transition. Key informants also noted that the Organization for Security and Cooperation in Europe (OSCE) was engaged on the subject, including through the same partner as A2I—Khoma. Internews reported taking the initiative to set up this engagement and broaden the organizations involved in the digital transition. USAID considered simply having Internews and Khoma be able to continue working with and having a dialogue with the government on the media and the digital transition in the conditions that prevail in Tajikistan as an important success of A2I in this area. AIB was also expected by USAID/Tajikistan to pursue the collective interests of independent outlets in digital. The domination of the organization by one person impeded this effort; this was not seen as a matter that Internews or USAID could address. Interviews with Internews Tajikistan emphasized that the switch to digital was entirely dependent on the state, which could switch anytime. However, the fragmentation of state agencies and the particular personal interests of bureaucrats were seen to have led to Gosteleradio having its own interests in extending the time to transition. A2I activities were considered as important in educating independent outlets on the issues as well as reaching key Tajik parliamentarians and Gosteleradio leaders about the importance and challenges of the transition. Internews noted that Khoma, funded through A2I, has the expertise and connections to be effective in supporting independent stations in the digital transition. The evolving timelines and plans for the transition were considered as impeding A2I programming and justifying a change in the objective away from targeting support for the digital transition. Staff expected that if the switch was done now, half the independent media would die. They said that independent media are not prepared to address the challenges of digital. Moreover, the state is “purposefully creating barriers for independent stations to join” the digital multiplex and setting the cost per hour at a high level that is “totally unacceptable for them to pay” as they do not have enough revenue. In Tajikistan, managers of all six television outlets interviewed that had worked with A2I on the digital transition saw limited results from Internews’ A2I work with them on this issue. Managers of six of the seven television outlets interviewed viewed the upcoming switch as threatening to their survival. The one station that did not have this perspective nevertheless recognized that they were not prepared; their strategy of purchasing their own multiplex to use in their region had been resourced at a bit more than half the level needed to purchase this equipment. The station did not have the resources to purchase the rest of the equipment needed and was thus not prepared to broadcast digitally. The station did not note or discuss the legal framework/political environment and whether it was compatible with the strategy of having their own multiplex. CONCLUSIONS Performance data from A2I is not adequate to demonstrate that the project is meeting its targets for Objective 2. Internews support under A2I has been important in increasing the resilience of independent media. It has also made contributions to understanding the challenges and needs of independent outlets in shifting to digital broadcasting. Yet, A2I partners do not have a comprehensive understanding of the project and its potential avenues to support them in the digital transition or competitiveness. A2I partner IMOs need support to manage and survive the shift to digital broadcasting in Kazakhstan or Tajikistan. 14 USAID and Internews staff believe that Internews is meetings its objective in increasing the resilience of independent media to survive the impending, treaty-mandated shift to digital broadcasting (FY 2014-2017) and increasing resilience and competitiveness of independent media in the face of the changing environment (FY 2018). The digital transition is driven by governments and threatens the survival of independent TV stations. A2I work to support the survival of independent media with the shift to digital broadcasting was not seen as ensuring the survival of these outlets in Kazakhstan or Tajikistan, as the design and implementation did not focus on the area and thus did provide enough support to address the challenges. The future shift to digital broadcasting, with decisions in the hands of governments not supportive of IMOs, was considered as a threat to the ability of these outlets to provide citizens with access to civically-relevant information. Available information offers little guidance to help media outlets during the transition to digital. The digital transition is a critical priority for independent TV stations that most outlets are not prepared for and which many see as threatening their survival. A2I provides modest support on this critical challenge. Regional TV outlets have particular concerns about the financial side of the transition, and some even fear closure as a result of a lack of funding to produce quality content and decreased revenues from the advertising market. RECOMMENDATIONS Internews should revise its indicators to more completely tell the story of A2I’s contributions to support partners in resilience, competitiveness, and the digital transition. USAID and Internews should continue to focus on supporting the resilience and competitiveness of IMOs in Kazakhstan and Tajikistan even with limited ability to influence government decisions on the transition. USAID should work with Internews to develop approaches to support the digital transition for independent TV stations within the framework of the A2I and CAMP programs. The survival of partner stations appears threatened by this transition, which would reduce the ability of citizens to access civically￾relevant information that is the goal of the program. USAID and Internews should assess together what more can be done to support partners in this critical area. Internews should develop comprehensive information products on projects, make this information publicly or more widely available, and brief partners and potential partners on approaches and potential support. More information can help partners assess options and develop and implement plans to survive the digital transition. Internews should focus more efforts on strengthening the resilience of IMOs to be able to successfully make the digital transition. As part of potentially supporting the shift to a digital system in Tajikistan through a private, alternative multiplex in the north of the country, Internews should work closely through Khoma with interested outlets, the AIB, and the state regulator to develop a legal framework that can ensure such a system is securely placed in the country’s laws. Internews should also work closely through Khoma with independent television outlets in the rest of the country on advocacy within the legal and regulatory framework to find ways for these outlets to make the digital transitional within this framework and system. 15 4.1.3 Objective 3 Strengthened diversity and professionalism of voices in traditional and “new” media (FY 2018 Strengthened media and digital literacy, diversity and professionalism of voices in traditional and “new” media) This objective covers the wide-ranging A2I activities to support professional development in the media as well as initiatives to increase diversity in the sector. Internews set the following four expected results in the AMEP: 1) improved quality of news and information produced by professionals with new and/or improved skills; 2) a 20 percent average increase in original content by partner stations (annually); 3) increased diversity of funding sources in targeted media companies and/or an increase of 10 percent in advertising income (annually); and 4) a 25 percent growth in the audience for partner online information outlets. FINDINGS Internews did not measure some performance indicators for this objective, such as percentage increase of revenues for targeted media outlets (following business sustainability consulting) for all outlets supported. These indicators were measured for a smaller set of targeted partner stations and media outlets. Strengthened professionalism, which was also to be measured by the level of quality news and information produced by professionals with new and/or improved skills, was also not measured systematically each year. The assumption was that the amount of training provided, by definition, increased professionalism. Performance data from the project measured gross capacity increased by the number of person training days provided. Person training days provided in Kazakhstan and Tajikistan over the four years had a moderate start (but one that exceeded the lower, stated Year 1 targets), showed tremendous growth in Year 2 (again exceeding targets), before declining by about half for Year 3 (although still exceeding targets), and declining again by about half in Kazakhstan while remaining steady in Tajikistan in Year 4 (exceeding the target for the two countries together). Measures of online audience growth through Google analytics showed large audiences and high rates of growth for the audiences of the outlets measured in Years 1-3 of the program in Kazakhstan and smaller audiences and less growth in Tajikistan. For Year 4, A2I developed some indicators in the new AMEP that measured improved professionalism and set targets. A2I was very close to meeting or exceeding these indicators for Year 4. While performance data under this objective in A2I shows that targets set by Internews were exceeded, these targets are not very useful as they do not measure how activities have led to increased diversity or professionalism. Regarding diversity, Internews reports being “thoughtful” about gender equality in the development, leadership, and participation in all A2I events in quarterly and annual reporting. As the Year 4 Annual Report notes, the gender landscape is particularly challenging in Tajikistan for women. This challenging environment was the reason that only 31 percent of training days from A2I went to women in Tajikistan in Year 4. By contrast, in Kazakhstan, 64 percent of training day participants in Year 4 were women. Kazakhstan USAID and Internews’ A2I staff interviewed noted the many ways that Internews has supported both traditional and “new” media in strengthening their diversity and increasing professionalism. USAID interviewees noted the influence of Internews in the sector in spreading appreciation for inclusion. USAID key informants highlighted Internews’ efforts to work with outlets to create content that covers multiple points of view. Internews A2I support for diversity was seen as more than support for women in the sector; the approach was recognized to cover youth, the disabled, and other perspectives, including those of the lesbian, gay, bisexual, and transgender (LGBT) populations. Internews’ staff interviewed noted ways that Internews had supported both traditional and “new” media in strengthening their diversity and increasing professionalism. Management and staff emphasized that A2I 16 used a large number of different approaches—face-to-face training, mentoring and coaching through residencies, webinars, conferences, learning labs, and Master Classes—to reach different audiences and different types of learners. Internews, under A2I, solicited proposals from partners and funded multi￾media campaigns on innovative topics in Kazakhstan. These topics, such as sexual harassment, were seen as having raised awareness of these issues as well as having supported content production and sharing. In Kazakhstan, through the A2I project Internews staff encouraged greater representation of national and cultural diversity in shared media content and use of diverse multimedia tools and genres. However, some editorial offices recognized that they would benefit from greater diversity with respect to ethnic representation or representation of younger people. One interviewee noted a lack of diversity in the media industry with representation of ethnic or sexual minorities: “A lack of diversity is holding us back; ethnic minorities and LGBT must have the same rights.” With respect to gender diversity in Kazakhstan, media managers in the editorial offices in visited stations asserted that gender equality is not an issue for them. The substantial number and roles of women in the media in Kazakhstan was reflected in the participation statistics for the A2I evaluation, with women substantially outnumbering men among the ET’s key informants in Kazakhstan. In fact, only six of the 21 managers interviewed, four of the 14 beneficiaries polled online, and seven of the 16 participants in the FGDs were men. At the same time, some data for the evaluation noted that issues remain for women. One male participant in the FGD in Uralsk described the media industry as largely “sexist,” with women being portrayed in traditional and sexualized roles as some sort of “bio-machines whose only purpose in life is to give birth to children…and obey their husbands...What we lack is the right kind of content that will portray a woman as a leader in society, capable of taking independent decisions...and even when they choose a woman as a speaker, they tend to portray her in a patronizing manner.” To promote gender equality, 10 NGOs and media outlets launched a women’s empowerment initiative, Korgau123, aimed at fighting sexual harassment at workplaces. Thanks to the success of the campaign, the Association of Entrepreneurs was established on the initiative of the leader of the NGO that led the Korgau123 effort. Another result was that Kazakhstan moved to add sexual harassment to its ethics code. The Crisis Center Union, which fights against domestic violence, praised Internews for coming up with the idea of the project and helping to develop it further. With respect to raising professionalism, the FGDs, interviews, and online survey participants appreciated the support of “the school of Internews” for their own professional development in a wide range of media specializations. Informants showed continued interest in developing business-related skills, financial and legal issues, advertising, and attracting and engaging with digital audiences across multiple platforms. Commenting on A2I supported development program, a media manager said in the online survey: “It inspired us to change and grow professionally. It made us take a broader look at the role of the media and what the CSR [corporate social responsibility] means, share experiences....Every seminar adds to our knowledge of the media industry, increases our professionalism. According to informal feedback from peers, NGOs and members of the public in the region, we are presenting information in a much better way and look more professional now.” Tajikistan In Tajikistan, interviews with USAID and PAS informants emphasized the value of Internews’ extensive training through A2I to media outlets, journalists, and even bloggers and other new entrants into the media sphere in the country. One informant particularly noted the value of reaching new, younger journalists that have different approaches and want to do more, in contrast to older journalists who “seem already to have found their path in life” and are not looking for new approaches or change. 17 Internews’ staff interviewed in Tajikistan noted not only the historical role of Internews in providing training in the media but also the importance of practical, A2I trainings through a variety of methods for furthering the professionalism of the media under current conditions in Tajikistan. Interviews with managers from A2I-supported media outlets found great appreciation for the “school of Internews,” the battery of training that they and their colleagues had benefited from over many years of Internews’ programming in the region—not just from A2I training over the last four years. Eight of the nine outlets interviewed noted important support from Internews through A2I to them in training and advice, particularly through residencies, in the last four years. The one outlet that did not note this assistance over the period was new; they expressed appreciation and valued earlier Internews training support which had helped develop their management and staff when they worked for other outlets. One station manager in Tajikistan noted that independent outlets, including this outlet, face a “crisis in cadres, which is also then a creative crisis, which also comes from an educational crisis.” Another station in Tajikistan credited Internews’ support for staff development and Internews’ ideas about programming for the development of their own quality daily talk show, which the station noted “people talk about” which is a “very big thing” for the station and region that other stations do not have. CONCLUSIONS The targets and indicators measured by Internews for diversity and professionalism do not provide a comprehensive assessment of progress towards meeting Objective 3. Performance data on engagement with A2I exceeded the targets set by Internews but did not measure how activities have led to increased diversity or professionalism. Internews has successfully supported independent news outlets in Kazakhstan and Tajikistan in their professional development as well as promoting diversity in the media sector in a variety of ways through the A2I project. The A2I project encouraged representation of national and cultural diversity and gender equality in both countries through shared media content and use of diverse platforms and modalities to reach beneficiaries and to engage media audiences. RECOMMENDATIONS Internews should work with USAID to develop indicators that measure strengthened diversity and professionalism in more useful, comprehensive ways. Internews should develop comprehensive information products on projects, make this information publicly available, and brief partners and potential partners on approaches and potential support. USAID, through Internews, should continue to support gender equality through media campaigns and creation of gender role narratives that contribute to women’s empowerment. USAID, through Internews, should support coverage and inclusion of ethnic and other minorities and LGBT populations in the media. 4.1.4 Objective 4 Improved legal and policy environment for media (FY 2018 Improved legal environment for media and legal support to journalists) This objective mainly covers the work of Internews staff in Kazakhstan and A2I partner Khoma in Tajikistan in two directions: to provide advice and individual consultations with journalists and media outlets on legal issues, and to work with partners and stakeholders on legal and policy reform to support independent media. Internews set the following four expected results in the AMEP: 1) three new policies that strengthen the rights and viability of non-state media are adopted (at least one in each participating 18 country); 2) attempts at censorship or media interference by the state are prevented; 3) 50 percent of journalists and non-state media outlets are able to access affordable legal support when needed; and 4) media representatives participate in three legislative processes aiming to reform the legal enabling environment for media (at least one in each participating country. FINDINGS Project reporting on Objective 4 covered both advocacy for an improved legal and regulatory environment for media and legal counseling and support for journalists or media facing undue prosecution. Internews’ reporting prominently featured the activity and successes of the Media Law Expert in Kazakhstan and A2I partner Khoma in Tajikistan on both sub-objectives. In Tajikistan, the OSCE also funded Khoma to take the lead on some of the key work done in the country towards the digital transition. Quantitative performance data from the project tallied the number of outlets supported and the number of instances legal support or consultations were provided to outlets. The numbers of outlets supported in Kazakhstan and Tajikistan exceeded the targets originally set by A2I for the countries in Years 1-3. The project reports the number of legal support consultations as 42, 77, 30, and 99 for Kazakhstan and 98, 227, 166, and 95 for Tajikistan for Years 1-4. Interviews with USAID and A2I staff and project reporting noted the less propitious legal environment for independent media and greater state interference in the media in Tajikistan compared to Kazakhstan. The extended modified project developed new targets and detailed measurements for the number of policies, regulations, and administrative procedures where United States Government (USG)-assistance or assisted partners were included in the analysis, stakeholder consultations, drafting, approval, and/or implementation for Year 4. A2I had more processes with which its staff or partners were engaged in Kazakhstan than in Tajikistan for Year 4. Kazakhstan USAID interviewees characterized this objective “as the most difficult objective to achieve” in Kazakhstan. USAID noted that supporting the development and improvement of the legal environment for independent media was an area where USAID had a comparative advantage relative to other donors. USAID interviewees emphasized that it is very important to continue Internews support in improving the legal and policy environment for media and providing legal support to journalists through USAID, given the continued changes in government rules and regulations in the country, which key informants see the government undertaking in order to create obstacles and challenges to independent media. Interviews with Internews’ A2I staff asserted that A2I legal support is very good for outlets and journalists because they lack information and are not well-informed on the many laws that affect the media, or on how they are changed almost annually. A2I-funded consultations are also very important when outlets and journalists have legal problems. A2I reporting, as well as staff in interviews, noted numerous examples where legal support reduced the level of pressure on media outlets and individual journalists so that they were able to continue to work. Legal support covers not only clear limits and threats to freedom of expression but also broader risks to media outlets from efforts to bankrupt or liquidate outlets. Internews staff recognize that the policy work under the objective is on both technical questions like regulations on types of advertising (e.g., alcohol and beer) and on more political issues like the use of Freedom of Information legislation—and that even technical issues can be part of political pressure against the independence of media outlets. Interviews with stations in Kazakhstan revealed that they greatly valued targeted support provided by the Internews Media Law Advisor, especially during residencies at their outlets. According to two interviewees, the legal advice is especially appreciated now that the situation in the country is getting “more oppressive” and “any attempts at dissent get suppressed.” 19 Interviews with media outlets in Kazakhstan revealed great appreciation for the efficient Internews￾Kazakhstan consultancy support for particular legal challenges they faced. Media outlets also benefited from targeted support during residencies, help with court cases, and legal advice during webinars. Key informants also expressed appreciation of Internews-Kazakhstan efforts into amending restrictive media legislation. According to the interviewees, the legal advice is especially appreciated now that the situation in the country has become more oppressive and a lack of knowledge about laws and by-laws might result in journalists being jailed, and their media outlets blocked or closed. The most recent case was on November 15, 2018 when a Kostanai newspaper, Nasha Gazeta, was blocked for the critical articles it published, with authorities denying any involvement. One outlet that received significant support from the Internews-Kazakhstan Legal Advisor based on their extensive legal issues noted that “very restrictive laws have been adopted, the rights of journalists are being infringed on...the lines between administrative liability and criminal liability are getting blurred.” In this context, the outlet benefitted from “colossal” help from the Internews Legal Advisor. While this manager was grateful for and praised individual legal support, this manager’s perspective was that here had been “zero results” with regard to Internews’ legal advocacy for the sector as a whole. The ET’s interview with the media watchdog Adil Soz showed that in a period from January to September 2018, 22 journalists were jailed, five journalists received threats, and 91 media outlets were either suspended or closed down. Adil Soz also reported that content blocking and distributed denial of services attacks are common. These issues are particularly difficult as they are not legal; police and prosecutors’ offices deny engaging in these activities. Editors of blocked publications cited particular stories that they believe lead to retaliation by state actors. USAID and Internews noted limited support from media associations in the country for legal reform and representation of the sector. Partner IMOs were not satisfied with the performance and advocacy of associations vis-à-vis the legal and policy issues they face in the sector. FGD participants also noted the need for help with legal technicalities in which they are not experts—and the detrimental effects that a lack of knowledge can have: “There are so many by-laws and amendments, even if you are not violating the law on media, you are probably violating some other law. It is more difficult to work now as anyone can claim his honor and dignity being damaged....In fact, they do everything to fray your nerves....In the end they make you just drop the story.” Tajikistan In Tajikistan, interviews with USAID and USG informants noted the importance of assistance through A2I partner Khoma to their ability to provide leadership from civil society in engaging with the government to support the development of the overall environment for independent media. A2I support was also seen as valuable for funding Khoma’s education and outreach work to make legal knowledge more widespread and accessible plus Khoma’s assistance providing legal protection through consultations for individual outlets and journalists. Internews’ A2I staff interviews in Tajikistan noted that support towards this objective was mostly delivered through Khoma since Internews’ engagement directly in legal and policy matters would be seen negatively by the regulator and government as “interference by Western NGOs.” Internews staff lauded Khoma legal counseling and engagement/advocacy work with the regulator Goskomteleradio. Internews also noted that Khoma has the contacts, connections, and style to be effective in the conditions of Tajikistan. Interviewed staff from Khoma noted that all the work that has been done in Tajikistan on the legal environment, including the legal environment for the transition to digital broadcasting, has resulted from Khoma’s work through A2I and other internationally supported projects. Khoma’s staff stressed the value of this partnership to A2I and their role in it. Khoma emphasized that state structures recognize the expertise Khoma brings and want the help of the organization; state structures also prefer working with 20 national organizations with whom they already have good relationships with to having to fight with international organizations on media and press freedom issues. Interviews with stations in Tajikistan found that stations that had received legal counseling (based on their solicitation of support for particular issues that they faced) greatly valued the assistance of Khoma-funded assistance through A2I to help address their particular legal issues. Managers from seven of the nine outlets acknowledged receiving useful assistance or valuing Khoma’s leadership and advocacy for the sector. The other two outlets had not received legal assistance over the last four years through Khoma and did not note their recent advocacy efforts. Media outlets that benefitted from Khoma’s legal advice valued having proactive, comprehensive legal counseling and preparation for audits on issues and legal changes that helped minimize potential issues in the future even more than the general advocacy. Some outlets noted that legal consultations were not as useful when the issues they faced were illegal government harassment, such as the government unlawfully blocking web access to the outlet’s site. This reflects a larger problem of the ways the government can and does illegally interfere in the media environment, an issue viewed by partner outlet managers as being outside of Internews’ and Khoma’s control. USAID and Internews, as well as partner IMOs, were critical of the performance and actions of the AIB vis-à-vis the legal and policy issues they face in the sector. The telephone-survey of individual beneficiaries in Tajikistan found that both participant individual beneficiaries of legal assistance (see Table 1) appreciated the results of individual legal assistance through Khoma. One noted that “finally after 20 years we [the outlet] have proper papers” as a result of this A2I￾funded assistance. On the 1-5 scale, both rated the assistance as 5 (the highest). CONCLUSIONS A2I is meeting its targets for Objective 4. A2I beneficiaries valued the legal advocacy work for an improved legal and regulatory environment in both countries. A2I’s support for improving the legal and policy environment was appreciated in Kazakhstan, particularly by outlets that received support. A2I-supported legal advocacy and legal support provided by Khoma was valued in Tajikistan. An improved legal and regulatory environment is a critical priority for IMOs in Kazakhstan and Tajikistan. The variety of legal consultancy tools in A2I provides media outlets broader access to support to help them with the different legal challenges they face in both countries. Legal literacy is an area where Internews could provide more support through A2I in both countries. A broader approach and more partnerships could be considered as part of USAID and Internews support for legal reform. RECOMMENDATIONS USAID should continue to support legal assistance through Internews and its partners for media outlets and journalists in Kazakhstan and Tajikistan as the services are needed by independent outlets and individual journalists. In addition, Internews and its partners can effectively provide these services. USAID, through its partners in both countries, should continue to support the needed outreach and advocacy work towards an improved legal and regulatory environment for independent media and journalists in both countries. Internews and its partners should continue to support media outlets or journalists facing undue persecution through targeted residencies and legal counseling. Internews and its partners should also continue to provide more general information through webinars and seminars. 21 Internews should continue to provide legal literacy training for media managers. It should also consider creating a question and answer (Q&A) page on legal issues for the newsroom/website or circulating a newsletter among media managers containing the most up-to-date legal advice. Internews should, jointly with other legal advocacy groups, continue to work towards bringing about changes to legislation to move closer to international media standards. Internews should consider setting up and supporting fora with legal advocacy NGOs to examine specific legal challenges media outlets or journalists face in each country. 4.2 EQ2: WHAT GAPS IN ADDRESSING THE PROGRAM OBJECTIVES STILL EXIST? AND TO THE EXTENT POSSIBLE, DESCRIBE WHY/HOW GAPS EXIST. As with EQ1, a limited set of informants had the knowledge and understanding of A2I to address the question of gaps. To address whether there are gaps and their origins, informants first need to know and understand program objectives. USAID and Internews staff knew objectives and could address questions of gaps, as could some of A2I’s main partners. However, most A2I partners—even those from outlets that work closely with Internews—did not have the detailed information about A2I objectives needed to be able to address questions of gaps. The conceptualization of gaps that most partners had was whether there were needs that they had which were not being met by A2I – whether or not these needs were part of the objectives of A2I or not. Performance data was not useful to address this EQ. FINDINGS Kazakhstan Both USAID and Internews’ A2I staff interviewed in Kazakhstan identified a lack of regular audience viewership research as a gap. This view was shared by nearly all media managers interviewed who said that the research could help them improve their programming and attract larger audiences. “We don't do measurements, therefore no big name advertising companies come to us. And we don't know who watches shared content, if any. So, we are forced to work under the so-called goszakaz [state orders], which guarantees you a steady inflow of money,” one FGD participant was quoted as saying. Internews’ staff appreciated USAID’s flexibility and the support that enabled A2I to do the one study on seven outlets as well as ways to use this information for advertising and resilience. However, Internews also noted that more regular and more systematic research was needed. Internews staff felt that A2I had gaps in several areas. A2I staff noted that they lacked the technical specializations that they sought to use in the Newsroom 2.0 activities under A2I to target some kinds of specialists. This gap was being addressed in CAMP. Another gap identified was that the Innovation Lab had limited finance for grants. The solution of helping participants with their own fundraising was considered insufficient help for Innovation Lab winners or participants as not enough of them were able to find funding. More broadly limited finances under A2I was also seen as a gap, especially in the unexpected context of the economic crisis. Internews staff felt these limits—in particular in supporting residencies to support media convergence—and felt that only having limited technological equipment and distance support rather than additional residencies created gaps. The limited amount of professional development and training though A2I was also seen as inadequate since so many people are leaving the profession and there is so much turnover. In addition, the continued need to retrain those who remain in the field was noted, as technologies and the situation in Kazakhstan change. Staff noted that Internews can do more to address these gaps in the CAMP project. Funding limits were also noted as driving the modalities used under A2I, with fewer face-to-face methods and more webinars and distance consultations. Internews noted that the tight budget left no funds for rapid responses to new changes, where A2I might want to quickly form a round table or hold a conference on an emerging issue or opportunity. Furthermore, Internews staff noted that the media sector lacked the effective associations to address the needs of the sector with the government, particularly in new media. Internews also asserted that A2I had concentrated on supporting 22 media content towards competitiveness, but needed to do more to support the development of management and business skills of media outlets to support their competitiveness as well. This was a gap now being filled in CAMP. Kazakh language support was also seen as a gap. A2I had limited staff capacity to address this area, and the subgrant for further Kazakh language support was only funded in Year 1 of the project. Internews management felt that A2I was too lean in staffing for all components, particularly in legal support and the Kazakh language. Internews interviewees also noted that coordination in the DRG sector was limited—and saw this as a gap. Internews suggested that USAID should take the lead and do more to encourage coordination and cooperation across implementers in the sector. Internews managers also considered not working with government media as a gap. This gap was being addressed in CAMP, which works with financing government media managers and journalists because they are colleagues in the sector and face some of the same problems and opportunities. In interviews and online surveys, media outlets identified their lack of knowledge about the A2I program, the trainings available, or the selection criteria for joining them as additional gaps. Managers from nine out of the 21 stations interviewed described a lack of engaging Kazakh language content as a gap, as well as the lack of Kazakh language experts and lawyers. Two media managers noted a “huge gap” between how the Kazakh language and Russian media outlets cover the same events and called for “a systematic study” into the differences. They suggested a training focus on Kazakh language journalism was needed to offset the strong Russian influence on the media. Two journalism lecturers and three media practitioners highlighted the gaps between the skills of journalism grads and the demands of the media industry. They noted that regional journalism lecturers do not have any of the multimedia skills they are supposed to be teaching and asked for Internews’ assistance in closing that gap and developing training modules for journalism faculties. Two media managers asked Internews to help develop a cloud-based solution for sharing large raw files from Concord Exchange, the modality for sharing established through A2I under Objective 1, citing the challenges in sharing large files as a gap. One FGD developed in such a way as to identify yet another gap. Participants discussed a lack of “a shared platform for journalists to discuss arising issues with state officials....and help them through discussion to nudge officials in the right direction” and asked Internews to help fill this gap. Tajikistan Interviews with USAID staff in Tajikistan emphasized the need for more targeted assistance in country rather than that provided regionally in A2I. USAID emphasized that this gap was filled with CAMP. Other USG interviewees were not familiar enough with the objectives of the A2I program to address the question of gaps. Internews staff interviews in Tajikistan noted the limited reach of the program to universities on journalism training. Internews’ had begun addressing this gap, but, in the view of Internews staff, these initial efforts were not substantial enough. Internews staff emphasized that the gaps were now being filled by CAMP— but were concerned that in the context of the huge brain drain that journalism faces in Tajikistan, this support would also not be sufficient. Internews staff also saw the limited number and capacity of local Tajikistani experts in areas of the media as a problem, and the fact that A2I did not address this problem as a gap. The solution suggested was more training, including for A2I staff, to develop a pool of local experts that would be large enough and strong enough to endure even with the ongoing brain drain in the media field. The limited emphasis in A2I on face-to-face methods was also seen as a challenge. Given the conditions in Tajikistan that limit Internet outreach and limit remote approaches, additional in-person assistance was seen as needed and the limited resources allocated to this area as a gap. This lack affected communications with partners, limited program development as partners “will not always tell you through 23 telephone that they have problems and will hesitate,” made it hard to show that Internews is “thinking about them and supporting them,” and inhibited the approaches to addressing their real problems. An annual partners meeting was suggested as one way to partly fill this gap. Two interviews with managers of media outlets in Tajikistan noted gaps; the other seven did not have the knowledge of program objectives to provide data. One outlet manager noted two gaps where assistance had not been enough to meet their needs—in A2I’s support for the digital transition and support for training their staff. The other outlet saw A2I’s lack of support for the development of a measurement system for the audience of independent outlets—which they needed to be able to more effectively sell advertisements since they would be able to provide evidence of their audience—as a gap. CONCLUSIONS A2I’s partners and beneficiaries would benefit from a more comprehensive understanding of USAID and Internews media assistance programming. This more comprehensive understanding could be used to develop stronger consultative, collaborative ways for Internews and partner outlets to work together on shared priorities and ways to meet them. USAID and Internews need a consultative process to address key areas in media assistance that they and some partners view as gaps. Internews could then help fill the priority gaps that are identified. RECOMMENDATIONS USAID should make more information about media assistance programming public. Internews should provide more comprehensive information about media assistance programming, including the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. USAID and Internews should consider developing consultative processes to focus some program resources on areas that are considered gaps. This process should be conducted in the context of the CAMP project. USAID and Internews should consider bringing some partners into this discussion how to address key areas in media assistance. 4.3 EQ3: GOING FORWARD, HOW, AND IN WHICH AREAS, SHOULD USAID￾FUNDED MEDIA ACTIVITIES BE CONDUCTED TO SUPPORT JOURNALISTS AND OTHER MEDIA ACTORS IN REPORTING OBJECTIVE INFORMATION TO THE PEOPLE OF KAZAKHSTAN AND TAJIKISTAN? While few stakeholders, partners, and beneficiaries were qualified and able to address questions about meeting objectives (EQ1) and gaps (EQ2), interviewees all had views about future priorities for USAID￾funded media activities to support journalists and other media actors reporting objective information. Many interviewees framed their views around their years of experience with Internews’ assistance in the region. A2I was viewed as providing much less support than in the period prior to 2013 when Internews had a much larger award from USAID and was correspondingly seen as much more active in providing media assistance. The desk review and performance data were not useful to address this EQ. FINDINGS Kazakhstan Both the USAID and Internews staff interviewed highlighted the value of ongoing support for regional independent media in content production and professional development. 24 In the FGDs, electronic surveys, and interviews, media managers identified some of the skills and techniques they particularly want their journalists to develop: digital skills; multimedia storytelling; fact checking; critical thinking; the ability to access, analyze, and clean open data and visualize and compress the information; and the ability to engage with different audiences across multiple platforms. One participant followed up with their hopes for what could come next: “A storytelling course, a detailed one. I am dying to put it all into practice! Having completed an online course on infographics and then sharing the knowledge laterally, I can safely say that I prefer face-to-face trainings.” Five of the 21 managers of stations interviewed emphasized the need for business-related training and training for advertising sales. Another three managers focused on the need for training to create more technologically capable cameramen, film cutters, and photographers. Two media managers urged Internews to give priority to training media outlets that do not take “goszakaz” and are free from government control. Twelve (12) of the 21 station managers, however, expressed preference for customized USAID-funded assistance from experts through residencies, which they saw as the most effective support for the broad needs of their outlets across a range of areas. E-mail survey beneficiaries noted both the benefits of activities and ideas for improvements. For example, one noted that “Master classes were very informative, the only drawback was that due to a high number of participants, there was not much time for Q&As or practical exercises. Working in smaller groups could have been more productive.” The FGD in Almaty consolidated the thinking of participants around a need for “some kind of common platform” for media actors to engage, share, and move forward together. The discussion focused on the need of participants in the media sector to share electronically online, as well as to analyze and fact-check this information in order to use it. Almaty participants felt this would be especially useful for regional media outside of Almaty. Almaty participants also felt that support to make city government more accessible should be a priority. With respect to how USAID-funded media activities should be conducted, the online survey showed that, out of the 14 beneficiaries surveyed online, 12 preferred webinars as the quickest and most effective way to reach larger numbers of journalists in a flexible way that professionals can access at their convenience at a time that best suits them (rather than in a fixed time and place as in face-to-face training). One person interviewed online and one in an FGD asked for joint trainings with the media and “government press secretaries—as you get the impression that those who put questions and those who answer them live under different laws.” Asked about the areas they need help with in the future, three different FGD participants also highlighted the challenges and need for support to work with the state on access to information. “We find ourselves in an information vacuum, something that USAID can help us with. In recent years it has become impossible to work with the internal affairs directorate or the prosecutor's office...as they have to ask for the prosecutor-general's approval all the time.” – Participant 1 “It was easier to obtain civically-important information five years ago. True, now we have e-portals, some open data sources...but when it comes to accessing civically-important information about budget processes, security, economic decisions that government and parliament take, getting access to that kind of information has become more difficult, and the situation is getting worse. ” – Participant 2 “USAID can help arrange seminars, workshops, conferences/round tables between journalists and state officials and also with MPs [Members of Parliament] to discuss legislative work...and bringing the bills in compliance with international norms. This also applies to access to information, which unfortunately we don't have at the moment.” – Participant 3 25 One FGD participant and one electronic survey participant asserted that the “best help USAID could give is to provide us with targeted grants to produce multimedia products,” that help would be, they argued, “very concrete and measurable. Tajikistan USAID interviews in Tajikistan sought more focus in media assistance on Tajikistan—“on our context, our capacity, our experts, and local success stories.” PAS interviews emphasized reaching younger journalists and managers with the drive and orientation to run the media as a business rather than as public relations or propaganda. Two interviews with managers of media outlets in Tajikistan noted gaps; the other seven did not have the knowledge of program objectives to provide data. One outlet manager noted two gaps where assistance had not been enough to meet their needs—support for the digital transition and support for training their staff. As previously stated, the other outlet saw the fact that A2I had not supported the development of a system to measure the audience of independent outlets (needed to effectively provide audience size and sell advertisements) as a gap. Interviews with station managers in Tajikistan prioritized support for the digital transition; seven of nine managers sought a priority for future USAID-funding in this area. Two sought additional training and technical assistance; their priority was having a larger, more robust Internews program “like Internews had prior to A2I.” One manager prioritized grants for outlets (including but not only grants to support the digital transition). All of the 18 individual beneficiaries reached through the telephone-survey found a desire for more of the same support or similar support to what they had received through A2I. For example, one individual legal assistance beneficiary sought additional legal training while another beneficiary of individual legal assistance focused on the need for more training in management to support the future development of their outlet. And individual beneficiaries of the Media and Social Innovation Lab sought more follow up trainings, plus grants from A2I to implement the ideas they had worked on at the lab. FGDs with beneficiaries in Tajikistan mentioned an appreciation for training. Legal training was noted as especially important since journalists and outlets must follow the law, but have a hard time doing so as information and knowledge of the law is generally lacking. FGD participants mentioned that they also sought financial support and equipment assistance going forward. Participants also supported more training. CONCLUSIONS Internews’ work towards the goal and four objectives of A2I are important in supporting the provision of objective information in the two countries. Through A2I, Internews’ diffuse reach to diverse beneficiaries (individuals and organizations) limits the magnitude of the support for particular beneficiaries. USAID and Internews need a process to determine priorities in media assistance in these countries. The digital transition is a critical priority for independent TV stations that most outlets are not prepared for, which threatens their survival. To date, Internews support through A2I for the digital transition has been limited and modest. RECOMMENDATIONS USAID should continue to support media assistance in Kazakhstan and Tajikistan towards the A2I’s goal and its four objectives. Internews should continue with A2I in its fifth and final year, and should develop and implement additional programming in media assistance towards this goal and these four objectives in these two countries. 26 USAID and Internews should consider focusing future assistance through Internews on media outlets and professionals. As part of focusing assistance, USAID and Internews managers should work together to focus on priorities to support selected IMOs and media professionals in both countries. USAID should work with Internews to develop approaches to support the digital transition for independent TV stations within the framework of its USAID programs. Internews should provide additional assistance to independent TV outlets with the critical challenges they face with digital transition. 4.4 EQ4: GIVEN THE MEDIA ENVIRONMENT IN CENTRAL ASIA AND GOAL OF THE PROGRAM, WAS THE DESIGN AND STRUCTURE OF THE ACTIVITY APPROPRIATE TO SUCCESSFULLY ACHIEVING THE GOAL? Similar to EQ2 on gaps in the A2I project, the ET anticipated that few stakeholders, partners, and beneficiaries would have the knowledge of program design and structure to address this EQ. Thus, the ET did not ask about whether the program design was appropriate in all interviews—only in interviews where station managers asserted that they knew and understood the design. Individual beneficiaries were not asked through e-mail or the telephone about the design and structure—although some volunteered information relevant to addressing this EQ. FGD participants, however, participated actively in discussing gaps based on their understanding of the program areas they were questioned about earlier in the FGD. However, they overwhelmingly placed their understanding in the context of the media environment in their country and seldom were able to speak directly to the design and structure of A2I. These participants discussed whether A2I had gaps in terms of its ability to affect their national and local contexts in these areas. Performance data and the desk review more broadly was not useful to address this EQ. FINDINGS Kazakhstan USAID interviews felt that the design and structure of A2I was appropriate. Internews managers and staff also asserted that the design and structure of A2I was appropriate for achieving the goal and four objectives of the program. All but five of the 21 managers interviewed agreed that the design and structure of the A2I program is appropriate for a more limited resource environment, the environment in which USAID and Internews now operate. Overall, interviewees rated A2I’s design and structure more critically than FGD and electronic survey participants. Five out of the 21 interviewed said they preferred the prior design; these five criticized the current program as not well structured, a bit “haphazard,” and “fragmented.” These five media managers suggested improvements to the A2I program through carrying out a training needs assessment, allowing time to practice what was learned, staying focused on the A2I objectives, and “taking a holistic view of stations’ training needs.” Media managers sought more support to manage the challenges of the digital transition. Four interviewees and two out of the 14 participants in online surveys highlighted a need for a better awareness campaign about the training opportunities and objectives of A2I. For example, one interviewee said they felt as if they were “left to their own devices” in engaging with the program. The focus group beneficiaries in Kazakhstan that rated the design and structure in terms of increasing citizens’ access to socially relevant information gave a score of 2.8 out of 5.0. Almaty participants did not rate this area as they felt they did not have enough information to do so, leaving this rating based on 10 participants from the other two FGDs. Uralsk participants were much more critical, rating the design and structure a 2.0 out of 5.0, compared to Karaganda. That A2I was considered to have only modestly 27 changed the situation with respect to information provision was seen by some participants as inferring that the design and structure of A2I was not ideal. Aggregation of all participants that rated the design and structure for regional cooperation to support competitiveness from the three FGDs with beneficiaries in Kazakhstan provided a 4.2 rating—the highest of any area rated. Almaty participants were the most positive (providing a rating of 4.8 out of 5.0), with Uralsk participants again being the most critical with a rating of 3.3. Ratings were clearly shaped by recognition of the importance of support in this area and appreciation for A2I activities. FGDs with beneficiaries in Kazakhstan that endeavored to rate the design and structure in terms of supporting the transition to digital broadcasting found few participants were able to rate A2I in this area. Only five people were willing to provide a rating, with only Uralsk having more than one participant to do so. The overall rating of 2.8 as well as the unwillingness to rate this area based on a lack of information shows the overall lack of knowledge which was seen as a problem for the design and structure of A2I. Participants noted limited preparedness of outlets and the population and were critical of the lack of Internews leadership and resources devoted to this area. Aggregating all participants that rated the design and structure for promoting professionalism and diversity, including the role of women, and professionalism of traditional and “new” media in FGDs with beneficiaries in Kazakhstan providing a rating of 4.0 out of 5.0. Surprisingly, only six people rated this area in the three focus groups. Internews attention to gender was recognized by participants, who also noted that the media sector in the country has many women leaders. FGDs with beneficiaries in Kazakhstan rated the design and structure in improving the legal and political climate for media overall as 2.3—the lowest of the five areas rated. As with regional cooperation, participants felt they had adequate information to comment on this area and did so. Almaty participants were the most critical, with the three participants giving the design and structure a score of 1.3. Uralsk and Karaganda participants were larger in number and more positive, with scores of 3 and 2.7 respectively. Some FGD participants in Uralsk and one media manager interviewed by phone highlighted the design and structure of the program in this area positively as “a good mix of professional skills training and legal advice.” One noted, “Of all the instruments they use the most effective one is [the] legal assistance they provide” and second “workshops that teach you professional skills and media campaigns which they conduct when one of us comes under pressure from authorities.” The two data journalism training participants reached were very enthusiastic about this training. One said, “That was a mind-blowing experience. My books are all in bookmarks, I also keep links to sites...I have been applying what I learned in analyzing the data and cleaning it...The best training ever for me...can't ask for more.” The other participant noted that “Data journalism training was helpful in terms of identifying the goals we should aspire to and arming us with relevant tools, but there was not much time to practice what we learned.” Tajikistan USAID interviewees in Tajikistan noted the design and structure was “generally” appropriate to successfully achieving the goal of the program given the media environment. Other USG interviews were not familiar with the design and thus did not comment. Internews staff interviewees compared the A2I design to the previous “huge portfolio” of Internews and found A2I wanting. The design was considered as problematic—as hard for Internews to tell clients: why they are meeting them less, why they are providing less technical support, why they are “not able to provide as much as they wanted from lack of resources,” etc. The A2I design’s focus on remote methods was seen as problematic given the conditions in Tajikistan and the need to demonstrate that Internews cares about its partners by being physically present. With the cost extension of the project, activities moved away from the online school of journalism to more face-to-face methods which staff saw as more appropriate in Tajikistan. Internews Tajikistan staff saw the regional design as appropriate as Tajikistan needs to both share and learn from others, in conditions where “it is no secret” that Tajikistan is “last among the Central 28 Asian countries.” The design was also seen as appropriate when planned and the country was not in an economic crisis. However, the economic crisis then made the design dated. Staff praised the regional Internews office and USAID for supporting annual work planning changes in Tajikistan that were used to make adjustments to A2I to take into account the crisis. Two of the nine media managers interviewed in Tajikistan were able to address program design; seven managers felt they did not know the program well enough to answer the question. Both managers that answered the question assessed the A2I design and structure as too small. These managers framed their understanding of the design compared to the prior Internews program, which was seen as much larger [although managers did not know the United States (U.S.) dollar size of the award or the distribution of funds across countries, components, and activities]. One of these media managers characterized the A2I project as too modest in size and scope, noting that for their station, “the assistance is not a big meal of plov, but a small bit of butter with bread.” Focus group beneficiaries in Tajikistan rated the design and structure in terms of increasing citizens’ access to socially relevant information overall as 3.7 out of 5.0. Comments from the participants, however, made it clear that these ratings focused more on the general environment around access to information in the country where “people generally do not have access to information.” A2I was considered to have only modestly changed this situation. This thus suggested to some participants that the design and structure was not ideal. Aggregating all participants that rated the design and structure for regional cooperation to support competitiveness from the three FGDs with beneficiaries in Tajikistan provided a 3.4 rating. Ratings were clearly shaped by recognition of the importance of support in this area and appreciation for A2I activities but were placed in the context of not being enough support given the challenges independent media face in Tajikistan, especially relative to earlier larger Internews assistance. FGDs with beneficiaries in Tajikistan rated the design and structure in terms of supporting the transition to digital broadcasting overall as 3.0 out of 5.0. Participants noted limited preparedness of outlets and the population and were critical of the lack of Internews leadership and resources devoted to this area. Dushanbe participants were notably less critical than those in the other two FGDs. Aggregating all participants that rated the design and structure for promoting professionalism and diversity, including the role of women, and professionalism of traditional and “new” media in FGDs with beneficiaries in Tajikistan provided a 4.1 rating. Dushanbe participants were notably more critical than others. Internews attention to gender was recognized by participants, who also noted results such as the existence of more women journalists now. FGDs with beneficiaries in Tajikistan rated the design and structure in improving the legal and political climate for media overall as 3.7 out of 5.0. Participants recognized both Internews achievements from work in this area but also noted how rights continue to be violated and the tools now available for legal protection are not able to address the issues in this area successfully. CONCLUSIONS Accumulated findings suggest that the design and structure of A2I was broadly appropriate to achieving the goal of increased access to civically relevant information. The design and structure of media assistance projects like A2I could be explained more comprehensively to beneficiaries, potential beneficiaries, and stakeholders. The A2I training program targets a wide range of people, not only IMOs and professionals. IMOs have many needs that are not met in a comprehensive way with this design and structure of the A2I project. 29 Moving away from addressing the digital transition with the revision of A2I risks reducing attention to this critical challenge for IMOs. RECOMMENDATIONS USAID and Internews should continue implementing A2I with this design and structure in its final fifth year of implementation (FY 2019). USAID should make more information about media assistance programming public. Internews should provide more comprehensive information about media assistance programming, including the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. USAID should consider additional diplomatic efforts with other bilateral and international partners to support IMOs vis-à-vis legal frameworks and government policies on the digital transition. 4.5 EQ5: TO WHAT EXTENT WERE THE STRENGTHS AND WEAKNESSES RELATED TO PERFORMANCE OF THE IMPLEMENTER AND THE USAID TEAM (PARTICULARLY THEIR MANAGEMENT EFFECTIVENESS AND EFFICIENCY)? Quantitative performance data was not useful to address this EQ. Internews did however provide information through quarterly reporting on “challenges.” Internews also provided this information through informal discussions and telephone/e-mail contact plus weekly, bi-weekly, and monthly reporting to their Agreement Officer Representative (AOR) in Kazakhstan and the Activity Manager in Tajikistan. Few A2I beneficiaries were expected to know about the management done by USAID or Internews for A2I. USAID staff, Internews staff, and outlets with substantial contact with the program were asked questions to gather data towards answering EQ5. FGD questions on management did collect some data, although many participants declined to answer a question in this area based on their limited engagement and knowledge with A2I management. The e-mail and telephone surveys were not designed to collect data to use to answer this EQ. Internews managers noted that the change in A2I when Kyrgyzstan is taken out of the regional program due to the separation of USAID Missions was a hindrance that hampered their efficiency. This change led to several changes in planning and implementation that had to be done in ways that were not timely. FINDINGS Kazakhstan USAID managers emphasized the practices USAID uses to link everyone in program implementation from USAID and Internews through quarterly digital video-conference calls. USAID interviews noted the challenges Internews had under A2I in its first year of implementation in Kazakhstan with staffing, as key staff left over dissatisfaction with shifts in the program and the budget under A2I compared to the prior Internews program. These issues were seen to be managed effectively by the new leadership, which had the challenging work of moving away from production themselves, reducing the emphasis on the traditional media, and reducing the size of the staff. However, Internews program management was not engaged with Internews Kazakhstan, the legacy institution of Internews in the country, over continued personality issues. USAID emphasized that Internews management was always very responsive to USAID. USAID managers characterized Internews as efficient, as appropriately focused on indicators and how they are measured to be able to answer “so what” questions about the program. Annual improvement in indicators was seen as showing efficiency. USAID managers also reported doing their own mid-term review of the project and assessing efficiency. USAID also noted that Internews was proactive in suggesting changes to how A2I delivered activities and the kinds of activities under the program based on efficiency criteria. Internews was praised for checking with their headquarters management about whether activities 30 were effective, and whether they should do something different or how to act differently. These processes were considered as part of work planning and the generation of lessons learned. Internews staff interviewed voiced appreciation of the partnership with USAID in both countries under cooperative agreement and spoke highly of USAID's contribution to A2I program and its “attentiveness to detail, and helpfulness.” A2I was modified and extended with a cost-extension in FY 2017. This modification adjusted the program in areas where it was seen as less effective in reaching its objectives. Internews staff viewed the revisions of the program with the cost extension at the start of FY 2018 as efficient, and helpful to Internews to modify the program to be more effective given the changed context. This was particularly noted in the change in Objective 2 from a focus on the digital transition to a broader objective around supporting the overall resilience of outlets, where A2I support on audience research and advertising was seen as more immediately effective. Internews managers also noted that USAID was starting to actively meet with some of the main government stakeholders in Kazakhstan to support A2I’s work, which was very useful. Asked about the quality and effectiveness of USAID management, interviewees, FGD and electronic survey participants valued the high level of support that USAID has shown throughout, highlighting its “professionalism,” “flexibility,” and “creativity.” “They have robust procedures, forms, and templates so you don't rack your brain, thinking how to write this or that report...They are eager to help,” an FGD participant was quoted as saying. According to two out of the 21 participants interviewed, USAID has been “keeping low profile, but recently stepped up its activities.” Another interviewee said they are aware of the support provided by USAID but choose not to mention its name so as not “to be accused of being pro-American” by social media commentators, a view that seemed to resonate with many others. Asked about the effectiveness of project implementation by Internews and consistency of delivery, interviewees rated its role more critically than FGD and electronic survey participants, who hailed Internews management as “effective,” “professional and knowledgeable,” and “truly helpful.” According to one interviewee, Internews “seems to be doing a lot, but the results are zero.” According to two out of the 21 participants interviewed, “Internews acts now more like a training hub than a focal point that highlights issues and lobbies for change” and “digitalization maybe an important issue but the issue of rights is more important.” Twelve (12) of the 21 station managers expressed preference for customized USAID-funded assistance from experts through residencies, which they saw as the most effective support for the many different needs of their outlets across a range of areas. When asked about what A2I techniques are most effective in the delivery of international support to journalists and other media actors, FGD participants had diverse answers and no consensus. Participants overwhelmingly validated the techniques used in the activities they participated in (e.g., Master Class participants praised master classes, fact checking training participants lauded training on fact checking). Some FGD participants in Almaty were particularly critical of A2I. As shortcomings, they highlighted an apparent lack of a comprehensive, integrated strategy for conferences/activities and a lack of handouts that, in their view, made for poor information sharing. One participant noted, “I don’t know about the whole program, as I attended only a couple of workshops...There was so much information, one presentation followed another and during the third presentation you simply switched off...as your memory can take only that much. I wish we had some handouts.” A second participant asserted: “I attended only two trainings but there was so much information to mull over and analyze…you drown in this flood of information. “ All survey beneficiaries in Kazakhstan asserted the opposite about A2I activities they participated in and said that they were well organized. For example, both Media and Social Innovation Lab participants reached called the activities very well organized and praised the ongoing support after the meeting from Internews (as did the cross-border exchange participant). 31 Tajikistan USAID sources asserted that the Internews team worked very well together and with USAID. The Internews team were viewed as good colleagues and “never obstacles to achieving program objectives.” The careful ways that Internews coordinated with USAID and that USAID managed between USAID/Central Asia and USAID/Tajikistan were viewed as evidence of management effectiveness by USAID. Internews staff noted their “excellent” relations with USAID as colleagues that consulted together on “everything” related to the media. This relationship was not only with their country manager but also with the U.S. Embassy and USAID in Tajikistan more broadly. Internews staff noted ways they were able to adjust features of the design that proved problematic in conditions of economic crisis through good understanding and cooperation with their regional office in Almaty and with USAID. Interviews with stations in Tajikistan found managers from four of the eight outlets that received A2I assistance over the period were willing to speak to the management performance of Internews. Managers of these four outlets were positive about the performance of the Internews’ A2I Tajikistan team. The telephone survey of individual beneficiaries in Tajikistan found that all 18 beneficiaries rated the management of the diverse activities they had participated on as a 5 on the 1 to 5 scale, with 5 being best. These training beneficiaries called Internews’ trainings very well organized. One compared this level of organization to other trainings and asserted that they chose to attend Internews’ trainings because they were stronger than the alternative from another unnamed organization. Two legal assistance beneficiaries felt that Internews and Khoma had provided good management in this area. FGDs with beneficiaries in Tajikistan noted the professionalism of Internews as a strength, but the limited transparency of Internews as a weakness. Participants particularly noted a lack of transparency in reporting project results. This focus on limited transparency may have been encouraged by the questions asked in the FGDs. Many participants were explicit in noting that they did not have enough information about the strategy of Internews or the performance of their staff to assess their (or USAID’s) efficiency and effectiveness. Participants returned to the earlier discussion of A2I’s four objectives and strategy to again note perceptions of the limited engagement and results of A2I in supporting the digital transition. Participants raised concerns that activities focused on the same sets of participants and should reach out more to new beneficiaries. Some participants noted a lack of feedback mechanisms with Internews compared to how the program had managed in the past. Another participant argued Internews should repeat longer-term monitoring of partners to assess their use of Internews-supported assistance and its results. As in Kazakhstan, media outlet managers interviewed were the most enthusiastic about the support provided to them through residencies (for outlets that had received a residency through A2I) or that could be provided to them through residencies (for outlets that knew about but had not benefitted themselves from a residency). When asked about what A2I techniques are most effective in the delivery of international support to journalists and other media actors, FGD participants in Tajikistan were enthusiastic supporters of whatever assistance they had participated in: residencies, trainings, cross-border exchanges, and even webinars. However other participants felt webinars were not effective in Tajikistan due to the poor quality and high costs of the Internet. The FGD participants with this view had usually not participated in web￾based trainings through A2I. CONCLUSIONS The USAID Agreement Officer Representative (AOR) and Country Manager engage with Internews A2I staff and manage the grant with the appropriate level of involvement. Dialogue and partnership between USAID and Internews in media are managed effectively. 32 Internews appears to manage the A2I grant acceptably through a Regional Director, Country Directors, small national staffs, a few partners, consultants, and engagement with the Internews home office. Given the wide range of the larger past Internews assistance and the still extensive activities of the A2I project, many partners, beneficiaries, and stakeholders have suggestions, ideas, and proposals for additional activities, directions, and support. These suggestions are less about strengths and weaknesses of management and more about opportunities and alternatives. RECOMMENDATIONS USAID should continue to work through grants with Internews to provide media assistance. In general, USAID managers should participate in and observe as many Internews media assistance activities as possible, with the exception of national-level media association and government dialogues. Internews should make more information available to partners and stakeholders about Internews programming, including program goals and objectives, delivery modalities, grants and training opportunities, and selection criteria. Internews should develop fora for project beneficiaries and potential beneficiaries to discuss not only the strategy and program approaches for projects like A2I but also how Internews proposes to (and has) managed for effectiveness. Internews should endeavor to work more with and through partnerships with national organizations that work in media, particularly in working to improve the legal and regulatory environment. USAID should work through developmental and diplomatic channels in support of Internews’ and national efforts towards a more open legal and regulatory environment for independent media in Tajikistan and Kazakhstan. 4.6 EQ6: TO WHAT EXTENT WERE THE STRENGTHS AND WEAKNESSES RELATED TO THE PROGRAM BEING REGIONAL VERSUS DISCRETE ACTIVITIES FOR EACH COUNTRY? USAID, Internews, and partner and stakeholder interviews were used to gather data relevant to this question, as were FGDs. Of the email and telephone surveys, only cross-border exchange participants were asked about regional implementation compared to national implementation. Performance data was not useful to address this EQ. FINDINGS Document review showed that Internews’ quarterly and annual reporting on “challenges” to USAID provided some data relevant to this EQ. Reporting on challenges by Internews noted the particular challenges that journalists, media outlets, and CSOs face in each of the countries separately. Review of these challenges over the four-year period (16 quarterly or annual A2I reports) showed both similarities and differences in the issues Internews reported on as challenging for the program. The specific features of legislation on the mass media in each country had aspects in Kazakhstan and Tajikistan that Internews focused on to support outlets and media actors. However, as these specifics differed, A2I’s support had to be delivered separately in the two countries targeting these different legislative and legal issues by the two Internews A2I offices and partner Khoma in Tajikistan. Internews noted from Year 1 and onward through Year 4 that high cost, limited accessibility, and low Internet speed from the monopolist provider Tajiktelecom in Tajikistan consistently hampered A2I’s efforts to implement activities through online methods, particularly webinars, media awareness campaigns, and the online school of journalism. To address this challenge, Internews used more face-to-face approaches in Tajikistan instead of online methods and, in some cases, provided Internet services to beneficiaries. 33 USAID/Central Asia management argued that regional programming was superior from learning and engagement across the countries. The most prominent examples were that Tajikistan can learn from the experience of the other two countries in the program through A2I. However, regional programming had even more benefits for cross-national learning as even Kyrgyzstan could learn from the other two. The knowledge from other countries and even healthy competition between outlets and media actors in the countries was seen by USAID as productive. This recognition was also accompanied by USAID staff noting the importance of local news and information for local independent stations, as audiences were seen to want and watch local news. Regional programming was also the only way that A2I could be funded through regional funds, and blend country-specific appropriations with regional funding. USAID and A2I managers felt that A2I had balanced the benefits of these two types of programming well. Given that “needs and levels” are different, some of the trainings conducted could stay country specific. Kazakhstan When asked, USAID, Internews, partner outlets, and beneficiaries in Kazakhstan noted important benefits from regional programming and largely saw regional programming as a positive feature of A2I. Internews interviews noted that regional programming brought benefits for the program as a whole and for Kazakhstan. This was particularly noted for support on the digital transition, where Kyrgyzstan had made the change. In content exchange however, Internews managers noted that Kazakhstani outlets only wanted Kazakhstani content through the content exchange, not Tajikistani content. Internews managers in Kazakhstan noted broader benefits from regional programming for Tajikistan through A2I. Regional programming was viewed as having continued utility and designed into the new CAMP three-country program. Interviews with station managers found that 16 of the 21 interviewed in Kazakhstan spoke in favor of regional events rather than country-focused events. Regional activities were seen as positive because they enabled managers to jointly shape the media market, network, and learn from the mistakes of others. Five interviewees on the other hand preferred country-focused programming for Kazakhstan from A2I as “the levels and needs are different” in Kazakhstan and Tajikistan. One station manager suggested expanding the number of states involved in the program and adding media outlets from Russia’s Uralsk as most of the Tajik migrant traffic goes via Kostanai to Russia’s Uralsk; another TV manager suggested adding Uzbekistan to regional exchanges as it develops “dynamically.” Outlet managers interviewed in Kazakhstan also recognized that conditions differed among Central Asian countries and that there were important benefits to targeted programming from A2I focused on Kazakhstan-specific needs. Almaty FGD participants asserted that implementing A2I as a regional program was positive. Uralsk FGD participants were not as positive about implementing A2I as a regional project and noted that content and examples from A2I were sometimes not useful for them as the information was from other countries where conditions were different from Kazakhstan. For example, one participant asserted that Kyrgyzstani programming on bride kidnapping was irrelevant to them. This critical perspective felt that the regional content distracted participants and Internews from focusing on the concrete, specific issues and opportunities that existed in media development in Kazakhstan. Participants noted however that this problem also had Kazakhstani dimensions: people in Almaty were said to not necessarily be interested in news and information from Uralsk. Participants in cross-border exchanges, the only individual beneficiaries asked about regional programming, appreciated cross-national modalities as a strength of A2I. 34 Tajikistan USAID, Internews, partner outlets, and beneficiaries in Tajikistan noted important benefits from regional programming and viewed programming that connected them with other countries of the region as a positive feature of A2I. USAID and USG interviews in Tajikistan noted that in the current environment, regional programming was a big advantage. Broadening activities acted as a “shield” to enable participation by people from Tajikistan and avoid government interference, as well as promote learning. However, USAID sources also emphasized the importance of balancing regional programming with national programming, and how the two types of programming were complementary. Other USG sources interviewed asserted “the more regional, the better” for programming. Internews’ staff in Tajikistan interviewed asserted that regional programming was a positive. In their view, Tajikistan not only needs to learn from the experience of neighboring countries, but also “partner outlets needed to go somewhere for lessons.” However, staff also noted that “due to language barriers, we need local experts.” Internews Tajikistan interviewees felt that they had balanced these needs and benefits in A2I. Having Kyrgyzstan drop from the program was seen a negative, as Tajikistan is “probably closer to Kyrgyzstan in terms of media development than Kazakhstan” which had made it “easier for us to learn from Kyrgyz colleagues than Kazakhs for us.” Khoma interviews found that staff felt that because it was “absolutely a different situation in Tajikistan,” a regional program was not ideal. Khoma noted that the focus of programming should be different for Tajikistan because the situation is different and asserted that there was too much attention to other countries in the program. This left Tajikistan getting less attention, which they saw as a problem. That said, learning from Kyrgyzstan on the digital transition through A2I was seen as a positive way to learn and working with Kyrgyzstani colleagues more generally was seen as a way to inspire Tajikistanis “that it was up to us to change the situation.” In interviews with station managers in Tajikistan, eight of the nine answered the question about regional programming when asked. Furthermore, seven of the eight who answered viewed the regional nature of the program as a positive. One media manager in Tajikistan noted that regional integration presented possibilities to show programming that was otherwise “taboo in their country.” With Internews support, the station was able to show unique content from Kyrgyzstan on the positive example of joint work between the mass media and prosecutors on preventing violent extremism in Kyrgyzstan. This manager noted that their coverage helps the population to understand these issues and ways of addressing them. The manager asserted that this leads the population to wonder why we do not have this kind of collaboration in Tajikistan. Regional integration was thus a benefit not only in showing what needs to change but also modeling how changes can be made through the examples of neighboring countries with similarities to Tajikistan. One station manager called cooperation across “Central Asia very good because we did not know anything about them and how do they do things” which makes “exchange very useful.” Content sharing across the region was seen as a way to learn about the changes in other countries, a way to acquire and show high quality analytical programming, and present new ideas. Regional media outlets and program beneficiaries noted unmet needs for increased knowledge of other regions of Tajikistan and the potential to meet this gap through A2I. Tajikistani station managers expressed an additional view on regional programming in their context. Two of the eight managers volunteered that they would benefit from additional regional programming and sharing across Tajikistan, noting that their programming and the populations they serve have limited access to independent information about other regions of Tajikistan. The telephone survey of individual beneficiaries in Tajikistan found the one cross-border exchange participant asked about regional compared to national activities remained enthusiastic about regional programming. Individual beneficiaries in other categories were not asked this question. 35 FGD participants in Tajikistan saw both positives and negatives from regional collaboration, and by extension from the regional strategy of A2I. One participant noted, for example, that “the advantage is the exchange of experience, and disadvantage is the fact that it is difficult to organize joint projects.” In Khujand, all participants asserted regional programming was an advantage. Dushanbe participants were less in agreement. One participant from Khatlon asserted that it would be “better for us if the project were intra￾regional, republican—carried out between the regions of the republic” and particularly sought grants and management for activities through media outlets in the regions of Tajikistan. FGD participants from all three discussions brought up and recommended regional cooperation within the country, noting that regional coverage across the Republic by journalists and independent outlets was weak and needed. CONCLUSIONS Regional programming through A2I has had important benefits in its support to independent media in Kazakhstan and Tajikistan. In some areas, the specific features of Kazakhstan and Tajikistan make national programming more appropriate. RECOMMENDATIONS USAID should continue to support Internews initiatives that connect media beneficiaries across Central Asian countries. USAID should consider supporting additional initiatives to share regional news and content across regions of Tajikistan. Internews should continue to provide both regional and national programming and determine a balance between these two different ways of combining beneficiaries. Internews should consider developing programs and a mechanism to support cross-Tajikistan sharing and cooperation of local news and information by independent outlets in the country. This type of sharing could be modeled on the existing regional sharing mechanisms and networking of Internews. Internews should continue to host regional events on topics of common interest to media actors in different countries of the region. Internews should also continue to conduct country-specific events and assistance through A2I on topics and information that target the interests and needs of specific countries. This particularly includes assistance delivered in the Tajik and Kazakh languages. 36 ANNEXES 37 ANNEX 1: STATEMENT OF WORK SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK C.1 TITLE OF ACTIVITY The USAID/Central Asia “Access to Information” (A2I) Activity Evaluation C.2 PURPOSE The purpose of this mid-term evaluation is to test primary objectives set forth in the A2I cooperative agreement. The evaluation will provide pertinent information, analysis, and judgments that assist USAID to learn what has been accomplished against the intended results of the cooperative agreement, as well as will give recommendations on how to achieve any results not yet attained during the final stages of the award. Specifically, the evaluation will help USAID better understand results from its media investments in Kazakhstan and Tajikistan. Activities in the Kyrgyz Republic were part of A2I in the first three years of the program, but the Kyrgyz portion of this activity will not be evaluated under this task order. C.3 BACKGROUND Central Asia is at a crossroads with respect to media development. The diversification of traditional broadcast news outlets and the proliferation of “new” digital, satellite and social media platforms have been embraced to various degrees by the people and governments across the region. Despite these developments, the availability of quality, civically-relevant programming remains low and greatly overshadowed by Russian-broadcast news and entertainment programs that reflect a worldview no longer fully relevant to Central Asians. In May 2017, Kyrgyzstan converted to digital television programming. Kazakhstan and Tajikistan postponed the transition to digital television until 2018 and 2020, respectively. Incomplete legislative frameworks, pernicious or incoherent government policies, and a lack of preparation by non-government stations jeopardized the future of television in Central Asia. Urgent support was required throughout this process to prevent the sudden disappearance of the already limited non-state media voices. While non-state media has struggled to survive and remain relevant, public trust in mass media has been eroding. This is largely due to low professional journalism standards and poor business practices in the industry. In addition, while digital tools generally enhance the ability to gather, process and disseminate news, the increased use of technology placed additional demands on journalists in terms of capacities, skills, and experience. C.4 PROJECT INTENT The A2I program’s period of implementation is from October 1, 2014 – September 30, 2019 with a total estimated cost of $4,999,390. The goal of the program is to improve governance outcomes by increasing citizen access to civically-relevant information. The program is regional, with regional and country-specific activities focused on Kazakhstan and Tajikistan. Activities in the Kyrgyz Republic were part of the program in its first three years, but the Kyrgyz portion of this activity will not be evaluated under this task order. Good governance is typically viewed as a two-way interchange between governance institutions and citizens. Access to civically-relevant information allows citizens to play an active role in the “feedback loop” with governance institutions -- be it by voting, advocating for improved service delivery, or speaking out against injustice. Access to information is a critical ingredient to participate fully in civic life and is fundamental to local and national government accountability. Collectively, constructive interactions between citizens and the state on good governance issues lead to sounder policy, more responsive government institutions, and improved social services, simultaneously reducing the potential of social, ethnic, and other forms of conflict. 38 The A2I program aims to increase citizen access to civically-relevant information through four complementary objectives: • Increased regional cooperation to improve the competitiveness of non-state media outlets through content generation and sharing; • Increased resilience of independent media to survive the impending, treaty-mandated shift to digital broadcasting across Central Asia in 2015; • Strengthened diversity and professionalism of voices in traditional and “new” media; and • Improved legal and policy environment for media. C.5 EVALUATION PURPOSE This is a mid-term performance evaluation that will help determine what components and project aspects are working well, which are not, and why, as well as to make informed decisions on the direction of the media portfolio in Central Asia. The purpose of this evaluation is to gauge the achievement of the primary objectives set in the A2I cooperative agreement. The evaluation should provide pertinent information, analysis, and judgments that assist USAID to learn what has been accomplished against the intended results of the cooperative agreement, as well as will give recommendations on how to achieve any results not yet attained during the final stages of the award. The evaluation will help USAID better understand its media investments in the region, notably in Kazakhstan and Tajikistan. C.6 EVALUATION QUESTIONS The evaluation will address the following key questions: • To what degree has Internews met the four objectives as stated in the Cooperative Agreement? • What gaps in addressing the program objectives still exist? • Going forward, how, and in which areas, should USAID-funded media activities be conducted to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan? • Given the media environment in Central Asia and goal of the program, was the design and structure of the activity appropriate to successfully achieving the goal? • To what extent were the strengths and weaknesses related to performance of the implementer and USAID team (particularly their management effectiveness and efficiency)? • To what extent were the strengths and weaknesses related to the program being regional versus discrete activities for each country? C.7 DATA ANALYSIS METHODS This evaluation will be qualitative in nature with some secondary, quantitative data analysis. Prior to the start of data collection, as part of the evaluation work plan, the evaluation team will develop and present, for USAID review and approval, a data analysis plan; what procedures will be used to analyze qualitative data obtained from key informant and other sources; and how the evaluation will weigh and integrate qualitative data from various sources with quantitative data from project performance monitoring records to reach conclusions about the effectiveness and efficiency of the A2I Program in Kazakhstan and Tajikistan. The contractor shall have flexibility in proposing the most appropriate methodology to accomplish the tasks outlined in the most efficient way. However, the contractor will address the following: 39 Task 1: Desk review The contractor will conduct a desk review to gain a general overview of the media sector in Central Asia, particularly in Kazakhstan and Tajikistan and identify areas (as related to the A2I objectives and evaluation questions) that merit closer attention once the team begins its fieldwork. Reading materials will be available to the team upon award. The contractor shall also review a broad range of other relevant non￾project related documents such as peer-review articles and other publications and incorporate the review findings into the data collection instruments, the data collection design, and data analysis and interpretation. USAID will provide the evaluation team with a package of background materials, including: • USAID Central Asia Regional Development Cooperation Strategy (2015-2019) • A2I Cooperative Agreement (2014) • A2I Work Plans, Years 1-3 • A2I Annual Reports, Years 1-3 • A2I Quarterly Reports, 2014 – 2017 • A2I Initial Performance Monitoring and Evaluation Plan, 2014 • A2I Annual PMEP Report, Years 1-3 • Internews CAR Media Assessment, April 2017 • Media Sustainability Index 2014, 2015, 2016 and 2017 for Tajikistan and Kazakhstan The final evaluation report shall triangulate information obtained from the pre-field work desk review and from the field to articulate A2I contributions to the objectives as well as any opportunities to strengthen achievements during the remaining time frame of the program and beyond. Task 2: Draft description of overall methodology and work plan In conjunction with USAID/Central Asia, the contractor will develop a plan for conducting their fieldwork using information from the desk review and this SOW. The work plan will clearly outline the methodological approach the team will use. The work plan shall be submitted for review to the USAID/Central Asia COR for approval prior to arrival in country. The work plan shall include a fieldwork schedule, including a preliminary list of all proposed districts for travel and key organizations the evaluation team plans to meet. Task 3: Conduct a field data collection The in-country data collection includes, but is not limited to, facilitated discussions through interviews with the following counterparts: KAZAKHSTAN Internews Network Partner CSOs Other CSOs Partner media outlets Government of Kazakhstan Donor counterparts Business counterparts Journalists, media specialists (beneficiaries) 40 TAJIKISTAN Internews Network Partner CSOs Other CSOs Partner media outlets Government of Tajikistan Donor counterparts Business counterparts Journalists, media specialists (beneficiaries) The evaluation team will spend at least three weeks in the region with the evaluation team visiting at least four cities in Kazakhstan and three cities in Tajikistan. Travel to Kyrgyzstan is not included in the budget or SOW. The identification of informants/respondents and scheduling data collection must be arranged as far in advance as possible to maximize efficient use of time while in the field. USAID/Central Asia will provide introductions and contact information to facilitate the data collection. 41 ANNEX 2: EVALUATION INSTRUMENTS Key Informant Interview Questions 1. Please tell us about your background and experience in the media here. 2. How did you first learn about Internews and the potential to benefit from the A2I project? 3. What support have you or your station/media outlet received from A2I over the past four years— and what activities of A2I have you participated in? 4. What have been the results of Internews’ work with you through A2I to: OR What have been the results that you know of from Internews’ work through A2I (for stakeholders rather than direct partners): a. improve the competitiveness of non-state media outlets through content generation and sharing? b. support the resilience of independent media in the shift to digital broadcasting c. strengthen the diversity, including the roles of women, and professionalism of voices in the media? and d. improve the legal and policy environment for media? 5. Are there effects in terms of increasing citizen access to civically-relevant information? 6. Based on your experience with A2I, do you see gaps that remain in the program in these four areas? What can programs like A2I do to fill these gaps? 7. What would you recommend for USAID-funded media activities to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan? 8. How would you prioritize these recommendations? 9. What techniques are the most effective in the delivery of international support to journalists and other media actors? Why are these techniques more effective than others? 10. Based on what you know of A2I and the media environment in Central Asia, do you think the design and structure of A2I was appropriate to successfully achieve the four goals of the program? 11. Do you see strengths and weaknesses in Internews’ management of A2I? What have been the strengths and weaknesses in their effective or efficient management of A2I? 12. A2I was developed and has been managed as a regional program linking Kazakhstan, Tajikistan, and Kyrgyzstan versus discrete activities for each country. Has the implementation of A2I as a regional program versus having discrete activities for each country affected program results? Have there been added benefits to A2I through regional implementation? If so, what are these benefits? 13. Has implementing A2I regionally led to any weaknesses in the program? 14. Finally, is there anything else we have neglected to ask about the A2I program or your experience with it that we should discuss? 42 Focus Group Discussion Questions Based on your experience with the Internews A2I program, what techniques are the most effective in the delivery of international support to journalists and other media actors in your country? Why are these techniques more effective than others? What would you recommend for USAID-funded media activities to support journalists and other media actors in reporting objective information to the population in the future? How should USAID activities support journalists and other media actors to increase the reporting of objective information in the future? [FOR EACH SUBQUESETION, GO AROUND ROOM TO GET A NUMERICAL ESTIMATE FROM EACH PARTICIPANT (WHEN POSSIBLE) BEFORE LEADING A DISCUSSION OF WHY OR WHY NOT] Based on your experience with A2I, on a scale of 1 to 5 (5 being best), do you think that the design and structure of A2I’s work was appropriate to successfully: ● Increase citizen access to civically relevant information? ● Support regional cooperation via content generation and sharing to improve the competitiveness of independent media? ● Support the shift to digital broadcasting? ● Increase diversity, including the roles of women, and professionalism in traditional and “new” media? ● Improve the legal and policy environment for media? Why or why not? What do you see based on your experience with the project as the main strengths and weaknesses in how USAID and Internews managed the implementation of the A2I project? A2I is a regional program. Has the implementation of the project as a regional one—instead of one that focuses on only one country—had benefits or weaknesses in terms of its support to you and your organizations? How and why? 43 Kazakhstan: Questions for Individual Beneficiaries Online Media School for Journalists Participants How did participating in A2I Online Media School help your skills or your media outlet? In your opinion, how well was the A2I Online Media School managed by Internews? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Webinar Participants How did participating in an A2I webinar help your skills or your media outlet? In your opinion, how well was the webinar managed by Internews? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? MEDIATOURS Master Class Participants How did participating in an A2I MEDIATOURS Master Class help your skills or your media outlet? In your opinion, how well was the MEDIATOURS Master Class managed by Internews? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Data Journalism Training Participants How did participating in an A2I Data Journalism Training help your skills or your media outlet? In your opinion, how well was the Data Journalism Training managed by Internews? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? A2I Regional Conference Participants from Kazakhstan How did participating in an A2I conference help your skills or your media outlet? In your opinion, how well was the A2I conference managed by Internews? What else would you suggest for projects like A2I to further support your skills development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Individual beneficiaries of legal assistance How did Internews-provided legal assistance help you or your outlet? How was the quality of the legal assistance you received on a scale of 1 to 5? Why? In your opinion, how well was the legal assistance you received managed by Internews? 44 What else would you suggest for projects like A2I to further support legal assistance in the area of mass media? What else would you suggest for projects like A2I to further support the development of your own legal knowledge and skills in this area? Tajikistan: Questions for Individual Beneficiaries Media and Social Innovation Lab Participants How did participating in an A2I Media and Social Innovation Lab help your skills or your media outlet? In your opinion, how well was the A2I Media and Social Innovation Lab managed by Internews? What else would you suggest for projects like A2I to further support your skills development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Cross-border exchange participants How did participating in an A2I Cross-Border Exchange help your skills or your media outlet? In your opinion, how well was the A2I Cross-Border Exchange managed by Internews? What else would you suggest for projects like A2I to further support your skills development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? What difference did it make to you or your media outlet from Internews’ working in this regional way? If there were benefits, what were they? If there were costs, what were they? Conference attendees How did participating in an A2I conference help your skills or your media outlet? In your opinion, how well was the A2I conference managed by Internews? What else would you suggest for projects like A2I to further support your skills development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Media master class beneficiaries How did participating in an A2I Master Class help your skills or your media outlet? In your opinion, how well was the A2I Master Class managed by Internews? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? Individual beneficiaries of legal assistance How did A2I-funded legal assistance through Khoma help you or your outlet? How was the quality of the legal assistance you received on a scale of 1 to 5? Why? In your opinion, how well was the legal assistance you received managed? What else would you suggest for projects like A2I to further support legal assistance in the area of mass media? 45 What else would you suggest for projects like A2I to further support the development of your own legal knowledge and skills in this area? A2I Training Participants How did participating in an A2I Training help your skills or your media outlet? How was the quality of the training you received on a scale of 1 to 5? Why? Has, and how has, the training encouraged more diversity in the media? What else would you suggest for projects like A2I to further support your development in this area? What would you prioritize now for projects like A2I to further professional development for you and others in your profession? 46 Focus Group Discussion Questionnaire Based on your experience with the Internews A2I program, what techniques are the most effective in the delivery of international support to journalists and other media actors in your country? Why are these techniques more effective than others? What would you recommend for USAID-funded media activities to support journalists and other media actors in reporting objective information to the population in the future? [FOR EACH SUBQUESETION, GO AROUND ROOM TO GET A NUMERICAL ESTIMATE FROM EACH PARTICIPANT (WHEN POSSIBLE) BEFORE LEADING A DISCUSSION OF WHY OR WHY NOT] Based on your experience with A2I, on a scale of 1 to 5 (5 being best), do you think that the design and structure of A2I’s work was appropriate to successfully: ● Increase citizen access to civically relevant information? ● Support regional cooperation via content generation and sharing to improve the competitiveness of independent media? ● Support the shift to digital broadcasting? ● Increase diversity, including the roles of women, and professionalism in traditional and “new” media? ● Improve the legal and policy environment for media? Why or why not? What do you see based on your experience with the project as the main strengths and weaknesses in how USAID and Internews managed the implementation of the A2I project? A2I is a regional program. Has the implementation of the project as a regional one—instead of one that focuses on only one country—had benefits or weaknesses in terms of its support to you and your organizations? How and why? 47 ANNEX 3: LIST OF KEY INFORMANTS INTERVIEWED BY ORGANIZATION Kazakhstan USAID, AOR A2I Internews Central Asia, Chief of Party (COP) A2I Internews Kazakhstan, Country Director Internews Kazakhstan, Legal Consultant Minber NGO, Director Crisis Centers Union, Chief Executive Officer (CEO) ARGO (Civil Society Development Association), CEO ARGO, Editor-in-Chief ARGO, Partnership for Innovations Program COP Radio Mediametrix, Consultant Informburo.kz, Editor-in-Chief CSR (CSO) Otyrar TV, Acting Director Otyrar TV Rabat, Editor-in-Chief Rabat, Correspondent TVK Channel, Deputy Director, Correspondent TVK Channel, CEO Aigak TV Capital KZ Almata TV, CEO Baribar.kz, CEO Adil Soz, media watchdog Adil Soz, media watchdog Kazmediaindustry Public Foundation KazPressClub Exclusive.kz Ministry of Information and Communications Tajikistan USAID, Country Manager A2I U.S. Department of State, Public Affairs Service Manager U.S. Department of State, Public Affairs Service Staff Internews Tajikistan, Country Director Internews Tajikistan, Staff Internews Tajikistan, Staff Khoma, Director Khoma, Staff Regar TV Mavji Ozod Tiroz TV Azia multimedia SM 1 Radio 7 Sitora 1 ANNEX 4: FINDINGS, CONCLUSIONS, RECOMMENDATIONS MATRIX Main Findings Key Conclusions Recommendations Evaluation Question (EQ) 1: To what degree has Internews met the four objectives as stated in the Cooperative Agreement? ● Independent media outlets can discuss A2I support but do not know the specific objectives of A2I or the targets for meeting objectives. ● A2I partners do not have a comprehensive understanding of the project and its potential avenues to support them. ● Internews should develop comprehensive information products on projects, make this information publicly available, and brief partners and potential partners on approaches and potential support. Objective 1: ● Performance data from A2I demonstrates that the project is meeting its targets for Objective 1. ● Independent media outlets (IMOs) that are A2I beneficiaries stressed the value of quality, free content shared through A2I in supporting their competitiveness by increasing their audience at low cost. ● USAID managers and A2I staff noted continued partner operation and use of shared content by partners as evidence for meeting the objective. ● IMOs in Kazakhstan preferred to share more raw content, particularly with legal changes demanding more local content. ● IMOs in Tajikistan preferred to share more finished content based on capacity constraints for production and challenges with Internet use. ● A2I is meeting its targets for Objective 1. ● Internews’ assistance towards the objective of improving the competitiveness of non-state media outlets through content generation and sharing [Fiscal Year (FY) 2014-2017] and increasing access to information through regional cooperation, learning, and content sharing (FY 2018) has important benefits for independent regional media outlets. This assistance also leads to increased access to information for the population in both countries. ● IMO partners of A2I vary in their preferences for finished and unfinished content based on different capacity and constraints in the two countries. ● USAID, through Internews, should continue to support A2I’s regional cooperation in content generation and sharing to support the competitiveness of independent news outlets in the two countries. ● Internews should continue to work with partner outlets on ways to better share different types of content across different platforms as well as on ways to monetize this sharing for competitiveness. ● Internews should support content sharing of both finished and unfinished content as partner outlets have different preferences. 2 Main Findings Key Conclusions Recommendations Objective 2: ● Indicators for Objective 2 developed by the project are not useful for understanding progress towards the digital transition. ● USAID changed the objective and targets for FY 2018 to focus on competitiveness. ● Internews measured new FY 2018 indicators on the audiences of IMOs’ outlets for a small sample of assisted stations, which does not adequately measure the overall situation. ● USAID managers and A2I staff noted continued operation and engagement on digital transition and competitiveness by partners as evidence for meeting the objective. ● Surviving the digital transition is a critical priority for independent television (TV) stations. ● Independent TV stations noted that they remain largely unprepared for this impending digital transition. ● Managers of independent TV stations noted their lack of preparedness threatens the survival of their outlets. ● Performance data from A2I is not adequate to demonstrate whether the project is meeting its targets for Objective 2. ● Internews should revise its indicators to more completely tell the story of A2I’s contributions to supporting its partners in resilience, competitiveness, and the digital transition. ● A2I’s support has been important in increasing the resilience of independent media and has made contributions to understanding the challenges and needs of independent outlets in shifting to digital broadcasting. ● The digital transition threatens the survival of independent TV stations. ● A2I partner independent media outlets need support to manage and survive the shift to digital broadcasting in Kazakhstan or Tajikistan. ● The future shift to digital broadcasting threatens the ability of these outlets to provide citizens access to civically-relevant information. ● USAID should work with Internews to develop approaches to support the digital transition for independent TV stations. ● As part of potentially supporting the shift to a digital system in Tajikistan through a private, alternative multiplex in the north of the country, Internews should work closely with interested outlets through the media development organization Khoma, the Association of Independent Broadcasters (AIB), and the state regulator to develop a legal framework that can ensure such a system is securely placed in the country’s laws. ● Through Khoma, Internews should work closely with independent TV outlets in the rest of the country on advocacy within the legal and regulatory framework to find ways for these outlets to make the digital transitional within this framework and system. 3 Main Findings Key Conclusions Recommendations Objective 3: ● Not all performance indicators for Objective 3 were systematically and annually measured. ● Some quantitative targets have been exceeded or almost met. ● USAID, A2I staff, A2I partners, beneficiaries, and stakeholders noted many ways that A2I has supported both traditional and “new” media in strengthening their diversity and increasing professionalism. ● A2I beneficiaries valued project support in professional development and promotion of national and cultural diversity in shared media content. ● A2I beneficiaries appreciated the project’s support in promoting gender equality. ● Internews A2I support for diversity was seen as more than support for women in the sector; the approach was recognized by partners and beneficiaries to cover youth, the disabled, and other perspectives, including those of lesbian, gay, bisexual, and transgender (LGBT) populations. ● Performance data does not provide a comprehensive assessment of meeting Objective 3. ● Performance data on engagement with A2I that counts the number of individual beneficiaries with direct involvement in A2I activities exceeded the targets set by Internews, but does not measure how activities have led to increased diversity or professionalism. ● Internews has successfully supported independent news outlets in Kazakhstan and Tajikistan in their professional development as well as promoting diversity in the media sector a variety of ways through the A2I project. ● The A2I project encourages national and cultural diversity, and gender equality, through shared media content and use of diverse multimedia tools and genres as well as professional development of media managers ● Internews should work with USAID to develop indicators that measure strengthened diversity and professionalism in more useful, comprehensive ways. ● USAID, through Internews, should continue to support the professional development of media professionals, particularly in management. ● USAID through Internews should continue to support gender equality through media campaigns and creation of gender role narratives that contribute to women’s empowerment. ● USAID through Internews should continue to support coverage and inclusion of ethnic and other minorities and LGBT populations in activities and in the media. 4 Main Findings Key Conclusions Recommendations Objective 4: ● Performance data from A2I demonstrates that the project is meeting its targets for Objective 4. ● USAID, Internews, and A2I partners and beneficiaries noted the support for improving the legal and policy environment through A2I consultants in Kazakhstan. ● A2I partners and beneficiaries, as well as USAID and Internews, recognized the important support for legal and policy reform as well as individual legal support provided by Khoma through A2I in Tajikistan. ● IMOs noted the value of targeted support through residencies and webinars and help with court cases. They highlighted legal literacy as a priority. ● USAID, Internews, and partner IMOs were not satisfied with the performance and actions of independent media associations vis-à-vis the legal and policy issues they face in the sector. ● A2I is meeting its targets for Objective 4. ● A2I’s beneficiaries valued the legal advocacy work for an improved legal and regulatory environment in both countries. ● A2I support for improving the legal and policy environment was appreciated in Kazakhstan, particularly by outlets that received support. ● Legal support provided by Khoma was valued in Tajikistan. ● An improved legal and regulatory environment is a critical priority for IMOs. ● Partner media outlets benefit from the variety of legal consultancy tools (information provision, consultancies, and residencies) Internews provides through A2I, which help them address the various legal challenges they face. ● Legal literacy is an area where A2I could provide more support. ● A broader approach and more partnerships could be considered as part of USAID and Internews support for legal reform. ● USAID should continue to support legal assistance through Internews and its partners for media outlets and journalists in Kazakhstan and Tajikistan as the services are needed by independent outlets and individual journalists. Internews and its partners can effectively provide these services. ● USAID, through its partners, should continue to support the needed outreach and advocacy work towards an improved legal and regulatory environment for independent media and journalists in both countries. ● Internews and its partners should continue to support media outlets or journalists facing undue persecution through targeted residencies and legal counseling. Internews and its partners should also continue to provide more general information through webinars and seminars. ● Internews should consider expanding legal literacy training for media and developing additional delivery methods. ● Internews should, jointly with other legal advocacy groups, continue to work towards bringing about changes to legislation to move closer to international standards. Internews should consider setting up and supporting fora with legal advocacy organizations to examine specific legal challenges media outlets or journalists face in each country. 5 Main Findings Key Conclusions Recommendations EQ2: What gaps in addressing the program objectives still exist? And to the extent possible, describe why/how gaps exist. ● A limited number of A2I partners and beneficiaries had a sufficiently comprehensive understanding of the A2I program to be able to address questions about A2I in terms of gaps. ● USAID, Internews, and some partners identified particular gaps in Kazakhstan: more knowledge about the program, more audience research, more grants, more residencies, more professional training, a rapid response capability for A2I, more management training, more Kazakh language support, more Democracy, Human Right and Governance (DRG) coordination, work with government media, work with journalism training at universities, cloud-sharing, and stronger associations to represent collective media concerns to government. ● USAID, Internews, and some partners identified particular gaps in Tajikistan: more knowledge of the program, more country-based programming, university journalism training, face-to-face methods, the digital transition, and audience research. ● Some of these gaps are or may be addressed in the new Central Asia Media Program (CAMP) programming. ● A2I partners and beneficiaries would benefit from a more comprehensive understanding of USAID and Internews media assistance programming. ● USAID and Internews need a consultative process to address key areas in media assistance that they and some partners view as gaps. Internews could then help fill the priority gaps that are identified. ● USAID should make more information about media assistance programming public. ● Internews should provide more comprehensive information about media assistance programming, including the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. ● USAID and Internews should consider developing consultative processes to focus some program resources on areas that are considered gaps. These processes should be conducted in the context of the CAMP project. ● USAID and Internews should consider bringing some partners into this discussion how to address key areas in media assistance. 6 Main Findings Key Conclusions Recommendations EQ3: Going forward, how, and in which areas, should USAID-funded media activities be conducted to support journalists and other media actors in reporting objective information to the people of Kazakhstan and Tajikistan? ● A2I partners and beneficiaries recognized the importance of getting more objective, civically￾relevant information to the public in Kazakhstan and Tajikistan, their roles in doing so, and A2I assistance in this regard. ● A2I partners and beneficiaries valued the support received through A2I through a wide range of activities, across numerous cities, and to different categories of people (including journalists, media managers, bloggers) and organizations [such as independent TV stations, Internet sites, and civil society organizations (CSOs)]. ● A2I partners and beneficiaries suggested many priorities for more and additional assistance through USAID via Internews going forward, especially regarding support for the digital transition. ● A2I’s support for the goal and four objectives of A2I is important in supporting the provision of objective information in the two countries. ● A2I’s reach to diverse beneficiaries (individuals and organizations) limits the magnitude of the support for particular beneficiaries. ● The digital transition is a critical priority for independent TV stations that most outlets are not prepared for, which threatens their survival. A2I support to date for the digital transition has been limited and modest. ● USAID should continue to support media assistance in Kazakhstan and Tajikistan towards this goal and these four objectives. ● Internews should continue with A2I in its fifth and final year, as well as develop and implement additional programming in media assistance towards this goal and these four objectives in these two countries. ● USAID should work with Internews to develop approaches to support the digital transition for independent TV stations within the framework of its USAID programs. ● Internews should provide additional assistance to independent TV outlets with the critical challenges they face with digital transition. ● Internews should support TV outlets to be more effective in how they work though media associations on this key area of common concern (as well as in other areas). 7 Main Findings Key Conclusions Recommendations EQ4: Given the media environment in Central Asia and goal of the program, was the design and structure of the activity appropriate to successfully achieving the goal? ● USAID and Internews staff felt the design and structure of A2I was appropriate. ● IMO managers interviewed that knew the A2I design and structure felt the design and structure of A2I was appropriate; however, they sought more assistance in content generation and sharing towards improved competitiveness, digital transition, training and mentoring to improve professionalism, and improving the legal and policy environment as well as legal protection as had been provided under the larger, prior program. ● Internews adjusted implementation in Tajikistan to move away from online approaches that were problematic due to conditions in the country. ● A2I partners and potential partners, beneficiaries and potential beneficiaries, and stakeholders had limited information about the design and structure of A2I, which left them uncertain about program priorities and what support was available from A2I. ● IMOs that worked with A2I understood the activities they had participated in under A2I. ● Of the specific areas examined, partners and beneficiaries were relatively critical about the extent to which the design and structure led to increased citizens’ access to socially relevant information. ● IMOs often did not see a comprehensive way that A2I was supporting their priority needs. They have many needs that have not been supported through A2I. ● IMOs recognized the digital transition as a threat to their survival. They noted limited Internews support for this critical area. ● Accumulated findings suggest that the design and structure of A2I was broadly appropriate to achieving the goal of increased access to civically relevant information. ● The design and structure of media assistance projects like A2I could be explained more comprehensively to beneficiaries, potential beneficiaries, and stakeholders. ● The A2I training program targets a wide range of people, not only IMOs and professionals. ● IMOs have many needs that are not met in a comprehensive way with this design and structure of the A2I project. ● Moving away from addressing the digital transition with the revision of A2I risks reducing attention to this critical challenge for IMOs. ● USAID and Internews should continue implementing A2I with this design and structure in its final fifth year of implementation (FY 2019). ● Internews should provide more comprehensive information about media assistance programming, including the goals, design, structure, and modalities of the program as part of engaging partners, potential partners, and stakeholders on shared objectives. ● In its last year of implementation, the A2I program should continue to provide support for media professionals and others. ● USAID and Internews should consider focusing media assistance on professionals and IMOs to concentrate the results of program assistance. ● USAID should consider directing media assistance towards supporting targeted needs of IMOs in these two countries. ● Internews should consider focusing more on IMOs that have many needs and develop widely-known application and selection processes for additional residencies and A2I staff support to selected IMOs. ● USAID should consider additional diplomatic efforts with other bilateral and international partners to support IMOs vis-à-vis legal frameworks and government policies on the digital transition. ● USAID and Internews should consider developing additional programming to support IMOs with the digital transition. 8 Main Findings Key Conclusions Recommendations EQ5: To what extent were the strengths and weaknesses related to performance of the implementer and the USAID team (particularly their management effectiveness and efficiency)? ● USAID managers asserted that A2I management was effective and appreciated A2I’s comprehensive sharing of information with USAID. ● USAID, Internews, and many of the main A2I partners and beneficiaries characterized the management as effective and efficient, including in collaboration and communications. ● Beneficiaries lacked a comprehensive understanding of the goals, objectives, modalities, and results of A2I. Many beneficiaries placed their understanding of A2I in the context of earlier, larger USAID-funded Internews projects in the region. ● USAID, Internews, and main program partners noted that support for the digital transition and the legal and policy environment were the most difficult areas for A2I as they depend on the governments of the two countries. ● Internews managers noted that USAID was starting to actively meet with some of the main government stakeholders in Kazakhstan to support A2I. ● The USAID Agreement Officer Representative (AOR) and Country Manager engage with Internews A2I staff and manage the grant with the appropriate level of involvement. ● Dialogue and partnership between USAID and Internews in media are managed effectively. ● Given the wide range of the larger past Internews assistance and the still extensive activities of the A2I project, many partners, beneficiaries, and stakeholders have suggestions, ideas, and proposals for additional activities, directions, and support; these suggestions are less about strengths and weaknesses of management and more about opportunities and alternatives. ● There may be additional efforts that are warranted to support a stronger legal and regulatory environment for independent media, including through the transition to digital technologies. ● USAID should continue to work through grants with Internews to provide media assistance. ● In general, USAID managers should participate in and observe as many Internews media assistance activities as possible—with the exception of national-level media association and government dialogues. ● Internews should develop fora for project beneficiaries and potential beneficiaries to discuss not only the strategy and program approaches for projects like A2I but also how Internews proposes to (and has) managed for effectiveness. ● Internews should endeavor to work more with and through partnerships with national organizations, particularly in working to improve the legal and regulatory environment. ● USAID should work through developmental and diplomatic channels in support of Internews’ and national efforts towards a more open legal and regulatory environment for independent media in Tajikistan and Kazakhstan. 9 Main Findings Key Conclusions Recommendations EQ6: To what extent were the strengths and weaknesses related to the program being regional versus discrete activities for each country? ● When asked, USAID, Internews, partner outlets, and beneficiaries in Kazakhstan and Tajikistan noted important benefits from regional programming and largely saw regional programming as a positive feature of A2I. ● Outlet managers interviewed in Kazakhstan also recognized that conditions differed among Central Asian countries and that there were important benefits to targeted programming from A2I focused on Kazakhstan-specific needs. ● Regional media outlets and program beneficiaries in Tajikistan noted unmet needs for increased knowledge of other regions of the country and the potential to fill this gap through A2I. ● USAID and A2I managers felt that A2I had balanced the benefits of regional and national programming well. Given that “needs and levels” are different, some of the trainings conducted could stay country specific. ● Regional programming through A2I has had important benefits in its support to independent media in Kazakhstan and Tajikistan. ● In some areas, the specific features of Kazakhstan and Tajikistan make national programming more appropriate. ● USAID should continue to support Internews initiatives that connect media beneficiaries across Central Asian countries. ● USAID should consider supporting additional initiatives to share regional news and content across regions of Tajikistan. ● Internews should continue to provide both regional and national programming and determine a balance between these two different ways of combining beneficiaries. ● Internews should consider developing programs and a mechanism to support cross￾Tajikistan sharing and cooperation of local news and information by independent outlets in the country. This type of sharing could be modeled on the existing regional sharing mechanisms and networking of Internews. ● Internews should also continue to conduct country-specific events and assistance through A2I on topics and information that target the interests and needs of specific countries. This particularly includes assistance delivered in the Tajik and Kazakh languages.