Photo Credit: Aung Tun KANN LET 2, OTI/BURMA FINAL EVALUATION REPORT March 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Lyndal Barry, Mary Callahan, and Aung Tun for Social Impact, Inc. KANN LET 2, OTI/BURMA FINAL EVALUATION EVALUATION REPORT USAID/Office of Transition Initiatives (OTI) AID-OAA-I-13-00006 QOl 1-OAA-15-00012 PDQ III TO 10 Submitted: March 2019 Lyndal Barry (Team Leader) Mary Callahan (Senior Policy Expert) Aung Tun (Monitoring & Evaluation Specialist) Social Impact, Inc. 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201 Telephone: 703.465.1884 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS Abstract....................................................................................................................................................i Acronyms................................................................................................................................................ii Executive Summary ...............................................................................................................................iii Introduction.............................................................................................................................................1 Evaluation Purpose and Evaluation Questions..........................................................................................5 Evaluation Methods and Limitations........................................................................................................6 Findings and Conclusions...................................................................................................................... 11 Recommendations.................................................................................................................................33 Looking Ahead......................................................................................................................................37 Annex A. Evaluation Design Matrix ...................................................................................................... 39 Annex B. Interviewee List..................................................................................................................... 40 Annex C. Data Collection Tools ............................................................................................................ 43 Annex D. Document Review sources..................................................................................................... 57 Annex E: Roles of CSO in Kann Let 2-Supported Transition ................................................................ 58 Annex F. Evaluation Scope of Work...................................................................................................... 59 LIST OF TABLES Table 1: Approximate Distribution of Data Collection Events by Location...............................................8 Table 2: Stakeholder Categories ..............................................................................................................8 Table 3: Interviewees by Objective..........................................................................................................9 LIST OF FIGURES Figure 1: Kann Let 2 Intervention Sites 3 Figure 2. Kann Let 2 Activities Per Objective 12 Figure 3. Kann Let 2 Funding Per Objective 12 i | Kann Let 2 Final Performance Evaluation USAID.GOV ABSTRACT This Final Evaluation constitutes an independent performance review of USAID/OTI Burma’s Kann Let 2 (KL2) program from March 24, 2016, to March 23, 2019. The findings highlight the strategic approach, relevance, and impact of KL2 efforts to deepen and sustain the ongoing reform process in Burma and provide a concise explanation of achievements and lessons learned. The report is intended to inform future United States Government (USG) assistance, primarily the two new USAID/Burma Mission programs. Using a highly effective management strategy that involved empowerment of program development officers and grant managers, Development Alternatives Incorporated (DAI) implemented 233 Kann Let 2 activities, having spent $15.9 million, as of September 2018. From its desk review and extensive field work (including 98 interviews), the Final Evaluation Team found that the KL2 program identified the critical impediments standing in the way of reform in Burma, supported the work of groups aiming to build intercommunal harmony, expanded inclusion in the formal and informal peace processes, helped to foster and defend the space of civil society, and contributed substantially to US foreign policy. The evaluation recommends that future USAID assistance in Burma continue to support as much flexible, well-informed, locally-contextualized programming as possible, while retaining and utilizing KL2’s significant analytical skills to address the barriers to advancing reform and peace. USAID.GOV Kann Let 2 Final Performance Evaluation | ii ACRONYMS ARSA CBO Arakan Rohingya Salvation Army Community-Based Organization CR Country Representative (OTI) CSO DAI Civil Society Organization Development Alternatives Incorporated D&G Democracy and Governance EAG Ethnic Armed Group EAO Ethnic Armed Organization EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion ICH ICC ICC KII KL M&E NCA NGO NLD NRPC OTI Intercommunal Harmony Intercommunal Conflict International Criminal Court Key Informant Interview Kann Let Monitoring and Evaluation Nationwide Ceasefire Agreement Nongovernmental organization National League for Democracy National Peace and Reconciliation Center USAID Office of Transition Initiatives PCG PDO PPMP PPR PA PLC PM Peace Creation Group Program Development Officer Program Performance Management Plan Program Performance Review Project Assistant Program Level Change Project Manager PSF QA Paung Sie Facility Quality Assurance RA STA SI Rolling Assessment Senior Technical Advisor Social Impact, Inc. SMT STEP SOW Strategic Management Team Smart Traveler Enrollment Program Scope of Work SPR SRS TL Strategic Performance Review Strategic Review Session Team Leader UPC UK UN UPDJC US Union Peace Conference United Kingdom United Nations Union Political Dialogue Joint Committee United States of America USG USAID United States Government United States Agency for International Development iii | Kann Let 2 Final Performance Evaluation USAID.GOV EXECUTIVE SUMMARY PROGRAM BACKGROUND In September 2012, the United States Agency for International Development’s Office of Transition Initiatives (USAID/OTI) launched the first Kann Let (KL)1 program in Burma2 to increase participation and inclusion of key stakeholders in the reform and peace processes as well as to address critical impediments to the transition, primarily the situation in Rakhine State, intercommunal conflict (ICC), and the formal peace process. Development Alternatives Incorporated (DAI) implemented this task order from September 2012 to August 2016. The first phase of the KL program was implemented with a budget of USD $18,477,124.37. In March 2016, OTI issued a new task order to DAI, KL2, to continue deepening and sustaining reforms and fostering legitimate processes for pursuing peace. USAID/OTI continued to refine its program objectives in 2017, emphasizing support of the peace process and efforts to address the drivers of ICC, while also promoting intercommunal harmony (ICH) and support for fundamental freedoms. KL2 is scheduled to end on March 23, 2019. The program operated with a USD $12,900,705.60 budget through September 2018. EVALUATION PURPOSE USAID/OTI engaged Social Impact, Inc. (SI) to conduct an independent performance evaluation of USAID/OTI Burma’s KL2 program. This evaluation was carried out in response to the OTI Burma Final Evaluation Scope of Work (SOW) issued by OTI. The evaluation covers the KL2 program in Burma from March 24, 2016, to March 23, 2019. The evaluation examined the strategic approach, relevance, and impact of KL2 efforts to deepen and sustain the ongoing reform process in Burma in order to provide a concise explanation of program achievements and lessons learned. The report is intended to inform future United States Government (USG) assistance, primarily the two new USAID/Burma Mission programs, one of which started in the Fall of 2018. EVALUATION DESIGN AND METHODOLOGY The evaluation team (ET) used a qualitative evaluation design consisting of a document review, key informant interviews (KIIs), focus group discussions (FGDs), and direct observation (DO). Field visits to project implementation sites to conduct KIIs and FGDs were undertaken. A total of 98 stakeholders were engaged by the ET across KIIs and FGDs. EVALUATION QUESTIONS - CONCLUSIONS EQ 1. HAS THE PROGRAM IDENTIFIED THE RIGHT CRITICAL IMPEDIMENTS TO THE ADVANCEMENT OF PEACE AND REFORM IN BURMA? TO WHAT EXTENT WERE THOSE WITHIN ITS ABILITY TO IMPACT OR INFLUENCE? The ET concluded that OTI/Burma and KL2 identified context-appropriate “critical impediments,” around which programming was meticulously carried out and adapted at key junctures over the life of the 1 “Kann Let”, the name of the project, in Burmese, means “Offering a Hand” and implies a “Helping Hand”. It comes from President Obama's inauguration speech in which he spoke of Burma, N. Korea, Iran, etc. and said that if they opened or unclenched their fists, that the US would be there offering a hand as a friend/partner. 2 In this report the term, “Burma,” is usually used as it was used by the United States Government in the report documentation. Myanmar is used for names or titles where relevant. The Government changed the name of the country in 1989 from Burma to Myanmar. It also changed other names that were used during the British colonial time back to their original Burmese names. Names for ethnic groups are used also according to project nomenclature, for example “Karen” is the name used by ethnic armed organizations whereas government uses “Kayin” as an official name for the ethnic group. USAID.GOV Kann Let 2 Final Performance Evaluation | iv program. The work of KL2 brought even greater clarity than the first phase of KL did to the concept that peace and ICH are of as great importance to United States of America (U.S.) interests as promoting formal democratic reform, the latter having been the singular objective of U.S. policy for some 20 years prior to OTI’s work starting up in Burma in 2012. Interviewees noted that in addition to the organizational, programmatic, and policy advocacy achievements of KL2, the body of knowledge created around critical impediments to peace, federalism, and ICH will continue to bear fruit for the USG and for Burmese organizations working in these areas for some time to come. EQ 2. HAS THE KANN LET PROGRAM EMPLOYED APPROPRIATE TOOLS AND APPROACHES TO SUPPORT THE PEACE PROCESS IN BURMA? The ET concluded that the long-term impacts of KL2’s support to the peace process will not be easily measurable for years, and possibly decades. However, the ET found that OTI/Burma and KL2 employed and adapted tools and approaches that were applicable to the complex and rapidly changing situation of the peace process. These were effectively utilized by KL staff, who were empowered DAI to make decisions regarding grants and delegated responsibility and power to efficiently conduct their work. KL2 was able to leverage relationships with partner organizations and individuals built during KL1 and deepened and focused their work with the groups resulting in more substantive inputs into the peace process. KL2 fostered general directions and processes, such as improving the inputs and outcomes of political dialogue through work on issues such as security and federalism (including nurturing a movement around fiscal federalism). KL2’s support for women resulted in increased and improved gender inclusion in the peace process. KL2 has tried to ensure that this work has been handed over to partner organizations and future USAID-sponsored projects to ensure their sustainability, however, there are concerns that without KL2’s rapid funding mechanisms future programs may not be able to deliver as efficiently as KL2 did. 3 EQ 3. TO WHAT EXTENT HAS THE PROGRAM BEEN EFFECTIVE AT MITIGATING DRIVERS OF ICC IN BURMA? The ET concluded that KL2 identified plausible drivers and sensible geographic focus areas and has developed a strong portfolio of partners and activities to address these drivers. KL2 has plausibly contributed to the observed reduction in ICC during the review period in Burma (and in Rakhine State), and that it prevented violence from occurring or from escalating. The program has established a track record and a range of partners that the follow-on program will be able to mobilize to continue context￾appropriate and risk-aware programming after OTI/Burma has closed. KL2 has been flexible enough to withstand external shocks and challenges that have led other donors to suspend their activities in this programming space – an obvious example being the August 2017 Arakan Rohingya Salvation Army (ARSA) attacks in Rakhine. EQ 4. TO WHAT EXTENT HAS THE PROGRAM EFFECTIVELY SUPPORTED CIVIL SOCIETY ORGANIZATIONS TO ENGAGE IN AND INFLUENCE THE ONGOING DEMOCRATIC TRANSITION PROCESS? KL2’s timely, responsive, and flexible actions in support of democratic transition via robust civil society engagement significantly and broadly influenced political reform. The entire KL2 program supported civil society to serve as frontrunners addressing challenging objectives, such as the peace process, ICH, and fundamental freedoms. The civil society activities were particularly crucial because the NLD-led government is not attending to those issues inclusively. This major accomplishment – the engagement of civil society throughout reform processes – required the KL2 team to adjust to Burma’s unpredictable context via reliance on its national staff and expertise, in collaboration with international expertise and myriad civil society partners, to take emerging transition opportunities. 3 The new program has been designed to have fast, flexible programming available, including in-kind grants. v | Kann Let 2 Final Performance Evaluation USAID.GOV EQ 5. HOW HAS OTI CONTRIBUTED TO THE ADVANCEMENT OF U.S. FOREIGN POLICY IN BURMA BEYOND THE KANN LET PROGRAM ITSELF? There is broad agreement that OTI/Burma and KL2 brought significant knowledge and credibility to the USG presence in Burma. Compared to other donors in Burma, the U.S. Embassy was noted by Burma experts as well as grantees for its responsiveness to even the slightest shifts in the political and social landscape and was widely considered to have the best grasp of the dramatic events that would unfold, particularly after the 2017 ARSA attacks in Rakhine State. The ET concluded that the Embassy and USG more broadly benefited considerably from the contextual knowledge KL2 programs generated. EVALUATION QUESTIONS– RECOMMENDATIONS GENERAL RECOMMENDATIONS: • USAID and the U.S. Mission to Burma should retain and utilize KL2’s significant analytical skill and focus in order to address the barriers to advancing reform and peace. KL2’s intensive learning process is a model for democracy and governance program, particularly as the 2020 election approaches, and as the peace process and Rohingya refugee repatriation likely stall. Such a learning modality provided KL2 up-to-date and insightful contextual analysis of shifting and emerging impediments to reform. USAID should be aware that the peace process and ICH will be impacted by the partisan environment in the lead up to the 2020 election. Electioneering is likely to see space for civil society organizations (CSOs) shrink further; increased risks of intercommunal conflict as populist forces rally support in advance of the 2020 election; and the stalling of the peace process as attention diverts to the election. • Follow-on programs should pick up as much flexible, well-informed, locally-contextualized work as possible, including the creation of small, in-kind mechanisms for grant provisions, specifically to support networks and smaller community-based organizations (CBOs) and CSOs, in a timely fashion; the search for partners from outside Yangon; and outreach to out to ICC and peace partners for advice on how to mitigate the risks of inflamed religious- or ethnic-based conflict resulting from the competition that is beginning to unfold in advance of the 2020 election. EQ 1. HAS THE PROGRAM IDENTIFIED THE RIGHT CRITICAL IMPEDIMENTS TO THE ADVANCEMENT OF PEACE AND REFORM IN BURMA? TO WHAT EXTENT WERE THOSE WITHIN ITS ABILITY TO IMPACT OR INFLUENCE? • USAID and the U.S. Mission to Burma should undertake programming that engages the Bamar majority – whose non-engagement was appropriately identified as a critical impediment by KL2 – on peace, ICH, and fundamental freedoms. These kinds of activities require small amounts of funding to local groups or networks, in recognition that threats to peace, fundamental freedoms and especially ICH vary immensely from one location to another. EQ 2. HAS THE KANN LET PROGRAM EMPLOYED APPROPRIATE TOOLS AND APPROACHES TO SUPPORT THE PEACE PROCESS IN BURMA? • USAID and the U.S. Mission to Burma should continue to support the work of local programs; and should identify alternative, follow-on donor funding and in-kind capacity for support to the peace process. • USAID and the U.S. Mission to Burma should ensure that as many relevant peace process support and implementing partner organizations are referred to future USG funding mechanisms or other peace support mechanisms. USAID.GOV Kann Let 2 Final Performance Evaluation | vi EQ 3. TO WHAT EXTENT HAS THE PROGRAM BEEN EFFECTIVE AT MITIGATING DRIVERS OF ICC IN BURMA? • USAID and the U.S. Mission to Burma should ensure continued support to ICH and Rakhine State programming through a mix of short-term and longer-term interventions, with particular emphasis on context-sensitive, agile, and creative ICH programming in Rakhine State, with the expectation that it will continue to be unstable and prone to sudden, major crises for years to come. Additionally, ICH programming will be important in rural and peri-urban areas, which are currently the target of nationalist groups across the country, and among the nation’s youth. • USAID and the U.S. Mission to Burma should ensure that in-kind grants are made by follow-on projects to support ICH. These were especially relevant to support partners and networks and were found to be particularly well-suited for Burma in the area of ICH, where much of the strong work is done at local levels by more loosely-organized and less formal CSOs and networks. • USAID and the U.S. Mission to Burma should recognize that “inter-faith” has become a buzzword, and risks exacerbating tensions within Buddhism, which can lead to further conflict. Intra-faith work is therefore relevant, but difficult because it “opens up hidden wounds” within Buddhist communities. In general, activities can be more effective when not explicitly branded “inter-faith” or “communal harmony” or not even focused mainly on achieving those goals. • USAID and the U.S. Mission to Burma should continue to support rumor management and response activities, which fuel conflict, both online and offline, across Myanmar. EQ 4. TO WHAT EXTENT HAS THE PROGRAM EFFECTIVELY SUPPORTED CIVIL SOCIETY ORGANIZATIONS TO ENGAGE IN AND INFLUENCE THE ONGOING DEMOCRATIC TRANSITION PROCESS? • USAID and the U.S. Mission should continue to recognize that civil society is of fundamental importance to Burma’s transition, and should maintain and expand support for CSOs’ work on peace process, ICH, and fundamental rights issues – at least leading up the 2020 elections. The weakening of civil society would represent a serious setback to the democratic reform process. • Data-driven and evidence-based CSO activities should continue and are crucial to Myanmar’s transition. Data-driven and evidence-based advocacy is now a major conflict resolution tool for Burma’s complex emerging issues. This kind of approach helps Burma move forward in many areas, including peace, ICH, and fundamental freedom issues. EQ 5. HOW HAS OTI CONTRIBUTED TO THE ADVANCEMENT OF U.S. FOREIGN POLICY IN BURMA BEYOND THE KANN LET PROGRAM ITSELF? • USAID and the U.S. Mission to Burma should engage its Democracy and Governance partners in an OTI-like analytical process of evaluating political and social change via regular reviews and strategy sessions, specifically charting the shifting landscape of critical impediments to peace and ICH. Only by carrying out such systematic learning can practical, time-sensitive knowledge of the context be generated for U.S. policy decision making. • For purposes of sustaining the goodwill of Burma’s civil society, OTI/Burma and USAID should convene meetings to introduce KL2 partners at risk of being left out of future funding opportunities to other donors. This should be a purposeful undertaking and considered high priority. 1 | Kann Let 2 Final Performance Evaluation USAID.GOV INTRODUCTION BACKGROUND AND CONTEXT Burma is in the midst of complex, simultaneous, and multi-dimensional transitions from authoritarian rule to democracy, from armed conflict to peace, and from a centrally-controlled, planned economy to market￾led economic policies. Since 2011, the country has also been stepping out of isolation to integrate into the regional and broader international community. With a few notable exceptions, the reform and peace processes have been directive and top-down with limited public participation. Although general elections in November 2015 and the sweeping victory for Aung San Suu Kyi’s National League for Democracy (NLD) marked a turning point in Burma’s democratic transition, military power remains steadfast with control over key ministries and veto power in parliament. The NLD-led government has demonstrated limited willingness to have international actors or civil society engage on the democratic reform process. Two additional Ethnic Armed Groups (EAGs) have since signed the Nationwide Ceasefire Agreement (NCA) with the government, joining the eight EAGs that signed in October 2015. However, there have been few obvious gains for the signatories as military interference has blocked public dialogues and consultations, and the peace process has been plagued with delays, military violations of the ceasefire agreement, and intensified conflict in Kachin and northern Shan States. The complex human rights and humanitarian situation in Rakhine State and intercommunal conflict (ICC) across the country threaten Burma’s transition. Ongoing intercommunal tension and violence in Rakhine State culminated most recently in two events in October 2016 and in much larger scale in August 2017, when Muslim militant attacks on border guard, police, and military posts in northern Rakhine State prompted counter military offensives and systematic clearance operations against the Rohingya that have led to the displacement of nearly 800,000 since October 20164 and a total of more than 900,000 Rohingyas refugees are now in Bangladesh.5 A recent State Department report cited violence that was “extreme, large-scale, widespread and… geared towards both terrorizing the population and driving out the Rohingya residents.”6 The Rohingya crisis has continued through an intractable process, with international community demands for accountability and commitment from the Government of Myanmar to the safe, voluntary, and dignified return of the Rohingya population from Bangladesh. In September 2018 the United Nations (UN) Human Rights Council International Fact-Finding Mission released the most damning account to date of the humanitarian crisis in Burma, in which it accused some military officers of genocide in Rakhine State, and war crimes and crimes against humanity perpetrated against Rohingya and other ethnic minorities in Shan and Kachin States.7 On the same day as the UN report was released, Facebook banned the Commander-in-Chief of the Tatmadaw, along with 52 pages associated with the military, to prevent the spread of hate and misinformation. Furthermore, on the same day, the 4 The Bureau of Democracy, Human Rights and Labor (DRL), ‘Documentation of Atrocities in Northern Rakhine State’, Bureau of Intelligence and Research Report, p.7. See https://www.state.gov/j/drl/rls/286063.htm 5 Inter Section Coordination Group, Situation Report Rohingya Refugee Crisis, Cox’s Bazaar, January 2019, https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/iscg_situation_report_januar y_2019.pdf. Note that the Government of Bangladesh estimates are higher. Prime Minister Sheikh Hasina stated at a United Nations Forum that there are 1.1 million Rohingya refugees in Bangladesh. This figure includes refugees that arrived in Bangladesh prior to 2017. See Matthew Pennington, “Bangladesh point finger at Myanmar for Rohingya 'genocide'”, Associated Press, September 27. Accessed at: https://www.apnews.com/547ffef101804bba8170cf044d122e51 6 The Bureau of Democracy, Human Rights and Labor (DRL), “Documentation of Atrocities in Northern Rakhine State”, Bureau of Intelligence and Research Report, p.7. See https://www.state.gov/j/drl/rls/286063.htm 7 United Nations Office for Human Rights (UNOCHR), “Report of the independent international fact-finding mission on Myanmar”, Human Rights Council Thirty-ninth session 10–28 September 2018. Accessed at: https://www.ohchr.org/Documents/HRBodies/HRCouncil/FFM-Myanmar/A_HRC_39_64.pdf USAID.GOV Kann Let 2 Final Performance Evaluation | 2 International Criminal Court announced the commencement of preliminary investigations on the crime of forced deportation, using jurisdiction based on Bangladesh being a signatory of the Rome Statute.8 PROGRAM DESCRIPTION In September 2012, the United States Agency for International Development’s Office of Transition Initiatives (USAID/OTI) launched the Kann Let (KL) program in Burma to increase participation and inclusion in reform and peace processes as well as to address critical impediments to the transition, primarily the situation in Rakhine State, ICC, and the formal peace process. Development Alternatives Incorporated (DAI) implemented this task order from September 2012 to August 2016. The first phase of the KL program was implemented with a budget of USD $18,477,124.37. In March 2016, OTI issued a new task order to DAI, KL2, to continue deepening and sustaining reforms and fostering legitimate processes for pursuing peace. USAID/OTI continued to refine its program objectives in 2017, reiterating support of the peace process and efforts to address the drivers of ICC, while also promoting intercommunal harmony (ICH) and support for fundamental freedoms. KL2 is scheduled to end on March 23, 2019. The program has operated with a USD$12,900,705.60 budget through September 2018. KL is part of the United States government (USG) effort to support a sustainable and inclusive peace in Burma. Figure 1 (next page) shows the breadth of KL2 interventions. MISSION GOAL AND OBJECTIVES Consistent with the USG’s Integrated Country Strategy for Myanmar/Burma (2017-20), the USAID/OTI Burma program supports specific Mission Goals and Objectives: Mission Goal 2: Myanmar/Burma makes meaningful progress toward peace and national reconciliation and begins to construct a shared national identity. ● Mission Objective 2.1: Achieve an inclusive national ceasefire and development of an effective political dialogue aimed at achieving a federal union. ● Mission Objective 2.2: Reduce religious and communal conflict and build inclusive, tolerant national identity, with focus on Rakhine State. USAID/OTI aims to foster legitimate processes for pursuing peace and advancing reforms through efforts that seek to enhance the ability of key stakeholders to engage in the peace process; to reduce the influence of drivers of ICC and strengthen the capacity to promote ICH;9 and to strengthen engagement on areas affecting fundamental freedoms. Within each objective, target issues, anticipated program-level changes, approaches, target actors and target areas have been identified (see KL Program Level Change (PLC) Matrix).10 8 International Criminal Court, “ICC Pre-Trial Chamber I rules that the Court may exercise jurisdiction over the alleged deportation of the Rohingya people from Myanmar to Bangladesh” ICC Press Release, 6 September, 2018. Accessed at: https://www.icc-cpi.int/Pages/item.aspx?name=pr1403 9 Although implementation of this objective is nationwide, it also addresses target issues specific to Rakhine State. 10 OTI sets strategy and priorities through a program level change (PLC) matrix, which outlines a strategic framework for high priorities. This framework was designed to assist staff organize programming and evaluate ideas. OTI systems and processes encourage allow deviations from the PLC matrix if justified. 3 | Kann Let 2 Final Performance Evaluation USAID.GOV Figure 1: KL2 Intervention Sites KANN LET 2 OBJECTIVES AND PROGRAM-LEVEL CHANGES USAID/OTI has sought to deepen and sustain reforms and foster legitimate, inclusive peacebuilding by focusing on the following objectives and program level changes (as of January 2018 following a Rolling Assessment): Objective 1: Peace - To enhance the ability of key stakeholders to engage in the peace process ● PLC 1: Increased diverse, civil society input to inform peace process discussions. ● PLC 2: Increased preparedness of government and ethnic representatives for more substantive peace process discussions. ● PLC 3: More inclusive consultations to inform development of and build momentum for “Interim Arrangements” as mentioned in the NCA. Objective 2: Fundamental Freedoms – To strengthen engagement on areas affecting fundamental freedoms USAID.GOV Kann Let 2 Final Performance Evaluation | 4 ● PLC 1: Increased opportunity and capacity for civil society and government engagement on critical impediments to fundamental freedoms. Objective 3: ICH - To reduce the influence of drivers of ICC and strengthen the capacity to promote ICH ● PLC 1: Increased capacity, relationships, and actions among key stakeholders to prevent ICC. ● PLC 2: Increased opportunity for mutually beneficial interaction between communities. Kann Let 2 supports key peace process assistance and community-level tolerance work across all 14 states and regions of Burma. The program implements mostly small grants to help key stakeholders deepen and sustain reform and foster legitimate inclusive peacebuilding by focusing on the following objectives: to enhance the ability of key stakeholders to engage in the peace process; to strengthen engagement on areas that affect fundamental freedoms; and, to reduce the influence of the drivers of intercommunal conflict and strengthening capacity to promote intercommunal harmony. The program has focused on local levels where it develops activities with key actors in civil society, faith-based organizations, religious groups, women’s groups, academia/nascent think tanks, local government, networks, and community-based organizations to strengthen their participation in peacebuilding and democratic reform. The majority of activities consist of awareness-raising, information sharing, policy￾strengthening, capacity-building, technical assistance, and media activities. 5 | Kann Let 2 Final Performance Evaluation USAID.GOV EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE USAID/OTI engaged Social Impact, Inc. (SI) to conduct an independent performance evaluation of USAID/OTI Burma’s KL2 program from March 24, 2016, to March 23, 2019. The evaluation examines the strategic approach, relevance, and impact of KL2 efforts to deepen and sustain the ongoing reform process in Burma. The report provides a concise explanation of achievements and lessons learned and is intended to inform future USG assistance, primarily the two new USAID/Burma Mission programs, one of which started in the Fall of 2018 The primary audience for this final performance evaluation is USAID/OTI, including its Burma-based team and OTI HQ staff in Washington, DC. The evaluation report is expected to be shared with staff at the USAID/Burma Mission and the United States of America (U.S.) Embassy in Rangoon. With OTI’s approval, the report may also be shared with implementing partner DAI as well as local stakeholders including Burmese government and ethnic representatives, non-governmental organization (NGO)/civil society organization (CSO) staff, and others linked to KL2’s work. EVALUATION QUESTIONS The evaluation seeks to inform the following evaluation questions (EQs), as posed in the SOW: 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? ● In answering this question, the evaluators consider whether the program appropriately adapted its strategy as the context evolved and seek to determine whether these same areas are likely to merit similar emphasis in future USG programming in Burma. 2. Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? ● In answering this question, the evaluators consider which tools and approaches have been the most and least effective at facilitating engagement among beneficiaries; whether different or additional approaches should be taken in future USG programming; and if the program has met the expectations of key actors in the peace process. 3. To what extent has the program been effective at mitigating drivers of ICC in Burma? ● In answering this question, the evaluators consider which approaches have been most effective at changing the attitudes and behaviors of target groups; whether there are lessons or findings from other fields (e.g., human psychology, media literacy) that should be incorporated in future programming or interventions; and, whether there are lessons for other programs seeking to promote tolerance or counter speech that is a catalyst for violence. 4. To what extent has the program effectively supported CSOs to engage in and influence the ongoing democratic transition process? ● In answering this question, the evaluators consider whether the program’s support was timely, targeted and critical; identify any missed opportunities; and offer recommendations for ways that the USG can be more proactive as the reform process continues. 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? ● In answering this question, the evaluators seek to identify lessons that can be learned from the close collaboration between OTI and other parts of the U.S. Mission to Burma. USAID.GOV Kann Let 2 Final Performance Evaluation | 6 EVALUATION METHODS AND LIMITATIONS DATA COLLECTION METHODS The evaluation team (ET) used a qualitative evaluation design consisting of a document review, key informant interviews (KIIs), focus group discussions (FGDs), and direct observation (DO). Field visits to project implementation sites to conduct KIIs and FGDs were undertaken. A total of 98 stakeholders were interviewed by the ET across KIIs and FGDs. The ET used protocols to guide its KIIs, FGDs, and DOs (these protocols can be found in Annex C. Data Collection Tools). These formal, semi-structured data collection protocols are composed of questions that were derived from the Evaluation Questions, as well as from the ET’s document review, its discussions with OTI and DAI staff, and its evaluation design knowledge. The ET conducted approximately 20 days of data collection across three areas where KL conducts activities: Yangon, Mandalay and Myitkyina (Kachin State). Table 1 lays out the distribution of data collection events by location, and Table 2 (on the following page) sets out the distribution and number of KIIs, FGDs, and DO’s. Please see Annex B. Interviewee List for the list of respondents. The ET averaged four to five KIIs per day, and conducted 6 FGDs during fieldwork, and conducted one DO prior to the fieldwork. Document Review: The ET conducted a review of documents produced by OTI and DAI to better understand KL2 program design and implementation, extract findings relevant to the EQs, and inform data collection protocol development so that instruments appropriately supplement or cross-check against information in the background documents. The ET reviewed quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments (RAs), Program Performance Reviews (PPRs), other information from OTI/Burma’s activity database, and other existing data. Two cluster evaluations were also reviewed. The full set of documents reviewed for the evaluation are detailed in Annex D. Document Review Sources. Key Informant Interviews: Findings from KIIs contributed to the ET’s responses to all EQs. KIIs were conducted one-on-one or in small groups with representatives of the target groups below: 1. U.S. Government officials; 2. OTI/DAI staff; 3. Grantees and Beneficiaries where possible/appropriate; and, 4. Others (including experts, analysts, community leaders, political party representatives, media, etc. Interview protocols that were adjustable for different types of respondents guided each KII (See Annex C. Data Collection Tools); as well, KIIs were semi-structured, meaning that the ET skipped questions or inserted follow-up questions depending on the nature of responses provided by key informants. Interviews lasted between 1-1.5 hours and many were conducted in Burmese, with consecutive translation. 7 | Kann Let 2 Final Performance Evaluation USAID.GOV Focus Group Discussions: In Burma, FGDs are a relatively new phenomenon. Given the relatively low levels of trust between individuals of one organization and another in Burma, the ET determined that FGDs were inappropriate for most grantees and beneficiaries, given the likely scenario that individuals would be unwilling to share forthright feedback in front of little- or even well-known peers. However, given the team-work of KL’s staff, FGDs were held with KL (DAI) staff in Yangon and three groups of beneficiaries of CSO activities in Mandalay. The number of participants in the FGDs ranged from two to six per FGD. These reflected on program implementation, challenges, and successes. The FGDs with KL staff were conducted in Yangon following completion of fieldwork in Mandalay and Myitkyina and helped the ET ground truth its findings. Direct Observation: In one instance (prior to the fieldwork period), the ET conducted DO to see an activity being implemented. A DO data collection instrument was developed to enable sound documentation of the event (See Annex C. Data Collection Instruments). Outbrief meetings with KL2 staff and OTI/Burma plus USAID staff: An outbrief was designed and facilitated by the ET in close association with the KL2 senior management team. The presentation was delivered to two audiences: first, the senior management team of KL2, and second, OTI/Burma and USAID staff. The ET shared a preliminary readout and key findings with the attendees. Key questions for the discussion with the groups generated feedback on the evaluation methodology, findings, and next steps. SAMPLING The ET determined the sample of locations, organizations, and individuals for KIIs, FGDs, and DOs through a multi-level, purposeful sampling approach applied to documentation in the OTI database. While the selection of the sample was informed by recommendations from OTI and DAI, the ET also applied several criteria in the selection process to maximize the diversity and relevance of its sample. For example, the ET selected the locations for the KIIs and FGDs utilizing the following criteria: ● Selection of regions in which organizations that serve both Bamar majority and ethnic nationalities are associated with ceasefires (Mandalay, Yangon (where the ET interviewed people from Karen state) versus states where there has not been a ceasefire and EAGs there have not signed the NCA (e.g. Myitkyina, Kachin State) ● Selection of urban cities (Rangoon and Mandalay), where there are intercommunal tensions and conflict and where policy level work on the formal peace process takes place, versus rural towns (Myitkyina) which is closer to armed conflict and directly affected by it. ● Selection of areas in which populations include Bamar populations and grantees versus areas in which populations and grantees are most likely to be from ethnic nationality groups. ● Selection of locations in which there has been intercommunal tensions and violence (Mandalay and villages/towns nearby, and Rangoon) versus areas that are closer to armed conflict or ceasefire areas (Myitkyina). ● Selection of locations where there have been a substantial number of KL2 activities: Yangon 120 activities, Myitkyina 34 activities, and city-wide in Mandalay 32 activities with another 28 activities nearby. In these geographical locations the ET chose interviewees based on a sampling of grantees carrying out large and small activities under the following three KL2 objectives: peace, fundamental freedoms, and ICH. The ET selected groups of interviewees for FGDs as these groups either worked at the same levels and the same thematic areas for KL2, for example, Grant Managers and Program Development Officers (PDOs) working on peace and ICH. Or groups that had been engaged by implementing partners of the KL2 program in groups, for example, beneficiaries of peace and ICH training. USAID.GOV Kann Let 2 Final Performance Evaluation | 8 Table 1: Approximate Distribution of Data Collection Events by Location TABLE 1. APPROXIMATE DISTRIBUTION OF DATA COLLECTION EVENTS BY LOCATION LOCATION KIIS FGDS DIRECT OBSERVATION Washington, D.C./other 11 0 0 Rangoon 49 4 1 Mandalay 12 2 0 Myitkyina 6 0 Total 78 6 1 Table 2: Stakeholder Categories TABLE 2. STAKEHOLDER CATEGORIES U.S. GOVERNMENT OTI/BURMA AND DAI GRANTEES ● U.S. Deputy Chief of Mission ● Political Officers (3) on Rakhine and Peace ● Public Affairs Section Chief ● Assistant Public Affairs Officer ● USAID/B Mission Director ● USAID/B D&G Director In Washington, D.C.: ● Burma Desk Officer Democracy, Rights and Labor, State Department (previous) ● Burma Desk Officer, Bureau of Conflict Stability Operations, State Department ● OTI/Burma Acting Country Representative ● OTI/Burma Acting Deputy Country Representative ● Former KL2 Chief of Party, acting Chief Of Party ● KL2 program heads of Peace and ICH ● KL2 program development officers (4 FGDs with Peace and ICH PDOs and Grant Managers, separately) In Washington, D.C. or elsewhere: ● Former OTI/Burma CR ● Former Deputy OTI/Burma CR ● OTI Asia Team Leader (current and previous) ● OTI Program Manager (current and previous) ● Chief, Fields Program Division [Names Redacted] BENEFICIARIES OTHERS [Names Redacted] [Names Redacted] 9 | Kann Let 2 Final Performance Evaluation USAID.GOV In these geographical locations the ET chose interviewees based on a sampling of grantees carrying out large and small activities under the following three KL2 objectives: peace, fundamental freedoms, and ICH. Table 3: Interviewees by Objective TABLE 3. INTERVIEWEES BY OBJECTIVE PEACE ICH FUNDAMENTAL FREEDOMS 64 49 43 The ET was unable to achieve complete sex parity in this evaluation’s data collection. Many stakeholder categories included few female participants, making such parity difficult. However, the ET sought out representatives when possible across genders through direct invitation, scheduling interviews at convenient times for respondents, and providing transport as needed. DATA ANALYSIS The Team Leader oversaw and managed systematic analysis of qualitative data. The ET’s data analysis approach utilized data triangulation to crosscheck results and applied several analysis methods to provide evidence for the evaluation findings and conclusions. DATA QUALITY AND PRELIMINARY ANALYSIS ET members took mostly electronic notes for KIIs and FGDs in real time, cleaning and sharing electronic summaries using Dropbox throughout fieldwork. Team members carried out internal debriefs at least every two days during fieldwork to discuss: (a) evidence collected, patterns, and discrepancies regarding the EQs, and (b) any adjustments that the evaluation schedule needed. DATA TRIANGULATION The ET employed analytical triangulation approaches to develop findings and conclusions. Methodological triangulation enabled the ET to cross-verify and cross-validate the findings that emerged from using the data collection methods and data sources to identify correlations between findings and determine KL2’s overall effectiveness. Asking related questions across and within stakeholder categories enabled greater cross-verification, as did cross-checking findings against desk-review documentation. DATA ANALYSIS METHODS The ET employed several data analysis methods to identify key findings within the data. The type of data analysis depended on the specific data being assessed (e.g., content analysis for qualitative KII and FGD data). Analysis methods used included the following: 1. Content Analysis – Content analysis for intensive review of KII and FGD data to identify and highlight evidence of KL2’s results (or lack thereof) that was relevant to the EQs. 2. Trend Analysis – Trend analysis to examine stakeholders’ perceptions on trends in KL2’s impact on the issues identified in the five EQs. 3. Comparative Analysis – The ET undertook comparisons of KL results across different parts of Burma to assess either convergence or divergence in perspectives. USAID.GOV Kann Let 2 Final Performance Evaluation | 10 BIASES AND LIMITATIONS The ET combatted bias as much as possible by using multiple sources of data to triangulate on an EQ. Combining information from multiple sources (i.e., documents and interviews significantly mitigated the effect of biases on the analysis. Potential bias risks in data collection and analysis included the following: 1. Response bias means that key informants may have been motivated to provide responses that would be considered socially desirable or influential in obtaining donor support. Response bias is connected to cultural and social norms and affected by gender and social ranking; if the first person who speaks in an FGD is the most senior in the group, other participants might take his/her cues from this person and only echo his/her responses. The ET alleviated response bias to the extent possible by using appropriate probing questions, asking for supporting evidence for statements, and using appropriate facilitation techniques to encourage participation and honesty by all respondents. 2. Selection bias, an inherent risk when implementers help to facilitate contact with project grantees, was mitigated throughout the research. Understanding that any implementor may have had an interest in suggesting the most active, responsive, or engaged stakeholders, meaning that the ET may only hear from key informants or grantees who report positive experiences, the ET drew material from the database and from their own contacts to ensure a more broad-based pool of respondents. Additionally, to mitigate the risk of selection bias, the ET asked DAI to identify under-performing grantees to understand the communal and programming dynamics that have failed to produce positive outcomes. Finally, because the OTI database did not contain any contact information on grantees, the ET was dependent on KL2 staff and OTI/Burma for contact information, which was in some cases inaccurate or out of date. 3. Unavailability of key peace stakeholders during the evaluation. There was notable lack of availability of the key actors who were involved in the formal peace process, which inhibited research on the second evaluation question. As the evaluation field work took place in the first weeks of November when there were several last-minute, important peace events taking place, including the high-level peace summit between October 10-15 and the follow-on meetings that went on sporadically over the next six weeks, the ET could only meet with two signatory EAO representatives. The ET compensated for this, where possible, by interviewing civil society leaders who had worked closely with EAOs. 4. Balancing interviews with Grantees and Beneficiaries to ensure the maximum number of interviews possible. The ET tried, where possible and appropriate, to interview beneficiaries of grantees. When evaluating a single project, the ET interviewed available beneficiaries. However, in this evaluation that is examining an overall program with over 200 activities under it, the timeframe for fieldwork does not allow for extensive focus on beneficiaries. In some areas affected by armed and ICC, beneficiaries were not be able to be interviewed as it would draw undue attention from parties to the conflict to their activities. Other beneficiaries, such as EAGs, were unavailable due to the meeting schedule for the formal peace process. 5. Beyond the activity documentation in the OTI database, the ET was given only a small range of OTI/KL2 documents to review. In OTI’s preparation for the final evaluation, fewer than 20 such documents were shared with the ET. For example, the Final Evaluation ET received only three Rolling Evaluation notes to file for the entire period of the KL2 program, those for January and March 2017 and January 2018. That left the Final Evaluation ET highly dependent on data collection through interviews, which has the limitations listed above. The ET found the database extensive and useful for examining project level programming. 11 | Kann Let 2 Final Performance Evaluation USAID.GOV FINDINGS AND CONCLUSIONS GENERAL FINDINGS DAI’s management style of empowering PDOs and Grant Managers to make many decisions regarding grants was widely praised. Many grantees reported that KL2 staff knew the rules and regulations of the organization, and that they had been delegated enough power and responsibility to enable them to make faster granting and other key decisions. PDOs themselves noted that they felt confident initiating design and facilitation at the outset of the grant process and were comfortable reaching out to the Strategic Management Team (SMT) and USAID staff at various levels whenever it was necessary. For example, one PDO offered that she felt the SMT trusted her to “transform ideas into reality” [Interview, November 13, 2018]. The KL2 mechanism is appropriate for the current Burma context particularly as it is: flexible, rapid, responsive, and locally-adapted, in that it was made user-friendly for Burmese grantees and for KL staff. The mechanism was adaptable, and adaptations were made based on constant and regular review processes (RAs, SRSs, etc.) that provided KL2 with sound analysis that gave guidance for changes in programming, as well as allowing for proactive programming. KL staff were recognized across the board for excellence in contextual knowledge. Grantees noted that national staff had in-depth knowledge of the rapidly changing landscape of the peace process. This enabled the staff to be more helpful in designing projects with partner organizations and to apply international ‘Do No Harm’ principles to their work on projects.11 KL in-kind support, micro-grants, and one-off granting tools were particularly well-suited for Burma, given that much CSO work is done at local levels by more loosely organized and less formal CSOs and networks, and often needs to adapt to a fast-changing context. KL advice and direct support enabled select media outlets to amplify key messages around peace, ICH, and fundamental freedoms with reach in both Burma heartland and ethnic areas. Although by the start of KL2, several media projects were being picked up and implemented in Burma, KL2 continued to support limited programming with wide-reach and excellent production levels in new formats that were popular with the public in both ethnic and Burma heartland areas including a radio drama and a debate format series, which was very new to Burma [Interview, November 2, 2018].12 The preliminary findings of Grantee research indicated that engagement with characters in the series has encouraged some listeners to feel more open to those of different ethnicities and religions. KII’s about the programming revealed it was able to increase understanding among Bamar communities of ethnic minority groups’ views on key aspects of the peace process, and to raise awareness and increase knowledge of it.13 KL identified strategic directions and approaches for support in the thematic areas of peace, ICH, and fundamental freedoms. For example, ensuring a focus on reaching Bamar heartland regions, and not only ethnic minority areas. 11 The "Do No Harm" principles integrate conflict-sensitivity into projects. See Mary B. Anderson, “Do No Harm: How Aid Can Support Peace - or War,” Lynne Rienner, 1999. 12 Implementing partner audience research estimated that listenership was approximately seven million, with 5.7 million of these regular listeners. 13 Note: the implementing partner audience research is due for release in March 2019, however, the preliminary findings were shared with the ET. The CSO is analyzing data from a nationwide quantitative household survey conducted in December 2018, and a quantitative survey of 800 people in the Ayeyarwady region (a Bamar heartland area). This measures the reach and impact on attitudes and behaviors of the program, updating research from 2016. See OTI Database. USAID.GOV Kann Let 2 Final Performance Evaluation | 12 EVALUATION QUESTION 1 Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? EQ1 FINDINGS KL2 focused the bulk of its 233 activities and $15.9 million in funding on critical impediments to advancing the peace process and ICH in Burma, as can be seen in Figure 2 and Figure 3, while the limited engagement on fundamental freedoms was more opportunistic in nature. Given the complexity of the context, it is impossible to determine conclusively that the critical impediments identified by KL2 were the “right” ones, but the ET research found they were appropriate for the evolving situation and were systematically reviewed, adapted and tested via regular analytical exercises performed by KL2. The ET found evidence that KL2 identified major barriers to the consolidation of a democratic and prosperous Burma. 14 The program identified the following critical impediments (CI) by program objective (PO) in its January 2018 Rolling Assessment exercise: • PO: Peace - To enhance the ability of key stakeholders to engage in the peace process – CI: Weak foundation for substantive political dialogue – CI: Lack of preparedness for inclusive political dialogue – CI: Lack of momentum to support “Interim Arrangements” within the NCA • PO: ICH - To reduce the influence of drivers of inter-communal conflict and strengthen the capacity to promote ICH – CI: Lack of understanding on critical ICH issues and concepts and on how ICC impact all segments of society – CI: Limited opportunities for interaction between communities in Rakhine State • PO: Fundamental freedoms – To strengthen engagement on areas affecting fundamental freedoms – CI: Limited engagement on issues related to Fundamental Freedoms These 2017 written formulations of critical impediments had undergone changes, based on the collective analytical findings of Rolling Assessments and Strategic Review Sessions in the year and a half before. These regular stock-taking exercises allowed OTI and KL2 staff to work effectively in a complicated and 14 The ET received “notes to file” on only three Rolling Assessments, and one PLC matrix that was not finalized, and thus faced limitations in documenting program developments around critical impediments. See point 5 in ‘Biases and Limitations’ section. Figure 2. Kann Let 2 Activities Per Objective Figure 3. Kann Let 2 Funding Per Objective 13 | Kann Let 2 Final Performance Evaluation USAID.GOV dynamic landscape created through a top-down reform process. In the January and March 2017 Rolling Assessments, KL2 and OTI/Burma staff showed timely awareness of the political, cultural and historical challenges faced by reformists in Burma, in particular: ● the lack of preparedness of the NLD government, which came to power just as KL2 began in 2016, to sustain the momentum achieved thus far in the peace process; ● the lack of inclusiveness and consultation by the NLD government on issues of peace, ICC, law￾making, etc.; ● increasing restrictions on media freedom and the quality of information flow, particularly given the omnipresence of Facebook in a digitally illiterate society; ● a growing rift between civil society and the NLD government; and ● the sustained influence of Ma Ba Tha and related extremist nationalist activities, especially in rural and peri-urban areas.15 The ET’s document review and interviews with partners, KL2 staff, embassy personnel, beneficiaries and other experts found that KL2 succeeded in rolling out activities that were well-timed, well-targeted and aligned with the critical impediments identified. The critical impediments were decades in the making, and therefore could not be overcome through simple technical solutions within a window of two or three years. Nonetheless, KL2 focused on moments of opportunity and crisis to design and implement programs that chipped away at these critical impediments. Opportunities, such as international peace day or an Aung San Suu Kyi speech on civil society led to responsive KL2 programming that took advantage of momentary openings. Crises such as the ARSA attacks of 2016 and 2017 led to activities that took into account the increasingly precarious environment for vulnerable communities. Through its analysis of OTI documents, the database, and KIIs, the ET saw evidence of strategic and intentional re-thinking of assets and opportunities, well-designed staff-development activities to promote critical analysis regarding whether adaptation was needed, and adaptation to the changing environment. Many of the grantees and all the KL2 staff interviewed were clear on the barriers to reform they were engaging. For example, one grantee noted that KL2 alone among donors was responsive to a moment of opportunity for his organization to launch activities for a peace process-related intervention; he specifically identified the lack of preparedness on the part of many actors for political dialogue as the impediment the activities would address [Interview, November 3, 2018]. Similarly, a KL2 grant manager working on ICH told the ET that given the limitations on activities in Rakhine State, “we have found that religious leaders, VTAs [village tract authorities] and other influential persons are the main stakeholders who can influence ICC” [Interview, November 13, 2018]. The ET found evidence that by early 2017, the KL2 program had identified contextual changes that needed to be addressed in its operations and undertook strategic adaptation in response. For example, the January 2017 Rolling Assessment exercise identified the following trends: ● Emergence of decentralized nationalist groups (in the wake of Ma Ba Tha’s decline), which are using disinformation and peace process-related issues/topics as hooks to build support and advance interests; ● The motivation and willingness to participate in the peace process is waning, as confidence in the process continues to decline; ● Most people do not understand the peace process; ● Questions about NLD loyalties and role of Tatmadaw (and role of NLD and military generally); ● General frustration with the performance of the NLD and GOB generally (noting that things in Magway Region were very tense); and 15 The Association for Protection of Race and Religion (known by its Burmese-language acronym, Ma Ba Tha) was the largest Buddhist nationalist organization until it was officially disbanded. It currently exists in various forms under different names. USAID.GOV Kann Let 2 Final Performance Evaluation | 14 ● Armed conflict in Shan and Kachin States is very serious and, not surprisingly, people there are not enthusiastic supporters of the peace process. Both KL2’s January and March 2017 rolling assessments focused in on Burma’s shifting political and conflict context by revisiting the program’s PLCs. For example, the July 2016 PLC, “Increased capacity and relationships among key influencers (including government) to promote tolerance” was changed in 2017 to “Increased capacity and relationships among key influencers (including government) to prevent ICC.” This tightened the focus of the program to one explicitly centered on a conflict-prevention approach, which reflected the new understanding of how potentially violent nationalist groups were now operating given the government’s inability to deal with the volatility. Likewise, in recognition of how slowly the NLD government was moving on a legislative agenda, another PLC was changed from “Increased civil society engagement on law implementation, proposed legislation or policies affecting fundamental human rights” to “Increased civil society engagement on proposed legislation or policies affecting fundamental human rights.” Programming shifted accordingly, with several activities coming online. Interviewees noted that KL2’s activities and partnerships allowed for smart interventions in Rakhine State after the ARSA attacks of October 2016 and August 2017, and in non-Rakhine ICC settings such as Meiktila and Bago. All these timely responses to changes in Burma’s context and partner innovations (e.g., around shifting from inter-faith to intra-faith activities) helped KL2 and the USG to refine their understandings of barriers to peace and reform. Issues KL2 staff, U.S. Embassy personnel, and independent experts observed that there were legal restrictions that made it difficult to work directly on major critical impediments to peace and ICH, most notably the legal and policy prohibitions against working with major stakeholders, including the military. As one embassy staff member noted, “The real limitation OTI faces is in the area of civil-military relations’ tensions. With the legacy of authoritarian rule, there will be ongoing influence that [the] military has over every aspect of society.” This was not something that OTI could address in its short window of engagement. The ET was limited in its ability to assess critical impediments in two ways. First, in KIIs, KL2 staff and grantees did not really use the terminology, “critical impediments,” but rather were more focused on PLCs. Second, and more importantly, OTI provided limited documentation (three Rolling Assessment notes to file, from early 2017 and 2018) on which to base the ET’s analysis. KIIs helped to fill in some of the gaps but could not substitute for the more systematic information typically contained in program documentation. EQ1 CONCLUSIONS For purposes of advancing reforms to bring about a more prosperous, open, democratic and inclusive Burma, OTI/Burma and KL2 identified context-appropriate “critical impediments,” around which programming was meticulously carried out and adapted at key junctures over the life of the program. The work of KL2 brought even greater clarity than the first phase of KL to the concept that peace and ICH are of as great importance to U.S. interests as those promoting formal democratic reform, the latter having been the singular objective of U.S. policy for some 20 years prior to OTI’s work starting up in Burma in 2013. Interviewees noted that in addition to the organizational, programmatic and policy advocacy achievements of KL2, the body of knowledge created around peace, federalism, and ICH will continue to bear fruit for the USG and for Burmese organizations working in these areas for some time to come. Looking forward, USG programming should continue to address what will likely be a shifting range of critical impediments to peace, ICH and the advancement of fundamental freedoms. Numerous interviewees – including grantees and independent experts – as well as the ET itself are concerned that 15 | Kann Let 2 Final Performance Evaluation USAID.GOV the closeout of OTI/Burma will leave a gaping hole when it comes to U.S. engagement on critical impediments to reform. Although follow-on programs will be building intercommunal and civil society capacity and resilience, some of OTI/Burma’s contributions will be lost in future initiatives. For example, a narrow focus on three states risks leaving all ICH and peace-related activities elsewhere in the country to other programs, which expressed concerns in a key informant interview about overhead funding, experience, and technical expertise to take on these sensitive programs in a manner that replicates KL2’s success. KL2 was resourced with appropriate overheads required for high levels of staffing and learning activities, demonstrated flexibility, and expertise in Burma on peace and ICH programming. Elsewhere, there is no apparent post-KL2 home for the fast, flexible, in-kind, support to organizations that can enable ethnic armed organizations (EAOs) to function as serious players in the peace process. Nor is there obvious follow-on support for social media monitoring, which was a major achievement of the KL2 program; the social media monitoring groups operate in a dangerous environment and need flexible and responsive partnerships with donors, the likes of which KL2 was best suited. Another loss will be the granular knowledge that KL2 PDOs and Grant Managers developed on critical impediments to achieving peace, ICH and fundamental freedoms, which was harnessed by OTI/Burma to inform Embassy and Washington, D.C. policy debates. Relatedly, it is not clear who will take up the continuous assessment process that OTI/Burma undertook to ensure that its programming was adapting to the changing national and local environments. EVALUATION QUESTION 2 Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? KL2 started in a time of extreme flux in the political context, launching only a week before the NLD assumed its official leadership of the government, and three months after the November 2015 elections. Over the first 18 months of KL2, the NLD struggled to effectively manage the peace process – unable to build important relations, trust and confidence with the Tatmadaw and EAOs, as well as CSOs and the international community. Simultaneously, there was the ongoing ramp up of activities during this time, as all parties to the conflict were required to participate in the political dialogue process under the NCA, which included national dialogues based on five thematic areas and Union Peace Conferences. EQ2 FINDINGS The ET’s program document review and stakeholder interviews yielded findings that indicate that the KL2 Program did employ tools and approaches that were appropriate for Burma’s context and supported the key stakeholders. Furthermore, the ET found that appropriate tools and approaches were employed in line with available project analysis documents and anticipated PLCs defined by internal and external context analysis and by regular exercises including Rolling Assessments (RA), Program Performance Reviews (PPR), and Strategic Review Sessions (SRS). Interviews with those involved with KL2 under its peace objectives were mostly positive regarding feedback on the project, even though funding had ended for most organizations at some point over the previous year, and there was for most no hope of future funding, with KL2 closing in March 2019. Even the number of peace process organizations’ representatives willing to be interviewed (64) was notable, given what a busy time it was in the formal peace process, and is most likely a testament to the strong and supportive work conducted by the OTI and DAI staff. In addressing this EQ, which was also posed during the mid-term review of this project, the ET faced two main challenges: ● First, the ‘peace process’ is often referred to as either the formal process in which peacebuilders have wrestled with the key parties over common principles to enable procedural approaches. These USAID.GOV Kann Let 2 Final Performance Evaluation | 16 constitute a “peace architecture” formally documented as the NCA. The NCA has been accepted by approximately half of the EAOs, the Government and the Tatmadaw, and was ratified by parliament before former President Thein Sein handed over power to the Aung San Suu Kyi-led National League for Democracy (NLD) after the first quarter of 2016. However, seven of the remaining EAOs (notably those controlling approximately 80% of combined troop strength - an estimated 60,000+ soldiers) formed an alliance, the Federal Political and Negotiation and Consultative Committee (FPNCC).16 There is still large-scale fighting continuing in Shan and Kachin States in the north, and a state of heightened security remains, particularly in Rakhine State, following attacks in August 2017 on police outposts by ARSA and subsequent military clearance operations. ● The formal peace process has all but stalled, with little incentive for the belligerent parties remaining outside the NCA process to legitimize the proceedings. The NLD has been criticized for its weak and inconsistent handling of the process in its first two years in power, which has intensified distrust between the parties, while leaving CSOs only a limited and narrow role in the formal process. By 2017, a year into KL2, there had been three rounds of Union Peace Conferences that had yielded 51 points to be incorporated into a Union Accord. However, while all of these were approved by the military, none go beyond the 2008 Constitution that they have vowed to defend. Outside of the formal peace process, and as stated in the mid-term review, peace is also comprised of “hundreds, maybe thousands, of different mini-processes and micro-negotiations that are aimed at dozens if not hundreds of different definitions of ‘peace’.”17 These are key building blocks to forge trust and understanding between often disparate groups in order to build countrywide support for the formal processes at higher levels. ● A second challenge involved this EQ’s terminology. Specifically, when asked about “tools and approaches,” most interviewees seemed confused by this term and did not use this terminology in their responses. Interviewees frequently requested clarification from the ET about whether this meant specific KL tools and approaches or their own. The ET communicated the intention of the phrasing used in the question and allowed interviewees to respond to either KL2 mechanisms used for grant￾making or defined approaches they had used to implement their projects or activities – or both. Achievements KL successfully navigated the shifting nature of the peace process funding landscape, finding strategic opportunities and filling gaps where other donors came up short. A good example of this was KL2’s engagement with and around a program that was established by the major international donor countries to ensure coordination around support to overall peace programming in Burma. The Embassy and USG began supporting the program in December 2016, and this affected the KL2 project significantly.18 The program was establishing its structures and mechanisms between 2016 and 2017, with its first call for 16 The Alliance includes: United League of Arakan/Arakan Army (ULA/AA), Kachin Independence Organization/ Kachin Independence Army (KIO/KIA), Myanmar National Truth and Justice Party/Myanmar National Democratic Alliance Army (MNTJP/MNDAA), Palaung State Liberation Front /Ta’ang National Liberation Army (PSLF/TNLA), Shan State Progress Party/Shan State Army (SSPP/SSA), Peace and Solidarity Committee/Shan State East National Democratic Alliance Association (PSC/NDAA) and United Wa State Party/Army (UWSP/UWSA) 17 This was a challenge referred to under EQ 2 on peace in the Mid Term Review of the KL1 program, Dr. Mary Callahan and Aung Tun, “Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016”, USAID/OTI, 2016. 18 The OTI Office in Washington D.C. funded a one million USD program through a contribution agreement between USAID and UNOPS, (separate from the KL or DAI-SWIFT task order) that was executed in December 2016. Although the funding was not through KL2, according to OTI staff, “lessons learned (largely through OTI’s support of KL to the peace process) could be shared with the program, and guidance provided using these and USG analysis” (Interview, October 29, 2018). The total OTI commitment to the program to date has been $3 million and OTI currently sits on the program’s Board. 17 | Kann Let 2 Final Performance Evaluation USAID.GOV proposals made in April 2017 and only three small grants of under 100,000 USD approved by August 2017. The program did not have a fast granting nor micro-grant mechanisms, as KL2 did.19 At first, KL2 shared information with and referred some grantees to another program. At the time, however, the NCA-mandated national dialogues were being rapidly negotiated and scheduled between the key parties to the formal peace process. This meant that funding had to be made available fast to ensure that the stakeholders in the process could participate in the dialogues, which in turn were key to informing the political dialogue process, through the Union Peace Conferences and the development of the Union Peace Accord. The key stakeholders were receiving confirmation of these events with little notice and had minimal capacity to manage preparations and logistics. They also needed rapid support in terms of funding, capacity and organization of these activities (see Achievements below regarding this point). This situation coincided with another program providing assistance to stakeholders across the board, and new partnerships meant that there were due diligence and administrative procedures to be conducted. During this time, KL2 picked up projects that would be assumed to be funded by the program. Some of KL2’s existing partners turned to KL for continued funding because KL2’s flexibility and quick-funding mechanisms proved indispensable to major actors involved in these activities. The KL2 mechanism was able to stand in, by providing crucial support to the dialogues through funding of CSOs and experts. Having both KL2 and another program supporting grants at this crucial time offered complementarity and meant that more organizations could be funded for their work on the dialogues in particular. KL2 was able to leverage relationships and networks originally fostered during KL1 to provide timely support to the formal and informal components of the peace process. The work of KL1 was partly focused on bolstering and improving the bargaining power of the weaker parties at the negotiating table, in particular CSOs and EAOs, on ceasefires and preparation for future political dialogue, with a focus on meetings, workshops, trainings, research platforms and cultural events on federalism, pluralism, and diversity.20 KL2 then assisted many of these same groups in conducting more substantive work on the content of the political negotiations. 21 The NCA-mandated political dialogue process started in the beginning of 2015, and the NLD-led government was initially slow to respond to the schedule. Although other stakeholders, including CSOs and EAOs, were preparing to participate in the dialogues, their capacity in this area was generally very low, with few having experience in dialogue or policy work. KL2 responded proactively to this situation by working with leading CSOs and INGOs they had supported in KL1 to research and navigate the complicated political dialogue process. By mid-2017, KL2 had shifted its approach to provide more significant inputs into the ethnic, regional and issue-based political dialogues.22 It provided strategic capacity building on the five themes allowed for discussion to both individuals and CSOs conducting trainings, and CSOs and EAOs as beneficiaries of these.23 KL2 then backed this with negotiations training to ensure that the information transferred would be more likely to be more effectively used at the talks. As the political dialogue part of the process included the production of policy papers at all levels of the dialogues, this demonstrated a honing of the 19 The program has two main forms of grant mechanism: small grants at under 100,000 USD, or larger grants over this amount. Currently, the program rarely makes smaller or one-off grants. 20 See KL1 project examples (OTI Database), that resulted in “…far more sophisticated thinking about the major challenges that lie ahead on a path towards federalism”. Dr. Mary Callahan and Aung Tun, “Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016”, USAID/OTI, 2016, pp.19-20. 21 OTI’s database lists 53 distinct awardees under the "Peace" objective for KL2 (not counting "Community of Practice."); of those, 20 (38%) were also KL1 partners. 22 Under the Framework for Political Dialogues there are different types: Ethnic, Region and Issue based. Ethnic-based dialogues are exclusive to a particular ethnic group (e.g. Chin or Karen), and involve discussions on prominent issues facing that particular ethnic-group. Region-based dialogues are exclusive to the inhabitants of a particular local areas, where key issues within that State or Region are discussed (although these dialogues are not necessarily referring to State/Regional boundaries). Issue-based dialogues are led by CSOs and involve the discussion of specific issues such as land and natural resources. 23 Under the Framework for Political Dialogue, Political Dialogues focus on five thematic areas: Political Affairs; Social Affairs; Economic Affairs; Land and Natural Resources Affairs; and Security Affairs. USAID.GOV Kann Let 2 Final Performance Evaluation | 18 capacity building already being conducted with these groups through KL2.24 This KL2-supported policy￾level work not only informed and enabled these groups to better contribute to discussions, it also further supported the capacity of Burma’s nascent think tanks and political and peace advisors.25 As one of the senior project staff stated, “We looked at who the key stakeholders like government and EAOs listen to – searching out technical access points as there is no single entity that all groups will listen to. We recommended that [the] team take more of an approach of supporting relevant think tanks and advisors.” From the ET’s interviews with USG personnel, grantees/beneficiaries and other experts, the consensus was that KL2 staff recognized opportunities for partnerships with actors who worked in collaboration with other stakeholders to design and implement activities that responded to emerging opportunities in the peace process context. These included projects and activities that supported both the formal NCA￾mandated peace process, and the informal micro-peace processes. The political dialogue opened the opportunity for non-security sector actors to engage in discussions on the issue of security publicly for the first time in decades, as this was previously seen by the military as solely within their purview. Security sector reform (SSR) had proven to be an extremely sensitive area for the military, in particular during the ceasefire negotiations leading up to the signing of the NCA in 2015, and was to remain an area that the military attempted to control during political dialogue.26 With KL2 support, one CSO was able to conduct hundreds of workshops and consultations with key signatory EAOs and CSOs, as well as regional governments and parliaments in ethnic states and Bamar heartland areas. KL2 had seen the emerging opportunity to support public awareness and advocacy around sensitive security issues and had supported one of the CSOs working in this field when few other donors were supporting this type of security sector work. Supported projects were relevant and responsive to the demands of the tight schedule of the peace process; examples of such projects included continued and more focused work on the content of the negotiations, and on awareness raising around peace-process related issues such as security and SSR. The KL1 project had started to focus on federalism as a key issue to the political dialogue process and had partnered with CSOs, and through them, and other experts, EAOs, as well as some parts of government to ensure a basic understanding and knowledge on this issue. Federalism has been at the heart of disputes between Burma’s Bamar majority and other ethnic nationalities since before independence, and prior to 2015 the government and Tatmadaw saw federalism as causing the disintegration of the country.27 At the start of the formal political dialogue process in 2015, the KL2 team noted that there was a pluralism of ideas around federalism with very different interpretations of it and how it could work in the country context. The ET also noted that there were very different interpretations of federalism between ethnic groups and the Bamar majority, and that there was low knowledge of the issue especially with the 24 These policy papers address issues which explore potential solutions to some of the challenges of the negotiations. Policy papers are produced by many different stakeholders, including communities, women’s groups, youth groups, EAOs, researchers and technical experts. They are produced at every level of the process: for the preparatory phase of national dialogues (Public Consultations); for National Dialogues themselves; and also, for the thematic working groups of the Union Peace Dialogue Joint Committee (UPDJC). After these are presented at National Dialogues, they go to the UPDJC Working Committees (which were set up under the Framework for Political Dialogue), with specific points from these papers then being discussed and used to inform future debate at Union Peace Conferences and the Union Peace Accord. 25 These changes were solidified in the 2017 program document and the Program Level Changes and Theory of Change adjusted accordingly. These were again examined and adjusted in 2018. 26 The 2008 Constitution defines a structure in which the military retains near total autonomy in the sphere of security affairs. 27 Note that it was not until January 2015, when the then-President Thein Sein stated in his Independence Day speech that there was an agreement to establish “a union based on federalism”. This was the first time the use of the word ‘federalism’ had been officially used by the government. On October 15, 2015, the NCA was signed, committing the Tatmadaw to establish “a union based on the principles of democracy and federalism…that fully guarantees political equality, the right to self-determination, and democratic practices based on the universal principles of liberty [and] equality”. (See Global New Light of Myanmar, “Main objective of our reforms is to enable people to enjoy peace and development democratically”, April 1, 2015; available at: http://globalnewlightofmyanmar.com/main-objective-of-our-reforms-is-to-enablepeople-to-enjoy-peace-and-development￾democratically/; and, the Nationwide Ceasefire Agreement, 2015, Article 1a). 19 | Kann Let 2 Final Performance Evaluation USAID.GOV majority group that could seriously impede acceptance of a future peace accord and federal union.28 As a former EAO leader stated, “The areas of focus need to be more on central Burma, not just ethnic areas, and KL recognized that and supported it. Many ethnic people think that federalism is only an ethnic issue - but that is wrong… Federalism is also a Bamar issue, and this knowledge should be given to [them]. They really are interested in… self-determination… [which] is the same for ethnics as it is for Bamar. The people in central Burma certainly won’t be accused of supporting separatism – yet, they do most likely want some form of self-determination. This is not only an ethnic issue. there is a need to work in Bamar areas in the seven regions. It's very important” [Interview, November 10, 2018]. KL2 therefore supported projects and partners who not only focused their activities on ethnic minority areas but also in Bamar dominated regions. These included CSO-led workshops, seminars, the production of printed materials, and media projects that enhanced Bamar understanding particularly of how federalism was relevant to them, and not only ethnic minorities. KL2 was one of the few international donors that accepted and acted upon this risk to the overall process of the Bamar majority not accepting an issue that they saw as a purely ethnic minority matter. KL2 also strategically supported several projects around fiscal federalism, which practically demonstrates how federalism can be managed administratively and engaged in by all key stakeholders in the country. The approach employed by KL2 of working with an individual expert on the issue who then worked with a lead research INGO, think tanks, CSOs, women’s organizations, EAOs and government, to refine curricula and enable some continuity and country relevance in what was being taught, was contextually pertinent. Although the expert was employed to work with one organization initially, KL2 staff re-strategized and arranged for the expert, where possible, to work with a lead research INGO and multiple CSOs, including leading women’s organizations, think tanks and those providing assistance to EAOs and government bodies (OTI Database). The feedback from these organizations indicates that KL2 through support to curriculum development and workshops for key peace stakeholders, fostered a movement and interest across all grantees that had participated in the activities on this important topic. The ET also found that KL2, through a specific grant to OTI from the USAID’s Women Peace and Security Fund, was able to directly increase women’s participation, representation, and contributions to key peace process discussions and events in the peace process.29 KL2’s analysis indicated that few CSOs and academics had experience with security issues, and women in Burma in particular had low knowledge of these issues. As a leading CSO expert in this area stated, “KL2 worked with a gender-focused CSO, with SSR expertise, “…we realized there were only a few key women working on the peace process and in politics… and few were working on women peace and security. [Security] was an area that needed further discussion publicly [as] …prior to this - only men were being consulted on this” [Interview, November 2, 2018]. KL2 supported a multi-pronged approach to this support, including: training women on key themes related to the peace process, such as Federalism, SSR, and Economic Policy; and removing barriers to attendance of dialogue events by providing funds for transport, accommodation, and child-care. This support resulted in an increase in women’s participation from 17 to 22 percent in the last UPC. The KL2 staff had also recognized and acknowledged that increases in women participation do not necessarily make for better contributions to the negotiations. They therefore worked with CSOs and national experts to conduct gender research on salient topics from the Framework for Political Dialogue’s five thematic areas, as well as capacity building. Particularly impressive was that KL2 supported one CSO in developing curricula and conducting gender-sensitive 28 The KL1 Mid-Term Review recommended continued support for projects that focused on Bamar majority areas - Dr. Mary Callahan and Aung Tun, “Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016”, USAID/OTI, 2016. 29 In late 2017, OTI received $700,000 of FY2016 resources from a grant of USD1,097,925 from a USAID program to increase women’s participation, representation, and contribution, to issues under discussion in the peace process. At the time of the evaluation, OTI had initiated twenty relevant activities. USAID.GOV Kann Let 2 Final Performance Evaluation | 20 security sector activities with CSOs, EAOs, political parties, and government at both national and regional levels. The ET heard from interviewees from CSOs, EAOs, political parties, and government how these activities had built knowledge of security and other key thematic issues both with women and men who were involved in important positions in consultations for the national dialogues. This was also evidenced by the number of specific requests the organization received for training (see OTI Database). Some of the activities that interviewees considered most effective included: ● Support for national and international experts to work with CSOs and EAOs, in particular through workshops, forums and production of documentation to bolster and provide more significant inputs into negotiations around ceasefires and political dialogues (OTI Database). According to a former EAO leader, who is now an advisor with a CSO and was interviewed for the evaluation, “with KL2 support, we were really able...to [help] the EAOs better prepare for the Panglong Conferences and political dialogues. We were the main show in town supporting signatory EAOs to really make substantive contributions to the peace process because we made sure that the EAOs understood the terms and issues that they were going for and helped them agree on terms and better prepare for the Conferences.” A key indicator of the success of this work was that the exact wording produced by two KL2-supported CSOs in publications printed for all parties to the conflict can be found in the actual Union Peace Accord points. In turn, the hundreds of thousands of booklets published that contained this wording, as well as the workshops given by these organizations to explain these points, were utilized by the key players and influenced policy (see OTI Database). ● The KL1 project had started to focus on support for training ethnic CSOs and stakeholders in the peace process on federalism as a key issue in an introductory way, explaining the basic theory and showing how federalism in different countries worked. The KL2 project supported CSOs and advisors in running workshops and other activities on federalism that were also aimed at bringing the Bamar majority to an understanding that federalism was an issue that was important for them (OTI Database). Five Bamar KIIs responded who had taken part in different iterations of these activities from separate locations confirmed that they and their colleagues had understood that federalism was an ethnic issue and was not important for the majority prior to the workshops. Following the activities, they indicated a change in understanding of federalism, stating that it was now an issue of high priority for them. Facilitators of these sessions and project reports also verify this new understanding. The ET was particularly impressed with the creativity of one project in particular which utilized Bamar literary festivals to introduce ideas on federalism to the Bamar public (OTI Database). These were not as successful as anticipated and were fraught with issues (as senior KL2 staff explained “We worked on a few grants on this – and understood the importance of it - but had to unfortunately turn it off way too soon. We can’t be effective in this without it being a top priority – and at the time it was not in our manageable interests”). However, the ideas and learning generated from these have inspired further thought amongst writers and artists on taking these ideas forward in future, as stated by a CSO head interviewed, “The understanding of the writers and the public in Burman areas of federalism improved. Most in the pre-tests said that they would not accept federalism as they equated it with separatism. After the workshop most said that they could support federalism… Some of the talks really improved though after the workshops – some became more political and contained more political comment than before and were more interesting and well informed” [Interview, Nov 3, 2018]. ● KL2 also strategically supported several projects around fiscal federalism that focused on curriculum and publications design and workshops and forums on fiscal federalism, conducted by CSOs and experts for other CSOs (with a focus on women’s organizations), EAOs, some government departments and Members of Parliament. According to KIIs, these were popular with both these implementing organizations and beneficiaries, as it brought practical understanding of how federalism worked administratively (OTI Database). Notably, KL2 brought in a global expert on the topic at the 21 | Kann Let 2 Final Performance Evaluation USAID.GOV request of a CSO grantee, who was also able to work across the grantees on refining and harmonizing work on the complex topic (a ‘Community of Practice’ grant). As one beneficiary stated of this work, “before we understood federalism as a theoretical concept, but fiscal federalism is practical. It’s a difficult thing to explain in Burma as many people have a low understanding of how governments work, but we need to know it to make a federal country work.” ● The ET also found that KL2 activities focused on women,30 including subsidized trips for CSO, EAOs, and political party representatives to attend peace conferences, dialogues and peace educational opportunities, increased women’s participation, representation, and contributions to key peace process discussions and related events, such as KL2 projects supported by the Women’s Participation Fund (OTI Database). The results of this support for participation are clearly evidenced by the increase in numbers of women who could attend the last Union Peace Conference and were directly supported by KL2. However, as stated above, the more important results can be found in the awareness raising, advocacy, advising and policy papers produced by women’s organizations on serious issues that women previously had little access or involvement in, such as workshops and training curricula on security sector reform from a gendered perspective. A prolific facilitator of these activities explained, “The KL staff were very committed to the concept of women, peace and security. They really understand gender issues and really listened to us and wanted to follow our ideas and advice on doing this in Myanmar. They supported our work and really tried to help us think through our planning and strategy.” The United Nations Secretary General also praised OTI’s work in Burma on this issue in the 2018 Security Council Report on Women Peace and Security, in an annual report on work conducted globally that contributed to the UN Resolution 1325 (2000) on women’s participation in peace processes in October 2018, which is also a testament to the success of the activity.31 Issues Although OTI was given special clearance in 2014 by the USG to work with several EAOs, sanctions in place over the trade of narcotics forbade work with some of the leaders of the most powerful of these organizations.32 In particular, restrictions on working with the Tatmadaw following the events of August 2017 in Rakhine State33 made it even more difficult to engage with the most powerful key actor in the peace process. The NLD-led government’s ambivalence toward allowing support from the international community for the key players in the peace process also made work on the process challenging. Given 30 These activities were made possible through a specific grant to OTI from the USAID’s program. 31 According to the UN Secretary General’s report: “It is also critical that we end the frequent practice of bringing women into processes late or, too often, as tokens. I encourage creative and practical steps to eliminate barriers, such as travel expenses and the need for childcare, mobility and translation. This includes through the establishment of rapid-response funding mechanisms, with capacities to approve requests on short notice, thereby empowering women to seize critical opportunities in peace processes and related events. In Myanmar, the women’s participation fund established by the United States Agency for International Development with the global development firm DAI is one such example.” United Nations Security Council, “United Nations Secretary General’s Report on Women, Peace and Security”, 9th October 2018, page 9, paragraph 33. Accessed at: https://undocs.org/S/2018/900 32 This includes sanctions against individuals working with the United Wa State Army (UWSA) included the U.S. Department of Treasury’s Office of Foreign Assets Control Specifically Designated Nationals and Blocked Persons List pursuant to the Foreign Narcotics Kingpin Designation Act (Kingpin Act).and others. See www.treasury.gov. 33 Attacks by ARSA on 25-30 Border Guard Posts, targeting government officials and Rohingya viewed as government collaborators, on August 25th, 2017. These resulted in clearance operations by the armed forces to rout out the insurgents, and the civilian populations were caught in the middle. Since, several international accountability mechanisms have found members of the Tatmadaw to be complicit in war crimes including ethnic cleansing and genocide. The U.S. “Documentation of Atrocities in Northern Rakhine State” report, researched by the Bureau of Intelligence and Research and funded by the Bureau of Democracy, Human Rights and Labor (DRL), found that the violence in Northern Rakhine State was ‘extreme, largescale, widespread and seemingly geared to both terrorizing the population and driving out the Rohingya residents’. See https://www.state.gov/j/drl/rls/286063.htm USAID.GOV Kann Let 2 Final Performance Evaluation | 22 these points, the challenging context and a lack of capacity or mandate to address deep, underlying structural causes and drivers of armed conflict, the program was somewhat limited in what it could achieve overall. In the transition from KL1 to KL2 there was staff turnover at the Embassy and USAID that had an impact on KL2. The US Ambassador and OTI Country Representative departed in April 2016 which left a gap in strategic directions for KL2. The USAID/OTI DCR and the USAID/OTI Field Program Manager took turns as Acting Country Representative for KL2, but shortly afterwards they both left as well - the DCR in July 2016 and the FPM in September 2016. These departures meant that administration and implementation slowed down on KL2 for the start-up phase, and strategic decisions about the way forward had to wait until permanent leadership arrived. In August 2016 a new, permanent Country Representative arrived to Myanmar. Although without previous USG, USAID, OTI or Asia experience, his learning curve was substantial and the strategic guidance for programming on KL2 was impacted accordingly. In late January 2017 a new Deputy Country Representative arrived, with no prior OTI experience but with experience in similar work at the US State Department, and in late February 2017 a second Deputy Country Representative arrived, with no prior OTI experience but with experience in similar work with other offices within USAID, leaving both to pick up the OTI model quickly on the job, as their roles played a key role moving the program forward through the challenging transitions. In comparison to KL1, programming in areas of peace process support slowed down during this staff transition, and some KL senior staff described the start of programming for KL2 under these conditions to be “less strategic and that programming was agreed to often more on a whim than sound analysis.” [Interview, November, 2018] This took a toll on the morale of the staff, both international and national, for several months. Eventually, a permanent OTI CR arrived in January 2018 with OTI and Myanmar experience, and programming stabilized. Simultaneously, as internal and external context analysis and strategic reviews helped shape the vision, the pace of programming picked up and staff morale improved [Interviews, November, 2018]. EQ2 CONCLUSIONS The ET concluded that the longer-term impacts of KL2’s support to the peace process will not be easily measurable for years, and possibly decades. However, the ET found that OTI/Burma and KL2 employed and adapted tools and approaches that were applicable to the complex and rapidly changing situation of the peace process. KL2 was able to leverage relationships with partner organizations and individuals built during KL2 and deepened and focused their work with the groups resulting in more substantive inputs into the peace process. KL2 fostered general directions and processes, such as improving inputs and the outcomes of political dialogue through work on issues such as security and federalism (including nurturing a movement around fiscal federalism). KL2’s support for women resulted in increased and improved gender inclusion in the peace process. KL2 has worked to try to ensure that this work has been handed over to partner organizations and future USAID-sponsored projects to ensure their sustainability, however, there are concerns that without KL2’s rapid funding mechanisms that these programs may not be able to deliver as efficiently as KL2 did. EVALUATION QUESTION 3 To what extent has the program been effective at mitigating drivers of ICC in Burma? The ET sought to gauge current understandings of the root causes of ICC and noted that these had not changed since the mid-term review and matched the analysis in the various KL2 program documents. The ET then asked further questions to better assess the program’s effectiveness at reducing the influence of 23 | Kann Let 2 Final Performance Evaluation USAID.GOV these drivers. Two cluster evaluations on ICC and hate speech were completed in time for the write-up phase of the ET’s work in November, thus bolstering the research available to the ET in these areas. 34 EQ3 FINDINGS Several plausible causes or at least contributing factors to violence were identified by interviewees: underlying anti-Muslim sentiment in Burmese society; renascent Buddhist nationalism; new freedoms of expression; a sudden rise in social media usage, particularly the use of Facebook; lack of effective non￾violent dispute resolution mechanisms; weak response capacity of the police; and political manipulation. Interviewees also stressed that they saw rural and peri-urban areas as the current target of nationalist groups. KL2’s work on ICH has been organized in two main strands: 1) activities specific to Rakhine State; and 2) activities addressing broader issues of ICH and ICC in targeted locations across the country. Initially, KL1 and KL2 had a standalone PLC for Rakhine programming, but as program analyses clarified that there were no major differences in strategy, approach and quality of intervention between the two, the Rakhine activities were subsequently integrated across the other PLCs as of January 2018. Achievements KL2 was able to respond well to the fast-moving crisis after the August 2017 attacks by the ARSA militant group and the brutal response by the Tatmadaw. Early on, KL2 had identified Rakhine State as a key area of ICC programming, and ICC therein as a major risk to the country’s stability and its political transition. This meant that it had developed partners, relationships and activities that could respond quickly to the current crisis and implement activities that would not have been possible if they were initiated only in the wake of the crisis. KL2’s flexibility and built-in risk-tolerance also facilitated that timely response. Its pre-existing activities in Rakhine faced some temporary disruptions following the onset of the crisis, but partners were able to resume activities when they felt able to do so, protecting the ICH space. In contrast, other donors in this programming space did not have the same latitude and suspended all projects in Rakhine State for an extended period after August 2017 in order to review risks and evaluate options. The evaluation interviews suggest that KL2 has plausibly contributed to the observed reduction in ICC during the review period in Burma (and in Rakhine State, specifically). The program has also likely prevented some violent incidents from occurring or from escalating. Through its KIIs, the ET also learned of a small number of concrete examples in which violence may have been averted. For example, a network supported by KL2 through an INGO had been trained in online rumor management and managed to quash a dangerous rumor that most likely would have ended in violence [Interview, November 14, 2018]. Rumor management and response activities are important, both online and offline, across Burma. In Rakhine State, it is particularly clear that rumors and insecurity are fueling the conflict, and it is vital to address these. Overall, there also has been a significant reduction in violent incidents over the period 2015-2018, compared to the period 2012-2014.35 However, it is not possible to establish any direct causal links between KL2 programming and the improved ICH or the prevention of outbreaks of violence. Nonetheless, factors that did contribute to this KL2’s achievements in preventing ICC violence included well-informed and context-sensitive programming; high-caliber and dedicated local and international staff who have built up a very good knowledge base of effective approaches; and a flexible, responsive 34 The “Kann Let Intercommunal Conflict and Rakhine Cluster Evaluation Report” evaluators interviewed 19 KL2 partner representatives in Yangon, Mandalay, Pyin Oo Lwin, and Sittwe, Ponnagyun and Mrauk-U (Rakhine State) between August and September 2018; and, the “Hate Speech and Social Media Cluster Evaluation Report” evaluator, interviewed over 26 persons in Yangon from KL2, partner organizations and U.S Government staff. 35 OTI/Kann Let Report on Intercommunal Conflict. USAID.GOV Kann Let 2 Final Performance Evaluation | 24 and risk-tolerant approach. The program has developed – and in many cases incubated – a very impressive set of partners implementing strong activities. The fact that a large proportion of the most impressive and successful initiatives and organizations working on ICH nationally and in Rakhine State are supported by KL2, in many cases as their first donor, is an indication of the success of the OTI/KL2 model and its effective implementation. Although measuring impact is extremely difficult with ICC programming of this kind, and the scale of the problem in the country is huge, KL2 has identified plausible drivers of ICC and sensible geographic focus areas and has developed a strong portfolio of partners and activities to address these drivers. It has done so through activities that promoted greater tolerance for diversity, including religious diversity; improved communication and coordination between groups working to prevent ICC; helped to monitor and respond to rumors, hate speech and dangerous speech; and helped to create capacity in at-risk areas for early warning and rapid de-escalation of trigger incidents. While finding evidence for a causal role in prevention of violent incidents will always be difficult, what can be stated with greater certainty based on interview data and the cluster evaluation on ICH is that the KL2 program did contribute to addressing several of the contributing factors to communal violence. For example, its activities have promoted greater tolerance for diversity, including religious diversity; improved communication and coordination between groups working to prevent ICC; have helped to monitor and respond to rumors, hate speech and dangerous speech; and have helped create capacity in at-risk areas for early warning and rapid de-escalation of incidents (see OTI Database). The choice of locations for relevant KL2 activities was sensible, especially given that it is virtually impossible to predict in advance where outbreaks of violence will occur. Its approach involved a combination of broad national interventions, together with targeting of locations where outbreaks of violence were more likely to occur, and where credible partners existed or could operate. These cases can also provide examples of partners effectively managing these tensions and preventing them escalating. For example: one partner targeted forums four conflict prone cities, Pyay, Lashio, Bamaw, and Myitkyina [Interview, November 15, 2018]. There are numerous specific examples to demonstrate that KL2 programming has contributed to positive changes in knowledge, attitudes and behaviors of beneficiaries. For example, one organization provided social cohesion training to officials of the Ministry of Social Welfare, Relief and Resettlement which had a concrete impact on behavior: there are now government champions for these ideas within the Ministry [Interview, November 14, 2018]. Additionally, there are also clear KL2 examples of this process of attitude change leading to changes in behavior: Two international organizations conducted a joint photography training and exhibition in which they brought photographers from around the country to Mandalay to document diversity. A young Rakhine Buddhist participant spent time documenting a Muslim family. This person initially stated that "I am Rakhine and… I was afraid [to take photos of Muslim community] due to the situation here in Myanmar. But then I decided to do it!”36 The photographer’s views on Islam completely changed and her knowledge of the religion and the lives of the community were deepened. As noted by the grantee, the photographer reported becoming friends with the family, a step that is challenging given the nature of the conflict and the situation in Rakhine State [Interview, November 2, 2018]. Another example of these changes is when one CSO (OTI Database) conducted an exposure trip to Myeik in 2018, comprising seven Rakhine community leaders and seven leaders of other communities. A KL2 staff member noted that one concrete outcome of the activity was to create a Viber group of participants to share ideas, which was created by a participant from Rakhine State who shares views with the group [Interview, November 13, 2018]. 36 Myanmar International Radio, “First ever National Geographic Photo Camp in Myanmar Exhibition happening at American Center”, Program Script, Yangon, Myanmar, September 28, 2018. Accessed at: http://miradio.com.mm/news/first-ever-national￾geographic-photo-camp-myanmar-exhibition-happening-american-center-yangon 25 | Kann Let 2 Final Performance Evaluation USAID.GOV More frequently, attitude and behavior change may be a long-term process that is difficult to measure, and therefore sets a very high bar for short-term programming. In many cases, the best that can be achieved is to challenge stereotypes, combat rumors, and gradually chip away at entrenched attitudes. KL2 has helped to achieve these smaller but important steps through its activities. The CSO-led discussion forums on nationalism, religion and politics, with extreme nationalists are a good example of the kinds of events and discussions that can contribute to attitude change. There was an observed impact: some participants did have their minds opened to ideas they had not considered or been exposed to before, even if they did not immediately agree with them. Other participants had been exposed to such perspectives before and were observed to be wavering; and some participants had very firm preconceived views and no amount of engagement will likely be able to change that. Overall, in a difficult programming space where few partners are working or even willing to work, KL2 has been successful in identifying an impressive portfolio of partners whose activities have had a positive impact on ICC based on the aforementioned examples, and who in many cases have demonstrated that these activities are sustainable. KL2’s local partners in Rakhine State are generally very effective at implementing activities that they have taken the initiative to develop and are generally very aware of the constraints they will face, how to navigate them, and the limits of what is possible. When these align with KL2’s objectives, the projects are generally implemented well despite difficulties inherent the context. KL2 has been a successful incubator of networks and local organizations, several of which it has been involved in creating through funding for other activities (events, brainstorming sessions, trainings, etc.). As well as being an incubator of organizations, KL2 has also been an incubator of ideas. This has happened through: (1) Cross-fertilization across implementing partners, such that successful ideas from one activity have been suggested by KL2 staff for incorporation in another activity with a different partner and/or in a different part of the country; and (2) The gradual accumulation of knowledge and ideas on good practice. This has happened because over time, with KL staff developing their own expertise and becoming “knowledge hubs.” In turn, this increasingly has enabled them to provide partners with good advice and ideas, including how to turn ideas into implementable activities. The staff of KL have become an important resource for the country in their own right. A good example of KL2 contributing to stronger relationships and the capacity of ICC stakeholders is through an urban-based youth network formed a regional network, that gained organizational skills and greater technical capacity to implement activities aimed at promoting ICH. Initiatives such as this are important, given the visible and influential role that youth37 have traditionally played in political activism in Burma; today, this is even more important, given that this is demographically a very young country undergoing rapid change and being governed by elderly leaders. Interventions that have achieved broad followings in promoting public discussion or sharing of information that promotes toleration include the series of debates, which according to the Cluster Evaluation on ICC and Rakhine are highly regarded by people involved in ICH work as well as more broadly among politically-engaged members of Burmese society, “They are seen as very good debates – lively, engaging, thought-provoking, and not shying away from sensitive topics,” noted the Evaluation. “They have revived a culture of substantive debate that had long been lost in Myanmar.”38 Cascading trainings are another means to reach and influence many people. One particularly successful activity involved youth leaders and youth in central Burma. One CSO conducted trainings on peace building and conflict resolution for youth leaders from different religious communities, ICH activists and others. The activity included a training-of-trainers component, which was seen by the partner as being particularly 37 Youth refers to people in the 15-24 years old age range while recognizing “youth programs likely engage a broader cohort of 10 and 29-year-olds; with the critical understanding that the transition from childhood to adulthood is not finite or linear and varies across and within countries.” USAID “Youth in Development: Recognizing the Demographic Opportunity,” October 2012 Washington D.C. 38 The live Facebook stream has served to greatly increase viewership. See Horsey, Richard, “Kann Let Intercommunal Conflict and Rakhine Cluster Evaluation Report”, USAID/OTI, October 31, 2018 USAID.GOV Kann Let 2 Final Performance Evaluation | 26 successful. These TOT alumni formed a training team, have done follow-up pro bono trainings with their resource-person team, and become sought-after and respected thought leaders among their contemporaries. KL2 successfully intervened to manage ICC-related online and offline rumor spreading. Two organizations worked together on a grant co-funded by KL2 on rumor management (OTI Database). One of these groups focused on trainings in rumor management, whereas the other focused on media literacy and peace mobilization systems online. The two organizations formed a community information management team in each area, which was trained in responding both online and offline to rumors [Interviews, November 14, 2018]. According to the ICC and Rakhine Cluster Evaluation, “There is clear evidence that the project was able to facilitate stronger relationships between key influencers. The project was therefore effective in creating stronger links between local officials and their communities.”39 KL2 successfully intervened to resolve some local incidents that could have resulted in violence. There were also successful initiatives to influence and build relations between key thought leaders and other important constituencies. One clear example is the five public fora conducted by a CSO on nationalism, religion and politics. These fora were presented by well-known writers and attended by more than one thousand participants. Another example was another CSO, which led a committee of organizations to convene a rapid response stakeholder meeting in an ICC-affected town that brought together local government authorities, members of civil society and other prominent local individuals. 40 The thinking behind the project was that communal harmony projects often focus on religious leaders and CSOs to solve conflict. Early Warning Early Response (EWER) projects supported by KL2 were useful and well executed. For example, one CSO produces regular “EWER Situation Updates” (OTI Database), which OTI/Burma and KL2 staff noted were widely circulated and helped to inform a broad range of stakeholders [Interview, October 29, 2018, and November 13, 2018]. Through these updates, the organization disseminates accurate, timely and detailed information on the ICC situation in both Rakhine State and the rest of the country. The content of the situation updates comes in part from a network of individuals and groups at the community level who contribute information and analysis to the CSO. This network, and the ongoing support for and engagement with it, are an important part of the CSO’s ability to conduct support for community responses to incidents of ICC. As a whole, KL2’s programs helped the USG navigate the August 2017 crisis and a highly sensitive ICC landscape. The information generated by the project made it possible for KL2 staff to provide the Embassy with “24/7 updates on the situation from the ground up, making the U.S. Mission one of the best informed in country,” according to Embassy staff [Interview, November 2018]. While these attacks and the subsequent military response caused temporary disruptions to KL2 activities, most partners and their programs were able to resume eventually, protecting the ICH space. Overall, KL2 was generally sensitive to the greater constraints on programming in Rakhine State, placing a lot more emphasis on parallel community activities and only limited joint activities; focusing on capacity building in recognition of the greater needs in Rakhine; and promoting tolerance for diversity given the context of extreme discrimination. Such an approach was wise. It was often the case that activities could be more effective when not explicitly branded “inter-faith” or “communal harmony” – or not even focused mainly on achieving those goals. Experience has shown that it is often better to bring people together under a non-threatening banner or one they are more interested in; whereas activities that 39 OTI/Kann Let Report on Intercommunal Conflict. 40 This meeting was urgently convened in response to increasing inter-religious tensions in the city, which had suffered from serious violence in 2013 that left dozens dead. In particular, pre-existing tensions were exacerbated by two incidents in mid-2017: the prosecution of a local Muslim leader for receiving Ramadan donations without official permission, and a forced entry into a Muslim home and attack on a man living there, perpetrated by a group of individuals known to be close to Ma Ba Tha. 27 | Kann Let 2 Final Performance Evaluation USAID.GOV are of more direct interest to a broader range of people – vocational training, technology and so on – can introduce sensitive topics more discreetly and indirectly. Issues Rakhine State presented particular challenges. Assessing success against strategic objectives in Rakhine is extremely difficult, both because any interventions will require a long time to show impact, and because over the period under review, the situation in Rakhine State deteriorated catastrophically. However, while impact may be hard to measure, the case can be made that without KL2, many of those partners would not exist, and the civil society space in the Rakhine would be much less encouraging One key challenge of working on ICC is that there are a limited number of partners. While KL1 was able to increase this number from a base of almost zero, further expansion is possible but will be constrained by the inherently difficult – and, in many communities controversial – nature of ICC and religious harmony work. Also, partners and communities tend to be reactive rather than proactive, keen to take action after violence, but less so as a preventive measure before violence. Some communities may feel that violence is unlikely, or a prospect they do not want to discuss, until it actually takes place. EQ3 CONCLUSIONS KL has identified plausible drivers and sensible geographic focus areas and has developed a strong portfolio of partners and activities to address these drivers. It has made a plausible contribution to addressing the drivers of ICC through activities that promoted greater tolerance for diversity, including religious diversity; improved communication and coordination between groups working to prevent ICC; helped to monitor and respond to rumors, hate speech and dangerous speech; and helped to create capacity in at-risk areas for early warning and rapid de-escalation of trigger incidents. At the same time, such activities can only ever be expected to have a small impact on what is a serious national problem requiring a sustained, nationwide response. The ET found that KL2 has plausibly contributed to the observed reduction in ICC during the review period in Burma (and in Rakhine State), and that it prevented violence from occurring or from escalating. The program has established a track record and a range of partners that the follow-on program will be able to mobilize to continue context-appropriate and risk-aware programming after OTI/Burma closes. KL2 has been flexible enough to withstand external shocks and challenges that have led other donors to suspend their activities in this programming space – an obvious example being the August 2017 ARSA attacks in Rakhine. Taking stock more than one year after the ARSA attacks, they have not had a catastrophic impact on the ability of partners to continue to do ICH activities successfully, although they have led to more popular anti-Muslim sentiment and to a much more cautious attitude from government – and northern Rakhine State is still reeling from the crisis and an extremely difficult context. The ET agreed with the findings of the cluster evaluation that the crisis in Rakhine State must be seen as a three-sided conflict, between the government/military, Rakhine Buddhist communities and Muslim communities, and that addressing the conflict requires working on all three sides.41 The ET also noted that there are significant risks involved in Rakhine State programming, and that conflict sensitivity is essential. Even though it emphasizes the intercommunal aspect of the Rakhine State crisis, it is the government that is the biggest obstacle to intercommunal interactions in Rakhine – building links, contacts and dialogue between communities contradicts its policy and instincts in favor of peace through 41 To date, the government prefer to focus on ICC and ignore the other two, whereas the international human rights community has tended to focus on tensions between Muslim communities and the state, and to some extent also on intercommunal relations, but much less on tensions between Rakhine communities and the state. USAID.GOV Kann Let 2 Final Performance Evaluation | 28 segregation, which have hardened as a result of the crisis. Nevertheless, with care and planning, and the right partner organization, it is possible to do – although expectations and ambition should be moderated. KL2 activities can only be expected to have a small impact on what is a serious national problem requiring a sustained, nationwide response. It is important to be realistic about what can be achieved through short-term donor programming. Often, the best that can be done is to challenge stereotypes, combat rumors, and start to chip away at entrenched attitudes – while offering a lifeline to keep moderate individuals and organizations in existence and engaged. These are all things that KL2 has achieved, the importance of which should not be underestimated. EVALUATION QUESTION 4 To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? EQ4 FINDINGS The ET heard many times in interviews with partners and Burma experts that KL2’s timely, responsive, flexible actions in support of democratic transition via robust civil society engagement significantly influenced political reform. KL2 broadly demonstrated this engagement in various ways through assistance to partner CSOs to: address critical issues in the transition (especially new KL partner CSOs); serve as key information and data sources; build advocacy activities into programming; and address organizational development issues, for example. The entire KL2 program also supported civil society by enabling them to address challenging objectives, such as the peace process, ICH, and fundamental freedoms. Those civil society activities were particularly crucial because the NLD-led government is not attending to those issues inclusively. More importantly, KL2 partners are worried that other donors will not pick up KL2’s successful activities after KL2 closes down. Nevertheless, KL2 succeeded in engaging civil society throughout reform processes by adjusting to Burma’s unpredictable context via the reliability and expertise of its national staff, and by collaborating with international experts and myriad civil society partners, to take advantage of emerging transition opportunities. By doing so, KL2’s support to and partnerships with civil society were timely, targeted, and critically important to support the country’s transition. Achievements There are several important findings regarding KL2’s work with civil society that should be highlighted. First, KL2 directly worked with over 93 CSOs on 193 activities, including 10 media-related interventions and 29 activities focusing on CSO networks. 42 Under the KL2 55 new partners were added that both moved beyond the “usual suspects” and targeted areas outside Yangon. Awards to these new partners represented about a third ($5,054,206) of the overall KL2 budget for the three primary objectives under this evaluation, with a roughly 60/40 split between cash and in-kind grants.43 Major interventions by strategic CSOs in support of transitions are summarized in Annex E: Roles of CSO in Kann Let 2- Supported Transition. KL2’s national staff were well positioned to implement these particular activities, given their local contextual knowledge and skills facilitating strategic and technical inputs to CSOs. Because of this approach, CSOs felt that KL2’s activities were more tailored to their needs, strategically oriented, locally owned, and largely free of foreign interference, all while enabling them to enrich key technical skills, such as preparation of grant proposals and development of new, innovative ways of thinking [Interviews, CSO leadership in Yangon, Mandalay, and Myitkyina, November/December 2018]. Additionally, KL2’s national staff involvement in a non-partisan review of documents prepared by the partners prior to their submission gave them greater confidence and made their preparations sound. 42 Data from OTI Anywhere Database retrieved in December 2018. 43 Data from OTI Anywhere Database retrieved in December 2018. 29 | Kann Let 2 Final Performance Evaluation USAID.GOV Partners also indicated that the KL2 mechanism’s support of their activities through a timeline with clear milestones was a very good way of managing with funding and activities. Second, KL2 clearly demonstrates that new CSOs are important in addressing local critical issues, such as ICH. Newly-emergent, locally-owned CSOs and CSO networks appear to be well-connected with local communities, thus putting them in a position to manage conflicts as they arose. They also represented an opportunity to expand existing KL1 networks and knowledge to reach new communities, and to bring in to KL2’s learning process new sources of information about complicated social and political issues. KL2 took advantage of opportunities to work together with CSOs on developing their ideas, proposals, and overall project management. Often, KL2 also funded partners’ overhead and core costs for existing operations, whereas other donors only offered project-focused funding. In addition, “KL2 fully recognized and funded” organizational development activities for such partners” [Interviews with civil society representatives and KL2 Staff, November/December 2018]. All of these factors directly supported CSOs’ sustainability in a time of critical political transition within Burma. Finally, KL2 support was extended to unregistered organizations, networks and even individuals when necessary, creating a greater impact on this transition. CSO partners expressed their appreciation for those actions. Third, partner CSOs were also a key information and data source that supported transition work in many ways, including through outreach activities to inform the general public and advocacy for stakeholders. KL2 actively engaged civil society on this front, utilizing them to help mitigate information risk, which was especially important because the government proved weak at communication. KL2 also supported the transformation of CSOs to be more data-driven and advocacy-oriented in addressing sensitive issues, such as ICH and the peace process. Team findings are also supported by recent research on CSOs’ contributions to transition, which notes that “the space for CSO contributions to peace negotiations between the Government of Myanmar (GoM) and EAOs has widened considerably.”44 KL2- supported CSOs played a vital advocacy role in this regard. Despite inexperience in advocacy work, KL2 partners were quick learners; the project gave them a chance to “test the waters” in advocacy via data- and evidence-driven initiatives [Interviews, November/December 2018]. Through efficient funding, development of curriculum, policy briefs, and manuals (e.g., the fiscal federalism manual), as well as publication of technical papers, KL2 supported peace process-focused advocacy efforts successfully. A similar track record was demonstrated through KL2’s ICC/ICH activities and Fundamental Freedom (Freedom of Expression) initiatives. Compared to KL1, the partners also identified youth as a niche target, since “members of that age group are more likely to be change agents,” compared with the older generations, on critical transition issues [Interviews, November/December 2018]. Additionally, given that information flow from different levels of authorities remained quite low, public outreach activities played a vital role in supporting the transition. Specifically, KL2 worked with both media and public diplomacy partners to influence public awareness through activities. These activities were all influential amplifiers of key messages on the peace process, ICH and fundamental freedoms. Furthermore, through these activities, KL2 partners were able to encourage audience members in the public to be more accepting of people of different religious and ethnic backgrounds.45 Issues The ET heard from many partner and expert interviewees that under the NLD-led government, the civil society space remains limited and appears to be shrinking, thereby creating risks for CSOs to operate as they have before. The popularly elected government feels it has the sole mandate to speak for the people and sees little role for civil society, even though CSOs are the ones who have closely worked with the 44 Michael Lidauer et al., “Unlocking Civil Society and Peace in Myanmar: Opportunities, Obstacles, and Undercurrents”, Paung Sie Facility, Page, 15-16. 2018. 45 Audience Surveys, preliminary findings from December 2018. Note that the complete findings will be released in March 2019. USAID.GOV Kann Let 2 Final Performance Evaluation | 30 grassroots for many years, especially in remote and difficult to access areas. The NLD-led government recently announced, for instance, that mandatory requests to authorities to hold open CSO meetings should be submitted at least 10 days advance.46 Paung Sie Facility (PSF) research also points out that one of the major difficulties for CSOs currently is their “relations with the State” [Interviews with CSO representatives, Yangon, Mandalay and Myitkyina; November/December 2018]. Specifically, interviewees indicated that this government has a limited understanding and acceptance central role CSOs play in performing advocacy work, which is key in helping to broaden democratic reforms. However, there were several exceptions that exemplified sound advocacy work in collaboration with Union Parliaments in Nay Pyi Taw with respect to improving freedom of expression. EQ4 CONCLUSIONS The CSO landscape is becoming bleaker and appears to be shrinking at a time when their inclusion is widely thought to be important for consolidation of transitional reforms. Meanwhile, many small, local CSOs feel that bigger national ones are taking up all funding and space, thereby limiting opportunities for locally grown ones to operate. Thus, it is important to spread funding around, and to build capacity for newly emerged ones, too. As a pillar of democratization, CSOs and their technical inputs are fundamentally necessary for the future of Burma’s transition, and must remain strong to defend and expand democratic reform. Without KL2, CSOs would not have been able to expand functionally or territorially, which would have negatively affected support for the transition. Key informants interviewed expressed that, compared with other donors, grantees tend to see the USG as the most committed to supporting a robust role for civil society in the country [Interviews, November, 2018]. This finding supports the USG’s Integrated Country Strategy’s mission objective 1.2, ‘to expand space for civil society and the media to support the democratic transition.’47 That also means, without USG’s ongoing support for CSOs, that Burma’s transition will most likely face more difficulties. It is important to note that since the NLD-led government assumed power, a considerable number of foreign donors had initially planned shifts in their support from civil society towards direct funding to the government (Union and State/Region Levels) and related governmental institutions in the hope they would achieve more effective delivery of government services. Although by 2017, this shift had not been fully implemented by some donors, and support where possible was re-directed to civil society assistance, this has been a discouraging sign for CSOs performing important work on the ground, and many feel it ignores their roles in the democratic transition and in building peace. EVALUATION QUESTION 5 How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? EQ5 FINDINGS The ET found that OTI brought deep knowledge of the environment to the Mission and Washington, DC, showed consistent consideration for a range of stakeholders, and recognized both opportunities and risks for both their and the Embassy’s engagement in a highly complex operating environment. OTI’s contributions to U.S. foreign policy were made primarily through the key role that OTI staff played within the Embassy country team, shaping political analysis, programming choices, legal interpretation, operational decisions, and policy formulation. 46 Quoted from a notification letter issued on November 8, 2018 by General Administration Department (GAD), Bopyin Township, Tanintharyi Region. 47 Quoted from Integrated Country Strategy, Approved August 30, 2018. 31 | Kann Let 2 Final Performance Evaluation USAID.GOV Although many interviewees questioned whether a U.S. policy – formal or informal – toward Burma existed, there was general agreement that US interests are to promote a stable, peaceful, prosperous democratic system in the country. Embassy personnel pointed the ET to the Integrated Country strategies (2015-2017 and 2017-2020) as the two primary statements on U.S. foreign policy to Burma. Inasmuch as the Integrated Country Strategies (ICS) focus on the Mission goals and objectives and identify responsibilities for achieving them, they lay out the whole-of-mission approach to supporting reform in the country. The two strategies addressed different contexts. In the 2015-2017 ICS, there was optimism that the government constituted after the 2015 election would be one that would continue and expand upon the political, economic, human rights, and peace reforms initiated by President Thein Sein (2011- 2016).48 By the time of the 2017-2020 ICS, a more sober reading of the context came to inform scaled￾back and differently directed objectives and sub-objectives. Throughout both, however, were common threads that U.S. interests aligned with promoting a more prosperous, open, democratic and peaceful Burma. Achievements According to Embassy personnel and USG officials in Washington, DC, OTI/Burma and KL2 helped to advance US foreign policy by: ● promoting inclusiveness in the peace process; ● sponsoring creative approaches to weaken the drivers of ICC; and ● where possible, working to expand fundamental freedoms. Succinctly put by a former KL1 staffer who is now running a major international NGO, “I think OTI impacts positively on people on the ground, and its main ability is to act like a quick fix” in a dynamic environment where other donors were not as responsive” [Interview, November 12, 2018]. Most notably, the staffer continued, KL2 built on the success of KL1 in increasing trust and relationships between the US government and communities in Burma. As observed by several interviewees, the granular knowledge developed by OTI/Burma and KL2 staff provided “the leading voice for how to proceed” on the peace process, intercommunal relations and other kinds of political analysis. According to a senior official at the U.S. Embassy/Burma, “After the 2017 [Rohingya] crisis, it [KL2] helped us to develop laser vision focus” on how the environment was changing so rapidly” [Interview, November 1, 2018]. Noted another Embassy official: “OTI supports concrete needs and responds with funding when approached with good ideas. That’s especially important when diplomacy is so challenging” [Interview, November 20, 2018]. One State Department officer in Washington, DC, observed that OTI helped the U.S. Embassy to develop “more consistent” policy and approaches than she had witnessed in other countries [Interview, November 9, 2018]. The impact of OTI/Burma and KL2 was appreciated in Washington, DC. As one State Department official noted, “OTI staff did a good job of representing the voices of their partners on the ground and inserting them into policy conversations,” that may have occurred in a less informed way if OTI was not involved [Interview, November 9, 2018]. Another State official said that OTI and KL2 helped to “ground truth” debates in Washington, where there has been a tendency to “oversimplify the complex realities” of Burmese political developments [Interview, November 27, 2018]. Issues Several lessons were learned from the close collaboration between OTI and other parts of the US Mission to Burma. First, the integration of OTI with Embassy personnel in several key policy-oriented 48 “A more democratic, inclusive and representative government – one that is responsive to the people – will not only help end decades of repression and fighting between the Burmese military and ethnic armed groups, but also solidify the U.S.-Burma bilateral relationship over the long run, and create the conditions for economic growth in Burma that will increase demand for U.S. exports and investment, ultimately creating jobs in both countries.” ICS 2015-2017, p.3. USAID.GOV Kann Let 2 Final Performance Evaluation | 32 committees at the Embassy was noted favorably by several Mission staff; OTI, drawing upon the knowledge developed by KL2 staff, was a “thought leader at the table,” said one Embassy staff member [Interview, November 14, 2018]. Second, an independent expert on Burma underscored the productivity of the management style at KL2; specifically, the in-kind mechanism was lauded as it enabled civil society to “do a lot of these activities fast, not worrying about logistics… and focusing on the content of their work” [Interview, November 12, 2018]. Another independent expert highlighted this flexibility and responsiveness as generative of an “immediate response” to local needs, which ultimately reflected well on “views of the U.S. and its policies” [Interview, November 10, 2018]. Third, the small grants and local knowledge enabled KL2 to do “great job of finding in hard context those opportunities, pry [them] open and see what works,” one Embassy staff member observed [Interview, November 14, 2018]. EQ5 CONCLUSIONS There is broad agreement that OTI/Burma and KL2 brought significant knowledge and credibility to the USG presence in Burma. Compared to other missions, the U.S. Embassy was noted by Burma experts as well as grantees for its responsiveness to even the slightest shifts in the political and social landscape and was widely considered to have the best grasp of the dramatic events that would unfold, particularly after the 2017 ARSA attacks in Rakhine State. 33 | Kann Let 2 Final Performance Evaluation USAID.GOV RECOMMENDATIONS OVERALL RECOMMENDATIONS 1. USAID and the U.S. Mission to Burma should retain and utilize KL2’s significant analytical skill and focus to address the barriers to advancing reform and peace. As the 2020 election approaches, and as the peace process and Rohingya refugee repatriation likely stall, democracy and governance programming should be based on careful contextual analysis of shifting and emerging impediments to reform. 2. USAID should be aware that the peace process and ICH will be impacted by the partisan environment in the lead up to the 2020 election. Electioneering is likely to see space for CSOs shrink further; increased risks to ICH programming as populist forces rally support; and the stalling of the peace process will result as attention turns to the election 3. Follow-on programs should pick up as much flexible, well-informed, locally-contextualized work as possible. 4. Follow-on programs should create small, in-kind mechanism for grant provisions, specifically to support networks and smaller CBOs and CSOs, in a timely fashion. USAID and the U.S. Mission to Burma should look for partners from outside Yangon, not just Yangon-based NGOs with field offices, to build multi-locational programs of activities. CSOs in ethnic states and non￾Yangon regions have very different visions, missions, capacities, and constituencies. These alternative perspectives may help to promote new kinds of networks that advance the new program and USG debates and strategic reassessments. 5. Follow-on programs should be resourced with appropriate overheads required for high levels of staffing and learning activities, demonstrated flexibility, and expertise in Burma on peace and ICH programming. RECOMMENDATIONS FOR EVALUATION QUESTION 1 Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 6. USAID and the U.S. Mission to Burma should undertake programming that engages the Bamar majority – whose non-engagement was appropriately identified as a critical impediment by KL2 – on peace, ICH and fundamental freedoms. These kinds of activities require small amounts of funding to local groups or networks, in recognition that threats to peace, fundamental freedoms and especially ICH vary immensely from one location to another. 7. USAID and the U.S. Mission to Burma personnel should reach out to ICC and peace partners for advice on how to mitigate the risks of inflamed religious- or ethnic-based conflict resulting from the competition that is beginning to unfold in advance of the 2020 election. RECOMMENDATIONS FOR EVALUATION QUESTION 2 Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 8. USAID and the U.S. Mission to Burma should continue to support work that increases women’s participation, representation, and contribution, to issues under discussion in the peace process. USAID.GOV Kann Let 2 Final Performance Evaluation | 34 If continued support is not possible through U.S. supported mechanisms, then a handover of activities should be negotiated. 9. USAID and the U.S. Mission to Burma should ensure that follow-up programming supports continuing work on fiscal federalism (and related matters) with the key stakeholders to the peace process (Government, Parliament, EAOs, Civil Society) to support the creation of an equitable federal system, build a social union, and develop models for revenue sharing, including on natural resources. Continued work on these issues could also be conducted by partner organizations with alternative funding recommended by USG. 10. USAID and the U.S. Mission to Burma should find ways to continue to support local programs, especially given the diminishing role afforded by government to civil society in the peace process. This will ensure that CSOs continue coordination around the peace process as well as the sharing of information and the forging of common positions. 11. USAID and the U.S. Mission to Burma should ensure that as many relevant peace process support and implementing partner organizations are referred to future USG funding mechanisms or other peace support mechanisms. 12. USAID and the U.S. Mission to Burma should identify alternative, follow-on donor funding and in-kind capacity for support to the peace process, specifically for EAOs whose needs exceed the capacity or will of other donors. RECOMMENDATIONS FOR EVALUATION QUESTION 3 To what extent has the program been effective at mitigating drivers of ICC in Burma? 13. USAID and the U.S. Mission to Burma should continue to support rumor management and response activities, which fuel conflict, both online and offline, across Burma. It should do this by identifying alternative, follow-on donor funding and in-kind capacity to support social media monitoring. There is no apparent home for this kind of programming in new programs. Additionally, the ET also recommends continued support for activities related to social media monitoring and reporting, including fact checking, public communications and media literacy training of trainers, organizational development, and online security. 14. USAID and the U.S. Mission to Burma should ensure continued support to ICH and Rakhine State programming through a mix of short-term and longer-term interventions. Short-term initiatives are crucial to try to prevent conflict and tensions from escalating, and to seize temporary opportunities to give support and respond to humanitarian or human rights emergencies. These efforts will need to be closely coordinated – perhaps through a working group of USAID implementers – and with others in this programming space, including the PSF. In this regard, greater sharing of data, research products and perspectives would be useful. It is also vital that these meetings be held before the end of the program so that projects can be handed over as seamlessly as possible to new USG and other donor programs. 15. USAID and the US Mission to Burma should continue to support context-sensitive, agile and creative ICH programming, particularly in Rakhine State, with the expectation that it will continue to be unstable and prone to sudden, major crises for years to come. Maintaining a strong set of local partner networks and organizations, with staff who understand the context and its limitations and opportunities, is vital. As the conflict in Rakhine State is a three-sided one, it will be important to involve the central state (government and military), the Rakhine, and Muslim communities including the Rohingya. It is important to work on all three dimensions of the conflict, and it will often be different partners who are best-suited to working on each. 35 | Kann Let 2 Final Performance Evaluation USAID.GOV 16. USAID and the U.S. Mission to Burma should support ICH programming in rural and peri￾urban areas, which are currently the target of nationalist groups across the country. Find ways to continue to target rural areas, for example through strategic support to village tract administrations and CBOs and their networks. Support to projects working with Rakhine State Government should also be encouraged. 17. USAID and the U.S. Mission to Burma should accept that real impact is likely to take a generation and will require longer-term interventions to contribute to this. For work in various thematic areas, USAID and the U.S. Mission to Burma should take into account that changing mind-sets and behavior could take generations, and that this should be factored into future programming in ICC-prone areas, including Rakhine State. 18. USAID and the U.S. Mission to Burma should ensure that “in-kind” grants are made by follow￾on projects to support ICH. These were especially relevant to support partners and networks and were found to be particularly well-suited for Burma in the area of ICH, where much of the strong work is done at local levels by more loosely-organized and less formal CSOs and networks. Follow-on programs should create small, in-kind mechanisms for grant provisions, specifically to support networks and smaller CBOs and CSOs, in a timely fashion. 19. USAID and the U.S. Mission to Burma should recognize that “Inter-faith” has become a buzzword, and risks merely preaching to the converted and also of exacerbating tensions within Buddhism, which can lead to further conflict. Intra-faith work is therefore relevant, but difficult because it “opens up hidden wounds” within Buddhist communities. In general, activities can be more effective when not explicitly branded “inter-faith” or “communal harmony” or not even focused mainly on achieving those goals. Experience has shown that it is often better to bring people together under a non-threating banner or one they are more interested in. Whereas activities that are of more direct interest to a broader range of people can introduce ICH issues discreetly. 20. USAID and the U.S. Mission to Burma should acknowledge that working with youth on ICH is important: Burma has a very young population, is undergoing very rapid change, but is led by aged politicians. Youth tend to be more open to new ideas and change. RECOMMENDATIONS FOR EVALUATION QUESTION 4 To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? 21. The USG should recognize that a robust and diverse civil society is of fundamental importance to Burma’s transition in that CSOs are an effective countermeasure to emerging transition setbacks. Despite the current bleak landscape for CSOs under the popularly elected NLD government, diverse voices on different complex issues (such as peace, ICC/ICH) need to be heard as much as possible. CSOs are indispensable in carrying those voices to respective political and policy circles. The weakening of civil society therefore would represent a serious setback to the democratic reform process. USG should recognize this, and also should maintain, support and expand support for CSOs’ work on peace process, ICH, and fundamental rights issues – especially during the lead up the 2020 elections. 22. Data-driven and evidence-based CSO activities should continue and are crucial to Burma’s transition. Data-driven and evidence-based advocacy is now a major conflict resolution tool for Burma’s complex emerging issues. This kind of approach will help Burma move forward in many areas, including peace, ICH, and fundamental freedom issues. For instance, fiscal federalism is a key to decentralization, and can be explained via data and evidence how it works now and should work in future. However, because advocacy is a new subject for both CSOs and the government, USAID.GOV Kann Let 2 Final Performance Evaluation | 36 it is important to find solutions that work in practice, and to manage expectations about the timeline and content of possible reforms that may result from evidence-based advocacy. 23. Funding for CSOs should be balanced between existing and newly emerged CSOs. Existing organizations need to strengthen their specific expertise in areas such as peace or ICH; they need to specify their area as well for their professional engagement with relevant stakeholders. In general, the more specific their expertise, the more effective they will be. In contrast, new CSOs engaged by KL2 were mostly locally driven; for instance, when a new ICC case happened in a remote village, a local CSO would emerge to address this. Because both have a role to play, funding should be balanced between the two, and should include fostering new initiatives or moving CSOs into new areas. 24. USAID and the U.S. Mission to Burma should increase engagement with CSOs outside of Yangon. This echoes the KL Mid-Term Review’s similar recommendation,49 and is still relevant now. Local CSOs feel that national NGOs, especially Yangon-based ones, are taking up many opportunities and much of the funding, thereby limiting resources and opportunities for local ones. In fact, many conflicts are locally driven, for which local CSOs are necessary because they are experts at understanding this context. For instance, CSOs in ethnic states and non-Yangon regions have very different visions, missions, capacities, and constituencies. They are the ones who understand about associated local risks there, too. These alternative perspectives may help to promote new kinds of networks that advance further KL/USAID related objectives. RECOMMENDATIONS FOR EVALUATION QUESTION 5 How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? 25. USAID and the U.S. Mission to Burma should engage its Democracy and Governance partners in an OTI-like analytical process of evaluating political and social change via regular reviews and strategy sessions, specifically charting the shifting landscape of critical impediments to peace and ICH. 26. For purposes of sustaining the goodwill of Burma’s civil society, OTI/Burma and USAID should convene meetings to introduce KL2 partners at risk of being left out of funding opportunities to other donors. These could include the PSF. This should be a purposeful undertaking and considered high priority. 49 “Kann Let’s future program should have better engagement with CSOs outside Yangon”. See Dr. Mary Callahan and Aung Tun, “Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016”, USAID/OTI, 2016, pages12-13. 37 | Kann Let 2 Final Performance Evaluation USAID.GOV LOOKING AHEAD In interviews, many grantees and independent experts on Burma offered their thoughts about the likely political, conflict and social environment, for the follow-on programs from OTI to take into consideration. One common theme, as will be seen below, is that the 2020 election campaign is likely to exacerbate existing social tensions across OTI’s objectives – peace, ICH, and fundamental freedoms. In the area of critical impediments to the peace process, the new one that lies in the horizon is this campaign for the 2020 election. As was the case in the lead up to the 2020 election, campaign politics will take elite and popular attention away from the peace process. As OTI understood, the peace process is a long-term undertaking, aimed at finding a path that accommodates or sidelines a complex array of conflict actors and political agendas. In 2015, peace negotiators ran out of time to bring non-state armed groups together around a common framework for dialogue, as national attention was diverted from the building of bridges between and among former enemies and toward divisive, partisan political competition during the run-up to the election. As several grantees noted, the logics of elections and peace processes are at odds: peace processes aim at conciliation and compromise while election campaigns, particularly in divided societies, rely upon showing up one’s opponents. Additionally, grantees and independent experts warned that armed conflict will continue in northern and northeastern Burma, and possibly in areas where the NCA is under fire. It will be important for new programs to ensure that all assistance to communities, civil society and the media in these areas be conflict-sensitive. In terms of ICH, the major issue on the horizon is potential Rohingya refugee repatriation to northern Rakhine State. According to grantees and independent experts, OTI’s follow-on programs should be aware that no large-scale repatriation is likely to take place, as the three-way conflict among Rakhines, Rohingyas and the Bamar-dominated Union government is likely to remain an intractable one. Repatriation of any sort is likely to get politicized by the three Rakhine ethnonationalist parties in the lead up to the 2020 election. Moreover, Rakhine State and nearby areas in southern Chin State will continue to be home to an armed anti-state movement, the Arakan Army, the popularity of which has grown in the aftermath of the 2016 and 2017 crises. Elsewhere in the country, populist political forces that will continue to mobilize around defense of the nation, race and Buddhism from non-Buddhist, “foreign” influences and Islam. According to grantees and independent experts, these nationalist groups are likely to become more prominent as the 2020 election campaign unfolds. To date, the most organized opposition party, the USDP, and its followers have fielded rallies, protests, and Facebook campaigns that showcase the threats that Burma allegedly faces from not only “terrorists,” namely the Arakan Rohingya Salvation Army, but also the meddling of western and UN leaders who seek – in its view – to undermine national sovereignty by spreading allegations about human rights abuses. Other political parties and perhaps even the NLD will seek to claim the mantle of defending the nation better than others, and it is possible that such campaign nationalism could foment intercommunal violence. It is also the case that this kind of populist appeal may serve to undermine Bamar and Rakhine support for the peace process. Grantees working in the area of prevention of ICC noted that communal violence can result from unpredictable events, or be the object of deliberate manipulation, but in either case it is very difficult to foresee its onset or path. That said, trends among nationalist forces and opposition political parties in their offline and online behavior suggest that communal, religious and nationalist claims will be center stage as the 2020 election approaches, thus raising the possibility that tensions along sectarian or communal lines could turn violent. Rohingya refugee repatriation will likely introduce one of a number of major schisms that divides the ruling party from Rakhine ethnonationalists, but also from those claiming to promote “national race protection” nationwide. USAID.GOV Kann Let 2 Final Performance Evaluation | 38 Across the board, interviewees were clear that Facebook will be a platform for the military, conflict actors, political parties and others to incite hate, suppress voting and possibly lead to offline action against vulnerable communities. Over the last few years, Burma’s almost overnight entry into the world of digital connectivity has created the fastest-ever uptake of Facebook in history. Indeed, surveys show that few in Burma use the internet for anything other than Facebook, gaming and pornography. With excessive dependence upon Facebook for information, Myanmar citizens are extremely vulnerable to the spread of disinformation and hate speech. In this ET’s judgment, neither Facebook nor the Myanmar government appear to have the will or the capacity to prevent negative impacts on communities targeted by online incitement, and there exists the potential for coordinated attacks against vulnerable minority communities. Grantees and independent Burma experts all recommend post-OTI funding be specifically targeted to social media monitoring. Barring a change in defamation, security, media, assembly and association laws, fundamental freedoms in Burma will continue to face restrictions, particularly in the lead up to the 2020 election. The extensive use of criminal defamation, official security and assembly laws, media and association regulations have led to many arrests of or lawsuits against journalists, protestors, civil society actors and others over the period of KL2. According to grantees and independent Burma experts, programs should be aiming for defense of existing civil society and media space wherever possible and be aware that more rather than less pressure will be placed on activists, journalists and citizens when they attempt to exercise their fundamental freedoms. 39 | Kann Let 2 Final Performance Evaluation USAID.GOV ANNEX A. EVALUATION DESIGN MATRIX EVALUATION QUESTION DATA COLLECTION METHODS & SOURCES ANALYSIS 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? ● Document review (RA, SRS, PPR, etc.) ● Database analysis ● FGD ● KII ● CR, DCR ● DAI Chief of Party, Deputy Chief of Party ● Content analysis ● Trend analysis ● Comparative analysis 2. Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? ● Document review (RA, SRS, PPR, etc.) ● Database analysis ● FGD - PDOs, Grant Managers in peace process ● KII ● CR, DCR ● DAI Chief of Party, Deputy Chief of Party ● Grantees working on peace process ● Other stakeholders in peace process ● Direct observation ● Content analysis ● Trend analysis ● Comparative analysis 3. To what extent has the program been effective at mitigating drivers of ICC in Burma? ● Document review (RA, SRS, PPR, cluster evaluation, etc.) ● Database analysis ● FGD - PDOs, Grant Managers on ICH ● KII: ● CR, DCR ● DAI Chief of Party, Deputy Chief of Party ● Grantees working on ICH ● Other stakeholders in ICH ● Direct observation ● Content analysis ● Trend analysis ● Comparative analysis 4. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? ● Document review (RA, SRS, PPR, cluster evaluation, etc.) ● Database analysis ● FGD - PDOs, Grant Managers ● KII: ● CR, DCR ● DAI Chief of Party, Deputy Chief of Party ● Other stakeholders in and around civil society and democratic reform ● Grantees ● Other stakeholders ● Direct observation ● Content analysis ● Trend analysis ● Comparative analysis 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? ● KII: ● CR, DCR ● DAI Chief of Party, Deputy Chief of Party ● Burma-based U.S. Embassy and USAID staff ● Washington-based OTI, USAID, and State Department staff ● Content analysis ● Trend analysis ● Comparative analysis USAID.GOV Kann Let 2 Final Performance Evaluation | 40 ANNEX B. INTERVIEWEE LIST NO. ORGANIZATION TYPE 1. Assistant Public Affairs Officer Interview 2. Burma Desk Officer, Bureau of Conflict Stability Operations, State Department Interview 3. Burma Desk Officer, Democracy, Rights and Labor, State Department (previous) Interview 4. Deputy Chief of Mission Interview 5. Political Officers - Human Rights Interview 6. Political Officer – ICH & Rakhine Interview 7. Political Officer Interview 8. Public Affairs Section Chief Interview 9. USAID/Burma Democracy & Governance Director Interview 10. USAID Burma Desk Officer Interview 11. USAID/Burma Mission Director Interview 12. OTI/Burma Acting Country Representative Interview 13. OTI/Burma Former Deputy Country Representative Interview 14. Kann Let/DAI/Acting Chief of Party Interview 15. Kann Let/DAI Field Advisor-Meiktila 16. Kann Let/DAI Field Advisor-Myikyina 17. Kann Let/DAI/Former Chief of Party Interview 18. Kann Let/DAI/M&E Officer Interview 19-20. Kann Let/DAI/Peace, Program Development Officer (PDO) FGD 21-25. Kann Let/DAI/Peace Program, Grants Manager (GM) FGD 26. Kann Let/DAI/Program Manager Interview 27-29. Kann Let/DAI/Inter Communal Harmony (ICH) Peace Program, Development Officer (PDOs) FGD 30-33. Kann Let/DAI/ Inter Communal Harmony Grants Managers FGD 34. DAI/Program Manager Interview 35. Former OTI/Burma Acting Country Representative Interview 36. Former OTI/Burma Deputy Country Representative Interview 37. Former OTI/Team Lead of Europe and Asia Interview 38. Former OTI Program Manager Interview 41 | Kann Let 2 Final Performance Evaluation USAID.GOV 39. OTI Asia Deputy Team Lead Interview 40. OTI Asia Team Leader Interview 41. OTI Burma/Program Assistant Interview 42. OTI Program Manager Interview 43-45. Grantee Interview 46. Grantee Interview 47. Other Interview 48-49. Grantee Interview 50. Grantee Interview 51-52. Grantee Interview 53. Other Interview 54. Independent Expert Interview 55. Grantee Interview 56. Grantee Interview 57-58. Other Interview 59. Other Interview 60. Other Interview 61. Beneficiary Interview 62. Beneficiary Interview 63-64. Other Interview 65. Grantee Interview 66. Grantee Interview 67. Grantee Interview 68. Grantee Interview 69. Other Interview 70. Grantee Interview 71. Grantee Interview 72-73. Grantee Interview 74. Grantee Interview 75. Other Interview 76. Grantee Interview 77. Grantee Interview 78. Other Interview 79. Other Interview 80. Beneficiary Interview 81. Grantee Interview 82. Grantee Interview 83. Other Interview 84. Grantee Interview 85. Grantee Interview 86-87. Other Interview 88. Grantee Interview USAID.GOV Kann Let 2 Final Performance Evaluation | 42 89. Other Interview 90. Grantee beneficiary Interview 91. Grantee beneficiary Interview 92-94. Grantee beneficiaries FGD 95-97. Grantee beneficiaries FGD 98. Grantee beneficiary Interview 43 | Kann Let 2 Final Performance Evaluation USAID.GOV ANNEX C. DATA COLLECTION TOOLS INTERVIEW TEMPLATE: USG/USAID Organization Code number Name(s)/Position(s) Date: Location: Time: Lead interviewer: Notetaker: Informed Consent and Confidentiality Narrative: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, D.C. area. We are collecting data about the Kann Let 2 program that launched in 2016. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Burma We selected you and other respondents to interview because we understand that you may have perspective on the Kann Let 2 program and/or on relevant subject matter. We expect the duration of this interview to be approximately one hour. We plan to ask you about the peace process in Burma, as well as program activities conducted by OTI’s implementing partner DAI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Burma’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Are you okay participating in this interview? (Yes?/No?) The lead interviewer should edit the template based on what is known about the interviewee, her/his position, and activities with which s/he may be familiar (summarize in “Pre-interview notes”). Choose the listed questions most appropriate for the interviewee(s) to ensure that each evaluation question is addressed. Pre- interview notes: Post-interview analysis or follow-up: For some stakeholders, we may need to explain some or part of the following: • Consistent with the U.S. Government’s Integrated Country Strategy for Myanmar/Burma (2017- 20), the USAID/OTI Burma program supports specific Mission Goals and Objectives: • Mission Goal 2: Myanmar/Burma makes meaningful progress toward peace and national reconciliation and begins to construct a shared national identity. USAID.GOV Kann Let 2 Final Performance Evaluation | 44 • Mission Objective 2.1: Achieve an inclusive national ceasefire and development of an effective political dialogue aimed at achieving a federal union. • Mission Objective 2.2: Reduce religious and communal conflict and build inclusive, tolerant national identity, with particular focus on Rakhine State. EQ1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 1. How were you involved in the Kann Let 2 process of identifying, targeting and programming according to critical impediments? 2. What are the strengths and weaknesses of the Kann Let program design in the transitional Burma environment? 3. What have been the major challenges to the advancement of peace and reform in Burma that Kann Let 2 focused on? 4. Did Kann Let 2 adjust its strategy – e.g., objectives, activities – appropriately when it encountered changes in Burma’s country context – e.g., new constraints or opportunities? If yes, can you provide examples? 5. Were the challenges that Kann Let 2 focused on within its ability to influence? Did this continue to be the case when it adjusted its strategy based on changes in Burma’s country context? EQ2: Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 6. What are the obstacles to inclusivity and preparedness in the peace process? 7. Has Kann Let 2 developed context-appropriate tools for supporting inclusivity in the peace process? If yes, please provide examples. If not, please explain why not. 8. Did Kann Let 2 miss opportunities for advancing inclusivity? 9. Has Kann Let 2 increased the preparedness of government and ethnic representatives for more substantive peace process discussions? If yes, please provide examples. If not, please explain why not. 10. Did Kann Let 2 miss opportunities for advancing preparedness? 11. What has Kann Let achieved vis-à-vis the peace process? 12. What peace process-related approaches or activities were the most difficult to evaluate? 13. What factors have contributed to achieving or not achieving peace-related program level changes sought by Kann Let? 14. What has Kann Let been able to accomplish on peace that other kinds of programs and funding could not? Give examples. 15. What Kann Let activities on peace do you recommend we study carefully, for lessons on effectiveness? 16. What U.S. policies and USAID or OTI management practices help or hinder Kann Let effectiveness vis-à-vis the peace process? 17. What advice about enhancing peace program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ3: To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? 18. What have been the key drivers of intercommunal conflict in Burma? 19. Which stakeholders could potentially build mutually supportive relationships that might prevent ICC? 20. Has Kann Let 2 developed context-appropriate tools for supporting increased capacity, relationships and actions among key stakeholders to prevent ICC? If yes, please provide examples. If not, please explain why not. 21. Did Kann Let 2 miss opportunities for improving capacity, relationships and actions to prevent ICC? 22. Has Kann Let 2 created opportunities for mutually beneficial interaction between communities? If yes, please provide examples. If not, please explain why not. 45 | Kann Let 2 Final Performance Evaluation USAID.GOV 23. Did Kann Let 2 miss opportunities for supporting mutually beneficial interaction between communities? 24. What has Kann Let achieved vis-à-vis prevention of intercommunal conflict and building mutually beneficial interaction? 25. What ICC-related approaches or activities were the most difficult to evaluate? 26. What factors have contributed to achieving or not achieving ICC-related program level changes sought by Kann Let? 27. What has Kann Let been able to accomplish vis-à-vis ICC that other kinds of programs and funding could not? Give examples. 28. What Kann Let ICC activities do you recommend we study carefully, for lessons on effectiveness? 29. What U.S. policies and USAID or OTI management practices help or hinder Kann Let effectiveness vis-à-vis ICC? 30. What advice about enhancing ICC program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ4: To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? 31. What challenges does civil society face in engaging in the democratic transition? 32. How has Kann Let identified and responded to opportunities to enhance the role of civil society in the democratic transition? 33. Has Kann Let 2 supported (or not) civil society’s role in the transition process? If yes, please provide examples. If not, please explain why not. 34. Has Kann Let 2 missed opportunities in which civil society’s role(s) could have been strengthened? Were there any lessons learned? 35. What are the strengths and weaknesses of Kann Let 2 program design with regard to civil society’s role in the transitional environment? 36. What methods do you recommend for strengthening the role of civil society in the democratic transition? What kinds of programming and activities should be undertaken? What should be avoided? EQ5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? 37. Has Kann Let 2 staff’s collaboration with the USAID Mission and U.S. Embassy in Burma over the life of the project contributed to furthering foreign policy goals in Burma? If yes, lease give examples. If not, please explain. 38. How have U.S. policies helped or hindered in the design and implementation of the Kann Let 2 program? Please give examples. USAID.GOV Kann Let 2 Final Performance Evaluation | 46 INTERVIEW TEMPLATE: GRANTEES Organization Code number Name(s)/Position(s) Date: Location: Time: Lead interviewer: Notetaker: Informed Consent and Confidentiality Narrative: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, D.C. area. We are collecting data about the Kann Let 2 program that launched in 2016. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Burma. We selected you and other respondents to interview because we understand that you may have perspective on the Kann Let 2 program and/or on relevant subject matter. We expect the duration of this interview to be approximately one hour. We plan to ask you about the peace process in Burma, as well as program activities conducted by OTI’s implementing partner DAI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Burma’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Are you okay participating in this interview? (Yes/No) The lead interviewer should edit the template based on what is known about the interviewee, her/his position, and activities with which s/he may be familiar (summarize in “Pre-interview notes”). Choose the listed questions most appropriate for the interviewee(s) to ensure that each evaluation question is addressed. Rearrange the order of the questions as appropriate to the stakeholder. Pre- interview notes: Post-interview analysis or follow-up: For some stakeholders, we may need to explain some or part of the following: • Consistent with the U.S. Government’s Integrated Country Strategy for Myanmar/Burma (2017- 20), the USAID/OTI Burma program supports specific Mission Goals and Objectives: • Mission Goal 2: Myanmar/Burma makes meaningful progress toward peace and national reconciliation and begins to construct a shared national identity. • Mission Objective 2.1: Achieve an inclusive national ceasefire and development of an effective political dialogue aimed at achieving a federal union. • Mission Objective 2.2: Reduce religious and communal conflict and build inclusive, tolerant national identity, with particular focus on Rakhine State. 47 | Kann Let 2 Final Performance Evaluation USAID.GOV EQ1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 1. What have been the major challenges to the advancement of peace and reform in Burma that Kann Let 2 focused on? 2. Through support from Kann Let 2, do you think your activities are effectively addressing one or more of these challenges? Which challenge(s)? EQ2: Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 3. What activities did you undertake relevant to the peace process? How did you settle on these activities? 4. How did the Kann Let 2 staff support the development of your peace-related activities? Do you have any feedback on the process you would like to share? 5. What do you think have been the main obstacles to the peace process in Burma? How do you think your activities are contributing to overcoming these obstacles? 6. Have your activities developed tools or methods for supporting inclusivity in the peace process? If yes, please provide examples. If not, please explain why not. 7. Were there other ways your activities could have been more inclusive if you had received additional Kann Let 2 support? If yes, please provide examples. 8. Have your Kann Let 2-supported activities increased the preparedness of government and ethnic representatives for more substantive peace process discussions? If yes, please provide examples. If not, please explain why not. 9. Were there other ways your activities could have helped government and ethnic representatives be more prepared for more substantive peace process discussions if you had received additional Kann Let 2 support? If yes, please provide examples. 10. Were there any activities that you feel weren’t successful in meeting their objectives. If yes, why not? 11. Do you have recommendations for other approaches to supporting inclusivity and better preparedness in the peace process? 12. What factors have contributed to your activities achieving or not achieving peace-related program level changes sought by Kann Let 2? 13. What has Kann Let 2 been able to accomplish on peace that other kinds of programs and funding could not? Give examples. 14. What activities on peace do you recommend we study carefully, for lessons on effectiveness? 15. What advice about enhancing peace program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ3: To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? 16. What activities did you undertake relevant to ICC? How did you settle on these activities? 17. How did the Kann Let 2 staff support the development of your ICC-related activities? Do you have any feedback on the process you would like to share? 18. What do you think are the key drivers of intercommunal conflict in Burma? How do you think your activities are contributing to overcoming these obstacles? 19. Have your activities developed tools or methods for supporting increased capacity, relationships and/or actions among key stakeholders to prevent ICC? If yes, please provide examples. If not, please explain why not. Were there other ways your activities could have improved capacity, relationships and actions to prevent ICC if you had received additional Kann Let 2 support? If yes, please provide examples. 20. Have your activities facilitated opportunities for mutually beneficial interaction between communities? How so? If yes, please provide examples. 21. Were there other ways your activities could have supported mutually beneficial interaction between communities if you had received additional Kann Let 2 support? If yes, please provide examples. USAID.GOV Kann Let 2 Final Performance Evaluation | 48 22. Were there any activities that you feel weren’t successful in meeting their objectives. If yes, why not? 23. Did your activities have any unintended results or consequences, if any? If yes, what caused these? 24. Do you have recommendations for other approaches to preventing ICC? 25. Which stakeholders could potentially build mutually supportive relationships that might prevent ICC? 26. What factors have contributed to your activities achieving or not achieving ICC-related program-level changes sought by Kann Let 2? What has Kann Let 2 been able to accomplish vis-à-vis ICC that other kinds of programs and funding could not? Give examples. 27. What ICC-related activities do you recommend we study carefully, for lessons on effectiveness? 28. What advice about enhancing ICC program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ4: To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? 29. What challenges has your organization faced in engaging in the democratic transition? 30. With Kann Let 2 support, do you think your organization has been able to play a strong role in the democratic transition? If yes, please provide examples. If not, please explain why you think this is. 31. Were there other ways your organization could have played a stronger role in the democratic transition process if you had received additional Kann Let 2 support? If yes, please provide examples. 32. What are the most significant strengths of Kann Let 2 with regard to your organization in this transitional environment? What could Kann Let 2 do better in the future? 33. What recommendations do you have for strengthening the role of your organization further in the democratic transition? What kinds of programming and activities should be undertaken? What should be avoided? 34. How is the activity ‘locally owned’? Is there any evidence of this or examples? EQ5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? Not applicable to grantees. 49 | Kann Let 2 Final Performance Evaluation USAID.GOV INTERVIEW TEMPLATE: OTI/BURMA AND DAI Organization Code number Name(s)/Position(s) Date: Location: Time: Lead interviewer: Notetaker: EQ1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 39. How were you involved in the Kann Let 2 process of identifying, targeting and programming according to critical impediments? 40. What are the strengths and weaknesses of the Kann Let program design in the transitional Burma environment? 41. What have been the major challenges to the advancement of peace and reform in Burma that Kann Let 2 focused on? 42. How did Kann Let 2 adjust its strategy – e.g., objectives, activities –when it encountered changes in Burma’s country context – e.g., new constraints or opportunities? Can you provide examples? 43. Were the challenges that Kann Let 2 focused on within its ability to influence? Did this continue to be the case when it adjusted its strategy based on changes in Burma’s country context? EQ2: Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 44. What are the obstacles to inclusivity and preparedness in the peace process? 45. Has Kann Let 2 developed context-appropriate tools for supporting inclusivity in the peace process? If yes, please provide examples. If not, please explain why not. 46. Did Kann Let 2 miss opportunities for advancing inclusivity? 47. Has Kann Let 2 increased the preparedness of government and ethnic representatives for more substantive peace process discussions? If yes, please provide examples. If not, please explain why not. 48. Did Kann Let 2 miss opportunities for advancing preparedness? 49. What has Kann Let achieved vis-à-vis the peace process? 50. What peace process-related approaches or activities were the most difficult to evaluate? 51. What factors have contributed to achieving or not achieving peace-related program level changes sought by Kann Let? 52. What has Kann Let been able to accomplish on peace that other kinds of programs and funding could not? Give examples. 53. What Kann Let activities on peace do you recommend we study carefully, for lessons on effectiveness? 54. What U.S. policies and USAID or OTI management practices helped or hindered Kann Let effectiveness vis-à-vis the peace process? 55. What advice about enhancing peace program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ3: To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? 56. What have been the key drivers of intercommunal conflict in Burma? USAID.GOV Kann Let 2 Final Performance Evaluation | 50 57. Which, if any, stakeholders could potentially build mutually supportive relationships that might prevent ICC? 58. Has Kann Let 2 developed context-appropriate tools for supporting increased capacity, relationships and actions among key stakeholders to prevent ICC? If yes, please provide examples. If not, please explain why not. 59. Did Kann Let 2 miss opportunities for improving capacity, relationships and actions to prevent ICC? 60. Has Kann Let 2 created opportunities for mutually beneficial interaction between communities? If yes, please provide examples. If not, please explain why not. 61. Did Kann Let 2 miss opportunities for supporting mutually beneficial interaction between communities? 62. What has Kann Let achieved vis-à-vis prevention of intercommunal conflict and building mutually beneficial interaction? 63. What ICC-related approaches or activities were the most difficult to evaluate? 64. What factors have contributed to achieving or not achieving ICC-related program level changes sought by Kann Let? 65. What has Kann Let been able to accomplish vis-à-vis ICC that other kinds of programs and funding could not? Give examples. 66. What Kann Let ICC activities do you recommend we study carefully, for lessons on effectiveness? 67. What U.S. policies and USAID or OTI management practices help or hinder Kann Let effectiveness vis-à-vis ICC? 68. What advice about enhancing ICC program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ4: To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? 69. What challenges does civil society face in engaging in the democratic transition? 70. How has Kann Let identified and responded to opportunities to enhance the role of civil society in the democratic transition? 71. Has Kann Let 2 supported (or not) civil society’s role in the transition process? If yes, please provide examples. If not, please explain why not. 72. Has Kann Let 2 missed opportunities in which civil society’s role(s) could have been strengthened? Were there any lessons learned? 73. What are the strengths and weaknesses of Kann Let 2 program design with regard to civil society’s role in the transitional environment? 74. What methods do you recommend for strengthening the role of civil society in the democratic transition? What kinds of programming and activities should be undertaken? What should be avoided? EQ5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? 75. Has Kann Let 2 staff’s collaboration with the USAID Mission and U.S. Embassy in Burma over the life of the second task order contributed to furthering foreign policy goals in Burma? If yes, please give examples. If not, please explain. 76. How have U.S. policies helped or hindered in the design and implementation of the Kann Let 2 program? Please give examples. 51 | Kann Let 2 Final Performance Evaluation USAID.GOV INTERVIEW TEMPLATE: “OTHERS” Organization Code number Name(s)/Position(s) Date: Location: Time: Lead interviewer: Notetaker: EQ1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 1. Were you involved in the Kann Let 2 process of identifying, targeting and programming according to critical impediments? If so how? 2. What are the strengths and weaknesses of the Kann Let program design in the transitional Burma environment? 3. What have been the major challenges to the advancement of peace and reform in Burma that Kann Let 2 focused on? EQ2: Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 4. What are the obstacles to inclusivity and preparedness in the peace process? 5. Has Kann Let 2 developed context-appropriate tools for supporting inclusivity in the peace process? If yes, please provide examples. If not, please explain why not. 6. Did Kann Let 2 miss opportunities for advancing inclusivity? 7. What has Kann Let achieved vis-à-vis the peace process? 8. What has Kann Let been able to accomplish on peace that other kinds of programs and funding could not? Give examples. 9. What advice about enhancing peace program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ3: To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? 10. What have been the key drivers of intercommunal conflict in Burma? 11. Which stakeholders could potentially build mutually supportive relationships that might prevent ICC? 12. Has Kann Let 2 developed context-appropriate tools for supporting increased capacity, relationships and actions among key stakeholders to prevent ICC? If yes, please provide examples. If not, please explain why not. 13. Did Kann Let 2 miss opportunities for improving capacity, relationships and actions to prevent ICC? 14. Has Kann Let 2 created opportunities for mutually beneficial interaction between communities? If yes, please provide examples. If not, please explain why not. 15. What has Kann Let achieved vis-à-vis prevention of intercommunal conflict and building mutually beneficial interaction? 16. What has Kann Let been able to accomplish vis-à-vis ICC that other kinds of programs and funding could not? Give examples. 17. What advice about enhancing ICC program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ4: To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? USAID.GOV Kann Let 2 Final Performance Evaluation | 52 18. What challenges does civil society face in engaging in the democratic transition? 19. How has Kann Let identified and responded to opportunities to enhance the role of civil society in the democratic transition? 20. What are the strengths and weaknesses of Kann Let 2 program design with regard to civil society’s role in the transitional environment? 21. What methods do you recommend for strengthening the role of civil society in the democratic transition? What kinds of programming and activities should be undertaken? What should be avoided? EQ5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? NONE 53 | Kann Let 2 Final Performance Evaluation USAID.GOV FOCUS GROUP DISCUSSION PROTOCOL Focus Group Discussion Protocol FGD Place and Date: Moderator: Sex: ⃝Female (Number: _) ⃝Male (Number: ___ ) Location: ⃝Yangon ⃝Hpaan ⃝Myitkyina ⃝Mandalay Stakeholder Group: ⃝Grantees ⃝DAI team Welcome, Informed Consent, and Confidentiality: Good morning, and welcome to our session. My name is X, and I work for Social Impact, which is an independent research company based in the Washington, D.C. area. We are collecting data about the Kann Let 2 program. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. Thank you for taking the time to join us to talk about activities conducted through the Kann Let 2 program, in which you all have participated as grantees. You may or may not know each other; this will be an opportunity to know more people involved in [supporting peacebuilding, intercommunal harmony or civil society]. You were invited because we are interested in hearing your perspective about the Kann Let 2 program (past, present, and future). All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. There are no wrong answers but rather differing points of view. Please feel free to share your point of view even if it differs from what others have said. Keep in mind that we're just as interested in negative comments as positive comments, and at times the negative comments are the most helpful. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering or withdraw from the discussion at any point. We would appreciate if you please turn off your cellphones. [Questions chosen according to relevance for group assembled.] EQ 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? 1. How were you involved in the Kann Let 2 process of identifying, targeting and programming according to critical impediments? 2. What are the strengths and weaknesses of the Kann Let program design in the transitional Burma environment? 3. What have been the major challenges to the advancement of peace and reform in Burma that Kann Let 2 focused on? 4. How did Kann Let 2 adjust its strategy – e.g., objectives, activities –when it encountered changes in Burma’s country context – e.g., new constraints or opportunities? Can you provide examples? 5. Were the challenges that Kann Let 2 focused on within its ability to influence? Did this continue to be the case when it adjusted its strategy based on changes in Burma’s country context? USAID.GOV Kann Let 2 Final Performance Evaluation | 54 EQ2: Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? 6. What are the obstacles to inclusivity and preparedness in the peace process? 7. Has Kann Let 2 developed context-appropriate tools for supporting inclusivity in the peace process? If yes, please provide examples. If not, please explain why not. 8. Did Kann Let 2 miss opportunities for advancing inclusivity? 9. Has Kann Let 2 increased the preparedness of government and ethnic representatives for more substantive peace process discussions? If yes, please provide examples. If not, please explain why not. 10. Did Kann Let 2 miss opportunities for advancing preparedness? 11. What has Kann Let achieved vis-à-vis the peace process? 12. What peace process-related approaches or activities were the most difficult to evaluate? 13. What factors have contributed to achieving or not achieving peace-related program level changes sought by Kann Let? 14. What has Kann Let been able to accomplish on peace that other kinds of programs and funding could not? Give examples. 15. What Kann Let activities on peace do you recommend we study carefully, for lessons on effectiveness? 16. What U.S. policies and USAID or OTI management practices helped or hindered Kann Let effectiveness vis-à-vis the peace process? 17. What advice about enhancing peace program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? EQ3: To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? 18. What have been the key drivers of intercommunal conflict in Burma? 19. Which, if any, stakeholders could potentially build mutually supportive relationships that might prevent ICC? 20. Has Kann Let 2 developed context-appropriate tools for supporting increased capacity, relationships and actions among key stakeholders to prevent ICC? If yes, please provide examples. If not, please explain why not. 21. Did Kann Let 2 miss opportunities for improving capacity, relationships and actions to prevent ICC? 22. Has Kann Let 2 created opportunities for mutually beneficial interaction between communities? If yes, please provide examples. If not, please explain why not. 23. Did Kann Let 2 miss opportunities for supporting mutually beneficial interaction between communities? 24. What has Kann Let achieved vis-à-vis prevention of intercommunal conflict and building mutually beneficial interaction? 25. What ICC-related approaches or activities were the most difficult to evaluate? 26. What factors have contributed to achieving or not achieving ICC-related program level changes sought by Kann Let? 27. What has Kann Let been able to accomplish vis-à-vis ICC that other kinds of programs and funding could not? Give examples. 28. What Kann Let ICC activities do you recommend we study carefully, for lessons on effectiveness? 29. What U.S. policies and USAID or OTI management practices help or hinder Kann Let effectiveness vis-à-vis ICC? 30. What advice about enhancing ICC program effectiveness would you offer to OTI, DAI, the U.S. mission, and policymakers in Washington, DC? 55 | Kann Let 2 Final Performance Evaluation USAID.GOV EQ4: To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? 31. What challenges does civil society face in engaging in the democratic transition? 32. How has Kann Let identified and responded to opportunities to enhance the role of civil society in the democratic transition? 33. Has Kann Let 2 supported (or not) civil society’s role in the transition process? If yes, please provide examples. If not, please explain why not. 34. Has Kann Let 2 missed opportunities in which civil society’s role(s) could have been strengthened? Were there any lessons learned? 35. What are the strengths and weaknesses of Kann Let 2 program design with regard to civil society’s role in the transitional environment? 36. What methods do you recommend for strengthening the role of civil society in the democratic transition? What kinds of programming and activities should be undertaken? What should be avoided? EQ5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? 37. Has Kann Let 2 staff’s collaboration with the USAID Mission and U.S. Embassy in Burma over the life of the second task order contributed to furthering foreign policy goals in Burma? If yes, please give examples. If not, please explain. 38. How have U.S. policies helped or hindered in the design and implementation of the Kann Let 2 program? Please give examples. Wrap-up 39. Suppose that you were in charge and could improve the US assistance programs to the peace process, intercommunal harmony and civil society. What would you do differently? 40. Is there anything that anyone would like to share on the topics covered in this meeting? Are we missing something important? USAID.GOV Kann Let 2 Final Performance Evaluation | 56 DIRECT OBSERVATION PROTOCOL Observation Place and Date: Observation Site: Observer: Number of Participants: Purpose: The purpose of this observation is to (1) verify whether the training activity contributes to the objectives of Kann Let 2, as well as (2) verify the community’s involvement in the activity and the interactions of different stakeholders. Instructions: Please consider each item carefully before assigning a score. Unsatisfactory Needs improvement Acceptable Satisfactory Very satisfactory Not Applicable 1 2 3 4 5 n/a _____ 1. Relevance of the topic presented (aligned with OTI program objectives, aligned with local needs) _____ 2. Number of people attending the training _____ 3. Representation of diverse stakeholders attending _____ 4. Presence and participation of women _____ 5. Presence and participation of diverse range of persons _____ 6. Participant engagement in the activity _____ 7. Development of follow-up activities _____ 8. Overall grade 9. What are your first perceptions of the process observed? 10. What specific suggestions would you make concerning this activity? 11. Additional notes/comments: 57 | Kann Let 2 Final Performance Evaluation USAID.GOV ANNEX D. DOCUMENT REVIEW SOURCES PROGRAM/SOURCE DOCUMENT TITLE SUPPLEMENTARY DOCUMENTS KANN LET 2 (MARCH 2016 - MARCH 2019) USAID/OTI Kann Let Final Evaluation Scope of Work (09.12.2018) OTI Burma, General OTI Burma Program Document OTI Burma, Kann Let Kann Let 2 Learning Agenda OTI Burma, Kann Let Kann Let 2 Activities (sortable) - OTI database report as of Sept. 11, 2018 USAID-OTI Burma Programming in Rakhine State (6.19.18)] Overview of Burma’s Peace Process (9.13.18) OTI Burma, Program Performance Management Jan. 2017 Rolling Assessment (RA) Note to File Addendum to Jan 2017 RA Note to File OTI Burma, Program Performance Management May 2017 Program Performance Review Report (PPR) Burma SRS Report – June 2017 Updated PLC Matrix - July 24, 2017 OTI Burma, Program Performance Management Jan. 2018 RA Note to File Updated Program Level Changes (PLC) Matrix OTI Burma, Cluster Evaluations Women Peace and Security Cluster Evaluation ICH and ICH/Rakhine Due: 5-9 November, 2018 (In process) Social Media/Hatespeech Cluster Evaluation Due: 23-26 October, 2018 (In process) KANN LET 1 (SEPTEMBER 2012 - AUGUST 2016) Evaluation Report Kann Let 1 Mid-Term Evaluation Report USAID.GOV Kann Let 2 Final Performance Evaluation | 58 ANNEX E: ROLES OF CSO IN KANN LET 2-SUPPORTED TRANSITION STRATEGIC CSO PARTNERS CRITICAL TRANSITION AREAS APPROACH IMPACT ON TRANSITION Ethnic CSOs/ non￾ethnic CSOs with strong knowledge of the peace process Increased Awareness of Peace Process Among the Bamar Majority Exhibitions, Workshops, Literature Talks, Peace related Information Distribution and Panel Discussions on peace process, etc. Without Bamar majority’s understanding of peace process, it is hard to achieve a federalist state Ethnic/Think Tank type CSOs with strong technical expertise on peace￾related issues Increased preparedness of government and ethnic representatives on more substantive peace process discussions Technical Papers, Workshops, and related research-briefs on selected thematic areas Without peace, democratic transition stalled Local CSOs and networks of CSOs on conflict￾resolutions Increased capacity, relationships, and actions among key stakeholders to prevent ICC/ICH Network Building, Workshops, Capacity Building, Early Warming Systems, related research, and Awareness Raising including anti-hate speech and enriched social media skills etc. Without addressing ICH, transition hard to move forward Ethnic/Think Tank type CSOs with strong technical expertise on peace￾related issues Increased diverse, civil society input to inform peace process discussions Workshops, Consultations, Knowledge, Expertise & Experience Sharing including experts abroad Without diverse inputs on peace process, peace hard to achieve Local CSOs and networks of CSOs on conflict￾resolutions Increased Opportunity for Key Influencers to Advocate for Tolerance, and Promote Dialogue and Mitigate Conflict Capacity Building, Technical Skills enrichment, Workshops, Coordination Actions among different religions leadership Without tolerance and promoting dialogue, hard to achieve peaceful coexistence, then hard to manage transition Media, Interested CSOs, Think Tank Type CSOs Increased Communication Channels About Reform Processes Public Debates, Participatory policy making workshops, Research/Technical papers, Public Forums etc. Without transparency, accountability would be lacking, then trust in transition mired as a result 59 | Kann Let 2 Final Performance Evaluation USAID.GOV ANNEX F. EVALUATION SCOPE OF WORK PDQIII Task Order #10 Activity #21: Burma Final Evaluation SCOPE OF WORK PERIOD OF PERFORMANCE: September 7, 2018 to February 4, 2019 BACKGROUND Burma faces complex, simultaneous, and multi-dimensional transitions from authoritarian rule to democracy, from armed conflict to peace, and from a centrally-controlled, planned economy to market￾led economic policies. The country is also stepping out of isolation to integrate into the regional and broader international community. With a few notable exceptions, the reform and peace processes have been directive and top down with limited public participation. Although general elections in November 2015 and the sweeping victory for Aung San Suu Kyi’s National League for Democracy (NLD) marked a turning point in Burma’s democratic transition, military power remains steadfast with control over key ministries and veto power in parliament. The NLD-led government has demonstrated limited willingness to have international actors or civil society engage on the democratic reform process. Two additional Ethnic Armed Groups (EAGs) have since signed the Nationwide Ceasefire Agreement with the government, joining the eight EAGs that signed in October 2015. However, there have been few obvious gains for the signatories as military interference has blocked public dialogues and consultations, and the peace process has been plagued with delays, military violations of the ceasefire agreement, and intensified conflict in Kachin and northern Shan. The complex human rights and humanitarian situation in Rakhine State and intercommunal conflict across the country threaten Burma’s transition. Ongoing intercommunal tension and violence in Rakhine culminated most recently in two events in October 2016 and in much larger scale in August 2017, when Muslim militant attacks on Border Guard Force posts in northern Rakhine State allegedly prompted disproportionate counter military offensives and systematic clearance operations against the Rohingya that have led to the displacement of more than 700,000 since August alone and a total of more than 1.1 million Rohingyas refugees into Bangladesh. Declared “ethnic cleansing” by the U.S. Secretary of State and “bearing the hallmarks of genocide” by the UN, the Rohingya crisis has continued through an intractable process with international community demands for accountability and commitment from the Burma government on the safe, voluntary, and dignified return of the Rohingya population from Bangladesh. In September 2012, USAID’s Office of Transition Initiatives (USAID/OTI) launched the Kann Let program in Burma to increase participation and inclusion in reform and peace processes as well as to address critical impediments to the transition, primarily the situation in Rakhine State, intercommunal conflict, and the formal peace process. DAI implemented this task order from September 2012 to August 2016. In March 2016, OTI issued a new task order to DAI, Kann Let 2, to continue deepening and sustaining reforms and fostering legitimate processes for pursuing peace. USAID/OTI continued to refine its program objectives in 2017, reiterating support of the peace process and efforts to address the drivers of intercommunal conflict, while also promoting intercommunal harmony and support for fundamental freedoms. Kann Let 2 is scheduled to end on March 23, 2019. USAID.GOV Kann Let 2 Final Performance Evaluation | 60 Consistent with the U.S. Government’s (USG’s) Integrated Country Strategy for Myanmar (2017-20) approved in February 2017, the USAID/OTI Burma program through Kann Let 2 supports specific Mission Goals and Objectives: ● Mission Goal 2: Myanmar makes meaningful progress toward peace and national reconciliation and begins to construct a shared national identity ○ Mission Objective 2.1: Achieve an inclusive national ceasefire and development of an effective political dialogue aimed at achieving a federal union; and ○ Mission Objective 2.2: Reduce religious and communal conflict and build inclusive, tolerant national identity, with particular focus on Rakhine State. USAID/OTI has sought to deepen and sustain reforms and foster legitimate, inclusive peacebuilding by focusing on the following objectives and program level changes (as of January 2018): ● Objective 1: To enhance the ability of key stakeholders to engage in the peace process; o Program level change 1: Increased diverse, civil society input to inform peace process discussions o Program level change 2: Increased preparedness of government and ethnic representatives for more substantive peace process discussions o Program level change 3: More inclusive consultations to inform development of and build momentum for “Interim Arrangements” as mentioned in the National Ceasefire Agreement ● Objective 2: To strengthen engagement on areas affecting fundamental freedoms, including freedom of religion, speech and expression; and o Program level change 1: Increased opportunity and capacity for civil society and government engagement on critical impediments to fundamental freedoms (opportunistic) ● Objective 3: To reduce the influence of drivers of intercommunal conflict and strengthen capacity to promote intercommunal harmony. o Program level change 1: Increased capacity, relationships, and actions among key stakeholders to prevent intercommunal conflict o Program level change 2: Increased opportunity for mutually beneficial interaction between communities – Rakhine State (opportunistic) As part of the program objective refinement process, which included various internal and external context analysis, USAID/OTI conducted strategic review exercises during the following Program Performance Management sessions: ● Rolling Assessment, January 2017 ● Program Performance Review, April 2017 ● Strategic Review Session, June 2017 ● Rolling Assessment, January 2018 (current) Recent evaluations conducted on the USAID/OTI Burma Kann Let program, mostly focused on Kann Let 2 include: ● Intercommunal Conflict Cluster Evaluation - August 2018 ongoing: This cluster is a subset of Objective 3 ● Rakhine State Cluster Evaluation - August 2018 ongoing: This cluster is a subset of Objective 3 ● Media Cluster Evaluation - September 2018 planned: This cluster is a subset of Objective 3 ● Women, Peace, and Security Cluster Evaluation - July 2018 finalized: This cluster is a subset of Objective 1 ● Mid-term Evaluation (Kann Let 1) - July 2016 finalized 61 | Kann Let 2 Final Performance Evaluation USAID.GOV PURPOSE The purpose of this activity is to conduct an independent performance evaluation of USAID/OTI’s second Kann Let program in Burma from March 24, 2016 to March 23, 2019. The findings will evaluate the strategic approach, relevance, and impact of Kann Let 2 efforts to deepen and sustain the ongoing reform process in Burma and will provide a product that concisely explains achievements and lessons learned. The final product will be used to inform future USG assistance that will take over the OTI portfolio, primarily the two new USAID/Burma Mission programs of which one starting will start in the Fall of 2018 and has the following objectives: ● Objective 1: Improve understanding of conflict dynamics, early recovery strategies, and approaches to working in communities affected by conflict ● Objective 2: Strengthen community participation and local ownership in the provision of essential services, community development initiatives, and peace and reconciliation efforts ● Objective 3: Reduce community vulnerabilities, enhance resilience to protracted crises and future shocks, support early recovery, and promote durable solutions for communities affected by conflict and displacement ● Objective 4: Enhance trust and reconciliation between communities and between communities and local governance actors The report will also be shared with the U.S. Embassy in Rangoon, local stakeholders, and other interested groups. EVALUATION QUESTIONS The report seeks to answer the following evaluation questions: 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma? To what extent were those within its ability to impact or influence? ● In answering this question, the evaluators should consider whether the program appropriately adapted its strategy as the context evolved and seek to determine whether these same areas are likely to merit similar emphasis future USG programming in Burma. 2. Has the Kann Let program employed appropriate tools and approaches to support the peace process in Burma? ● In answering this question, the evaluators should consider which tools and approaches have been the most and least effective at facilitating engagement among beneficiaries; whether different or additional approaches should be taken in future USG programming; and if the program has met the expectations of key actors in the peace process. 3. To what extent has the program been effective at mitigating drivers of intercommunal conflict in Burma? ● In answering this question, the evaluators should consider which approaches have been most effective at changing the attitudes and behaviors of target groups; whether there are lessons or findings from other fields (e.g., human psychology, media literacy) that should be incorporated in future programming or interventions; and, whether there are lessons for other programs seeking to promote tolerance or counter speech that is a catalyst for violence. 4. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing democratic transition process? ● In answering this question, the evaluators should consider whether the program’s support was timely, targeted and critical, identify any missed opportunities, and offer recommendations for ways that the USG can be more proactive as the reform process continues. USAID.GOV Kann Let 2 Final Performance Evaluation | 62 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? ● In answering this question, the evaluators should seek to identify lessons that can be learned from the close collaboration between OTI and other parts of the U.S. Mission to Burma. METHODOLOGY This evaluation will be non-experimental and largely qualitative in nature, but mixed methods may be used as appropriate. In answering the evaluation questions, the contractor shall utilize data that is disaggregated and analyzed by sex, whenever such data is available. Methodological specifics will be agreed upon among the evaluators, OTI/Burma, and OTI/Washington, and the evaluators are encouraged to suggest creative approaches. OTI’s activity database is a rich source of information on individual projects and should be utilized by the evaluation team. Possible methods for the evaluation include: ● Documentation review, e.g., cluster evaluations, quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments, and Program Performance Reviews (PPRs), perception surveys and other existing data, and OTI/Burma’s activity database; ● Facilitated workshop with key program staff to reflect on program implementation, challenges and successes; ● Field visits to the implementation areas; ● Interviews with key program stakeholders, including U.S. Embassy and USAID staff, community leaders, and beneficiaries; ● Focus group discussions and interviews with beneficiaries, grantees, and others; and ● Direct observation. TEAM COMPOSITION The evaluation team should be comprised per the specifications described below. The positions to be filled by Social Impact require USAID/OTI concurrence of the proposed candidates. In addition to three evaluators and one local translator engaged by Social Impact, OTI intends to field a staff member to participate in the evaluation as a fifth team member. Two senior evaluation specialists with research and/or evaluation experience in humanitarian assistance and political transition programming. At least one of the specialists should have extensive experience related to and a deep understanding of Burma’s political system, reform process, and modern history. One evaluation specialist shall be designated the evaluation team leader and will be responsible for liaising with USAID/OTI, the preparation of all deliverables, and leading briefings with relevant stakeholders. One mid-level evaluation specialist with research and/or evaluation experience in complex political and/or social change. This evaluation specialist should be a host country national and have local research and/or evaluation experience. Knowledge of OTI-type programming is preferred. The mid-level evaluation specialist will support the team leader and participate in the field review, interviews, the draft and final evaluation reports, and debriefs in the field only. The team will require the services of a separate local translator with Myanmar and ethnic language skills. OTI intends to identify a staff member to provide periodic support to the evaluation team. The staff member identified for this role can help the Evaluation Team better understand, and contextualize data in, the OTI operating model. Additionally, OTI intends to provide the team with initial support in navigating the OTI database to access critical background documents. TIMELINE 63 | Kann Let 2 Final Performance Evaluation USAID.GOV Phase Dates* Task Phase 1: Inception Phase Week of September 10 (timing dependent on team availability) Kick-off call w/ OTI team to refine evaluation questions, initial methodology, travel and logistics, etc. Database training, if deemed necessary, should be scheduled in next 1-2 weeks to inform design and sampling. September 3 - October 8 Reading on Burma context and Kann Let 2. Key materials to be provided by OTI. Orientation and key Washington-based interviews (may be done remotely) Due Friday, October 12 Draft Evaluation Inception Report and draft data collection instruments OTI to reply with comments by Friday October 19 (or sooner, if possible) Week of October 22 Revisions to the Inception Report/methodology. Revision process should include follow-up meeting/call with OTI team to discuss feedback to the Inception Report, refine methodology, travel and logistics, etc. Phase 2: Field Work Saturday, October 27-28 Travel to Rangoon, Burma (if not Rangoon-based) October 29 - November 14 Preliminary meetings with OTI and DAI, Field-based interviews and analysis, including site visits to locations outside Rangoon for interviews with non-Rangoon based grantees and beneficiaries, November 15-16 Initial analysis and briefing preparations; initial debrief on findings to OTI Senior Management Team, briefs staff and U.S. Mission as appropriate Friday, November 16 Travel, if applicable Phase 3: Reporting November 26 - December 21 Analysis and Reporting writing; submit first draft to OTI by COB Friday, December 21, 2018 December 24 - January 8 Receive comments from OTI by COB Tuesday, January 8, 2018 January 9 – 22 Revisions to the first draft. Submit second draft by COB Tuesday, January 22, 2018 Debriefs in Burma with OTI, DAI, and USG as determined by OTI SMT (end of week into next) January 23 – 28 Receive comments from OTI by COB Monday, January 28, 2018 USAID.GOV Kann Let 2 Final Performance Evaluation | 64 January 29 - February 4 Revisions to the second draft. Submit final report by COB Monday, February 4, 2018 Week of February 11 Debriefs in Washington, D.C. with OTI, DAI, and USG interagency as determined by OTI All dates listed above are illustrative and subject to change. Social Impact should propose an updated timeline, taking into account actual international and local travel dates and division of roles and responsibilities/LOE of each team member. A six-day work week is approved during the fieldwork for this activity. TASKS AND DELIVERABLES The evaluation team, under the direction of the team leader, is responsible for completing the following Tasks and submitting the following Deliverables: TASKS ● Refined or clarified evaluation questions (in collaboration with USAID/OTI); ● Materials and written summary of out-briefing to USAID/OTI and DAI before departing Burma; ● Materials for final debriefs (handout or presentation) and participation in debrief(s) with USAID/OTI, DAI, USG interagency, and other Washington stakeholders as determined by OTI; and DELIVERABLES ● Evaluation Inception Report, to include the following: ○ Summary of proposed data collection methods and analytical approaches; ○ Draft data collection tools; ○ Management framework, including divisions of roles and responsibilities among the Evaluation Team members and points of contact for liaising with OTI, and DAI; ○ Updated timeline with applicable LOE; ○ Initial interviewee list (to be developed in collaboration with USAID/OTI and DAI); and ○ Utilization plan for evaluation findings, based on initial conversations with OTI Washington and OTI Burma staff; ● Draft evaluation report for comments (content and structure), due December 21, 2018; and ● Final evaluation report, due February 4, 2019. Final report will be limited to 25-30 pages total, excluding annexes and the executive summary. All deliverables will require USAID/OTI concurrence prior to their finalization. The final evaluation report will be posted on USAID’s Development Experience Clearinghouse (DEC) website by Social Impact. Social Impact will be responsible for the following logistical matters with respect to the Evaluation Team members hired through this activity: ● Social Impact is responsible for all scheduling, arranging meeting locations, and confirming times/dates/locations with the team and interviewees and grantees. OTI and DAI will provide references, contact information, and can assist with scheduling as needed. While the evaluators are in country, OTI’s partner, DAI, will make available a conference room in its office for the evaluators to use in Rangoon. However, meetings in hotel lobbies, bars, and gardens are sometimes preferable and convenient. USAID/OTI will facilitate access to the U.S. Embassy for meetings with other USAID and Embassy officials and arrange conference rooms in the Embassy. 65 | Kann Let 2 Final Performance Evaluation USAID.GOV ● Social Impact shall arrange and purchase all international and U.S. travel for the Evaluation Team. TOCOR concurrence is required for all international travel not included in the original activity budget; ● Due to often complex banking and wire transfer of funds in Burma, Social Impact shall arrange cash advances to the Evaluation Team members if requested. ● Social Impact shall provide per diem (lodging and M&IE) for the Evaluation Team when traveling away from their home of record; ● Social Impact shall fund in-country air travel and ground transportation; ● Social Impact and the Evaluation Team will work with OTI and DAI to arrange interviews in Washington, D.C. and in Burma; ● Social Impact will obtain visas;50 and ● USAID/OTI will provide Social Impact with a list of Embassy-cleared hotels and transportation services. REPORT STRUCTURE The evaluation report (25 to 30 pages excluding annexes and the executive summary) should adhere to USAID guidelines and be structured as follows: ● Cover Page with photo ● List of Acronyms ● Table of Contents, which identifies page numbers for the major content areas of the report. ● Executive Summary (3 to 5 pages): should be a clear and concise stand-alone document that gives readers the essential content of the evaluation report, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. Thus, the Executive Summary should include: maximum of two paragraphs describing the program; summary of objectives and intended outcomes; conclusions including those related to meaningful under- or over-achievement; and evidence-based recommendations. ● Methodology: Describe the data collection methods used including strengths and weaknesses, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken. This section should also address constraints and limitations of the evaluation process and rigor, including what can and cannot be concluded from the evaluation. All actual or potential conflicts of interests among members of the Evaluation Team should be noted in this section. In accordance with ADS 203, the report should also state why a performance evaluation was conducted in lieu of an impact evaluation. ● Conclusions and Findings: The primary conclusions should be presented clearly and succinctly, followed by the relevant data and results. ● Synthesis, Recommendations, and Lessons Learned: This is space for the Evaluation Team to make concrete recommendations for project improvements/changes, pull out organizational lessons learned (both positive and negative), and generally comment on data and results. The Evaluation Team, in consultation with OTI, should decide whether this remains a section of its own, or whether recommendations follow specific conclusions and findings. ● Annexes: List of stakeholders interviewed with number and type of interactions; interview data on evaluation questions, the scope of work; qualitative protocols developed and used; any data sets can be provided in electronic format; other special documentation needed. 50 DAI is able to assist in arranging business visas for the evaluation team, if required. USAID.GOV Kann Let 2 Final Performance Evaluation | 66 POINT OF CONTACT The OTI/Burma Program Manager (Contact: dcha.otiburmadcteammaillist@usaid.gov) in Washington will serve as the point of contact for overall coordination of the Washington meetings. There will be some occasions where s/he arranges the meeting and location and others where s/he may provide contact information for the consultants to arrange their own logistics. While in Burma, the OTI team in Rangoon will identify a point of contact for in-country logistics and scheduling. This individual will be the key liaison coordinating with and communicating between OTI, DAI, and the Evaluation Team members. The TOCOR will remain the point of contact for all technical direction and requests requiring formal concurrence. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, D.C. 20523