MIDTERM EVALUATION USAID/Office of Transition Initiatives’ Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 July 7, 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mary Callahan and Aung Tun of Social Impact. Midterm Evaluation of the USAID/Office of Transition Initiatives’ Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 July 7, 2016 PDQIII Task Order #10: Q011OAA1500012 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 ii TABLE OF CONTENTS Acronyms............................................................................................................................................................................ iii Executive Summary ........................................................................................................................................................... v Evaluation Purpose and Evaluation Questions....................................................................................................... v Project Background ...................................................................................................................................................... v Evaluation Methods and Limitations........................................................................................................................ vi Findings and Conclusions......................................................................................................................................... viii Recommendations....................................................................................................................................................... ix Project Background........................................................................................................................................................... 1 Evaluation Purpose & Questions.................................................................................................................................... 4 Evaluation Methods & Limitations................................................................................................................................. 6 Findings and Conclusions................................................................................................................................................. 9 Question 1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence?.........................................9 Question 2: Has the Kann Let program employed the right tools and approaches to support the peace process in Burma?...........................................................................................................................................15 Question 3: To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma?...........................................................................................................................23 Question 4: Did Kann Let successfully support civil society to engage in and influence the reform process?.........................................................................................................................................................................28 Question 5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself?............................................................................................................................................35 Recommendations...........................................................................................................................................................38 Question 1: Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence?.......................................38 Question 2: Has the Kann Let program employed the right tools and approaches to support the peace process in Burma?...........................................................................................................................................39 Question 3: To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma?...........................................................................................................................40 Question 4: Did Kann Let successfully support civil society to engage in and influence the reform process?.........................................................................................................................................................................42 Question 5: How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself?............................................................................................................................................43 Annexes .............................................................................................................................................................................44 Annex I: Evaluation Scope of Work .......................................................................................................................45 Annex II: Evaluation Methods and Limitations.....................................................................................................51 Annex III: Data Collection Instruments.................................................................................................................52 Annex IV: Sources of Information...........................................................................................................................55 Annex V: Disclosure of Any Conflicts of Interest...............................................................................................58 Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 iii ACRONYMS ADS Automated Directives System BTI Burma Transition Initiative CSO civil society organization (implies Burmese organization) DAI Development Alternatives, Inc. DEC Development Experience Clearinghouse DKBA Democratic Karen Buddhist Army DP development partner DVB Democratic Voice of Burma EAG ethnic armed group EAO ethnic armed organization EQ evaluation question GoB Government of (Union of) Burma ICC intercommunal conflict INGO international non-governmental organization KNU Karen National Union LOE level of effort Ma Ba Tha Patriotic Association of Myanmar M&E monitoring and evaluation MIDO Myanmar ICT for Development Organization MPC Myanmar Peace Center MTE Midterm Evaluation NCA nationwide ceasefire agreement NGO non-governmental organization NLD National League for Democracy NPT Nay Pyi Taw NSAG non-state armed group OTI/Burma Office of Transition Initiatives, Burma PHC Population and Housing Census (2014) PO program objective PPR Program Performance Review PWD persons with disabilities RA Rolling Assessment RFTOP Request for Task Order Proposals SI Social Impact, Inc. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 iv SOW Scope of Work SM Scoping Mission SMT Senior Management Team (of Kann Let) SRS Strategy Review Session TOCOR Task Order Contracting Officer’s Representative USAID U.S. Agency for International Development USG U.S. Government Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 v EXECUTIVE SUMMARY The Social Impact (SI) Team is pleased to present this midterm evaluation of the U.S. Agency for International Development, Office of Transition Initiatives’ Burma (USAID/OTI Burma) Kann Let program. The purpose of this assignment is to conduct an independent midterm performance evaluation of USAID/OTI’s Kann Let program in Burma to date. This report concisely describes achievements and lessons learned. At the direction of USAID/OTI, the evaluation’s main audience is USAID/OTI staff in Burma and the Kann Let program as it enters the planning phase for its second task order. It is also intended to be used to inform future U.S. Government (USG) assistance and OTI programming in Burma and will be shared with the USAID/Burma Mission, the U.S. Embassy in Yangon, local stakeholders, and other interested groups as determined by USAID/OTI. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this activity was to conduct an independent performance evaluation of USAID/OTI’s Kann Let program in Burma to date and to offer guidance for future activities. On January 5, 2016, USAID/OTI issued a Request for Task Order Proposals (RFTOP) under the SWIFT IV indefinite quantity contract to procure a follow-on task order to continue programming in Burma for two or three additional years. The evaluation reviewed the strategic approach, relevance, and impact of Kann Let’s efforts to deepen and sustain the ongoing reform process in Burma. It has resulted in this product, intended to concisely explain achievements and lessons learned for use by OTI and its implementing partner in the second phase of the program. The specific evaluation questions were as follows: 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence? 2. Has the Kann Let program employed the right tools and approaches to support the peace process in Burma? 3. To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma? 4. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing reform process? 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? PROJECT BACKGROUND The U.S. Government recognizes Burma’s recent reform efforts as the most significant opportunity in several decades to engage with the people and government of Burma in the pursuit of democracy, peace, and human rights. In support of the broader U.S.-Burma Partnership for Democracy, Peace and Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 vi Prosperity,1 USAID/OTI has sought to deepen and sustain the reform process and foster legitimate, inclusive peacebuilding processes. Burma faces complex, simultaneous, and multi-dimensional transitions: from authoritarian rule to democracy; from armed conflict to peace; and from a centrally managed, planned economy to market-led economic policies.2 Important steps have been taken, such as signing bilateral ceasefire agreements and a partial nationwide ceasefire agreement (NCA), easing press and Internet censorship, and staging a credible election for a smooth transfer of power to a democratic opposition party that has waited 25 years to gain power. Nonetheless, reform and peace policies, which have been directive and top-down with limited public participation, remain fragile. In alignment with the USG’s Principled Engagement policy and the Integrated Country Strategy approved in January 2014, USAID/OTI is working to achieve the goal of deepening and sustaining the reform process and fostering legitimate, inclusive peacebuilding processes via the Kann Let (“Offering Hand”) program implemented by Development Alternatives, Inc. (DAI).3 Kann Let staff have worked with civil society, communities, ethnic leaders, and the Government of Burma to identify, develop, and implement more effective policies and laws, communicate priorities and concerns, support peacebuilding processes, and create the conditions necessary for Burma to become a truly inclusive, multi-ethnic union. In March 2015, Kann Let refined its program objectives, effectively creating new standalone program objectives for the four most significant critical impediments to deepening and sustaining reforms and fostering legitimate processes for pursuing peace: 1. To facilitate public engagement in the reform process 2. To reduce the influence of the drivers of intercommunal conflict 3. To enhance the ability of key stakeholders to engage in the peace process 4. To enable survivors of explosive ordnance and persons with disabilities (PWD) in conflict-affected areas to participate in social and economic life. EVALUATION METHODS AND LIMITATIONS The Midterm Evaluation (MTE) team was composed of Dr. Mary Callahan, U Aung Tun, U Than Htike Oo, and Jeanne Briggs (assigned by OTI): • Mary Callahan, an academic with 26 years of field experience researching political, social, and conflict issues in Burma, served as the team leader and senior evaluation specialist. As team leader, she was responsible for liaising with USAID/OTI, preparing all deliverables, and leading briefings with relevant stakeholders. Her deep and broad networks among stakeholders 1 According to USAID, “The Partnership for Democracy, Peace, and Prosperity involves a three-step process: 1. Affirming a joint statement of principles in support of the democratic transition, 2. Working together to develop a joint action plan prioritizing key areas, 3. Honoring commitments made in the joint action plan to benefit the people of Burma.” From USAID, The United States–Burma Partnership for Democracy, Peace, and Prosperity, 2012. Available online at https://www.usaid.gov/sites/default/files/documents/1861/USAID_Burma_framework_fact_sheet.pdf 2 Office of Transition Initiatives, Burma Fact Sheet, June 15, 2016. 3 Where reference is made to Kann Let, it is considered to include OTI and DAI staff as reflected in OTI’s “one team” approach. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 vii representing many sectors helped the team identify potential informants, control for bias, and place Kann Let’s challenges, achievements, and staff development into historical perspective. • Aung Tun, a native of Burma, served as a mid-level evaluation specialist with research and evaluation experience in Burma’s complex political and social change. Per the Scope of Work, he supported the team leader and completed the advance documentation review, set up and prepared the MTE team for interviews, wrote parts of the draft and gave inputs for the final evaluation reports, and conducted debriefs in the field both at USAID office and Kann Let/DAI office in Yangon. He supervised some of the interpretation for particularly sensitive conversations. He provided logistical support as needed. His wide-ranging experience served the team well as he was able to identify local experts working in OTI’s programmatic areas. He was able to set up interviews with insightful stakeholders in various field locations such as Mandalay and Taunggyi. • Than Htike Oo, an accomplished freelance journalist, served as the MTE team’s logistics officer, translator, and interpreter. His knowledge of elite and grass-roots level politics was also a substantive asset in the research process. • Jeanne Briggs was on assignment from OTI to guide the MTE team. She is a career Foreign Service Officer with USAID, with 15 years of experience including 9 years with OTI. Ms. Briggs participated in kick-off discussions with OTI staff, Social Impact, and the other two evaluation team members. She conducted pre-departure interviews with relevant OTI staff regarding their intentions for her role on the team and the objectives of the evaluation. While in Burma, she participated in stakeholder interviews and provided guidance to the other two members of the team on OTI policies and practices. She participated in field-based out-briefs, and reviewed drafts of the final written report. The conclusions and recommendations of the report are primarily those of the two independent evaluators. To investigate the above-noted research questions, the team used the following methodology: • The team conducted a thorough document and data review, e.g., quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments (RAs), and Program Performance Reviews (PPRs), perception surveys and other existing data, and OTI/Burma’s activity database, a rich source of information on individual activities; • Semi-structured key informant interviews with Kann Let program staff to allow reflections on program implementation, design and relevance, strategic approach and efficiency, and identifiable impacts; • Semi-structured key informant interviews with key program stakeholders, including partners (grantees) and beneficiaries, U.S. Embassy staff, and community leaders; • Field visits to three implementation areas for semi-structured key informant interviews (Yangon, Mandalay, and Taunggyi) with above-noted stakeholders; • Facilitated workshops with key program staff to reflect on program implementation, design and relevance, strategic approach and efficiency, and identifiable impacts. The document review and semi-structured interviews with 62 key informants proved to be an effective method of collecting data responsive to the research questions. The composition of the team included members skillful at crafting flexible, probative, intuitive, and streamlined approaches among the key informant interviews. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 viii Note that nearly all interview subjects were promised confidentiality and have been identified throughout this report by a generic reference (e.g., “partner,” “non-partner expert,” “embassy staffer,” etc.) and the date of their interview. The major limitation to this methodology was time and availability of key informants. Given the time constraints, the rapidity of political and conflict-related change as the evaluation team was in the field, and the breadth and number of Kann Let activities, the team chose to give priority to interviews with partners (especially local organizations), Burmese experts familiar with the challenges to achieve program objectives, and Kann Let staff and Mission personnel with experience working in Burma for at least a year. The schedules of those active in the many kinds of reforms that continue to unfold are unrelentingly busy, leading to more than a dozen cancellations of scheduled interviews, only a few of which could be rescheduled given the brevity of the field work. FINDINGS AND CONCLUSIONS The MTE team found through document review and interviews with partners, Kann Let staff, and Embassy personnel that the program succeeded in rolling out surgical and flexible activities that were well timed and well targeted for the highly complex Burma transition. Once the learning curve associated with any new program was scaled, the OTI leadership was particularly adept in moving its staff and partners from positions of ambiguity around addressing critical impediments to matching partners and networks to achieve skilled provision of assistance. The MTE team’s research into program documentation and interviews with a wide range of key stakeholders yielded findings that are largely positive across the five research questions. Regarding critical impediments, the MTE team concluded that despite the enormous complexity of structural barriers to reform, as well as the risks and opportunities associated with misreading the landscape, Kann Let staff gradually gained the knowledge and analytical skills needed to identify truly critical impediments. In their work, they tested sensible approaches, learned lessons from partners and others, and developed a coherent set of activities that responded to Kann Let’s increasingly clear definitions of objectives and critical impediments. Kann Let was possibly the most important vehicle for maintaining the USG principle of “balance” needed to create effective peace measures. USG agencies and personnel had to walk the fine line between messaging President Thein Sein’s government about the significant undertaking of the peace process and being unable to fund the formal architecture itself. Instead, OTI and Kann Let focused squarely on building the capacity of other legitimate stakeholders who were clearly at a disadvantage at the negotiating table. Inter-communal conflict (ICC), a critical impediment, challenged all development partners committed to reform. The MTE team found that Kann Let was repeatedly able to respond relatively quickly to the onset of intercommunal conflict, which came on suddenly in 2012 with outbreaks of deadly violence. The violence broke first in Rakhine State, followed by violence in other areas and the rise of a populist movement to “protect the national races and religion,” at the cost of the safety and rights of ethnoreligious minorities. Moreover, Kann Let found partners capable of undertaking more long-term, strategically focused activities aimed at reducing the influence of drivers of ICC. Early on, Kann Let staff helped others in the Embassy and the donor environment to recognize ICC as a definitive barrier to the future of political reforms and peace in Burma. Kann Let played an important role in shifting domestic and international discourse to take far more seriously the anti-Islamic movement and shrinking space for minority ethnoreligious freedoms, as it influenced ICC and violence. Furthermore, Kann Let realized the potential Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 ix that this all had in creating a negative impact on the 2015 election proceedings. The MTE team concluded that at this strategic level, Kann Let’s assessments, research, and unique relationships with partners were likely to have minimized harm in some instances by supporting highly localized, contextually appropriate outreach between pro-tolerance groups and pro–Ma Ba Tha4 individuals and groups. Kann Let’s support to and partnerships with civil society was timely, targeted, and critical, particularly among local organizations that were prepared to shift their roles to the unfolding reform environment (e.g., DVB debate series, improved media skills around the election, listening projects, youth interfaith exposure, etc.). The choice of grantees and participants was rich and balanced. This multi-faceted approach enabled Kann Let to create a multiplier effect where both civil society organization (CSO) and activity participants as well as the broader community benefited from Kann Let’s engagement. Additionally, Kann Let targeted a powerful set of networks that provided information on unfolding events, introduced new partners to each other, and expanded Kann Let’s potential impact. Pilots and tests were built out nationally via trusted partners, most of whom emanated from Yangon. In general, the MTE team’s conclusions align with other internal and external assessments already completed by OTI/Burma. The MTE team recognized how, at first, Kann Let was unfocused, trying to fix too many things in response to the idea that everything in the country seemed like it was stuck in an era of authoritarianism. But by 2014, Kann Let had become far more laser-focused. By then, the MTE team and other evaluators (in many assessments already conducted such as the 2013 Management Review) had defined, analyzed, and targeted impediments considered to be of the highest significance and most amenable to OTI-style interventions. Tools and approaches were wide-ranging and well considered, and evaluation and monitoring in the database also grew in quality. RECOMMENDATIONS The MTE team has made specific recommendations for each evaluation question, which draw from the findings and conclusions in the report. The complete list of recommendations is in the body of the report, but here we present the priority recommendations for Kann Let Phase II to consider. The MTE team’s recommendations across the five research questions vary, but at the heart of all of them is a suggestion to keep an open and critical mind when analyzing the political, social, and economic direction of the country under a mostly democratically elected civilian government. Evaluation Question 1 • Kann Let should continue its method of working closely with partners throughout the cycle of programming, particularly when designing activities. • Kann Let staff must retain and utilize their significant analytical skill and focus in order to address the barriers to advancing reform and peace in Burma under the new government, as well as in a conflict environment, which has transformed considerably over the last 8–10 months. • Kann Let’s Senior Management Team (SMT) should sustain its promotion of staff learning and debates around emergent issues, particularly discussions around critical impediments and Kann Let’s approach to address them. Program staff should continue to be directed to speak 4 Ma Ba Tha, loosely translated as the Patriotic Association of Myanmar, is an influential Buddhist organization that has recently gained prominence for many of its outspoken political positions. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 x frequently with partners, attend partner and other relevant activities, and seek out multiple stakeholders’ views. Evaluation Question 2 • Kann Let and OTI must continue to identify and collaborate with the right partners and promote flexibility in the partnership process, as former partners are finding themselves facing new kinds of risks and opportunities, while potential new partners will need attention as they navigate the new political landscape. • Kann Let should study the list of activities that the evaluators found to be successful under Phase I and consider how these could be replicated or scaled up under Phase II. • Given the loss in ground gained under the previous iteration of the peace process, Kann Let should support activities that improve the skills of negotiators from NCA signatory and non￾signatory groups. • The team’s major actionable recommendation is that Kann Let take very seriously its commitment to inclusivity, both among legitimate armed stakeholders who may or may not have signed the NCA and civil society and conflict-affected populations. In practice, Kann Let’s emphasis on inclusivity will mean staying engaged with stakeholders who appear to be left out of or diminished by the new government’s evolving approach to peace. Kann Let must seek platforms for cross-stakeholder dialogues and learning; it must promote forums for advocating to the new government for greater inclusion. Evaluation Question 3 • Kann Let should develop a particularly careful ICC strategy and vocabulary, beyond the obvious politicization of “Rohingya” vs. “Bengali.” • Facebook and other digital platforms must be taken on in a much more serious and systematic way, particularly as hate speech, rumors, and inaccurate information spread, while mobile providers are likely to be incentivized to make access to it free of charge for most. • We suggest Kann Let expand ICC activities to other potential victims of communal violence, not necessarily defined in religious or sectarian terms. Evaluation Question 4 • Kann Let’s future program should have better engagement with CSOs outside Yangon, which for ten or more years have included small grassroots organizations. These organizations could serve as an effective countermeasure to emerging issues. • Kann Let must maintain its close working relationship with civil society. We especially recommend that Kann Let look to the low-key kind of “capacity building” that many small, new CSOs have requested. Evaluation Question 5 • If OTI is interested in contributing to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself, it should seek expertise and deep knowledge for its senior management team. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 xi • OTI should recognize that if the OTI Country Representative is necessarily playing a larger representational and advisory role in the Mission that further management support for the OTI office may be needed in order to ensure sufficient engagement and analysis with program partners. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 1 PROJECT BACKGROUND The United States recognizes Burma’s five years of reform efforts as the most significant opportunity in decades to engage with the people of Burma and their government in the pursuit of democracy, peace, and human rights. Burma faces complex, simultaneous, and multi-dimensional transitions: from authoritarian rule to democracy; from armed conflict to peace; from a centrally managed, planned economy to market-led economic policies; and from isolation and disconnection from democratic nations to an unprecedentedly rapid re-integration into global affairs. In late 2015, eight ethnic armed groups and the government signed a nationwide ceasefire agreement (NCA), followed by peaceful parliamentary elections, which resulted in a landslide victory for the opposition. The new government, led by the National League for Democracy (NLD), enjoyed an unexpectedly smooth handover of power and is now firmly ensconced in the Union and state/region executive and legislative branches, with the exception of roles and responsibilities assigned to the military. USAID/OTI’s in-country programming began implementation by DAI in September 2012. On January 5, 2016, USAID/OTI issued a Request for Task Order Proposals (RFTOP) under the SWIFT IV indefinite quantity contract to procure a follow-on task order to continue programming in Burma for two or three additional years. That contract was awarded to the same implementer (DAI).5 DAI led the implementation of USAID/OTI’s Kann Let program, established in 2014 to 1) increase participation and inclusion in the reform and peace processes and 2) address critical impediments to the transition.6 To achieve these goals, USAID/OTI’s work in Burma focused on the following objectives as outlined in Kann Let’s theory of change, in order to address the most critical impediments: 1. facilitate public engagement in the reform process; 2. reduce the influence of the drivers of intercommunal conflict; 3. enhance the ability of key stakeholders to engage in the peace process; and 4. enable survivors of explosive ordnance and persons with disabilities (PWD) in conflict-affected areas to participate in social and economic life.7 5 Office of Transition Initiatives (Burma), Fact Sheet, February 2016. 6 Office of Transition Initiatives (Burma), Fact Sheet, February 2016. 7 For greater detail on Kann Let’s road map, see Burma Transition Initiative (BTI), Strategy Review Session/Rolling Assessment (SRS/RA), February 4–6, 2014, Report. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 2 USAID/OTI sought to address these priority objectives by confronting three interrelated contextual challenges: access to information (improve access to reliable and accurate information about reform and peace processes); capacity of civil society and the government to effectively engage in reform and peace processes (strengthen the skills and provide the tools that civil society and government require); and Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 3 opportunity to respond to Burma’s reform efforts (create forums and space for timely, meaningful government, civil society and community dialogue, discussion, and interaction).8 There was $18 million obligated to the Kann Let program with 343 activities distributed across Burma. There were 136 activities focused on strengthening the capacity of civil society members and government officials, 47 on increasing awareness of peace issues and democratic processes, 71 on creating opportunities for community dialogue and reform discussions, and 89 on improving access to accurate content on peace and reform processes and countering hate speech and rumors (see the map above). The purpose of this Midterm Evaluation (MTE) was to conduct an independent performance evaluation of USAID/OTI’s Kann Let program in Burma to date. In this document, we evaluate the strategic approach, relevance, and impact of Kann Let efforts to deepen and sustain the ongoing reform process in Burma. The MTE concisely describes and analyzes program strengths and weakness and offers recommendations to guide future strategic planning for OTI and Kann Let, as well as other future USG assistance. It will be shared with the USAID/Burma Mission, the U.S. Embassy in Yangon, local stakeholders, and other interested groups as determined by USAID/OTI. 8 Office of Transition Initiatives (Burma), Fact Sheet, February 2016. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 4 Evaluation Purpose & Questions EVALUATION PURPOSE The purpose of this Midterm Evaluation (MTE) was to conduct an independent performance evaluation of USAID/OTI’s Kann Let program in Burma to date. In this document, the MTE team evaluates the strategic approach, relevance, and impact of Kann Let efforts to deepen and sustain the ongoing reform process in Burma. The MTE report concisely describes and analyzes program strengths and weakness and offers recommendations to guide future strategic planning for OTI and Kann Let, as well as other future USG assistance. It will be shared with the USAID/Burma Mission, the U.S. Embassy in Yangon, local stakeholders, and other interested groups as determined by USAID/OTI. On January 5, 2016, USAID/OTI issued a Request for Task Order Proposals (RFTOP) under the SWIFT IV indefinite quantity contract to procure a follow-on task order to continue programming in Burma for two or three additional years. That contract was awarded to the same implementer (DAI).9 A performance evaluation was conducted in lieu of an impact evaluation with control groups in accordance with ADS 203. As a transition program, it would have been difficult to select control areas that had similar characteristics to intervention zones. Moreover, this is a midterm evaluation and limited time has passed for programmatic interventions to provide adequate information for an impact evaluation. EVALUATION QUESTIONS This report answers the following questions: 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence? In answering this question, the evaluators should consider whether the program appropriately adapted its strategy as the context evolved and seek to determine whether these same areas are likely to merit similar emphasis in OTI’s future programming in Burma. 2. Has the Kann Let program employed the right tools and approaches to support the peace process in Burma? In answering this question, the evaluators should consider which tools and approaches have been the most and least effective at facilitating engagement by key actors and stakeholders, whether the program needs any additional tools or approaches, and if the program has met the expectations of key actors in the peace process. 3. To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma? In answering this question, the evaluators should consider which approaches have been most effective at changing the attitudes and behaviors of target groups, whether there are lessons or 9 The above paragraphs were adapted from Office of Transition Initiatives (Burma), Fact Sheet, February 2016, as well as the Scope of Work. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 5 findings from other fields (e.g., human psychology, media literacy) that could be incorporated to enhance the effectiveness of future interventions, and whether there are lessons for other programs seeking to promote tolerance and/or counter dangerous speech that is a catalyst for violence. 4. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing reform process? In answering this question, the evaluators should consider whether the program’s support was timely, targeted, and critical; identify any missed opportunities; and offer recommendations for ways that OTI can be more proactive as the reform process continues. 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? In answering this question, the evaluators should seek to identify lessons that can be learned from the close collaboration between OTI and other parts of the U.S. Mission to Burma. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 6 Evaluation Methods & Limitations The Midterm Evaluation (MTE) team was composed of Dr. Mary Callahan, U Aung Tun, U Than Htike Oo, and Jeanne Briggs (assigned by OTI): • Mary Callahan, an academic with 26 years of field experience researching political, social, and conflict issues in Burma, served as the team leader and senior evaluation specialist. As team leader, she was responsible for liaising with USAID/OTI, preparing all deliverables, and leading briefings with relevant stakeholders. Her deep and broad networks among stakeholders representing many sectors helped the team not only identify potential informants but also control for bias and place Kann Let’s challenges, achievements, and staff development into historical perspective. • Aung Tun, a native of Burma, served as a mid-level evaluation specialist with research/evaluation experience in Burma’s complex political and social change. Per the Scope of Work, he supported the team leader and completed the advance documentation review, set up and prepared the MTE team for interviews, wrote parts of the draft and gave inputs for the final evaluation reports, and conducted debriefs in the field both at USAID office and Kann Let/DAI office in Yangon. He supervised some of the interpretation in particularly sensitive conversations. He provided logistical support as needed. His wide-ranging experience served the team well as he was able to identify local experts working in OTI’s programmatic areas. He was able to set up interviews with insightful stakeholders in various field locations such as Mandalay and Taunggyi. • Than Htike Oo, an accomplished freelance journalist, served as our logistics officer, translator, and interpreter. His knowledge of elite and grass-roots level politics was also a substantive asset in the research process. • Jeanne Briggs was on assignment from OTI to guide the MTE team. She is a career Foreign Service Officer with USAID, with 15 years of experience including 9 years with OTI. Ms. Briggs participated in kick-off discussions with OTI staff, Social Impact, and the other two evaluation team members. She conducted pre-departure interviews with relevant OTI staff regarding their intentions for her role on the team and the objectives of the evaluation. While in Burma, she participated in stakeholder interviews and provided guidance to the other two members of the team on OTI policies and practices. She participated in field-based out-briefs and reviewed drafts of the final written report. The conclusions and recommendations of the report are primarily those of the two independent evaluators. This evaluation made extensive use of document review and qualitative methods. In answering the evaluation questions, the team attempted to obtain data that is disaggregated and analyzed by sex, although it is recognized that a combination of Burmese cultural practices, political sensitivities, and capacity constraints hindered that disaggregation. Methods followed in the evaluation include: • Documentation review, e.g., quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments, and Program Performance Reviews (PPRs), perception surveys and other existing data, and OTI/Burma’s activity database, a rich source of information on individual projects; Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 7 • interviews with 62 key program stakeholders, including grantee partners and beneficiaries, U.S. Embassy staff, and community leaders; • semi-structured key informant interviews with USAID and DAI staff; • field visits to implementation areas for semi-structured key informant interviews; and • workshops with key program staff to reflect on program implementation, design and relevance, strategic approach and efficiency, and identifiable impacts. Semi-structured interviews with key informants proved to be the most effective method of collecting data that responded to the research questions. The composition of the team allowed for members to craft flexible, probative, and streamlined approaches among the key informant interviews. The semi-structured question guide is included as Annex III. Note that nearly all interview subjects were promised confidentiality and have been identified throughout this report by a generic reference (e.g., “partner,” “non-partner expert,” “Embassy staffer,” etc.) and the date of their interview. The semi-structured interviews targeted three kinds of stakeholders: 1. OTI/DAI/USAID staff in Burma and in Washington, DC; 2. OTI/Burma Kann Let partners (and beneficiaries where possible); and 3. other community leaders with Burma context knowledge (in USG, Embassy, other Embassies and development partners, civil society, think tanks, activists, and academia) The list of key informants that the team attempted to interview was based on independent lists compiled by SI, OTI and DAI, and the team members. The team rigorously reviewed the range of possible interviewees, and those with extensive Burma experience used their knowledge to limit the introduction of selection bias. The final list sampled from all sources prioritized key informants whose knowledge was most directly related to the evaluation objectives and questions. In principle, that meant that the team emphasized interviews with local partner organizations, Kann Let staff, and Mission personnel who had at least one year of relevant experience in Burma. The major limitation to this methodology was the timing of the evaluation and the brevity of the assignment. There has been almost no policy direction from the new Burma government, which was credibly elected in November 2015 and took over the parliament on February 1, 2016, followed by the executive branch on April 1, 2016. The areas covered by the first four research questions have been in limbo for eight months now, thus making it difficult to assess complicated processes and activities. This also hindered the evaluation team’s ability to get meetings onto the schedules of those active in the many kinds of reforms that continue to unfold. A total of 62 semi-structured interviews were eventually completed by the research team, in addition to facilitated workshops. The MTE team recognizes that limited availability of respondents may introduce a response bias in these findings and conclusions. The availability of a few partners was curtailed because their organizational leadership was trying to complete Kann Let Phase I grants and close out necessary paperwork while still having to devote staff resources, travel, and time to analyzing unfolding events. More recent partners and ethnic armed group leaders were almost entirely unavailable for interviews as they were occupied with meetings to try gain greater clarity of the road ahead. In consultation with the SMT of Kann Let, we Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 8 focused on a small sample of ethnic civil society group interviewees whom the team agreed could speak responsibly and forthrightly on their views of NCA signatories and non-signatories, as well as other peace stakeholders. In this way, they served as proxies, but their reputations and histories, along with our triangulation with other sources, gave us confidence in our findings. The team was aware of and expected the following response biases for all key informants: the staff of OTI, DAI, and USAID, whose career incentives and personal senses of accomplishment may have colored responses; partners, who may have been concerned about whether their responses would hurt or help their chances at future funding; and Burma experts, who may have difficult-to-discern agendas and only made themselves available if they had a particular interest in the evaluation. The inclusion in the team of evaluation experts who are well versed in domestic Burmese affairs was our main mitigation strategy against these biases; the same was true for key informants from OTI and USAID in Washington, DC, as the team member from OTI/DC effectively mitigated the possibility of oversampling of one point of view and missing other important insights. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 9 Findings and Conclusions The MTE team’s review of the program documentation and interviews with a wide range of key stakeholders yielded findings that were largely positive across the five evaluation questions. The team arrived at an analysis of program achievements but also found interview subjects to be unexpectedly candid when queried on program weaknesses. In general, many of the partners’ responses about program strengths noted the following: • items such as the nature of the OTI grant-making process; • the give-and-take discussions and negotiations with the program staff; • the helpfulness of the program staff with design and implementation of activities; and • the inclusiveness of the Kann Let staff and program partners in analyses of the unfolding and often confusing context. Partners notably compared this program favorably in relation to their constructive work with other donors. In terms of weaknesses, however, many partners noted the following: • USG restrictions on engagements with major stakeholders (such as the previous government, the Myanmar Peace Center, or MPC, and the military); • staff turnover under previous management, which left partners with an unclear path to realizing their objectives in a collaborative fashion; and • a sense that Kann Let’s commitment fell short of helping small civil society actors grow into organizations capable of making a broader impact. The MTE team’s recommendations across the five research questions vary, but at the heart of all of them is a suggestion to keep an open and critical mind when analyzing the political, social, and economic direction of the country under a mostly democratically elected civilian government. Space for work on peace, ICC, and civil society could close just as easily as open. Planning for Phase II of Kann Let should proceed accordingly. QUESTION 1: HAS THE PROGRAM IDENTIFIED THE RIGHT CRITICAL IMPEDIMENTS TO THE ADVANCEMENT OF PEACE AND REFORM IN BURMA, SPECIFICALLY THOSE WITHIN ITS ABILITY TO IMPACT OR INFLUENCE? The original critical impediments to reform in Burma were defined in the Scoping Mission report of 2012, which synthesized the many complicated, intertwined, and deeply structural “impediments to democratic reform” as follows: • Institutions and Structures • Culture and History Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 10 • Context and Process • External Factors The July 2012 Feasibility Study and Notification to Congress described the program goal as, inter alia, engaging both the Burmese government and civil society in the pursuit of peace and democratic reform. It was anticipated that the OTI program would initially focus on: • providing technical assistance and research support to the Government of Burma to help in shaping a rational and informed reform process; • broadening and deepening civil society engagement (advocacy) with the government in the reform process; and • supporting nascent peace processes and initiatives in order to increase public confidence and increased understanding between Burman and ethnic communities.10 The program’s broad strategy stated that OTI would work with civil society, communities, and the Government of Burma (GoB) to identify, develop, and implement more effective policies and laws; build capacity to communicate priorities and concerns; support peace building processes; resolve intra-ethnic disunity; and create the conditions necessary for Burma to become a truly inclusive, multi-ethnic union.11 However, despite the initial understanding that USAID would engage with government interlocutors and support government institutions involved in implementing democratic reforms, a shift in Department guidance and congressional oversight forced OTI (and the Mission) to put such programming, considered too politically sensitive, on hold.12 Given current uncertainty and sensitivities, Embassy Yangon (including OTI) established a practice of seeking policy concurrence from Embassy leadership on activities involving assistance, or even the perception of assistance, to the GoB). Eventually, Kann Let identified a range of other objectives, some of which were based on improved analysis of the context, to address critical impediments, such as the lack of an inclusive national identity, land issues, and poor public information. One credible election probably has not broken down most of the barriers to sustainable, peaceful reform. Indeed, the results thus far suggest that it may have created a range of new impediments, including lost momentum on reform, government indecisiveness, and widely held, unrealistic expectations of what the first legitimately elected government can accomplish. Early evidence from the NLD government is that the 10 Quoted in Management Review, 2013. 11 See OTI Burma Fact Sheet, April 2013 12 These included: • Negotiations Training for Ethnic Nationality Leaders in UNFC • Crisis Management Advisor for the Office of the President • Strategic Communications Advisor for the Office of the President • Support to Myanmar Peace Center Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 11 party’s control of the legislative (Union, all regions, and all but two states) and executive branches at all levels has not resulted in any widening of the space for democratic consultation with grassroots communities and civil society. The “transition” may have weakened pro-tolerance efforts, while the stance of the executive and legislative branches has set back progress in government–civil society–population consultations. The narrative of “landslide victory” and “mandate” has been invoked to silence concerns raised about this unexpectedly long period of drifting policy guidance from the GoB. This lack of direction, paralysis, and concentration of decision-making in government leadership may constitute a significant critical impediment going forward. The MTE team synthesized from comparative and research experiences that encouraging unity among and rights for particular religious, ethnic, or ethnoreligious communities may come at the expense of having more difficult conversations regarding citizenship rights more generally. In the long run, this very critical impediment may undermine program objectives. Burma’s transition is in a unique position, specifically in terms of national identity issues.13 This is also all about whether Burma’s citizens, civil society, and leaders should emphasize specific ethnic identity rights or broader citizenship rights. Take for instance the Zomi ethnic group in Chin state, which campaigned against Chin-ness in the 2014 nationwide census by promoting the “914 sticker campaign” (914 means that Zomi people should be coded as “others” rather than as “Chin,” which is coded 401).14 This shows that many potential conflicts, especially intra-ethnic disunity, are brewing for certain ethnically named states. Dozens, and maybe more, models of nation building and federalism are being debated now from wildly varying positions of political, cultural, and economic strengths in how to address Burma’s transition and its effect on its national identity and broader citizenship rights. At the same time, the democratic transition could be at an even greater risk if a singular national identity is promoted—especially at the expense of other identities—by continuing Burmese regimes’ long history of limited imagination on possibilities for broader citizenship rights. Therefore, pro￾democracy projects in Burma should address the gap between ethnic minority rights and citizenship rights, which is of course an obvious critical impediment in Burma’s transition. Finally, in terms of specific events on the horizon that could impact multiple approaches to the outlined critical impediments, the Ministry of Labor, Immigration, and Population has announced the pending release of religion and ethnicity figures from the 2014 Population and Housing Census (PHC). Twice in the last year, the now “former” director in charge of the census has warned that the public release of these statistics could “shatter the state’s peace and stability.”15 13 It is worthwhile to note that many Eastern European countries, formerly a part of the Soviet Union, had experience in solving in their national identities or identity issues when they had attempted to secure their democratic transitions. 14 All relevant ethnic codes can have access: http://chinstatemyanmar.blogspot.com/2014/02/chin-ethnic-group￾codes-for-2014-census.html 15 Dr. Nyi Nyi, quoted in Myanmar Times, 24 February 2016, http://www.mmtimes.com/index.php/national￾news/nay-pyi-taw/19137-census-data-could-shatter-transition-stability-says-official.html Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 12 Findings In this context, the MTE found alignment with other internal and external assessments already completed. At first, Kann Let was unfocused and trying to fix too many things. By late 2014, the evaluation team and other evaluators16 and interviewees concluded that Kann Let had become far more laser-focused on critical impediments and, most importantly, was far clearer on how it might be amenable to OTI-style interventions. One key finding was that despite some early lack of focus and confusion around critical impediments, the close cooperation among the Kann Let staff, the Kann Let SMT, OTI/Burma, and Embassy personnel strengthened partners’ analytical, strategic, and collaboration skills. The MTE team leader, who has worked in and on Burma for more than 25 years, was particularly aware of the improbability of such a finding given nearly 50 years of poor investment in the education system. She found in personal conversations with the staff that the SMT of Kann Let, as well as their own colleagues, provided them with almost daily opportunities to learn more about how to analyze current events and put them into comparative or historical perspective. Additionally, staff explained how the SMT and their colleagues provided opportunities to discuss and unravel the many interconnected impediments to reforms of target issues. Furthermore, staff and SMT worked collaboratively to discuss program-level changes and appropriate approaches, as well as how to assess and improve the “fit” of potential Kann Let partners, while remaining cognizant that Kann Let cannot solve all of Burma’s problems, even with the best￾intentioned of partners. The MTE team found that the Kann Let project is working in a complicated and continually evolving landscape and that the following issues raise significant concerns about the directionality of reform: • the unpreparedness of the new government for carrying momentum forward in the peace process; • the lack of inclusiveness and consultation by the past and new government on issues of peace, ICC, land issues, law-making, etc.; • restrictions on media freedom and the quality of information flow, particularly given the omnipresence of Facebook in a digitally illiterate society; • the growing rift between civil society and the new government; and • the growing influence of Ma Ba Tha and related extremist nationalist activities, especially in rural and peri-urban areas. Achievements The MTE team found through its document review and interviews with partners, Kann Let staff, and Embassy personnel that Kann Let succeeded in rolling out surgical and flexible activities that were well timed and well targeted. As partners, Kann Let SMT and program staff scaled the steep learning curve associated with any program. The leadership was particularly adept in moving its staff and partners from positions of ambiguity around how to define and address critical impediments to a skill level wherein they could recognize impediments that fell within OTI’s capacity and mandate that they find the right partners and networks who could adapt to the OTI style of assistance. Document review and interviews 16 Assessments shared with the MTE team, e.g., OTI/Burma Kann Let, Management Review, May 8–20, 2015, Yangon–Washington. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 13 suggest that Kann Let’s “critical impediments” were initially very broad and unmanageable—“a little bit of everything,” according to a former staffer. This was not surprising for a new program being constructed in an environment that was also building out what had been a skeletal embassy with no ambassador for 20 years and no preceding USAID presence. Among the many documents shared by OTI, Social Impact and Kann Let, the MTE team saw evidence of continuous and strategically intentional re-thinking of assets and opportunities, well-designed activities to promote staff development for critical analysis, and frequent yet cautious adaptation to the changing environment. Partners interviewed cited initial confusion over Kann Let’s objectives as well as modus operandi but told us they grew more confident in the Kann Let staff and approaches as time went on. This appeared to have been a function of both increasing familiarity with processes and increased trust built through the program work. Early on, the diverse critical impediments were objects of research and testing of pilot activities and evaluation, which in a number of cases resulted in the decision to sharpen the focus to other, more definitive impediments. Thus, time-expired (e.g., preparation for the 2014 Population and Housing Census) or unmanageable impediments (such as taking on the absence of a national identity) were reviewed and determined to be diminishing in threat, to constitute less significant barriers to sustainable reform, or to be outside Kann Let’s ability to influence.17 Interviewees noted that Kann Let’s activities and partnerships allowed for swift interventions in Rakhine State after the initial shock of 2012, in momentary windows of opportunity in the peace process, and in non-Rakhine ICC crises (e.g., right after the ICC violence broke out in Meiktila in 2013). All of these timely responses to changes and partner innovations helped Kann Let and the USG to refine their understandings of barriers to reform and to advance program objectives. Additionally, informants noted that Kann Let was able to recognize earlier than others the potential dangers of the rise of Ma Ba Tha and associated extremist elements. Partners particularly considered Kann Let funding to multiple, often-crosscutting, initiatives to have been vital to helping them reduce the influence of hate speech and other drivers of ICC, especially in the lead-up to the election. Several partners and Burmese experts noted that Kann Let’s support to media—which cut across critical impediment issues and targeted changes—resulted in a far more responsible ethnic and majority journalistic approach to covering the very divisive events leading up to the election. Furthermore, partners considered what Kann Let called “Ethnic National Conferences” to have been first-of-their-kind opportunities that—where Kann Let correctly analyzed—ethnic minorities lacked unity, which undermined their potential for participation in the peace process as key challenges.18 Partners involved in these activities, along with other events that brought together many different kinds of ethnic stakeholders, appear to have produced more resilient networks and expanded the perspectives of ethnic “leaders” (of armed, political, or civil society groups). These first-of-their-kind events represented first steps toward surmounting the impediment of years of disunity and distrust. 17 Evidence drawn from the OTI Anywhere database of activities, as classified by Critical Impediments. 18 OTI Burma DevtNote12 Ethnic National Conferences.pdf Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 14 That Kann Let was able to identify, analyze, and act in such responsive but strategic ways—despite the immense risks, the fluidity of the environment, and staff turnover—is a major achievement. As a former senior Embassy staff member told us, “This is the most complex operating environment I have ever been in… It is so easy to make a mistake here.” When the new quasi-civilian government took power in April 2011 after decades of military rule, most experts and citizens were skeptical about whether Burma was serious about reforms or more likely inclined to the window dressing associated with “illiberal democracy.”19 The country’s democratic transition has traveled a bumpy road on all fronts: the 2011– 2016 government tried but failed to achieve lasting peace; was caught unawares by unprecedented intercommunal violence but attempted some limited interventions; eliminated all press censorship but maintained a tight space for media freedom; created a market in land via two new laws and a new set of instruments to deal with land confiscation complaints; and opened itself to consultations with opposition on key legal reforms, including the elimination of some repressive laws, although much remains to be done. Nearly all of the partners interviewed considered one of Kann Let’s strengths to be its ability to work flexibly in the midst of all this risk and uncertainty. When the USG took the initiative to reengage with Burma in 2012, the critical question was pressing: where is the right entry point to help solve those emerging issues in Burma’s democratic transition? The answer was not obvious to most donors due to the unexpected and fragile nature of the transition. Given that the United States and a massive influx of new development partners found themselves in uncertain territory, the dilemma was how to shift from regime-change, pro-parallel system aid delivery to assistance that could strengthen the hand of the emergent reformists; attract fence-sitters to take risks in supporting democratization; and weaken the position of hard-liner, anti-reformists. Other donors with limited Burma knowledge rushed in and poured money into organizations, activities, and individuals of marginal relevance to the strengthening of reforms. Unlike those programs, Kann Let faced a range of restrictions on the use of U.S. funds. The MTE team, however, found through its review of program documentation and interviews with all three groups of subjects that OTI/Burma and Kann Let developed the capacity to identify and quickly empower the civil society sector, as it appeared (and in most cases proved) to be most effective at advancing U.S. priorities to “serve the people of Burma.”20 Across both the period of Kann Let Phase I and the range of critical impediments addressed, partners, Burmese experts, and Embassy personnel considered Kann Let to have flushed out both difficult-to-find knowledge on how the reform process was evolving and the right partners possessing the skills, protection, and capacity to intervene progressively. The MTE team recognizes the significance of this achievement in such a complex environment. Issues to Be Addressed Interviews with partners and Burmese experts in Mandalay and Taunggyi yielded a common concern that Kann Let was building out its ICC, civil society, and peace programming on the backs of Yangon-based organizations, including INGOs, at the cost of engaging directly with what one Burmese expert called “our hometown” organizations and networks. In activity documentation around ICC and peace—the two major areas called out by non-Yangon interviewees—there was some evidence that multi-locational programs did indeed appear to be carried out by Yangon-based groups in 19 See Fareed Zakaria, The Future of Freedom: Illiberal democracy at Home and Abroad, New York, W.W. Norton, 2007. The former military junta (up until 2011) and its successor quasi-civilian government under Pres. Thein Sein (2011–2016) referred to this system as “discipline flourishing democracy.” 20 Scoping Mission, p. 28 Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 15 Mandalay, Taunggyi, and many other locations. This Yangon-outward approach—whether intentional or not—was likely a practical solution to building a program from scratch in Kann Let’s first phase. CSOs with explicitly political experience were either headquartered or represented in Yangon. Additionally, as Kann Let hired staff, their Yangon-based applicants probably knew more about organizations in Yangon than elsewhere. One check that Kann Let devised to bring voices from afar into planning and strategy was the employment of “field advisors” in a number of states and regions. However, their time commitment is very limited, which raises questions about how effective they can be at informing SMT and program staff. The MTE team heard different visions for change, as well as critical issues to target outside of Yangon. They were not wholly incompatible with Kann Let’s findings and analysis, but the team concluded that partners and Burmese experts and their constituencies have experienced the transition in different ways. As Kann Let not only designs programs around identified critical impediments but also updates its own analysis of critical impediments, these alternative and localized views may not be sufficiently understood. Conclusions The MTE team concluded that if it were determined that Kann Let has succeeded only in raising the staff and partners’ critical thinking and strategic planning to address critical impediments to peace, that accomplishment alone would be one of the more enduring development outcomes in Burma. Despite the enormous complexity of structural barriers to reform, as well as the risks and opportunities associated with misreading the landscape, Kann Let staff gradually gained the knowledge and analytical skills to identify and act on the truly critical impediments. In their work, they tested sensible approaches, learned lessons from partners and others, and developed a coherent set of activities that responded to Kann Let’s increasingly clear definitions of objectives and critical impediments. The team particularly credits Kann Let management for promoting staff abilities and empowerment to the degree that they were capable of analyzing this environment so confidently and successfully. Other development partners have been far less effective at developing the critical skills and team spirit that Kann Let has. The result was clear: the accumulated knowledge and experience of Kann Let’s staff have promoted sound analysis of and new insights into the identification of and programming around critical impediments. QUESTION 2: HAS THE KANN LET PROGRAM EMPLOYED THE RIGHT TOOLS AND APPROACHES TO SUPPORT THE PEACE PROCESS IN BURMA? Of all the evaluation questions, this proved the hardest to evaluate, for two reasons. First, the “peace process” is constituted by hundreds, maybe thousands, of different mini-processes and micro-negotiations that are aimed at dozens if not hundreds of different definitions of “peace.” Spoilers and peacebuilders have wrestled over basic common principles for procedural approaches, while large-scale fighting continues in Rakhine and the north. The process announced by President Thein in 2011 and eventually given a domestically driven architecture of procedures in 2012 focused some of the stakeholders’ activities on working within that vision, but the complexity, endurance, and economics of decades of civil strife suggests that, as one interviewee noted, “There is a very long and bumpy road ahead.” That the State Counsellor has only recently begun to reveal her approach to a government-led process has left enough time for the landscape to become even more complicated and fraught with distrust. Kann Let can support activities directly and indirectly related to the peace process, but it has no capacity or mandate to address deep, underlying structural causes and drivers of armed conflict. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 16 The second challenge the MTE team faced was the absence of a central focus for a procedural architecture, creating an environment of scattered unpredictability. The new government’s struggle to create effective policies to assist in the peace process has resulted in the unavailability of many of the peace stakeholders for interviews, as the they are working tirelessly to define agendas and stay on top of shifting allegiances, opportunities, narratives, and options. Hence, our interview list came up short on peace stakeholders— particularly actual signatory and non-signatory representatives. This weakness very much undermines our ability to analyze the effectiveness of the full spectrum of tools and approaches to support peace activities. This shortcoming is all the more significant given OTI/Burma’s unique role since 2014 as the only USG agency permitted to work directly with ethnic armed organizations (EAOs). The evaluation team tried to approximate, where possible, by interviewing civil society leaders who had worked closely with signatory and non-signatory EAOs. Since independence in 1948, five high-level attempts of national-level peace negotiations have failed to produce a political solution to the grievances that fueled civil wars. As a result of these civil wars, tens of thousands of combatants and non-combatants have likely died, and credible accounts of serious and widespread human rights abuses have been reported. Life in these war zones was brutal, resulting in large￾scale displacement and refugee flight. Land confiscation, resource exploitation, smuggling, and human trafficking have been rife in areas of live conflict as well as in ceasefire areas. The histories of non-state armed groups (NSAGs) that organize themselves by ethnic nationality identity have followed multiple paths. Some have split several times, and some have chosen—in previous and current negotiations—to make bilateral ceasefire deals to end the fighting in specific territories in order to gain access to and control over services and resources. EAOs have varied in their methods of recruiting, financing, and regulating people, taxes, and trade. Their relationships with registered political parties or other non-state armed groups bearing the same ethnonym are often troubled, contingent, or non-existent. Communities in and around former and current bilateral ceasefire areas, as well as in territories home to recent warfare, are hopeful about the present round of peace negotiations. However, they remain wary about claims that development or a single credible election will deliver “peace dividends.” Given these context limitations, the Embassy regrouped and developed its policy of “balance” toward legitimate peace stakeholders. Achieving that balance was difficult, but OTI/Burma was crucial in designing a different peace engagement strategy. Several interviewees from OTI/Burma, Mission personnel, and the Kann Let SMT credited the OTI/Burma Country Representative for her guidance to the Embassy and USG on how to assess “reformists,” as well as incentives and disincentives on all sides of the emerging negotiations. According to a senior USAID official, the Country Representative’s awareness of the history and context “helped us find a way to engage with the Myanmar Peace Center, not by funding it, but by encouraging and messaging that we understood the significance of the moment. We would have had a hard time finding that balance without her knowledge from OTI, her ability to read Burma, and her access to [the] highest policy circles.” The former ambassador noted that this allowed the U.S. Embassy to focus support on other stakeholders and, particularly, “we focused like a laser on the ethnic side.” Findings With this context and limitations in mind, the MTE team found that Kann Let’s approach, which grew increasingly more focused, yielded networks, knowledge, trust, and channels of communication that could reasonably be expected to have contributed to progress in what will likely be a non-linear and very difficult 10- to 20-year peace negotiation process and the finalization of a comprehensive peace agreement. Per USG strategy, the principle for engagement was “balance,” which meant walking the fine line between messaging President Thein Sein’s government about the significant undertaking of the peace process and being unable to fund the formal architecture itself. Instead, Kann Let focused squarely on Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 17 building the capacity of other legitimate stakeholders who were clearly at a disadvantage at the negotiating table. Interview subjects praised Kann Let’s ability to work with the right partners, even as the most nascent opportunities emerged, and toward program-level changes that increased the capacity, awareness, networks, and trust of many different kinds of stakeholders in the process. At the same time, interviewees noted some issues they wished had been addressed differently. For example, partners, Burma experts, and USG personnel recognize that these activities were limited by the restrictions on supporting engagements with arguably the most significant actor in the peace negotiations—the military. Furthermore, early choices for the funding of one partner over another on raising awareness versus holding workshops to address the peace process may have led to missed opportunities to achieve the program-level changes sought. Achievements All three groups of interview subjects reported that Kann Let partnered with actors who worked in collaboration with other stakeholders to design sets of activities responsive to emerging opportunities. These activities addressed the twists and turns of the formal peace process, such as the expansion of the legal space for ceasefire monitoring; listening projects developed to increase channels of communication between communities and ethnic armed and non-armed elites; and state-level consultations on inputs to the eventual Political Dialogue.21 They also addressed indirect interventions to create a more fertile ground for peace, such as increasing public access to information about peace (in both Burmese and non-Burmese indigenous languages), introducing conflict sensitivity into CSO activities and trainings, and supporting the convening of ethnic national conferences. Kann Let found appropriate partners capable of earning the trust of a traumatized population and highly skeptical stakeholders to the actual process. To a very real extent, all of Burma is “conflict￾affected” after 70 years of warfare on its soil. Most armed, anti-state opposition has been domestic, although at different times, at least five foreign governments, as well as numerous foreign non-state actors, have supported one anti-state group or another. Because the country has been on a constant war footing since independence in 1948, state-society relations have always been characterized by an emphasis on protection of the state at all costs. Institutionally, that has placed the military in power for at least five decades. For these reasons alone, the continuation of armed conflict on Burmese soil constitutes the most significant impediment to sustainable, peaceful democratic reform. 21 Over the last few years, the conceptualization of Burma’s peace process has evolved to be composed of three stages or steps: 1) bilateral ceasefires between the government and an NSAG; 2) a nationwide ceasefire which, it was hoped, would attract all 16 recognized NSAGs to sign an agreement alongside the army (which would be a first in history; however, although the military signed this historic document in October 2015, only eight NSAGs signed); and, later, 3) a relatively ill-defined, extra-legislative convening of stakeholders in a “political dialogue,” wherein federal reforms could be debated and moved forward in such a way as to create the basis for a comprehensive peace agreement. 22 Data from the OTI Anywhere database retrieved in May 2016. Table 122 Four Main Target Issues No. Activities Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 18 As Kann Let developed programming to address the key impediment of ongoing armed conflict, its sophisticated understanding and ability to respond to opportunities evolved over time. Eventually, out of its internal Rolling Assessment in March 2015, Kann Let focused on the target issues in Table 1. Kann Let activities effectively addressed gaps in the peace process and OTI/Burma’s objectives in the country. These targets also reflect the Embassy’s focus on empowering peace stakeholders who were at a serious disadvantage in the process. Among partners, there was general appreciation for Kann Let’s support to civil society organizations to collaborate on the design of activities directly related to the formal peace process. These activities aimed to increase readiness for an eventual political dialogue; to promote coordination via a range of workshops and ethnic national conferences held to prepare for upcoming peace talks; to support activities that allowed for civilian participation in ceasefire monitoring and civilian protection; and to prepare communities, EAOs, CSOs and other stakeholders to assist in managing the immensely complicated tasks associated with interim arrangements. Partners and Burmese experts who are involved in the peace process also expressed how valuable it was that OTI/Burma and Kann Let supported activities that improved the skills of negotiators from NCA signatory groups, while also remaining committed to assistance to non-signatory groups. While the activity notes from the OTI database are clear that such exercises were fraught with challenges, they have proven to be good for learning how to fine-tune future activities. Some of the activities that interviewees considered most effective included:23 • Listening projects (e.g., BTI179 and BTI182), which the internal OTI evaluator told us in an interview were particularly successful at bringing ordinary individuals’ concerns to the attention of ethnic armed groups (EAGs). According to the team leader, these projects succeeded in “bringing them face to face and opening communication channels.” A grantee told us, “At first, the EAGs [were] hesitant to receive CBOs/CSOs but when they know it’s not to make trouble for them, they become more open. The Listening Project taught us that EAGs need more exposure. The second-line leaders need to travel and get more experience.” • Workshops inside the country and in Chiang Mai aimed at bringing communities together to learn about the peace process, the nationwide ceasefire agreement, and the political dialogue, and to formulate positions for input into eventual state-level political dialogues. One Kann Let partner held awareness seminars on political dialogue issues. The interviewee noted that “Because of the meetings, they came to know more about NCA and federalism. We came to get this conclusion from the letters received from some participants from the meetings. They send a letter of thanks for the activity.” • Support for media capacity building and programming to increase public awareness about peace, federalism, and the 2015 election in ethnic areas (e.g., BTI044, BTI058, BTI061, 23 These examples should not be considered exhaustive, given our very limited sample size and time constraints. Weak foundation for political dialogue 49 Lack of preparedness for joint elite peace negotiations 30 Inadequate implementation of existing ceasefire agreements 20 Challenges of implementing interim arrangements 7 (all retired) Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 19 BTI062, BTI175); one grantee said that without Kann Let’s support, his outlet could not have undertaken special projects aimed at remote audiences. • Meetings, workshops, trainings, and cultural events that focused on federalism, pluralism, and diversity, for a variety of different stakeholders. It was hard to determine that Kann Let’s interventions were solely responsible for what the evaluation team agrees was greater awareness on these issues, but there is a general sense among interviewees that at least a few of Kann Let’s partners were responsible for far more sophisticated thinking about the major challenges that lie ahead on a path towards federalism (e.g., BTI205, BTI351). According to one Burma expert interviewed, many of the actors in civil society and directly in the formal process had “long used ‘federalism’ as a catch-all category without any idea of how complex it was.” He said the level of understanding has improved considerably. Notably, in the activity notes of one such series of workshops that brought together political parties, two EAO leaders, and other organizations, an evaluator wrote of participants’ concern that they didn’t know “who represents who.” [BTI 246] • Support for consultations, design, and implementation of civilian ceasefire monitoring, which has gained some traction in several areas and resulted in a handful of agreements to take a humanitarian pause and let civilians get out of the fray (e.g., BTI002, BTI027, BTI 028, BTI141). Although there remain many problems with it (not the least of which is the lack of a communication channel to the official Joint Monitoring Committee), one partner said that no one could have imagined such mechanisms to be possible a few years ago. • Research and platforms for EAOs, civil society, and other stakeholders to address concrete issues—like mother tongue language, constitutional design, negotiation skills and comparative lessons on federalism—have delivered some concrete results (e.g., mother tongue language in education has gone from being unmentionable in 2011–2012 to being a far more widely accepted norm now.) [Sample activities include BTI202, BTI203, BTI205, BTI286, BTI287.] Other activities, such as in peace education or anti-ICC activities, were less effective, but even in those cases, program staff and at least one partner considered them absolutely necessary when designing alternative approaches afterward. • Testing activities not obviously related to peace but unquestionably addressing potential impediments for progress. One example in Shan State was an in-kind grant of a computer and printer to a nascent CSO that raised money to support communities’ visions on development. The grantee told us, “Our main goal is peace, but when we assessed the CSOs working on peace, we found a gap and decided to focus on development to support peace.” She showed us spreadsheets and other documentation of their activities to demonstrate how a simple equipment donation helped the organization continue to design, carry out, evaluate, and build out on its strategy. • Even very simple activities, such as Kann Let’s publication and distribution of the text of the NCA booklet, was widely cited as furthering the work of partners and others committed to education of EAO leaders and communities in and beyond conflict-affected areas. The MTE team asked partners, Kann Let staff, USG Mission personnel and Burmese experts to explain Kann Let’s success in pursuing its principle of “balance” and choosing appropriate tools and approaches. As is the case with other program objectives, one set of responses clustered around the contextual analysis and knowledge of OTI/Burma from the outset. One grantee noted that OTI/Burma had learned much from its pre-2012 work along the border, and thus was well placed to understand the significance of the emergent process as well as its limitations. The OTI/Burma Country Representative was also cited as crucial for developing Kann Let staff’s analytical capacity and knowledge about the range of peace stakeholders. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 20 A second cluster of responses on the principle of balance focused on the flexibility of Kann Let and its ability to move quickly when a partner identified an opportunity. Two different grantees ran into unexpected delays and barriers to implementation of their original activity design. They reached out to Kann Let for guidance, came up with alternatives to address the same target issue, and in the end were relatively certain that the redesign was more effective, if more challenging. One of these partners noted that Kann Let was unlike any other donor in terms of the speed with which it could deliver funds or in￾kind assistance. Another donor had withdrawn suddenly from a large event, and Kann Let was able to deliver support to save the event within a week. The fact that both of these partners interviewed tended to work in organizations with strategies that aligned with Kann Let’s program objectives probably allowed for the kind of trust needed to mobilize with such agility. A third implementer, based in Chiang Mai, considered Kann Let’s flexibility and responsiveness crucial to the research, capacity building, awareness raising, and other activities that their organization works on with multiple EAOs. Once Kann Let had the clear authority to pursue activities to strengthen the capacity of EAOs in the negotiations, this same partner said that Kann Let was the most responsive donor and especially appreciated the in-kind assistance. “If Kann Let didn’t do the logistics for us, no one else would have stepped up and we would have lost a chance.” A third factor, noted with particular reference to Kann Let’s influence in the last year or so, was the respect and trust that partners had developed for the Kann Let staff’s counterparts. This was often counter-posed to the early Kann Let period during which there was “too much staff turnover,” according to one partner, but also held up against other donors, whose staff could not be relied upon to understand the complexity of the peace process. Several grantees appreciated Kann Let’s practice of observing activities. One said, “They gave us great suggestions on how to improve and move on to next steps.” Issues to Be Addressed Key stakeholders, including partners and others, revealed two sets of disappointments in Kann Let. One set involved partners’ dismay that the USG was unwilling to fund some important and unexpected opportunities that arose to promote the skills and understanding of key stakeholders (e.g., reformists working on peace in the President’s office, EAO leaders needing help for the Summit at Laiza). From partners, Kann Let staff, and Burma experts, the MTE team heard complaints that they could not understand the parameters of what Kann Let could and could not do, as a result of policy restrictions from Washington. One major lost opportunity was OTI’s inability early on to support the Myanmar Peace Center. At that point (2012), OTI/Burma was among the only donors who understood that the challenges for ending decades of armed conflict were highly political and not amenable to infusions of context-blind technical assistance. The other major cluster of criticisms is most likely due to the complexity of the implementing environment and the extreme competition among donors around peace. One director from a large local non￾governmental organization (NGO), which had generally positive things to say about Kann Let, reported that both she and her staff were not clear about Kann Let’s role, among a logarithmically expanding donor and foreign implementer landscape: In implementation, it is very confusing as to who is doing what. And whether Kann Let or DAI are purely donors or implementing agencies. We don’t know who is who. This is especially true when it comes to in-kind support. One example: USAID invited non￾signatories for a meeting we hosted. Usually we do logistics. But DAI was directly managing logistics and travel. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 21 The interviewee also said that “EAOs were confused and thought they had to go to the U.S. Embassy” to get Kann Let support [May 10]. A couple of the peace grantees also were frustrated with how long it took to get a grant issued. In the words of one grantee, “It was so tiresome to negotiate fixed obligation grant—[it] took 12 months. That didn’t reflect well on DAI.” This partner was relatively sure the delay was a result of either turnover in personnel (he had had a series of different points of contact at DAI) or restrictions on assistance imposed by Washington, DC–based USG. The interviewee also noted, however, “Once we got it, there was so much flexibility and accommodation.” It is worth noting that this case was an exception. Kann Let’s choice of focusing on single-ethnicity “national” conferences, convenings of ethnic party leaders, and federations of EAOs, may have neglected a chance to target deep inter-ethnic distrust at community levels, as that constitutes a major impediment to progress toward peace. For example, awareness-raising activities were held in Shan State to share information on the peace process that included Pao and Shan EAO leaders and civil society groups. However, when we queried the grantee on whether relations were improving between Shan and Pao communities, she said, “very little.” When probed, she considered it quite possible that Shan-Pao’s fighting could break out not only over major political issues but also the tensions that have brewed in separate but adjacent communities for decades. These may be structural problems beyond the mandate of OTI or Kann Let, but community-level interethnic distrust remains a major barrier to crafting a stable peace. Another interviewee, not a partner but a Burmese CSO representative from southern Shan State, said that while EAO leaders may be gaining trust in each other via Kann Let activities, it is not clear the same can be said among diverse communities living in close proximity to each other. The MTE team found that OTI and Kann Let may have been delayed by Washington too long to work with EAOs directly (before congressional permission finally came through). The armed organizations were at a severe disadvantage entering the ceasefire negotiations and discussions over frameworks for political dialogues. A senior Embassy official was also frustrated with the delay in permissions to support EAOs, as it held up Embassy attempts to ensure assistance and programming was fully aligned with its principles of engagement. He said once that permission for support came online, “we could be more sure our policy was coherent” on peace. Kann Let is likely to find that the peace negotiations will be even less linear than the last iteration of the process. The new government continues to forge ahead with ad hoc and vague declarations of its decisions without any meaningful formal or informal consultations with the wide range of already￾mobilized peace stakeholders. It is increasingly clear that there is a lack of competence on peace-related issues in the new government. Aside from rhetorical commitments to Panglong24 and peace, most actions 24 The “Panglong Agreement of 1947” was a nine-paragraph agreement signed by Aung San (who at the time had no legal authority to commit the then-colony to anything) and representatives of three ethnic groups. Aung San promised autonomy in exchange for Chin, Shan, and Kachin leaders’ willingness to allow their territory to be part of the post-colonial entity of Burma. Had Aung San not obtained those signatures, Britain surely would have split the colony in two. Historically, “Panglong” has achieved mythical status, as did Aung San after his assassination several months after the meeting. For Kachins, especially, but also some Shans and Chins, it is a reminder of how Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 22 have resulted in the creation of structures with only form and little substance. The lag time between the January Union Peace Conference and the next conference, as well as the confusion over whether the existing agreements are to be scrapped or modified, have only introduced new kinds of challenges, such as difficulties between some signatories and non-signatories, fighting among multiple groups, and the escalation of offensives by the Defense Services. State Counsellor Aung San Suu Kyi’s views on the role of non-signatories now very much aligns with the position of the Defense Services, i.e., that those who did not sign the NCA can be involved in the peace dialogues but only under conditions they may find unfavorable. As one long-time peacebuilder (and Kann Let partner) advised, this decision means that EAOs will have to redesign their own institutions to increase their capacity under the new rules now emerging. The geographic location of the peace process in Nay Pi Taw rather than Yangon could present an opportunity for Kann Let. Conclusions The MTE team concluded that the impact of Kann Let’s support to the peace process will not be easily measureable for years, maybe decades, to come. As is the case with all five research questions, the peace process is in limbo until the State Counsellor and her advisors address the enormous range of substantive and logistical issues introduced by the transition of power. Although recent announcements of a skeletal architecture for an NLD-led process are a step forward, there is a long road ahead until enough of the substance and logistics are negotiated. Unfortunately, it is not clear this early in the new government’s tenure whether informal and formal consultations over agenda-setting will be its modus operandi, and that alone could set back negotiations significantly. For Kann Let, this may represent an opportunity to bring partners’ voices (and broader Embassy advocacy) to bear upon the hundreds of decisions that will add up to a peace process. On the other hand, it also represents a risk of missed opportunities if the USG policy, directed from Washington, is to await clear directions from the government on its peace strategy. Kann Let generally employed the right approaches to peacebuilding in the complicated and evolving context from 2014 to 2016, especially by involving local partners in collaborative design of activities. Interview respondents cited multiple successful activities they felt had furthered the peace process. Kann Let’s support of the US’s “balanced” approach to peace was generally successful because they had a strong understanding of the context and a strong rapport with the local partners and the OTI funding mechanism that allowed for rapid response as appropriate for the evolving landscape. Feedback on the main “tool” at Kann Let’s disposal—their rapid and varied funding mechanism—was mostly positive, but there were a few reports of confusion about the role of Kann Let in the widening donor landscape. There was generally appreciation for the rapid and flexible funding mechanism but disappointment with some perceived missed opportunities and the length of time to award for granting arrangements. Just-in-time, “in-kind” support was deemed very helpful in several reports. Burmese leaders break all their promises. For other ethnic groups not invited or not signing, there are decades of resentment of non-inclusion. And for the ethnic majority, including Aung San Suu Kyi, it represents the only moment in history where peace was achieved. She has long dreamt of convening an ahistorically-envisioned “Panglong 2” in which she will be the unifier of all national races, but she has developed little in the way of the administrative, logistical, and political bodies she will need to achieve that. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 23 NCA signatories will need a strong support team, while non-signatories will need a different institutional basis to make the most of their position. “That means the evolution of two parallel institutions and two parallel policymaking processes. How can we possibly help all of them to do what they need to do?” Furthermore, as she noted, once ethnic leaders decide on the organizational leadership of such institutions, “I worry that we will never be able to bring them together.” QUESTION 3: TO WHAT EXTENT HAS THE PROGRAM BEEN EFFECTIVE AT REDUCING THE INFLUENCE OF DRIVERS OF INTERCOMMUNAL CONFLICT IN BURMA?25 To address this evaluation question at an operational level, the MTE team asked interviewees from all three groups what they considered the drivers of intercommunal conflict. First, the MTE team sought to understand the root causes of intercommunal conflict. Next, the team probed further to assess the program’s effectiveness at reducing the influence of these drivers. Partners, Kann Let and OTI/Burma staff, Embassy personnel, and Burmese experts all warned against being overly reductionist in defining “root causes” or drivers of conflict. Most also were cautious about the degree to which causation could be proved. In general, we heard repeatedly that there are many drivers of ICC in Burma. In Yangon, Mandalay, and Taunggyi, interviewees pointed to a wide range of historical grievances and exclusionary experiences; the popular lack of understanding of broader context; longstanding socialization of hatred of the “other;” and the pervasive, pent-up anger after years of suffering under repressive regimes, which increased the likelihood of the scapegoating of ethnic or religious minorities when repression lessened. Some pointed to the unfortunate timing of the liberalization, coming as it did with the dramatic expansion of access to the market-dominant Facebook, which very quickly was used to mobilize fear, hatred, and antagonism toward “others.” Others pointed to factors such as the Chinese pipeline, a possible intentional strategy on part of the “dark forces,” a term used to refer to the outgoing military junta of Than Shwe and hardliners in Thein Sein government. Burma has faced years of built-up suffering, anger, and resentment that can be easily manipulated in terms of scapegoating religious minorities. Findings Achievements The MTE team found that Kann Let was repeatedly able to respond relatively quickly to the onset of intercommunal conflict (ICC), which came on suddenly in 2012. There were outbreaks of deadly violence first in Rakhine State, followed by violence in other areas, and the rise of a populist movement to “protect the national races and religion.” Moreover, Kann Let found partners capable of undertaking more long-term, strategically focused activities aimed at reducing the influence of drivers of ICC. However, in this environment, it is impossible to mount compelling evidence that any one or set of interventions can be definitively said to have held off the growth of drivers of ICC. The MTE team took note of anecdotal 25 The MTE team is cognizant that a separate, more intensive evaluation of Kann Let’s work on ICC has been commissioned. As a result, we focused squarely on the research question (and subsidiary questions posed), and our analysis is presented in a more concise manner than our responses to questions on civil society and peace. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 24 but persuasive evidence that the spread of anti-Islamic and anti-minority sentiments and activities continue to present threats to the reform process. The MTE team found that a major achievement of Kann Let was that none of the activities that the team evaluated appears to have done any harm. Given the enormous risks that its partners assumed, the significance of this achievement cannot be overstated. This finding stands in contrast with the actions of other donors and implementers. Once the violence began in Rakhine State in 2012, the mostly newly arrived donors all hunted furiously for interventions, under pressure from their capitals to do something, as though “doing something” was better than doing more nuanced analysis to avoid harm. In most cases, that meant donor-supported, high-level interfaith dialogues where senior religious officials spoke in Burmese about how each religion has tolerance woven through its sacred precepts. However, several partner interviewees noted that Buddhist leaders lost interest when it came to operationalizing activities that could decrease the tension. Most interviewees characterized these activities as theatrical but not necessarily effective. In addition, other donors provided technical assistance in Rakhine State and sought out “moderates” to support in order to bring about “reconciliation.” However, they did not recognize the consequence of aligning themselves to these moderates: they became more identifiable targets. Kann Let identified a wide range of partners who: 1. understood the sensitivity and deep structural drivers of ICC; 2. leveraged multiple sets of personal and organizational networks to roll out nuanced interventions; 3. developed along the way an instinct for identifying opportunities and direction-changing risks; and 4. could identify some ways in which their activities de-escalated the tensions that come out of the historical, geopolitical, cultural, and other causes of ICC. Many activities focused on youth, which is a sensible approach for reducing the short- and long-term drivers of intercommunal violence. Kann Let chose many kinds of partners, with activities that included Facebook monitoring and interventions against hate speech; creative pop-up shows on tolerance and pluralism; and low-tech, quiet dialogues among community leaders that resulted in the formation of conflict-mitigation committees. The latter appear to have delivered the most definitive results in reducing the influence of hate-mongering forces. In Mandalay and Taunggyi, interviewees told the team of several instances when these networks or committees were able to stop brewing tensions simply by having gained the knowledge of, trust of, and access to community leaders of multiple faiths. Some examples of activities that probably decreased the influence of ICC drivers: • Engagements with partners able to bring together a variety of different organizations and communities, such as tours for youth from one part of the country to another and quiet discussions among CSOs and more liberal representatives of Ma Ba Tha, particularly over women’s issues. From the evaluations of these activities, there appear to have been bridges built that have been sustained and mobilized to block violence. • Support to local organizations to carry out monitoring of Facebook postings associated with ICC. Although Facebook has done shockingly little to address its role in ICC in Burma, this monitoring exercise increased awareness of who—whether real names or aliases were used—is behind the particularly dangerous use of this medium. This information is then used to inform the planning of pro-tolerance activists, Kann Let, and other donors. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 25 • Identification of townships at risk for violence, partnered with local organizations capable of mapping the landscape and seeking pro-tolerance allies. A third achievement was cited by partners and Embassy personnel. They noted that OTI/Burma and Kann Let staff relationships with partners and their ability to identify the right partners at the right moment helped the Embassy and other development partners respond to crises and problems quickly, so as to prevent the possibility that suffering or frustration could be manipulated to escalate along intercommunal lines. The former USAID director noted, “OTI helped us find which organizations to work with on intercommunal conflict.” USAID was particularly cognizant of contributions to reduce potential drivers of conflict in Rakhine State when faced with the need to move quickly to address suffering. Asked for an example, the former Mission Director said, “when the floods happened in Mrauk U, OTI used [Kann Let’s] existing grants with organizations they knew could help us roll out emergency relief. This really mattered, as the two communities shared common needs and we could emphasize commonalty and tolerance.” The MTE team was convinced by program documentation and interviews that more youth, Facebook users, and a possibly emergent “liberal” segment of Ma Ba Tha have been exposed to messages, activities, and dialogues that promote tolerance and demonstrate the costs of ICC. However, as noted above, given the long historical and interconnected layers and linkages among causes, perpetrators, victims, and triggers of ICC, it was very difficult to conclude decisively that significant behavioral and attitudinal changes have been achieved among target groups. One well-respected Burmese academic and partner warned that it is unrealistic to think that anyone can systematically measure whether interfaith engagements change Buddhist nationalist minds about whether a person in Burma can be something other than a Buddhist and be a full citizen. Issues to Be Addressed There were two major barriers to success in relation to online messaging. The first was the poor state of digital literacy in Burma. As LIRNEasia and other research organizations have found, the country’s leap from 5 percent Internet saturation to 80 percent mobile saturation in the last three years has led to a reliance on Facebook for all information, however unreliable. Rohan Samarajiva of LIRNEasia argues that the mobile revolution, dominated by Facebook in many developing companies, means that “hate speech is enabled and accelerated. It’s tricky controlling this, partly because one man’s hate speech is another man’s free expression.”26 Second, the combination of the exciting novelty of connectivity via Facebook and the education system’s non-promotion of critical thinking skills creates the potential for great vulnerability. Anecdotally we heard from interviewees that Facebook users are as likely to click “like” on a hateful post 26 “Benefits and costs of mobiles in Myanmar,” November 24, 2014, http://lirneasia.net/2014/11/benefits-and-costs￾of-mobiles-in-myanmar/ Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 26 as they are on an anti-hate post27. One multi-activity partner that has worked against ICC with youth recommended that Kann Let focus in its next phase on the social context that gives Facebook such power in the realm of hate speech. He said, “Kann Let should focus on basic citizen information literacy and access in rural areas. People […] believe crazy rumors.” One Burmese expert raised the alarm that donors, including Kann Let, were missing the expansion of potentially violent extremism. He was concerned that it was hard to convince donors to take risks on promoting tolerance and undermining ICC, as “anger is so easy to mobilize against scapegoats.” He suggested that key scapegoats could include anger over land complaints, Chinese mega-projects, or more historically-based anti-Chinese sentiments. Another civil society representative (not a partner) said he was convinced that some areas of Mandalay could easily ignite. “It could be Facebook-rumor-driven, but more likely I think that the continued unmet expectations of voters and their frustration with the NLD will create moments when scapegoats will become the targets of all the anger.” When asked about whether the donor focus on clerics/laity interfaith platforms for discussions could mitigate the risk, he said, “I don’t think any of the interfaith dialoguing can stop violence against Muslims.” In nearly all geographies and among all Burmese experts, there was a nearly unanimous assertion that Kann Let and other donors have failed to take the risks and invest the resources necessary to counter the growing influence of Ma Ba Tha and other anti-Islamic, pro-Buddhist-nationalist leaders and groups in peri-urban and rural areas. Although they felt that some degree of conflict mitigation had been achieved in parts of the three urban areas we visited, they are concerned that not enough research, mapping, and risk analysis have been conducted on the less centrally located places that senior Ma Ba Tha and nationalist “patriotic” monks and laity are now frequenting. From local organizations in Mandalay and Taunggyi we heard some apprehension that Kann Let was relying too much on Yangon-based organizations to bring their tolerance messages out to localities that they did not understand particularly well. In Taunggyi, a group of Muslim elders (not grantees) explained the nuanced way in which they had mitigated over 300 disputes that could have turned violent over the last two years. They gave examples of how a formal committee established by the government with assistance from Yangon-based CSOs was impotent, particularly noting that Buddhist members of this formal conflict-mitigation group lost interest in the issue and stopped attending awareness and prevention meetings. They, as well as local organizations in Mandalay, were convinced that ICC is probably the most “local” of critical impediments to reform, and as such, should be handled by truly local organizations. A well-respected academic who has written extensively on the violence in Rakhine State, elaborated: “In every region of this country, there are some common sources of animosity—not just between Buddhists and Muslims, but also between other kinds of groupings—that have and will continue to lead to violence. But much more important are the narratives of grievance that are very particular to a specific village or township or bigger area or ethnic or religious group—and lots of times it’s several 27 The MTE team also would like to share our observation of the repeated use of the English term “scapegoats,” in the midst of Burmese-language interviews with partners and Burmese experts in Mandalay and Taunggyi. In both of those locations, the same interviewees cited NLD paralysis on protecting at-risk communities to be in and of itself a potential trigger of myriad tensions that could erupt in conflict perpetrated against minorities who might be defined in ethnic, religious, or ethnoreligious terms. 28 http://www.mpt.com.mm/en/mpt-and-facebook-to-bring-basic-internet-services-to-people-in-myanmar/ Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 27 of these. Without knowledge of the particulars, projects aimed at challenging the oversimplified characterization of “the other” are doomed to be irrelevant, at best, but more likely to do harm.” Conclusions While it was difficult for the MTE team to come to a definitive response to this evaluation question, discussion with interviewees indicated that Kann Let addressed head-on some drivers of conflict, such as the rumor-mongering on Facebook, hate-speech, and lack of intercommunal networks and trust. In most cases these were implemented through relatively small-scale pilots or testing, which then used the lessons learned to maximize opportunities to build a constituency for tolerance. At the same time Kann Let minimized the risks that seemed to expand as the country moved toward the 2015 election. The MTE team concludes that Kann Let has been successful in identifying the partners that can make it an effective force for reducing ICC in Burma. However, these partners need to represent at wider geography and diversity of perspective. There is also a need for a more in-depth understanding of whether the NLD government has demonstrated its understanding and approach to addressing the complex situation that can mitigate or sustain ICC. There is an opportunity to expand the effectiveness of approaches to reduce ICC through both low-tech and high-tech means. On the low-tech side, Kann Let has been very effective in fostering community dialogues with a just-in-time approach that has served to diffuse rising tension. On the high￾tech side, the MTE team found that Kann Let is operating in a changing digital space. There is the possibility that cellular providers in Myanmar are on the verge of expanding the tools that can be used by ICC promoters. In particular, MPT’s announcement in June 2016 that it will roll out Facebook Free Basics and Facebook Flex.28 Telenor also now provides Facebook and Viber access for free. Increased digital access will likely expand the already market-dominant position of Facebook and likely increase the potential for harm by those with malicious intent. The MTE team is aware that Kann Let’s support of anti–hate speech programming focusing on Facebook may generate knowledge about Facebook identities who promote rumors and encourage violence. However, the team could not find substantial evidence that these activities were likely to significantly weaken this high-tech driver of ICC without other kinds of interventions, such as those that promote research on the efficacy of currently programmed activities, the promotion of broader based digital literacy capacity building, or the advocacy for a major policy change by Facebook vis￾à-vis its unregulated Myanmar market. The MTE team concludes that Kann Let must ensure that it does not become complacent about new types and forms of extremism. The MTE team found that because Kann Let was among the first donors to recognize the import of extremist, nationalist social movements such as 969 and Ma Ba Tha, it was able to influence USG, Embassy, and other development partners’ strategic thinking about how to protect minorities at risk. In particular, Kann Let staff early on helped others to recognize ICC as an unexpected but definitively critical impediment to the future of political reforms and peace in Burma. As such, interviewees considered Kann Let to play an important role in the shifting domestic and international discourse particularly related to the anti-Islamic movement, the shrinking space for minority ethnoreligious freedoms, the threat of more violence, and the potential for a xenophobic, nationalist hijacking of the 2015 election proceedings. The team found that at this strategic level, Kann Let’s assessments, research, and unique relationships with partners were likely to have minimized harm in some instances. Through a wide variety of innovative and carefully crafted activities, Kann Let sponsored the activities of some 28 http://www.mpt.com.mm/en/mpt-and-facebook-to-bring-basic-internet-services-to-people-in-myanmar/ Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 28 stakeholders who have emerged as both prominent and (perhaps more importantly) off-the-radar champions of tolerance. The more than three weeks of field work led the team to conclude that there may be a misplaced sense of complacency among Kann Let staff, Embassy personnel, other donors, and a small number of Kann Let grantees that the threat from the extremist movement associated with Ma Ba Tha had passed. When we probed, we were told that Ma Ba Tha and its followers did everything they could to negatively impact the NLD’s election results but had failed miserably and therefore were now largely disheartened and no longer a political, social, or conflict factor. Another piece of evidence offered was the rise of “counter-movements,” such as Kann Let–sponsored activities like Panzagar and independent activities like the April 25, 2016, protest by 25 anti-nationalist activists at Hledan in Yangon. The evaluation team considers these anecdotes to be important, but we question whether small or “virtual” counter￾movements or Ma Ba Tha’s apparent electoral impotence have had an impact on drivers of ICC. This is a question that can be researched by empowering local organizations to develop databases on activities of pro- and anti-tolerance groups, clean that data, and collaborate with the many other small groups collecting data relatively unsystematically and with limited knowledge of research methods and data analysis. Our interviews with successful pro-tolerance partners overwhelmingly suggested that Islamophobic, extremist nationalism remains a major impediment to reform. In Phase II planning, Kann Let SMT and staff should not underestimate the potential for the growing influence of this direct driver of ICC. QUESTION 4: DID KANN LET SUCCESSFULLY SUPPORT CIVIL SOCIETY TO ENGAGE IN AND INFLUENCE THE REFORM PROCESS? Findings This evaluation team heard time and again in interviews with partners and Burmese experts that Kann Let successfully supported myriad civil society approaches to engagement in the reform process. More importantly, program documentation and Kann Let staff, as well as grantees and other observers, pointed to Kann Let–supported civil society activities that have opened channels for direct (and unprecedented) influence on political reforms broadly. Interviewees also thoughtfully and critically raised some areas for improvement and brought to our attention the potential for changes by the new government that might close off space for civil society participation in reforms. Achievements Kann Let worked with 374 civil society organizations directly on 348 activities to date. Many of these organizations and activities were simultaneously carving out space for civil society to help accelerate the ongoing reforms on multiple fronts: peace, intercommunal relations, land, human rights, legal reforms, freedom of expression, victim assistance, and gender issues. To some degree, the entire Kann Let program factors in a “CSOs-led” approach to addressing critical impediments and increasing space for non-elite participation in advancing reforms. See Kann Let’s tangible and strategic support to CSO partners and range of target issues engaged in Table 2. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 29 Table 2. Mechanisms of Civil Society Engagement: Examples of Tangible Kann Let-Supported Activities Strategic Partners or CSOs-Led Level of Engagement or Tactics Emerging Issues Approach Ethnic CSOs State-based dialogue activities Bringing ethnic armed groups’ leaders into direct talks with grassroots community and ensure the community voice is heard; NCA booklet distribution; understanding characteristics of federalism Workshops and public consultation (e.g., ethnic armed group listening projects); both direct consultations (e.g., KNU, DKBA); indirect consultations with communities and specific EAGs or communities too traumatized to speak directly to EAGs (e.g., SSPP) Recently returned exiled media and media development CSOs Broader public awareness activities on politically and socially sensitive topics Bringing debate culture or stimulating discussions on the table including intercommunal violence Special program such as debate TV program and local debate series—nurturing “agree to disagree” nature in societal matters Electoral support– interested media and CSOs Broader public awareness about electoral risks and increased availability of accurate information Disseminating information on prevention of abuses of religion in politics and preventing hate speech Journalism training on elections reporting including ethnic areas; online platform for election monitoring; promoting code of conduct for political parties and candidates, etc. Conflict-resolutions CSOs Dialogue between different religions Bringing different religious leadership to have dialogue for peaceful coexistence Interfaith dialogue; interfaith dialogue trainings and workshops Conflict-resolutions CSOs Counteracting hate speech and dangerous speech Disseminating anti-hate and anti-dangerous speech through various means including social media and posters Local CSOs provides anti-hate and anti-dangerous speech materials to greater numbers of the public via various means including posters and social media and using technological solutions Conflict-resolutions CSOs Awareness about “national identity” Addressing the limited awareness about finding a common national identity—which is a blow for nation building and peaceful coexistence Workshops and awareness trainings; literature; talks; small campaigns Ethnic CSOs who are greatly involved in peace process Dialogue between ethnic “elite negotiators” and all relevant stakeholders Support to overcome a key impediment in the peace process—the lack of preparation among EAGs Ethnic national conferences in ethnic areas including CSOs, EAGSs, Government, political parties, religious and community leadership Disabilities-interested CSOs Legal reforms for more inclusive disabilities rights Prompting relevant ministry (Social Welfare, Relief, Resettlement) to consult with persons with disabilities and achieving a general consensus Consultative workshops with all relevant stakeholders along with disabilities communities Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 30 Overall, Kann Let’s support to and partnerships with civil society were timely, targeted, and critically important, particularly among civil society organizations that were prepared to shift their roles to the unfolding reform environment (e.g., DVB debate series and electoral support as well as listening projects). The choice of grantees and participants was rich and balanced. This multi-faceted approach enabled Kann Let to create a multiplier effect whereby the CSO and activity participants as well as the broader community benefited from Kann Let’s engagement. Additionally, Kann Let targeted a powerful set of networks that not only provided information on unfolding events but also introduced new partners and expanded Kann Let’s potential impact. Historical perspective is crucial to understanding the extent of Kann Let’s achievements in the civil society arena. Given the state of civil society in, the political uncertainty of, and the risks embedded in the pre– Kann Let environment, the program’s ability to find suitable partners, help them develop their ideas, carry out strategic activities, and develop shared understandings of lessons learned was highly improbable. In March 2011, when the new quasi-civilian government took power under the 2008 Constitution, Burma was home to a society long accustomed to oppression, with individuals distrustful of all but relatives and inner circles. Few in the country believed any claims of “reform” by incoming president Thein Sein. However, even in such hostile conditions, civil society had not spent the period of junta rule (1988–2011) underground or in decline. In fact, as the Transnational Institute noted in 2011, despite a wide range of obstacles, “civil society in Burma today is diverse and dynamic, and has been creating its own space.” (Gaining Ground, p. 3) Most were service-oriented and apolitical for purposes of survival. And as the Scoping Mission correctly assessed in 2012, Burma’s civil society groups often reflected broader cultural and social structures. “They are often top-down, hierarchal, ageist, identity-based, premised on the leadership of one dominant personality and their internal operations often undemocratic.” Hence there were significant challenges ahead when OTI/Burma received clearance in July 2012 to launch a Burma program in the unexpected “window of opportunity” to “support and influence a peaceful transition to a more representative and responsive government in Burma.” [the USAID Action Memo] Today’s civil society in Burma is markedly more inclined toward political engagement, strategic policy development, and activities aimed at building bridges from the formerly silent communities to the local, region/state and Union-level authorities. Interviewees provided examples of the positive outcomes of Kann Let activities—small conflicts resolved, larger conflicts defused, marginalized individuals empowered, and relations re-established between segments of society that had grown apart. We asked partners, Kann Let staff, USG Mission personnel, and Burmese experts to explain Kann Let’s overall success in support to civil society engagement in and influence over reform content and direction. Responses clustered around four themes described below. First, numerous interviewees pointed to the contextual analysis and knowledge of OTI/Burma from the outset, and later Kann Let, in knowing where, when, and how to engage with civil society, which was contrasted with other donors who had little to no country expertise and brought inappropriate models and tools to work in Burma. The sensibility and trustworthiness of the OTI/Burma Country Representative, who had years of experience working on Burma, was cited by civil society partners, Kann Let program staff, and SMT and USG staff as necessary condition for Kann Let to help CSOs transform their ideas, outlooks, and organizations into ones that could be effective and nimble in engaging in the non-linear and relatively high-risk reform environment. It also helped to secure the role of Kann Let, USAID, and the Mission in achieving “quick wins” via effective engagement with trustworthy and capable civil society organizations across the country, including ethnic areas, where armed conflicts remain active (e.g., ethnic CSOs engaged in peace and state-based dialogue). Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 31 An example of where interviewees considered that this deep country knowledge paid off was the early (compared to other donors) focus on fostering civil society’s engagement in developing better laws, such as the Association Registration Law and the Anti-Violence Against Women law. Most of them had never had any such experience of advocacy on legal reforms, much less seeing their recommendations taken on board in revisions. To sum up this first point, OTI placed the right person in the right place at the right time. Doing so improved the chances that Kann Let could target innovative and resourceful CSOs that not only helped achieve program-level changes but also improved the capacity of Kann Let SMT and program staff to stay on top of emerging community needs, unexpected impediments, and new opportunities for interventions and partnerships. Second, partner interviewees invariably felt that Kann Let’s success was a function of its methodology of active and ongoing engagement with partners from the germination of ideas to the development of activities and through to evaluation. After two decades of small-scale support from donors, most organizations were unaccustomed to this kind of partnership model. Kann Let allows local CSOs to present initial draft ideas or concepts in terms of actual local needs and CSOs’ capacity and skills to address those pressing needs, and then Kann Let provides necessary assistance for further actions. This is a unique program both for Kann Let and its partner CSOs because other funders have many reservations to do so given that many of those issues are politically and socially sensitive—which many called “testing the water.”29 Many civil society grantees were very appreciative of Kann Let’s flexibility and advisory role in designing effective activities. CSO partners expressed sincere gratitude by saying “we appreciate the courage of the USAID and the U.S. Embassy”30 to recognize emergent opportunities and take risks, which “gave us more strength to try.”31 They invariably noted that Kann Let’s level of support and flexibility exceeded that of most other donors with whom they have dealt. A recurrent phrase came out of interviews with Kann Let’s CSOs partners: “without Kann Let’s flexibility, our program could not have been a success.”32 As a result of building up relationships through flexible and accommodating guidance, Kann Let was able to rely upon CSO partners to be frontrunners in designing responses to pressing or unexpected reform issues, sometimes risking their personal safety and organizational security to push for more space as they can in the limited political landscape. To encounter emerging issues—for instance, intercommunal violence mainly between, on the one hand, radical Buddhist monks and laity, and, on the other, Muslim communities and individuals—CSOs play a crucial role by taking bold steps such as bringing religious leadership together and making sure there is mutual understanding among all stakeholders and even in grassroots communities. These CSO-led actions have been unprecedented in Burma. Third, in the process of ongoing engagement, CSO partners noted the efforts that Kann Let management and staff made to follow the CSOs’ own agendas, as opposed to achieving outputs or outcomes set in some far-off capital. Although we expected to hear more complaints that the intensive communication requirements of Kann Let took too much time, in fact, most interviewees 29 Interview, Civil Society Leadership in Yangon, Mandalay and Taunggyi, U.S. Embassy officials, Kann Let Staff (April/May 2016) 30 Interviews, CSOs in Yangon (May 2016) 31 Interview, CSO representative in Yangon, but works mainly in remote ethnic areas. 32 Interviews, CSOs in Yangon, Mandalay, and Taunggyi (April/May 2016) Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 32 considered it necessary to help them achieve their goals and, especially, to mitigate risk in the rapidly changing environment. Although some negotiations with potential partners extended across a longer period of time than would be the norm for OTI, the sense of the partners and program staff interviewed was that whatever costs may have been incurred in timeliness were worth the benefits of getting the design and activity right. Fourth, Kann Let’s staff capacity has earned the trust of the partners sampled, which has facilitated improvements in both Kann Let programming and CSO development. Over time, Kann Let’s staff developed more confidence and competence in analysis and OTI methodology, as well as in their definitions of target issues, opportunities for program-level changes, and innovation around approaches and activities. Some of this improved analysis resulted from intentional and creative staff-development activities, but it was clear in interviews with program staff that they had learned an enormous amount from civil society organizations with which they had built partnerships. From CSO interviewees, they appreciated how much Kann Let listened to them, took on board their understandings and concerns, and reflected back analysis to help them achieve their goals. Issues to Be Addressed From our analysis of program documentation and interviews, there are three overarching issues Kann Let needs to address as it partners with civil society to increase engagement and influence in reforms. First, the historical legacies of repression continue to haunt CSOs. The concrete historical legacies and the trauma resulting from decades of political repression continue to hinder civil society engagement in reforms of all kinds. During president Thein Sein’s rule (2011–2016), CSOs continued to face numerous legal restrictions on their political, apolitical, and community-service activities, as well as the survival of their organizations. Many were legacies that went back a century to British colonial laws that reflected the regime’s preoccupation with security. To date, the new executive or legislative branches at the Union and state/region levels have delivered mixed messages on the dozens of anti-democratic laws that civil society actors have called out for repeal or amendment. On the one hand, one of the first legislative achievements of the NLD government was to repeal the State Protection Law. On the other, the NLD’s revision of the Peaceful Assembly Law sustained highly repressive components that undermine the potential for CSOs and activists to express popular concerns in public. Additionally, some CSO interviewees, as well as Burmese experts, remained very cognizant of the role of the security apparatus in managing state-society relations. Of particular concern was the continued power of the General Administrative Department within the military-led Ministry of Home Affairs to regulate, administer, and potentially repress CSO work at the township and village levels, as well as in state and regional affairs. Interviews with senior management and some Kann Let staff suggest there is awareness of the lingering influences of historical barriers to CSO progress. In its early days in office, the NLD government indicated a preference for Union-level government officials to set priorities from the top (and in particular, the Bamar-dominated NLD central executive) and then direct funding via a central Union body, with little consideration of grassroots needs and inputs. For example, one ethnic nationality human rights partner told us that the NLD “want to be practicing the essence of ‘federalism’ and to say they are respecting the voices on the ground, but instead they are centralizing everything and deciding who should be the [chief] minister and who should do everything.” Second, civil society representatives from outside Yangon perceive that most of Kann Let’s and other donors’ civil society support goes to Yangon-based organizations, across multiple program￾level change arenas. Although Kann Let’s civil society engagement achieved extensive geographical Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 33 coverage in a short time, organizations based outside of Yangon were disappointed that Kann Let programming appeared to have privileged Yangon-based organizations to expand into distant areas. There may be some practical reasons for such an approach by Kann Let. For example, Yangon is home to branch offices of many truly local (non-Yangon) CSOs, like Chin Human Rights Organization. Additionally, Kann Let has limited staff, most of whom hail from Yangon, and thus their inclination in Phase I may be have been to tap Yangon-based organizations known for their effectiveness and leadership. Beyond staff and resource constraints, infrastructural, linguistic, and experiential barriers also make it hard to locate suitable partners in distant areas. This issue was raised repeatedly in our field visits to Mandalay and Taunggyi, which led us to investigate reports and the database for more insight into geographical coverage. What we found from activity reports from places like Kayah and Shan States, as well as Magway and Mandalay regions, was that Kann Let activities in those areas were often part of a bigger multi-locational program carried out—with a few exceptions—by Yangon-based CSOs. Relatedly, CSOs outside of Yangon would also like to receive more direct support and engagement from Kann Let rather than through proxy organizations of Yangon-based CSOs or international NGOs.33 For instance, an emerging Mandalay-based CSO in conflict resolution area made clear, “if we accept money from Muslim community, we’ll be accused, but if from Buddhists, Muslims don’t trust. So we want more from international directly.”34 Third, small partners expressed disappointment that Kann Let did not provide support to help them grow into bigger roles. Small partners in Yangon and elsewhere learned a lot from their engagement with Kann Let, but some of the newer and smaller ones reported that it was hard for them to grow and develop organizational capacity. They appreciated the guidance Kann Let provided on strategic design for activities but were hungry for a similar kind of guidance on organizational development. When we probed for a clearer idea of what these groups concretely meant by “organizational development” or “capacity building,” it was apparent that most were not seeking an influx of technical assistance or funding to grow into large-scale NGOs. Instead, they were looking for opportunities to engage in the kind of back-and￾forth discussions that came out of their Kann Let engagement, particularly around thinking realistically about what their strengths and weaknesses were, how to define problems systematically, and how to match their strengths and networks to their visions. As one said, “We just need more opportunities for guidance on how to make our changes happen.” Further, as engagements with civil society have grown, so did the amount of information in the database. The MTE team noticed variation in the precision of activity notes, but over time, a considerable improvement in this section. It was not clear how all of this extremely valuable descriptive and analytical information in the database was aggregated and used. For example, we saw across different program areas notes and feedback in the activity notes about which individuals/groups can work with others. These data are very important in Burma, as institutional and organizational cultures are wedded to personalities and histories of either personally driven collaboration or distrust. Staff turnover within Kann Lett appears to happen at a higher rate than within other development partners. The reasons for this are not clear. For example, the MTE team learned that initially there was 33 Interviews, CSOs based in Mandalay and Taunggyi (May 2016) 34 Interview, CSO based in Mandalay (May 2016) Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 34 a commitment to a hiring policy that would not distort the salary scales among development partners (DPs) and local organizations. As the logarithmic expansion of job opportunities accompanied the arrival of development partners with less knowledge of the context and the potential to do harm, departing staff likely found better-paying jobs. The OTI model was brand new to staff, as were the analytically challenging requirements of implementing activities from concept through evaluation. One current staff member speculated that others who left may just not have been well suited for the kind of work assigned under Kann Let. Finally, we also heard polite criticism of an earlier Kann Let Chief of Party, although no one chose to elaborate. Given the strong performance of Kann Let in developing trust with civil society, a stable Kann Let team will be critical to the overall program’s continued success. Conclusions This evaluation, while limited in time and scope, reached the conclusion that Kann Let support to civil society not only enhanced its ability to influence the reform process but also empowered groups that were small, medium, and large; gained impressive coverage across the country; and focused on the rights of ethnoreligious minorities, women, people living with disabilities, and other highly vulnerable communities. Moreover, we found evidence that Kann Let’s activities achieved its objectives of increasing participation and influencing reforms. Support to the activities for the Civil Society Forum for Peace, for example, has greatly expanded the network of CSOs who were influencing peace dialogues at the local and Union levels. Another example is the support to CSO involvement in lawmaking and development technical working groups, which gained civil society representation it had never achieved before. It is hard to prove causality on all of these conclusions, but the team is convinced that Kann Let played an important role in several such milestones. Interview respondents expressed concern about the Yangon-centric approach of the Kann Let work. The evaluation team took these findings seriously and probed for more detail in many interviews. The team ultimately concluded that building multi-locational programs from Yangon-outward (to other states and regions, or in general terms to more “remote” areas) constituted a sensible and relatively lower-risk approach to quick wins in Kann Let’s first phase. Additionally, Kann Let’s network was of seminal importance in helping the international community respond to crisis conditions by seeking information from CSO partners and pointing appropriate kinds of assistance to the right actors in the right way (examples: 2013 violence in Meiktila; 2015 floods in Chin and Rakhine States; and for leading in Embassy conversations about the costs and benefits of Mission engagement in emergent opportunities, such as the 2015 election). Via an effective engagement with CSOs, Kann Let has well-identified hotspots and key players, influencers, and spoilers in advance of the 2015 elections, which was a crucial step for better political transition. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 35 QUESTION 5: HOW HAS OTI CONTRIBUTED TO THE ADVANCEMENT OF U.S. FOREIGN POLICY IN BURMA BEYOND THE KANN LET PROGRAM ITSELF? Of the five questions in scope of work, the MTE team spent proportionately the least amount of time on Question 5, as we understood that OTI had assigned a separate exercise to examine this specific issue. (That analysis has not yet been finalized in order to be incorporated into this evaluation.) Nonetheless, we heard feedback from all three groups of key informants about OTI’s advancement of U.S. foreign policy beyond the Kann Let program itself. Sometimes this feedback was the result of direct MTE team questions (especially to Group 1—Kann Let SMT, Kann Let staff, and other USG personnel), but often it came up largely unprovoked in the context of discussions with partners and Burmese experts. The latter often conflated OTI with USAID or DAI or Kann Let, but the team was able to glean their views on OTI’s contributions within and beyond Kann Let. Findings The team found that the primary achievements in U.S. foreign policy objectives to serve the people of Burma were heavily dependent upon the contributions of OTI’s first Country Representative in Burma, and later by the Deputy Country Representative and Field Program Manager. The common thread among different informants’ viewpoints was that OTI came in with deep knowledge of the environment, showed consistent consideration for a range of stakeholders not necessarily involved in Kann Let’s programming, and recognized both opportunities and risks for the Embassy and OTI engagement in a highly complex operating environment. The contributions that OTI made to U.S. foreign policy were through the key role that OTI staff played within the Embassy country team, shaping political analysis, programming choices, legal interpretation, operational decisions, and policy formation. Key informants on this question were expected to be those interviewees from the U.S. Embassy and USAID Mission in Yangon, but as noted above, Kann Let staff, partners, and Burmese experts often offered useful insights. Achievements Interviewees praised OTI and the Kann Let program for being at the forefront of shaping U.S. foreign policy in Burma. The most commonly cited strength was the rapid learning that occurred across the Mission and across agencies as a result of OTI’s deep understanding of the country. The environment was as difficult to read as any, and with the most minimal Embassy presence in 2011 and no USAID Mission in the country, OTI was the one agency with not only contextual knowledge but also networks of trust that spanned powerful leaders and organizations. The former director responsible for establishing the USAID Mission starting in 2012 noted the following: “The key to this was that the U.S. was able to find someone with deep expertise of the country, not someone with transitional experience in a bunch of other countries but in Burma. [The OTI Country Representative] had deep experience in the context and knowledge of actors and issues. This was essential to start up a mechanism like OTI and was a huge advantage to the Embassy, as it could draw upon her knowledge as the Embassy grew.” [April 27] Outside of specific programming choices that the OTI/Kann Let team made based on their analysis of the context and impediments to peace and reform, OTI staff also played a unique role within the USG country Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 36 team at the Mission, by serving not only on working groups but taking leadership roles on the most politically strategic ones. In particular, the outgoing OTI Country Representative, due to her expertise and experience in Burma as well as her close working relationships with the former Ambassador and Mission Director, filled a role unusual for most OTI Country Representatives in that she served as a senior advisor to the Embassy and USAID Front Offices, often asked to participate in discussions or provide advice on issues that would normally be outside the scope of a typical OTI Country Representative. She and her deputy country representative networked with development partners in multiple sector working groups, both to share contextual analysis and to find ways to balance the load on promoting reform. Particularly in the early stages of the establishment of the USAID Mission, the OTI Country Representative played a disproportionate role in shaping decisions on policy, strategy, programming, and legal considerations, and also mentored many of the incoming staff who had not previously worked in Burma. According to the former director of the Mission in Burma: “USAID had no experience inside the country. Many responsibilities fell on [the OTI Country Representative] to guide us and especially to mentor all our new incoming people, and to help all the short-term TDY-ers who held places open [until permanent staff were deployed] but had no knowledge of the country. This was true for both the Embassy and USAID. She helped us to grow our expertise and to focus on what we could do in such a difficult environment. Because we started from scratch, it mattered that it was [this person specifically]—with the trust she’d gained from local partners, the knowledge she had to recognize an opportunity and to help us make informed choices about risk. Her track record here ended up opening doors for us. It gave us effectiveness and greater legitimacy.” USG stakeholders, and Kann Let staff and partners cited the OTI Deputy Country Representative and Field Program Manager as demonstrating deep knowledge of the Burmese context and a sophisticated grasp of OTI programming approaches. Their partners appreciated their leadership and flexibility and recognized their critical role within the Embassy team. The result for OTI, according to the former USAID Mission director, is, “This Embassy has the ability to take a far more strategic look forward.” Key among the contributions mentioned by informants was the robust and rigorous political analysis conducted by OTI and Kann Let staff, which led to cross-Mission collaboration that—according to the former ambassador—allowed for its “agility, the wisdom to pick our spots and demonstrate U.S. legitimacy.” Some examples from other informants included recognizing the analysis and leadership of OTI and Kann Let in the Embassy during its early recognition in 2013 of ICC as a national phenomenon that reached beyond Rakhine State, as well as its identification of the emerging power of Ma Ba Tha. These examples ultimately led to ICC’s prioritization in policy, programming, and public diplomacy efforts, as well as in leadership in the donor community in Burma. U.S. Government stakeholders and peace-program staff and implementers cited the authorization of OTI to work with ethnic armed groups as a key component of the Embassy’s advancement of the USG policy of principled, balanced engagement with all legitimate parties to the conflict and the peace process. This ability, unique among most other donors in Burma, allowed the USG to promote context-appropriate programming and significant resources to support a much more balanced approach by the international community, the rest of which had been largely focused on assistance to government￾associated peace stakeholders. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 37 Issues to Be Addressed The contributions by OTI to the broader USG Mission at Post as noted above do not come without costs. Although OTI Country Representatives globally were increasingly spending less time on program management and more time on representation within the Embassy, the Burma case seemed to represent a particularly extreme example. The drawback of this unique opportunity to provide inputs on policy were that the Country Representative was not able to spend as much hands-on time with the OTI and implementing partner staff as would normally be the case. There were evidently circumstances in which the Country Representative was drawn into issues that were far beyond the purview of the OTI program. The evaluation team did not discern any major negative impacts on this point but notes that it was raised by several key stakeholders as a concern, and clearly the significant role that OTI played in USG policy in Burma came with trade-offs in regard to the traditional OTI management model. Conclusions The major conclusion of the MTE is that by placing the right person in the right place at precisely the right time, OTI had a major impact not simply on achieving its program objectives, but also in supporting the relatively seamless transformation of a tiny Embassy into a much larger one. OTI’s knowledge, methodology, and collaboration with both the Embassy and the new USAID Mission were described as “crucial,” in the words of the former ambassador, to navigating a highly complex, opaque environment while maintaining the commitment to USG policy. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 38 RECOMMENDATIONS QUESTION 1: HAS THE PROGRAM IDENTIFIED THE RIGHT CRITICAL IMPEDIMENTS TO THE ADVANCEMENT OF PEACE AND REFORM IN BURMA, SPECIFICALLY THOSE WITHIN ITS ABILITY TO IMPACT OR INFLUENCE? • Kann Let staff must retain and utilize its significant analytical skill and focus in order to address the barriers to advancing reform and peace in Burma under the new government, as well as in a conflict environment that has transformed considerably over the last 8–10 months. Those shifts must be analyzed carefully in the upcoming strategy review and in ongoing internal assessments, in order to ensure that Kann Let stays on top of shifting and emerging impediments. • Kann Let should continue its method of working closely with partners throughout the cycle of programming. In a period of apparent political contradictions—a legitimately elected government promoting some reforms while at the same time stifling dissent and centralizing control—staff and Kann Let’s SMT need to stay in close contact with former and potential future partners. Doing so will more likely sustain the relationships of trust established under Kann Let’s first phase, strengthen Kann Let staff’s understanding of evolving challenges in this multi-reform environment, and help to guide partners in their analytical, strategic, and organizational development. • Kann Let SMT should sustain its promotion of staff learning and debates around emergent issues. Program staff should continue to be directed to speak frequently with partners, attend partner and other relevant activities, and seek out multiple stakeholders’ views. The evaluation team believes this will also contribute to continued Kann Let prescience on finding the right partners on the right issues at the right time. • Kann Let should look for partners from outside Yangon, not only Yangon-based NGOs with field offices, to build multi-locational programs of activities. CSOs in ethnic states and non￾Yangon regions have very different visions, missions, capacities, and constituencies. These alternative perspectives may help to promote new kinds of networks that advance Kann Let’s four program objectives and bring new voices to bear upon Kann Let and USG debates and strategic reassessments. • If human and financial resources are available, Kann Let should consider opening at least one field office. We recognize that it is nearly impossible to choose a location that directly supports work across all major program objectives. For example, for ICC, Mandalay may be a strategic location, given the extensive influence of extremist nationalist forces in the central part of the country and in ethnically named states adjacent to Mandalay Region. However, that choice would potentially undermine popular and partner perceptions of Kann Let, given that it would likely be seen as overcommitting to the ethnic-majority Bamar communities rather than the ethnic nationality ones. However, we suggest serious consideration of Nay Pyi Taw as a new location now that the National Reconciliation and Peace Center will be located in Nay Pyi Taw, and now that the democratic opposition is firmly ensconced in government there. An office or representative there would be able to identify new partners, unexpected openings for interventions, and potential new approaches to increasing participation. • The MTE team recommends opportunistic and strategic engagement—perhaps initially via pilot activities—with GoB at the Union and state/region levels, and across the three branches Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 39 of government, when developing approaches to target issues under critical impediments. • Kann Let staff should reach out to ICC and peace partners for advice on how to mitigate the risks of inflamed religious- or ethnic-based conflict resulting from the release of the 2014 Population and Housing Census final ethnic and religious aggregate statistics. • Finally, resources permitting, we recommend Kann Let consider bringing partners from different target issue areas and geographical regions together for discussions particularly around critical impediments. QUESTION 2: HAS THE KANN LET PROGRAM EMPLOYED THE RIGHT TOOLS AND APPROACHES TO SUPPORT THE PEACE PROCESS IN BURMA? • Kann Let and OTI must continue to identify and collaborate with the right partners and promote flexibility in the partnership process, as former partners are finding themselves facing new kinds of risks and opportunities, while potential new partners will need attention as they navigate the new political landscape. In addition to being responsive to shifting conditions and resulting pressures to move deadlines forward or backward, promotion of sound and truly locally led initiatives that continue the practice of underplaying USAID/OTI’s brand should remain a priority. • Given the loss in ground gained under the previous iteration of the peace process, Kann Let should support activities that improve the skills of negotiators from NCA signatory and non-signatory groups. The current government is leaning toward an inferior status for non-signatory groups, which will only decrease the likelihood of any progress in the peace process. Negotiators on all sides will need support as they determine how best to deal with this and other likely obstacles that emerge. • Kann Let should study the list of activities that the evaluators found to be successful under Phase I and consider how these could be replicated or scaled up under Phase II. In particular, the civilian ceasefire monitoring holds promise. There remains much to be accomplished in this process, but partners thus far have demonstrated capacity not only for managing highly sensitive negotiations, but also for working with Kann Let to be flexible in altering its strategies as necessary and sharing lessons learned with monitoring groups from other parts of the country. • The team’s major actionable recommendation is that Kann Let take very seriously its commitment to inclusivity, among both legitimate armed stakeholders who may or may not have signed the NCA and civil society and conflict-affected populations. In practice, Kann Let’s emphasis on inclusivity will mean staying engaged with stakeholders who appear to be left out of or diminished by the new government’s evolving approach to peace. Kann Let must seek platforms for cross-stakeholder dialogues and learning and must promote forums for advocating to the new government for greater inclusion. • For Phase II, we highly recommend that Kann Let sustain its assumption that achievement of program-level changes under this objective will be difficult, as both partisanship and inexperience in the government will likely unsettle the relationships among legitimate stakeholders in the peace process for months to come. Realistically, given the complexity of the civil wars in Burma, a comprehensive peace agreement—complete with political solutions that involve a definition of “federalism” most can agree upon—are a decade or more off in the future. • We recommend targeted engagement with GoB at the Union and state/region levels, and across the three branches of government, on matters related to the complexity of the peace process. We recommend approaches that expose GoB leadership not necessarily to other countries’ peace processes, but rather to domestic achievements toward peace over the last four years, including the Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 40 wide range of citizens’ aspirations. These activities need to de-emphasize the partisan nature of the current skeletal government approach to peace. In order to sustain Kann Let’s commitment to balance, it should encourage activities that de-escalate the threat perception of the new GoB leadership and stakeholders’ voices who are not necessarily loyalist NLD leaders. • The MTE team suggests that Kann Let use its flexible programming approach to broaden and revisit the focus of its cross-cutting support to media as a tool to promote peace. The connectivity transition in Burma is unprecedented, with Burma experiencing the fastest-ever roll-out of cellular access and being the first straight-to-smartphone market in the world, where phones are largely being used to access Facebook. The uniqueness of the Burma connectivity landscape cannot be overstated and presents promising opportunities for reform as well as emergent dangers. The team encourages Kann Let to work with local organizations researching the impact of Facebook and connectivity, as well as others working on citizen journalism and digital literacy. To date, most donors’ funding to organizations seeking to mitigate conflict resulting from the digital transition have gone to two implementers (mainly Phandeeyar, for large investments, and Myanmar ICT for Development Organization (MIDO)), but there are numerous local groups throughout the country that are aware of the malicious consequences of the digital transformation. These local groups are capable of conducting local research and have ideas for progressive and preventative activities to address these consequences. Examples include Myanmar Fifth Estate, The Seagull, Myanmar Social Science Institute, National Enlightenment Institute, Enlightened Myanmar Research Foundation, Center for Diversity and National Harmony, Myanmar Book Aid Preservation Foundation, and government bodies like the Information and Public Relations Department (which runs thousands of libraries across the country). Note that we are not recommending the abandonment of support to mainstream media, which still has far to go, but rather a more serious recognition of the unanticipated power of social media. • Because the new peace architecture is to be anchored in Nay Pyi Taw (NPT) (not Yangon), we recommend opening an office in NPT where Kann Let can help bridge the knowledge and consultation gap between the new government and the large range of stakeholders mobilized under the previous government. • Kann Let might consider revisiting program-level changes related to increasing awareness and support for peace among non-minority populations living in areas far from recent conflict. In Phase I, Kann Let looked for partnerships for central Burma–based peace-related activities—such as peace education and dialogues across communities—but reported little uptake. Kann Let could not find suitable partners or much interest. The times may have changed, and in accordance with the CSO￾support recommendation to build at least some CSO multi-locational programs from the peripheryinward (instead of Yangon-outward, see below under Question 4), this may be more of a possibility under the new political landscape. • Kann Let should continue to do what it has done well so far, as the ambiguity around the new government’s strategy will likely persist: stay engaged, talk to partners and non-partners, maintain its commitment to regular staff analysis of evolving conditions, err on the side of taking on risk if the process appears to regain momentum, and seize opportunities to link peace-related programming to other kinds of activities. QUESTION 3: TO WHAT EXTENT HAS THE PROGRAM BEEN EFFECTIVE AT REDUCING THE INFLUENCE OF DRIVERS OF INTERCOMMUNAL CONFLICT Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 41 IN BURMA?35 • As OTI and Kann Let expand their view to define ICC in terms not simply related to Rakhine State, we suggest Kann Let expand ICC activities to other potential victims of communal violence, not necessarily defined in religious or sectarian terms. In particular, partners and Burmese experts expressed explicit concern for the potential of anti-Chinese violence to erupt—in Mandalay, perhaps as a result of long-standing land disputes, and in Taunggyi, over land issues as well as the ongoing civil wars and natural resource concessions. These are ICC issues that should be monitored throughout the next phase of Kann Let. • Kann Let should develop a particularly careful ICC strategy and vocabulary in Rakhine State, beyond the obvious politicization of “Rohingya” vs. “Bengali.” The Burmese expert and former Kann Let partner who has researched and written extensively on Rakhine State communal violence strongly argued against giving any activities in Rakhine State a label and using the term “reconciliation” in Burmese or English. He explained: “Rakhines feel strongly that that word signals that ‘Muslims’ will be helped to settle in ‘our ward or village.’ At the same time, Muslims hear it and think it means that they have to give in to all Rakhine Buddhist demands and that if they don’t, they have no future.” • Facebook and other digital platforms must be taken on in a much more serious and systematic way, particularly as mobile providers are likely to be incentivized to make access to it free of charge for most. Burmese citizens, whose formal education system for the last 40 years has suppressed critical thinking skills, are particularly vulnerable to mis- and disinformation, as compared to the citizens of other countries leaping across the digital divide. • Kann Let should consider carefully the evidence of whether the NLD government has demonstrated a well-informed awareness of the complexities that sustain ICC vulnerabilities. To date, the new government has been narrowly focused on narratives and vocabulary (i.e., “Rohingya” and “Bengali”) around Rakhine communities, but there is little evidence that broader anti￾communal-conflict initiatives will be a major priority. For example, interviewees working in this area had no idea where the NLD’s views stood on the “Four Laws” promulgated in 2015. Other partners emphasized the necessity of religious tolerance legislation, which has received some press but has seen little follow-through. It is too early in the new government’s term to conclude one way or another, but the MTE team recommends that Kann Let and OTI/Burma keep these issues on the Embassy agenda and use all methods possible to promote tolerance. • Kann Let should continue its method of working closely with partners throughout the cycle of programming. Staff and SMT need to stay in close contact with former and future partners working in this potentially explosive arena to ensure that the relationships of trust established under Kann Let’s first phase will be sustained or strengthened while also helping partners minimize risk. • The critical impediment of ICC requires Kann Let’s greatest commitment to flexibility in the partnership to protect partners whose risks are among the greatest in Kann Let’s portfolio. Thus far, any successes achieved by Kann Let have been dependent upon programming via locally knowledgeable organizations and limiting USAID/OTI’s branding upon these activities. • Investments in very basic capacity building of local organizations based in rural and peri-urban areas newly targeted by anti-minority, nationalist groups should be considered a priority. In 35 The MTE team is cognizant that a separate, more intensive evaluation of Kann Let’s work on ICC has been commissioned. As a result, we focused squarely on the research question (and subsidiary questions posed), and our analysis is presented in a more concise manner than our responses to questions on civil society and peace. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 42 more remote areas, local partners will need some degree of capacity development, which is not typically an OTI pursuit. However, in this case, what they will really need is some basic guidance about helping leaders and staff move from local knowledge of risks to visions of change, and then to promotion of strategic thinking on how to design activities that connect the two. The MTE team sees these requests for “capacity building” to be almost entirely consistent with the partnership model and not to represent requests for what USAID or other donors mean by “capacity building,” which would be more in the range of organizational development. • Kann Let should advise not only its partners but also Embassy personnel to take care around using the terminology of “reconciliation” in regard to ICC tensions in Rakhine State. We recommend Kann Let convene its partners working in this area to discuss this and other discourse issues that may inadvertently harm pro-tolerance activities and coalition-building. QUESTION 4: DID KANN LET SUCCESSFULLY SUPPORT CIVIL SOCIETY TO ENGAGE IN AND INFLUENCE THE REFORM PROCESS? • Kann Let’s future program should have better engagement with CSOs outside Yangon, which for ten or more years have included small grassroots organizations. These organizations could serve as an effective countermeasure to emerging issues. For instance, the complex nature of identity and the multiple underlying root causes of intercommunal tensions are not always readily apparent in what appear to outsiders to be peaceful towns and small cities. Is there a way for Kann Let to have significant impacts at the village and neighborhood levels? Those locations might include, for example, peri-urban new settlements and outlying villages surrounding potential flashpoints in Mandalay or Taunggyi, for which local Mandalay region–based or southern Shan State–based (respectively) CSOs have the best expertise and about which they expressed high levels of concern about the potential for violence.36 The Seagull’s pilot project (BTI341) in Chanmyathazi, Mandalay, would be a model for expanding pilots of highly localized interventions in areas being overlooked for ICC and conflict risks that may be home to emergent reform opportunities that otherwise may not be accessible. CSOs in ethnic states and non-Yangon regions have very different visions, missions, capacities, and constituencies. These alternative perspectives may help to promote new kinds of networks that advance Kann Let’s four program objectives. In particular, on some issues, such as inter communal tensions, state/region legislative reform or peacebuilding, CSOs from other parts of the country have opportunities to lead from their grassroots positions. We suggest consideration of building national programs from the periphery inward (toward the center), or at least empowering CSOs from smaller urban centers or more remote locations to scale their programs to have influence in central debates. • Kann Let must maintain its close working relationship with civil society. The evaluation team heard significant negative feedback about USAID’s larger-scale funding to an international NGO to build capacity in civil society. The MTE team especially recommends that Kann Let look to the low￾key kind of “capacity building” that many small, new CSOs have requested. Kann Let’s flexible, interactive, context-sensitive practices alone may serve their needs and help to continue to support partners as they evolve out of the historical repression they knew for so long. 36 These locations are not comprehensive, but rather reflect the evidence gathered on our short field visits to Mandalay and Taunggyi. There are probably areas overlooked for potential sources of intercommunal tensions in northern Shan State, central Shan State (Pao areas are rife with anti-Islamic literature and events), many of the regions, and probably Tanintharyi, Kayin, and Mon states. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 43 • With the new NLD-led government in power, Kann Let needs to be on alert about the potential for the closure of space for civil society to influence reform content and direction. The evaluation team was concerned that some program staff and grantees would assume that a democratically elected government creates a wide open door for civil society to enter into national, state/region, and local political affairs. However, most civil society actors are feeling increasingly concerned about losing the influence they fought hard to gain under the last government. It is too early to evaluate the accuracy of the alarm we heard expressed. These concerns are plausible, however, in light of developments and non-developments since the November 2015 election and should be taken seriously by Kann Let in planning Phase II. To date, most civil society interviewees consider that State Counsellor and NLD leader Aung San Suu Kyi has demonstrated little patience for or interest in collaboration with civil society. Instead, she has indicated that her electoral mandate (57% of the popular vote and 80% of parliamentary seats) justifies her party’s determination of national priorities, regardless of concerns raised by the media, civil society, or others. The MTE team recommends scenario planning in Kann Let internal assessments and meetings around the potential closure of space for civil society to seek funding, to deliver services, to engage with the government, and other possible developments. We heard one reasonable scenario in different versions, but it is essentially as follows: The new government appears intent upon dramatically changing funding channels. Many CSOs are concerned that their donors would shift civil society support to the popularly elected government directly. What could make it worse is that representatives of the new￾NLD government have told some interviewees that they prefer the modality of Chinese civic society operations in a sense that all funds go to the government for redistribution to government-friendly CSOs (essentially, new Government Organized Nongovernmental Organizations (GONGOs)). Again, it is far too early in the new government’s tenure to make any firm conclusions, and hence other scenarios should also be engaged. Kann Let must listen carefully to partners in civil society, monitor the twists and turns of the relationship between the new government and civil society, and keep this issue on the agendas of internal meetings and assessments. • Kann Let should reach out to its partners and others in civil society to explore opportunities to facilitate what will be very hard conversations among party, media, and CSO stakeholders about their changing roles, now that a credible election has led to a transfer of power to a more legitimate, mostly civilian government. Like the NLD, CSOs have spent years in opposition to an authoritarian government. Dialogues, gatherings of partners, and other facilitated conversations would be appropriate ways to promote a home-grown set of answers to hard questions, such as How does the presence of a more legitimate government change CSO’s agendas, self-perceptions, networks, and mindsets? In Phase II, maybe it is also time to consider support directly to the government to help facilitate a better understanding of the necessity of a strong civil society in a democracy. QUESTION 5: HOW HAS OTI CONTRIBUTED TO THE ADVANCEMENT OF U.S. FOREIGN POLICY IN BURMA BEYOND THE KANN LET PROGRAM ITSELF? • If OTI is interested in contributing to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself, it should seek expertise and deep knowledge for its senior management team. • OTI should recognize that if the OTI Country Representative is necessarily playing a larger representational and advisory role in the Mission that further management support for the OTI office may be needed in order to ensure sufficient engagement and analysis with program partners. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 44 ANNEXES Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 45 ANNEX I: EVALUATION SCOPE OF WORK PDQIII Task Order #10 Activity #9: Burma Midterm Evaluation Scope of Work PERIOD OF PERFORMANCE: February 5 to July 15, 2016 BACKGROUND Burma faces complex, simultaneous, and multi-dimensional transitions from authoritarian rule to democracy, from armed conflict to peace, and from a centrally-controlled, planned economy to market￾led economic policies. The country is also stepping out of isolation to integrate into the regional and broader international community. With a few notable exceptions, the reform and peace processes have been directive and top down with limited public participation. In September 2012, USAID’s Office of Transition Initiatives (USAID/OTI) launched the Kann Let program in Burma to increase participation and inclusion in reform and peace processes as well as to address critical impediments to the transition. Consistent with the U.S. Government’s (USG’s) Principled Engagement policy, the Integrated Country Strategy approved in January 2014, USAID/OTI Burma program broadly seeks to deepen and sustain reforms and foster legitimate process for pursuing peace. In order to support this goal, USAID/OTI has increased participation and inclusion in reform process and addressed critical impediments to the political transition, specifically reform and peace issues necessary for Burma to become a truly inclusive, multi￾ethnic union. In March 2015, Kann Let refined its program objectives, effectively creating new stand-alone program objectives for the four most significant “critical impediments.” USAID/OTI aims to deepen and sustain reforms and foster legitimate processes for pursuing peace through four Program Objectives: 1. To facilitate public engagement in the reform process; 2. To reduce the influence of the drivers of intercommunal conflict; 3. To enhance the ability of key stakeholders to engage in the peace process; and 4. To enable survivors of explosive ordnance and persons with disabilities (PWD) in conflict-affected areas to participate in social and economic life. PURPOSE The purpose of this activity is to conduct an independent performance evaluation of USAID/OTI’s Kann Let program in Burma to date. The findings will evaluate the strategic approach, relevance, and impact of Kann Let efforts to deepen and sustain the ongoing reform process in Burma and will provide a product that concisely explains achievements and lessons learned. The final product will be used to inform future USG assistance and OTI programming in Burma and will be shared with the USAID/Burma mission, the U.S. Embassy in Yangon, local stakeholders, and other interested groups. On January 5, 2016, USAID/OTI issued a Request for Task Order Proposals (RFTOP) under the SWIFT IV indefinite quanity contract to procure a follow-on task order to continue programming in Burma for two or three additional years. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 46 EVALUATION QUESTIONS The report shall seek to answer the following evaluation questions: 6. Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence? In answering this question, the evaluators should consider whether the program appropriately adapted its strategy as the context evolved and seek to determine whether these same areas are likely to merit similar emphasis in OTI’s future programming in Burma. 7. Has the Kann Let program employed the right tools and approaches to support the peace process in Burma? In answering this question, the evaluators should consider which tools and approaches have been the most and least effective at facilitating engagement by key actors and stakeholders, whether the program needs any additional tools or approaches, and if the program has met the expectations of key actors in the peace process. 8. To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma? In answering this question, the evaluators should consider which approaches have been most effective at changing the attitudes and behaviors of target groups, whether there are lessons or findings from other fields (e.g., human psychology, media literacy) that could be incorporated to enhance the effectiveness of future interventions, and whether there are lessons for other programs seeking to promote tolerance and/or counter dangerous speech that is a catalyst for violence. 9. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing reform process? In answering this question, the evaluators should consider whether the program’s support was timely, targeted and critical, identify any missed opportunities, and offer recommendations for ways OTI can be more proactive as the reform process continues. 10. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? In answering this question, the evaluators should seek to identify lessons that can be learned from the close collaboration between OTI and other parts of the U.S. Mission to Burma. METHODOLOGY This evaluation will be non-experimental and largely qualitative in nature, but mixed methods may be used as appropriate. In answering the evaluation questions, the contractor shall utilize data that is disaggregated and analyzed by sex, whenever such data is available. Methodological specifics will be agreed upon among the evaluators, OTI/Burma, and OTI/Washington, and the evaluators are encouraged to suggest creative approaches. OTI’s activity database is a rich source of information on individual projects and should be utilized by the evaluation team. Possible methods for the evaluation include: Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 47 • Facilitated workshop with key program staff to reflect on program implementation, challenges and successes; • Field visits to the implementation areas; • Interviews with key program stakeholders, including U.S. Embassy and USAID staff, community leaders, and beneficiaries; • Focus group discussions and interviews with beneficiaries, grantees, and others; • Direct observation; • Documentation review, e.g., quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments, and Program Performance Reviews (PPRs), perception surveys and other existing data, and OTI/Burma’s activity database. TEAM COMPOSITION The evaluation team should be comprised per the specifications described below. The positions to be filled by Social Impact require USAID/OTI concurrence of the proposed candidates. In addition to two evaluators engaged by Social Impact, OTI intends to field a staff member to participate in the evaluation as a fourth team member. • Two senior evaluation specialists with research and/or evaluation experience in humanitarian assistance and political transition programming. At least one of the specialists should have extensive experience related to and a deep understanding of Burma’s political system, reform process, and modern history. One evaluation specialist shall be designated the evaluation team leader and will be responsible for liaising with USAID/OTI, the preparation of all deliverables, and leading briefings with relevant stakeholders. • One mid-level evaluation specialist with research and/or evaluation experience in complex political and/or social change. This evaluation specialist should be a host country national and have local research and/or evaluation experience. Knowledge of OTI-type programming is preferred. The mid-level evaluation specialist will support the team leader and participate in the field review, interviews, the draft and final evaluation reports, and debriefs in the field only. In addition, the team will require the services of a local translator with Myanmar and ethnic language skills. TIMELINE and LEVEL OF EFFORT (LOE) The field work of the evaluation will take place over approximately three weeks. The team will travel to Burma on/about April 23 2016 and depart on/about May 15, 2016. In addition, the team will have a total of two weeks of LOE in advance of departure for reading and some Washington-based interviews in January and February. This activity will also include three weeks of LOE for writing and completion of deliverables. In total, OTI envisions up to 50 days of LOE per member of the evaluation team per the timeline below: Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 48 Dates* Task LOE March 1–31, 2016 Reading on Burma context. Key materials to be provided by OTI. Key Washington-based interviews. 10 days March 2016 (precise date TBD) Kick-off meetings/calls with OTI team to refine evaluation questions, methodology, travel and logistics; database training. 2 days April 23–24, 2016 Travel to Burma 2 days April 25–May 13, 2016 Field-based interviews and analysis, debriefs to senior management, staff and U.S. mission as appropriate. 17 days May 15–16, 2016 Travel from Burma 2 days May 17–29, 2016 Report writing; submit first draft to OTI by COB June 3, 2016 10 days June 16–27, 2016 Receive comments from OTI by COB June 23, 2016. Submit final report by July 7, 2016. 5 days June 14 – July 15, 2016 Debriefs with OTI, DAI and USG interagency as determined by OTI. 2 days TOTAL: 50 days LOE * All dates listed above are illustrative and subject to change. A six-day work week is approved during the field work for this activity. TASKS and DELIVERABLES The evaluation team, under the direction of the team leader, is responsible for completing the following Tasks and submitting the following Deliverables: • Interviewee list (to be developed in collaboration with USAID/OTI); • Evaluation plan, including summary of proposed data collection methods; • Summary of out-briefing to USAID/OTI before departing Burma; • Draft evaluation report for comments (content and structure), due June 3, 2016; • Debrief summary (handout or presentation); • Debrief(s) with USAID/OTI, DAI and USG interagency as determined by OTI; and • Final evaluation report, due July 7, 2016. All deliverables will require USAID/OTI concurrence prior to their finalization. The final evaluation report will be posted on the USAID’s Development Experience Clearinghouse (DEC) website by Social Impact. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 49 Social Impact will be responsible for the following logistical matters with respect to the three evaluation specialists engaged through this activity: • Social Impact is responsible for all scheduling, arranging meeting locations, and confirming times/dates/locations with the team and interviewees and grantees. OTI and DAI will provide references and can assist with scheduling as needed. While the evaluators are in country, OTI’s partner DAI will make available a conference room in its offices for the evaluators to use in Yangon. However, meetings in hotel lobbies, bars, and gardens are sometimes preferable and convenient. USAID/OTI will facilitate access to the U.S. Embassy for meetings with other USAID and Embassy officials, and arrange conference rooms in the Embassy. • Social Impact shall arrange and purchase all international and U.S. travel. TOCOR concurrence is required for all international travel not included in the original activity budget; • Social Impact shall provide per diem (lodging and M&IE) for the evaluation team in Burma; • Social Impact shall fund in-country air travel and ground transportation; • Social Impact will obtain visas;37 and • Social Impact and the evaluation team will work with OTI and implementing partners to arrange interviews in Washington, DC and in Burma. USAID/OTI will arrange international travel, provide per diem, secure lodging, fund in-country air travel, and obtain a visa for the OTI member of the evaluation team. Social Impact will, however, be responsible for including the OTI staff member when making ground transportation arrangements for the team while in Burma. REPORT STRUCTURE The evaluation report should adhere to USAID guidelines and be structured as follows: • Cover Page with photo • List of Acronyms • Table of Contents, which identifies page numbers for the major content areas of the report. • Executive Summary (2 to 3 pages): should be a clear and concise stand-alone document that gives readers the essential content of the evaluation report, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. Thus, the Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of objectives and intended outcomes; areas of meaningful under- or over-achievement. • Methodology: Describe the data collection methods used including strengths and weaknesses, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken. This section should also address constraints and limitations of the evaluation process and rigor, including what can and cannot be concluded from the evaluation. All actual or potential conflicts of interests among members of the evaluation team should be noted in this 37 DAI is able to assist in arranging business visas for the evaluation team, if required. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 50 section. In accordance with ADS 203, the report should also state why a performance evaluation was conducted in lieu of an impact evaluation. • Findings: The evaluation team should determine how best to organize this section based on the evaluation questions. • Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to discuss the data and results, and make concrete recommendations for project improvements/changes, pull out organization lessons learned, and generally comment on data and results. • Annexes: List of stakeholders interviewed with number and type of interactions; interview data on AER questions, the scope of work; qualitative protocols developed and used; any data sets can be provided in electronic format; other special documentation needed. POINT OF CONTACT The OTI/Burma Program Manager in Washington will serve as the point of contact for overall coordination of the Washington meetings. There will be some occasions where s/he arranges the meeting and location and others where s/he may provide contact information for the consultants to arrange their own logistics. While in Burma, the OTI team in Yangon will identify a point of contact for in-country logistics and scheduling. The TOCOR will remain the point of contact for all technical direction and requests requiring formal concurrence. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 51 ANNEX II: EVALUATION METHODS AND LIMITATIONS The major limitation to this methodology is time and availability. The team has a limited time frame to research the evaluation questions, and it is during a month when many developments are expected (or may capture the time of key informants). The schedules of those active in the many kinds of reforms that continue to unfold are unrelentingly busy. Moreover, traffic in Yangon particularly, but also other modes of travel, will undoubtedly introduce unexpected and numerous delays. It is also probably worth noting the likelihood of serious weather problems in Burma during the month of May (as has been widely reported to accompany the El Nino year underway). Biases include the challenge of triangulating data from key informant interviews with documentation and reporting from OTI/Burma. As a result of the limitations above, the team will have to self-consciously be aware of the possibility that key sets of stakeholders may be unavailable. The inclusion in the team of evaluation experts well-versed in domestic Burmese affairs is our main mitigation strategy against this bias; the same is true for key informants from OTI and USAID in Washington, DC, as the team member from OTI/DC will aid in mitigating against oversampling of one point of view and missing other important insights. Response biases can be expected from all key informants: from OTI, DAI and USAID staff, whose career incentives and personal senses of accomplishment may color responses; from partners, who may be concerned about whether their responses may hurt or help their chances at future funding; and from Burma experts, who may only make themselves available if they have a particular interest and may have difficult-to-discern agendas. Midterm Evaluation: USAID/OTI’s Contribution to Burma’s Nascent and Unexpected Transition, 2012–2016 52 ANNEX III: DATA COLLECTION INSTRUMENTS Documentation review and semi-structured interviews with key informants were the most effective method of collecting data responsive to the research questions. The composition of the team allows for members to craft flexible, intuitive and streamlined approaches among the key informant interviews. Such a methodology is particularly useful in a midterm performance evaluation as it avoids the pitfall of being overly reductionist or apolitically technical like many other evaluation approaches. Below is a comprehensive list of questions. However, in general, the MTE team reviewed interviewee backgrounds before interviews (activity reports in the case of partners; professional backgrounds in the case of USG and OTI/DAI staff; and known information about “experts”) before determining how to initiate the conversation, and then probe toward relevant research questions. Given our time limitations, we privileged inquiry related specifically to the issues that the Scope of Work (SOW) raised the research questions about KL’s performance; we did not attempt to press for measures of impact, although we usually sought examples of achievements and challenges. For some stakeholders, we may need to explain some or part of the following: USAID/OTI aims to deepen and sustain reforms and foster legitimate processes for pursuing peace through four Program Objectives (POs): 1. To facilitate public engagement in the reform process; 2. To reduce the influence of the drivers of intercommunal conflict; 3. To enhance the ability of key stakeholders to engage in the peace process; and 4. To enable survivors of explosive ordnance and persons with disabilities (PWD) in conflict-affected areas to participate in social and economic life. Evaluation questions (EQs): 1. Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence? 2. Has the Kann Let program employed the right tools and approaches to support the peace process in Burma? 3. To what extent has the program been effective at reducing the influence of drivers of intercommunal conflict in Burma? 4. To what extent has the program effectively supported civil society organizations to engage in and influence the ongoing reform process? 5. How has OTI contributed to the advancement of U.S. foreign policy in Burma beyond the Kann Let program itself? Crosscutting Themes OTI/DAI/USAID Informants Relevance and Design * Has the program identified the right critical impediments to the advancement of peace and reform in Burma, specifically those within its ability to impact or influence? * Has Kan Lett developed context-appropriate tools for supporting inclusivity in the peace and political reform processes, and for reducing the influence of drivers of intercommunal conflict? * Did the planned activities advance Kann Let objectives and program level changes? What activities are or were most or least responsive to the goals and desired program-level changes? * What are the strengths and weaknesses of the Kann Let program design in the transitional Burma environment? * What have been the major challenges to implementing the Kann Let objectives? * How has Kan Lett contributed to furthering US foreign policy goals in Burma? Please give examples. * How have US policies helped or hindered in the design and implementation of Kann Let? Please give examples. Effectiveness * How were partners identified for activity implementation? * What outcomes or outputs has Kann Let achieved? * What approaches or activities were the most difficult to evaluate in terms of program level changes on target issues? * What factors have contributed to achieving or not achieving program level changes sought by Kann Let? * What aspect of the Kann Let program has been the most successful in responding to events unfolding in Burma? * Have you documented the effects of the activities on behavior change among OTI/DAI staff, partners and beneficiaries? * (For Kann Let staff) What staff development initiatives have helped you do your job better? What other kinds of training would you like to receive to improve your performance? * What has Kann Let been able to accomplish that other kinds of programs and funding could not? Give examples. * What Kann Let activities do you recommend we study carefully, for lessons on effectiveness? * What US policies and mission/OTI/DAI management practices help or hinder Kann Let effectiveness? * What advice about enhancing program effectiveness would you offer to OTI, DAI, the US mission, and policymakers in Washington, DC? Crosscutting Themes OTI/DAI/USAID Informants Partnership and sustainability * What was the role of each set of actors (OTI, DAI, USAID, embassy staff, partners/grantees) in the process of promoting Kann Let goals? * What strengths and weaknesses did different partners contribute to enhanced collaboration, effectiveness, learning and sustainability? * In hindsight, should roles and responsibilities have been assigned differently? If so, how and why? * What are the strengths and weaknesses of the close collaboration among OTI, other parts of the US mission, and DAI for partnership development and sustainability? * What activities monitoring and evaluation mechanism has been implemented by DAI? Did it work properly? Why or why not? How could monitoring and evaluation of activities, program level changes and advancement toward objectives be strengthened? * How well did OTI perform its supervisory role? Why do you conclude this? How useful was OTI in the implementation and monitoring of Kann Let operations? What made it more or less useful? Please give examples. * What activities do you recommend we study carefully, for lessons on building sustainable and collaborative partnerships in complex transitions? ANNEX IV: SOURCES OF INFORMATION Number of subjects for semi-structured interviews: Yangon Mandalay Taunggyi KL partner 17 5 3 USG Staff/OTI/DAI staff 16 3 0 Burma experts 7 6 6 Please note that given the rapidly changing environment, some of the interviewees qualify under multiple categories. List of documents reviewed: 1. Interview list for midterm evaluation interviews 2. Kann Let Org Chart 3. Kann Let Staffing Turnover Data Graphic 4. Midterm Evaluation Kick­off Call Notes 5. DAI Quarterly Reports ­ 2013 & 2015 (Folder) USG (Folder) 1. Integrated Country Strategy 2. USG Peace Support Strategy (sub­folder) 6. USG ICC Strategy (sub­folder) 7. USG Rakhine Strategy (sub­folder) 8. Gender Strategic Paper 2015 9. Elections Scenarios PPMP (Folder) 10. PPM Events Inventory (Excel) 11. Strategy Evolution Paper Database Reports – Old DB prior to New Objectives and Migration to WBDB (sub­folder) 12. By OLD Objectives 13. Summary by Objective Jan 2014 14. Summary by Objective Jan 2015 15. Summary by Objective March 20, 2015 16. Graphic – Critical Impediments 17. Graphic – Program Level Changes 18. Critical Impediment: Census 19. Critical Impediment: Constitution 20. Critical Impediment: CSO Capacity 21. Critical Impediment: GoB Capacity 22. Critical Impediment: Inclusive Identity 23. Critical Impediment: InterCommunal Conflict 24. Critical Impediment: Lack of Inclusive Identity 25. Critical Impediment: Rakhine Situation 26. Critical Impediment: Lack of Public Information 27. Critical Impediment: Restrictive Laws 28. Critical Impediment: Victims Assistance 29. PLC: Womens Participation 30. PLC: Youth Engagement 31. PLC: Citizen Participation in Decision­making 32. PLC: Culture of Debate & Civic Responsibility 33. PLC: Access to Info 34. PLC: Border Org Engagement 35. PLC: Human Rights Decision Docs (sub­folder) 36. USG Scoping Mission Report – April 2012 37. OTI Go Memo – 2012 38. OTI/Burma Extension Memo for Follow­on program ­ 2015 Management Reviews (sub­folder) 39. Management Review Final Report – May 2013 40. Management Review Final Report – May 2015 Political Timeline 41. Political Timeline 2012 42. Political Timeline 2013 43. Political Timeline 2014 44. Political Timeline 2015 45. Political Timeline 2016 Field Visit Activity Reports 46. HPA­an Activities 47. Loikaw Activities 48. Mandalay Activities 49. Meiktila Activities 50. Taunggyi Activities Program Development Notes (Sub­folder) 51. PDN 02 Congressional Notifications 52. PDN 03 CCF and War Victims Funding 53. PDN 05 Large Meetings & Forums 54. PDN 07 Civic Engagement 55. PDN 08 Updating Critical Impediments 56. PDN 09 CTBPP 57. PDN 10 RA December 2014 58. PDN 11 Research 59. PDN 12 Ethnic National Conferences 60. PDN 13 RA March 2015 61. PDN 14 RA October 2015 Note: the Program Document was also updated in February 2015 for use at an OTI­wide Training. That version reflected the CR’s proposed new Program Objectives. For purposes of reviewing OTI’s strategy, the April 2015 update includes the new Program Objectives and related changes to the TOC as well as small context updates. The April 2015 version reflects the Team’s work and analysis from the March 2015 RA. 62. OTI/Burma Program Document ­ April 2013, August 2013, December 2013 63. OTI/Burma Program Document – December 2014 64. OTI/Burma Program Document – February 2015 (see note above) 65. OTI/Burma Program Document – April 2015 66. Program Performance Reviews (sub­folder) 66. Program Performance Review Final Report – November 2013 67. Program Performance Review Final Report ­ January 2015 Rolling Assessments (sub­folder) Note: Please see the Program Development Notes for Notes and Outcomes of Rolling Assessments. We use these notes to formally document guidance and direction to local staff. 68. Mini Rolling Assessment Results (FBO) – May 2013 69. Rolling Assessment Notes – September 2013 70. Strategy Review Session Final Report + Annexes (folder) – February 2014 71. Rolling Assessment Notes – June 2014 Reflections (sub­folder) 72. Ethnic National Conferences (Unformatted) 73. Independent Media Support (Unformatted) Association Law (Unformatted) 74. DVB Debates ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Mary P. Callahan Title Independent Consultant Organization Evaluation Position? XX Team Leader Team member Evaluation Award Number (contract or other instrument) PDQIII TASK ORDER #10: Q011OAA1500012 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) OTI Burma, as implemented by DAI I have real or potential conflicts of interest to disclose. Yes XX No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 2052