EVALUATION REPORT MID-TERM PERFORMANCE EVALUATION OF USAID/BURMA PROMOTING RULE OF LAW PROJECT February 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Malcolm Russell-Einhorn and Aung Tun for Social Impact, Inc. USAID Contract AID-486-I-14-00001; Task Order AID-492-TO-16-00001 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. PHOTO: HTTP://WWW.IRRAWADDY.COM/WP-CONTENT/UPLOADS/2012/12/6.-COURT-PHOTO.JPG USAID/Burma Promoting Rule of Law Mid-term Evaluation ii CONTENTS Acronyms............................................................................................................................................................................ iii I. Executive Summary.................................................................................................................................................. v II. Project Background..................................................................................................................................................1 III. Evaluation Purpose and Questions.......................................................................................................................3 IV. Evaluation Methods and Limitations ....................................................................................................................4 V. Findings and Conclusions........................................................................................................................................8 VI. Recommendations..................................................................................................................................................23 Annexes.............................................................................................................................................................................27 Annex I: Evaluation Statement of Work................................................................................................................28 Annex II: Data Collection Protocols and Questions..........................................................................................35 Annex III: List of Background and Desk Review Documents...........................................................................42 Annex IV: Grantee Trainings and Workshops through Base Period .............................................................45 Annex V: Evaluation Team........................................................................................................................................46 Annex VI: PRLP-Supported CSO Work: Areas of Accomplishment and Capacities Strengthened .......47 Annex VII: Photos of Project Sites..........................................................................................................................49 Annex VIII: Evaluation Work Plan...........................................................................................................................52 Annex IX: Evaluation Design Matrix.......................................................................................................................61 Annex X: Disclosure of any Conflict of Interest.................................................................................................66 Tables Table 1: Summary of Interviews and Focus Groups*................................................................................................5 Table 2: Legal Aid Services Provided By Grantees Cumulative Through 3-Year Base Period)....................14 USAID/Burma Promoting Rule of Law Mid-term Evaluation iii ACRONYMS A2JI Access to Justice Initiative AMEP Activity Monitoring and Evaluation Plan ASEAN Association of Southeast Asian Nations CDCS Country Development Cooperation Strategy CEDAW Convention on the Elimination of all Forms of Discrimination Against Women CLE Clinical Legal Education COR Contracting Officer's Representative CSO Civil Society Organization DANIDA Danish International Development Agency DRG Democracy, Human Rights, and Governance EQUI Evaluation, Quality, Use, and Impact ER Evaluation Report ET Evaluation Team EWP Evaluation Work Plan FGD Focus Group Discussion GAD General Administration Department GBV Gender-based Violence GOB Government of Burma IBA International Bar Association ICJ International Commission of Jurists ILAM Independent Lawyers Association of Myanmar IP Implementing Partner IT Information Technology JICA Japan International Cooperation Agency KII Key Informant Interview KWEG Kayin Women’s Empowerment Group LGBT Lesbian, Gay, Bisexual, and Transgender M&E Monitoring and Evaluation MP Member of Parliament NDI National Democratic Institute NGO Nongovernmental Organization NLD National League for Democracy OSCU Office of the Supreme Court of the Union PA Project Assistant PD Project Director PM Project Manager PMEP Project Monitoring and Evaluation Plan POVAW Protection from Violence Against Women Law PRLP Promoting Rule of Law Project QA Quality Assurance ROL Rule of Law SI Social Impact, Inc. USAID/Burma Promoting Rule of Law Mid-term Evaluation iv SOW Statement of Work TA Technical Assistance TIP Trafficking in Persons TL Team Leader UAGO Union Attorney General’s Office UNDP United Nations Development Program USAID United States Agency for International Development WON Women’s Organization Network USAID/Burma Promoting Rule of Law Mid-term Evaluation v I. EXECUTIVE SUMMARY Social Impact, Inc. is pleased to present this mid-term performance evaluation of the United States Agency for International Development (USAID)/Burma Promoting Rule of Law Project (PRLP). The evaluation’s main audience is the Mission staff in Burma, particularly the Democracy, Human Rights, and Governance team, as PRLP enters its final twenty months of implementation. It is also intended to be used to inform future USAID/Burma assistance, particularly as the Mission embarks on its first Country Development Cooperation Strategy (CDCS). I. Project Background In 2012, after the National League for Democracy and other non-military parties won seats in Burma’s Parliament for the first time under the 2008 constitution, USAID and other international donors perceived a need and opportunity to support the country’s prospective democratic transition. The project design set forth two complementary objectives: (1) to promote more effective, accountable, and accessible justice sector institutions; and (2) to increase legal literacy, access to justice, and participation of marginalized populations in target regions/states. The resulting PLRP, awarded in October 2012, is a five-year, $12.1 million project focused on support to the Office of the Supreme Court of the Union (OSCU), the Union Attorney General’s Office (UAGO), and the Union Parliament, while simultaneously advancing access to justice and legal awareness work and building the capacity of civil society organizations (CSOs) committed to such work. The scope is nationwide. II. Evaluation Purpose and Questions This mid‐term, formative evaluation is intended to help determine what project components are or are not working well and why, and to make modifications and mid‐course corrections, if necessary, to guide PRLP over its second half. This report answers the following questions posed by the Mission: 1. Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma? 2. Was the Project able to respond to new opportunities to advance its stated objectives? 3. To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? 4. Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? 5. How effective are the Project’s activities in coordinating with other USAID and other donors’ programs? III. Evaluation Methods and Limitations The evaluation team (ET) is composed of Mr. Malcolm Russell-Einhorn (team leader and senior evaluation specialist), Mr. Aung Tun (local evaluation specialist), and Mr. Than Htike Oo (interpreter, translator and logistician). The evaluation utilized a mixed-methods approach that included multiple qualitative data collection methods and an analysis of quantitative data. The data collection methods included:  Documentation review of project documents as well as non-project related secondary sources.  Key informant interviews with 110 individuals (some in group settings), encompassing 53 men and 57 women. These included USAID/Burma and PRLP personnel, PRLP grantees, representatives of several PRLP governmental and partner nongovernmental organizations (NGOs), several donor representatives, and several experts and observers within and outside of Burma.  Field visits and site observations with a pilot township court in West Yangon, pilot township and district courts in Mawlamyine, a pilot township court in Hpa-an, a pilot Attorney General law USAID/Burma Promoting Rule of Law Mid-term Evaluation vi office in Mawlamyine, justice (legal aid) centers in West Yangon and Mawlamyine, and the OSCU and UAGO in Nay Pyi Taw.  Focus group discussions with legal awareness trainees from two PRLP grantees in Hpa-an (two men and four women), with justice center attorneys and paralegals in Mawlamyine (four men and three women), and with law officers provided PRLP media relations/public information training at UAGO in Nay Pyi Taw (five men and five women). The ET made attempts whenever possible to ensure a gender balance among interviewees, and that interview questions addressed gender issues as per PRLP’s revised Gender Action Plan. Site visits were selected in consultation with both USAID/Burma and PRLP. Methodological and logistical limitations to the evaluation design included possible response bias, recall bias, selection bias due to non-random sampling, and the brevity of the data collection period (under three weeks); however, a substantial number of interviews were conducted according to a rigorous schedule (involving a total of 159 individual participants). Information and perspectives obtained were triangulated wherever possible through one or two additional interviews/sources. IV. Conclusions Evaluation Question 1: Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma 1. PRLP successfully engaged with a wide range of key justice system stakeholders, including the OSCU, several pilot courts, the leadership of UAGO, the Parliament, and a very wide range of CSOs, including legal aid organizations, the nascent Independent Lawyers Association of Myanmar (ILAM), and several groups engaged in citizen legal awareness and/or paralegal work. In particular, PRLP established a very solid working relationship with the OSCU, eliciting that institution’s collaboration on a three-year strategic plan, and an innovative pilot court initiative that opened the judicial system to greater public scrutiny and opportunities to improve access to justice. 2. The pilot court initiative acquired momentum in a short period of time, but lacks metrics to properly evaluate efficiency improvements and justice/equity concerns. Serious questions exist about whether the initiative’s existing metrics capture meaningful efficiency improvements, and whether stronger indicia of procedural fairness need to be introduced to accurately measure impact. 3. Whatever bundle of metrics is utilized, pilot court data need to be interpreted carefully and put it in their proper context. Case management efficiency improvements are only meaningful when they presuppose a just and respected judiciary. Given deep structural impediments in police and prosecutors’ ability to meet the pilot courts’ performance improvement standards, USAID/Burma and PRLP should calibrate expectations and look for improvement in areas where courts’ manageable interest is greater. That could include certain indicia of transparency, access, and equity (particularly regarding the treatment of women), even where overall standards of fairness may be adversely affected by the actions and behavior of police and law officers (and many judges). 4. The strengths of PRLPs civil society support and emerging work on issues of gender-based violence (GBV) and trafficking in persons (TIP) offer an opportunity to engage with the pilot court program in a more focused. Growing work on GBV and TIP opened up possibilities for broader and deeper engagement with a number of different state and civil society justice sector stakeholders. This work offers an opportunity to engage with the pilot court program in a more focused and meaningful way, promoting women’s empowerment generally with regard to female victims, witnesses, relatives, defendants, lawyers, and court staff. USAID/Burma Promoting Rule of Law Mid-term Evaluation vii Evaluation Question 2: Was the Project able to respond to new opportunities to advance its stated objectives? 1. PRLP responded effectively to multiple new opportunities to meet partner needs and advance its objectives. PRLP’s successful relationship-building as well as the quality and responsiveness of its work created new opportunities for engaging with the OSCU and UAGO, along with many different international NGOs and CSOs. These opportunities created a wide range of entry points for collaboration, ranging from the pilot court and law office initiatives with the OSCU and UAGO, respectively, to critical PRLP support for a national paralegal network and a policy advocacy coalition of 40 CSOs known as the Access to Justice Initiative. 2. Even where some new opportunities may appear at first to be one-off interventions, PRLP’s capacity-building and ‘connector’ work with and among various institutions created conditions for potentially more sustained development and collaboration in the years ahead, particularly within the CSO community. PRLP facilitated the development of organizations, coalitions, and/or processes that can generate new work methods and ways of collaborating in the construction of a new justice system infrastructure. For example, support to ILAM is a major step forward in creating a new, more open and encompassing platform for legal professionals—several of whose standing committees (including one on human rights) have the potential to advance reforms in the years to come. Perhaps the most impressive opportunities seized by PRLP are those involving facilitation of public consultation and evidence-based policymaking, which could prove transformative in a country accustomed to secrecy and compartmentalization. Evaluation Question 3: To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? 1. PRLP utilized a combination of grants funding and training to leverage a mix of access to justice, legal awareness, and research and policy advocacy activities on a large scale in a highly impoverished legal environment. This work was accompanied by strongly committed PRLP capacity-building assistance, which was highly appreciated by CSO grantees. This strong counterpoint of demand-side activities under Objective 2 is important if institutional reforms under Objective 1 are to gain longer-term traction with elites and entrenched bureaucrats wary of change. 2. Gaps in PRLP’s data collection tools exist, which hamper efforts to evaluate the immediate and intermediate results of PRLP and grantee access to justice and legal awareness work, particularly related to beneficiary satisfaction and understanding. Changes in PRLP’s Activity Monitoring and Evaluation Plan (AMEP) should help provide a fuller picture of progress in these areas. 3. Certain of PRLP’s legal aid and legal awareness activities feature training, outreach, and other work modalities that deserve additional study and evaluation to help guide future grant-making and other activities in these two areas. 4. PRLP did not provide significant support to informal and customary dispute resolution, even though the vast majority of disputes in Burma are handled informally, and these processes can often be improved through better information and representation provided to participants. 5. PRLP grants were limited in their geographic reach and diversity, although this was recognized and planned for, and plans are in process to award grants in more regions in the final two years. 6. Legal reform and justice work in Burma suffers from a lack of useful media coverage. As a discredited and mistrusted backwater of Burma society, the justice system lacks the kind of amplification that media coverage can provide. PRLP has thus far not supported such work, which USAID/Burma Promoting Rule of Law Mid-term Evaluation viii could involve more instruction in media relations for CSOs, but dramatically different and more rigorous training on legal topics for journalists. Evaluation Question 4: Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? 1. A thoughtful Gender Action Plan informs PRLPs technical work and focuses PRLP’s activities in several areas—particularly assistance to women on GBV and TIP issues—that leverage PRLP’s comparative advantages in legal aid and legal awareness-supported activities. 2. Due to the short duration of such work and gaps in data collection and analysis, it is difficult to ascertain whether tangible improvements in gender equality in these targeted areas are being registered thus far. Better data capture and evaluation would help evaluate women’s understanding of, satisfaction with, and tangible gains from, PRLP-supported activities. 3. PRLP spent little time supporting work on land rights issues, despite their significance to women. However, this might be possible in subsequent grant funding rounds (or perhaps through collaborative activities with implementing partners [IPs] Namati or the International Commission of Jurists, based on their past work on such topics). Evaluation Question 5: How effective are the Project’s activities in coordinating with other USAID and other donors’ programs? 1. Several informal mechanisms exist for effective PRLP coordination with other key donors and implementers, and PRLP is doing a good job of coordinating and sharing-information with other donors and implementers. 2. PRLP’s donor coordination has avoided significant IP collisions, overlap, and/or confusion. 3. There is room for USAID/Burma to convene forward-looking strategy sessions with its relevant implementers, something that may be more feasible as the CDCS process gets underway. V. Recommendations 1. Evaluation Question 1 1.1. Taking into account time and budget constraints, PRLP should expand its CSO work and publicize it to a greater degree as a matter of (a) fulfilling the statement of work’s vision of balancing supply of and demand for just and equitable justice services and ensuring a ‘sustainable role for civil society’ and (b) ensuring a balanced characterization of its overall objectives during the transition. 1.2. USAID and PRLP should consider ways of commissioning an analysis of different Burmese CSO approaches to legal awareness/empowerment work, using in-depth qualitative methods (including comparative case studies where applicable). 1.3. PRLP should incorporate court monitoring work into the pilot program, or encourage another implementer to undertake such work separately, to have more indicators of fair trial standards and court user satisfaction. 1.4. Based on the OSCU’s significant commitment to the pilot court program, reform progress in the individual pilot courts should be encouraged and closely monitored. Given potential resistance to certain procedural reforms in the pilot courts, the courts’ difficulties in securing cooperation from police and prosecutors, and opportunities to generate case disposition statistics at the expense of litigants/defendants’ and victims/witnesses’ rights, pilot court data need to be interpreted carefully and put it in their proper context. 1.5. PRLP should use upcoming GBV and TIP work to further anchor the pilot court initiative and address broader issues of women’s empowerment in the judicial system. 2. Evaluation Question 2 USAID/Burma Promoting Rule of Law Mid-term Evaluation ix 2.1. USAID/Burma should continue to provide flexibility and support to PRLP in work plan/programmatic approvals in the option period, given the complexity and uncertainty of the present transition. 2.2. PRLP should capitalize on the progress made thus far in forging stakeholder dialogue and acclimating participants to the importance of data to policy discussions and advocacy. 2.3. USAID and PRLP should properly frame PRLP’s progress in the context of a transition that is more fraught and challenging than most outsiders can imagine. 3. Evaluation Question 3 3.1. Given the lower profile of the Objective 2 work and the public’s legal literacy needs, PRLP should work with civil society partners to focus additional attention on civil society-related activities and accomplishments, including additional evidence of project-driven synergies. 3.2. PRLP should do more (via its revised AMEP) to capture qualitative and quantitative information about beneficiary access to, and satisfaction with, PRLP-supported legal aid and legal awareness￾raising activities. 3.3. PRLP should look for opportunities through new grantees to promote better linkages to effective community dispute resolution via community leaders and trusted local groups. 3.4. Future grants should favor engagement with CSOs in new states and regions, especially Northern Kachin and Northern and Southern Shan—areas that coincidentally have high TIP statistics. 3.5. PRLP should explore ways to encourage promising journalists to cover justice system matters in more depth and with more insight, including via specialized training modules and workshops. 4. Evaluation Question 4 4.1. PRLP should foster synergies with the pilot court program to address known problems with the lack of formal protective measures for women, weaknesses with judge and law officer insensitivity to GBV victims, and inadequate referral pathways for pertinent social services. 4.2. PRLP should ensure that changes to its AMEP are developed, approved, and put into use as soon as possible. This will ensure that PRLP, its grantees, and the general public can analyze and disseminate information about improvements in women’s perception of, access to, comfort with, and results obtained from, both legal awareness and access to justice activities. This might be remedied through indirect support for legal awareness and legal rights activities through CSOs under the small grants program. 5. Evaluation Question 5 5.1. PRLP should urge United Nations Development Program to encourage more open discussion and problem-solving relating to common issues faced in the justice system at its monthly rule of law sessions. PRLP should also consider hosting or co-hosting such sessions to obtain more influence over the agenda and make such meetings more useful and interactive. 5.2. USAID/Burma should consider facilitating more open discussion/brainstorming sessions with other IPs to explore possible synergies and share common challenges which may be more feasible in connection with the impending CDCS process. USAID/Burma Promoting Rule of Law Mid-term Evaluation 1 II. PROJECT BACKGROUND Invocation of ‘the rule of law’ (ROL) was widespread when Aung San Suu Kyi took her seat in the Burma Parliament in the spring of 2012 and agreed to chair a new Committee on The Rule of Law and Tranquility. The now-State Counselor mentioned the term continuously in 2012, as did many of her allies and supporters. After decades of military rule, the committee endeavored to adhere to the law by legislators, prosecutors, civil servants, the government, media, and other institutions. As part of the complex, simultaneous, and multi-dimensional political, economic, and social transitions in process in Burma since 2010, broad legal reform was viewed as a critical priority by a wide swathe of Burmese opposition politicians, civil society leaders, and international donors. In the excitement and cautious optimism of the time, it seemed opportune to begin efforts at legal reform, even within the constraints of the 2008 constitution and a government dominated by military or ex-military leadership in many senior positions.1 In this environment, the United States Agency for International Development (USAID) hoped to play a leading role in the legal reform process. In March 2013, USAID/Burma issued a tender for the Promoting Rule of Law Project (PRLP) to support “a set of interventions to promote and protect the rule of just law and civil liberties in Burma.” 2 PRLP includes two complementary objectives: “(1) to promote more effective, accountable, and accessible justice sector institutions; and (2) to increase legal literacy, access to justice, and participation of marginalized populations in target regions/states.” The intended approach was to “work strategically across key areas in the justice sector, balancing supply of and demand for fair and equitable justice services; provide assistance to identified needs and deepen engagement with justice stakeholders; and build local capacity to identify and address needs that will contribute to the achievement of project goals.” 3 The project’s scope was to be nationwide based on potential work with civil society organizations (CSOs) and marginalized segments of the populations throughout the country.4 Awarded to Tetra Tech DPK in October, 2013, PLRP is a five-year,5 $12.1 million project to promote ROL and access to justice in Burma as a means of supporting the country’s larger democratic transition. USAID crafted the project quickly, pairing the broad, general scope with a large number of anticipated results, and grounding those results in a contract with substantial Mission involvement. The Mission launched PRLP at a time when Burma just opened up to outside scrutiny, and when USAID’s only other substantial investment in peace, democracy, and governance was the Office of Transition programming (begun in September 2012), which focused on facilitating public engagement in the reform process and reducing drivers of intercommunal conflict.6 In the context of the United States (US) imperative to demonstrate rapid support for the transition, USAID put together a statement of work (SOW) with a particular focus on justice system apex institutions, such as the Office of the Supreme Court of the Union (OSCU), the Union Attorney General’s Office (UAGO), the Union Parliament, and the Constitutional Tribunal. This SOW also contained expectations that the project would work simultaneously on demand-side access to justice and legal awareness issues while also building the capacity of CSOs interested in such issues. PRLP’s broad scope and intentions are attributable to a number of higher-level policy unknowns— unknowns that, despite the 2015 election, still exist today about the motivations and political calculations animating individuals as well as key governmental and nongovernmental organizations (NGOs). PRLP’s scope included a large number of expected results (37), which posed a challenge, given the new landscape in which PRLP is operating and the significant exploratory, assessment, and relationship￾testing/building activities that were required to gain traction in any of the project’s main components.7 The PRLP budget provided general budget allocation guidance between Objective 1 (work principally with state institutions, 70 percent) and Objective 2 (work focused on civil society legal awareness, access to justice, networking, and capacity building activities, 30 percent). The large estimated allocation for USAID/Burma Promoting Rule of Law Mid-term Evaluation 2 Objective 1 in part reflected USAID hopes that PRLP would provide essential assistance to the Government of Burma (GOB) on development of a multi-year national legal reform strategy.8 PRLP was able to negotiate the above budgetary stipulations and conducted a number of assessments in 2014 that led to a wide range of interventions involving Burmese justice institutions and CSOs. The progress of these activities persuaded USAID to exercise its option period in late 2016 and extend the project for another two years. USAID/Burma Promoting Rule of Law Mid-term Evaluation 3 III. EVALUATION PURPOSE AND QUESTIONS Evaluation Purpose USAID/Burma requested this performance evaluation at the onset of the option period9 to identify the extent to which PRLP engaged successfully with justice sector stakeholders in Burma and responded to new opportunities to advance its stated objectives. The Mission also seeks to inform USAID/Burma’s and PRLP’s strategic decision-making on programming decisions for the remainder of PRLP, and any subsequent legal reform interventions that the Mission might seek to support. As such, this evaluation’s recommendations may help inform aspects of USAID/Burma’s upcoming Country Development Cooperation Strategy (CDCS) drafting process.10 This evaluation report (ER) will be shared with USAID/Burma, the U.S. Embassy in Yangon, local stakeholders, and other interested groups as determined by the Mission. Evaluation Questions This report answers the following questions posed by the Mission: 1. Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma. 11 2. Was the Project able to respond to new opportunities to advance its stated objectives? 3. To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? 4. Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? 5. How effective are the Project’s activities in coordinating with other USAID and other donors’ programs? In accordance with the tasks given to Social Impact, Inc. (SI) in the Mission’s evaluation SOW, its principal focus should be on questions one through three, with subsequent emphasis on questions four and five. USAID/Burma Promoting Rule of Law Mid-term Evaluation 4 IV. EVALUATION METHODS AND LIMITATIONS The evaluation team (ET) included the following members:  Mr. Malcolm Russell-Einhorn—team leader (TL) and senior evaluation specialist—is a senior governance, ROL, and public administration specialist with 24 years of overseas development assistance experience in comparative legal and regulatory reform and public administration. A Lecturer in International Relations and Senior Fellow at the McCormack Graduate School of Public Policy and Global Studies at the University of Massachusetts, Boston, Mr. Russell-Einhorn was responsible for liaising with USAID/Burma, preparing all deliverables, and leading briefings with the Mission. His ROL project management and evaluation experience were critical to the proper framing of this evaluation in comparative perspective as well as to the ET’s absorption of technical legal information and documentation.  Mr. Aung Tun—local evaluation specialist—has over six years of professional experience in political and social developments and consulting for donor organizations in Burma, particularly on government-civil society relations. For this evaluation, Mr. Tun assisted with the documentation review, helped conduct in-country stakeholder interviews, contributing to writing the ER as well as briefings for USAID stakeholders.  Mr. Than Htike Oo—interpreter, translator, and logistician—has more than nine years of experience as a journalist, publishing articles in English and Burmese, and maintains extensive skills in administration and project management with international organizations. Mr. Htike Oo helped set up all of the interviews and focus group discussion (FGD) arrangements, handled all travel and other logistics, and translated interlocutors’ responses in the Burmese language in the interviews and FGDs (fewer than a dozen were conducted in English). Data Collection Methods The evaluation utilized a mixed-methods approach that included several qualitative data collection methods and an analysis of quantitative data on various aspects of the current justice system and work accomplished by PRLP thus far (see Table 1 for data collection statistics). 12 Data collection methods included:  Document review of PRLP annual and quarterly reports, the Activity Monitoring and Evaluation Plan (AMEP), sub-grants documentation, and various expert assessments commissioned by PRLP. The ET also reviewed non-project related background reports, articles, and books on the Burmese justice system and current political context. See Annex III for a full list of documents reviewed.  Key informant interviews (KIIs) occurred in Yangon, Hpa-an,13 Mawlamyine, and Nay Pyi Taw— with 110 individuals (some in group settings), encompassing 53 men and 57 women. These interviews included USAID/Burma and PRLP personnel, PRLP grantees, representatives of several PRLP governmental and partner NGOs, donor representatives, and legal and political experts and observers within and outside of Burma. Interviews were conducted using a semi-structured questionnaire. The ET also conducted anonymous, confidential interviews with a current court clerk and a retired judge, respectively.  Field visits and site observations included a pilot township court in West Yangon, pilot township and district courts in Mawlamyine, a pilot township court in Hpa-an, a pilot Attorney General law office in Mawlamyine, justice (legal aid) centers in West Yangon and Mawlamyine, and the OSCU and UAGO in Nay Pyi Taw.  FGDs conducted with legal awareness trainees from two PRLP grantees in Hpa-an (two men and four women), with justice center attorneys and paralegals in Mawlamyine (four men and three USAID/Burma Promoting Rule of Law Mid-term Evaluation 5 women), and with law officers provided PRLP media relations/public information training at UAGO in Nay Pyi Taw (five men and five women). Table 1: Summary of Interviews and Focus Groups* Location Dates Stakeholder Groups Individuals Total Interviewed Yangon January 8-11  USAID (9)  PRLP staff (16)  CSO partner staff (6)  Donors staff (3)  Local expert (3) 37 Hpa-an January 12-13  CSO partner staff (2)  Pilot Court judges/staff (7)  CSO beneficiaries (FGD) (6)  Local expert (1) 16 Mawlamyine January 14-17  Pilot Court judges (8)  CSO partners staff (3)  UAGO officials (7)  CSO beneficiaries (FGD) (11)  Local expert-Deputy Speaker of State Parliament (I) 30 Nay Pyi Taw January 18-19  UAGO officials-1 st meeting (5)  UAGO officials-2 nd meeting, legislative vetting (4)  UAGO officials/ beneficiaries (FGD) (10)  OSCU judges-1 st meeting (7)  OSCU judges-2 nd meeting, legal aid (5)  OSCU judges and staff (3rd meeting, case management) (5)  OSCU judges (4th meeting training/information technology) (5)  CSO partner (1) 42 Yangon January 22-24  PRLP staff (6)  CSO partner staff (8)  International partner staff (2)  Pilot Court (9)  Local expert (3)  USAID (3) individually)  PRLP staff (1)  Implementing Partners (2) 34 Total 159* * Includes individuals who may have been interviewed more than once in separate meetings. All interviewees and FGD participants were informed about the objectives of the evaluation and given the opportunity to verbally consent to be interviewed and promised confidentiality. For sensitive topics, the ET ensured data collection methods did not present potential harm to respondents.14 Accordingly, these subjects are identified throughout this report via a generic reference (e.g. “partner,” “CSO grantee representative,” “PRLP staffer,” etc.) and the date of their interview. See Annex II for copies of the consent form and data collection tools used. USAID/Burma Promoting Rule of Law Mid-term Evaluation 6 Sampling The list of key informants that the ET sought to interview was based on independent lists compiled by the ET members and PRLP. The KIIs and FGDs selected represent a purposive sampling approach based on the suggestions stipulated in the Mission’s SOW for this evaluation, information gleaned from the ET’s document review, potential interviewees’ depth of knowledge of PRLP, time constraints posed by the geographic locations of PRLP activity, and the manageable interest of the ET interviewees. The ET took into account gender considerations in setting up meetings with other implementing partners (IPs); international NGOs; CSOs and CSO grantees, including consultations with PRLP’s court, civil society, and grant teams; and staff responsible for PRLP’s Gender Action Plan.15 Site visits were selected in consultation with both USAID/Burma and PRLP. The ET benefited from its research on the Burmese legal system over the past century, interviews with several knowledgeable legal experts, special confidential interviews with a current court clerk and retired judge, and two members of the 88 Generation movement. Analysis Interview notes taken during the field work were distilled independently by each of the ET members immediately following the USAID/Burma out-brief and then collated and harmonized. For each evaluation question, responses were triangulated with document information and other KII and/or FGD responses. The number of male and female respondents was tallied and their respective answers were analyzed for critical differences, particularly in the mixed FGDs. Additional documents were requested from PLRP staff where necessary to fill in key information gaps or ambiguities. Limitations A major limitation to the evaluation’s methodology was the timing of the evaluation and the brevity of the field work, which spanned a total of 20 days. The evaluation was originally scheduled for the summer of 2016, but due to delays in USAID’s release of the solicitation and then availability of the ET, was delayed until January 2017. Consequently, this limited interview opportunities, such as with Parliamentary members or staff, since Parliament was not in session.,16 However, the ET was able to conduct a substantial number of KIIs within the allotted timeframe and virtually all relevant facets of PRLP were addressed, with the exception of relevant Parliamentary informants.17 Due to the lack of a random sampling approach, the evaluation faces potential selection bias regarding the chosen respondents and geographic location for site visits. However, any selection bias was mitigated to some degree through consultations with USAID/Burma, PRLP staff, and a range of external legal and political experts. However, there were some constraints imposed simply by virtue of the relatively small number of individuals and organizations who were familiar with the specific activities of PRLP (many of whom are project partners or beneficiaries). Selection bias as to geographic sampling was minimized due to the vast majority of PRLP’s partners and grantees being located in a limited number of areas of the country, most of which were accessed by the ET during its three weeks in the field. The ET acknowledges that due to the large number of individuals interviewed with direct ties to PRLP, there is still a possibility of response bias (i.e. desirability bias) with respect to the findings and conclusions. The inclusion of a local evaluation expert as a member of the ET who is familiar with domestic Burmese affairs was a key mitigation strategy against possible response biases. In addition, interviews with independent Burma observers and legal experts with whom one could pose key questions about trajectories of change in the country as well as where expected and unexpected obstacles could be encountered helped reduced any potential of response bias. Although recall bias could have posed a potential problem with some interviewees (e.g. a tendency of some respondents to confuse one activity with another or blend different experiences into a composite USAID/Burma Promoting Rule of Law Mid-term Evaluation 7 memory and response), this risk was minimized by having the ET clearly state its purposes and questions and by having respondents confirm otherwise unclear or vague responses. USAID/Burma Promoting Rule of Law Mid-term Evaluation 8 V. FINDINGS AND CONCLUSIONS This section of the ER contains findings and conclusions for each of the five questions posed by the evaluation SOW. Evaluation Question 1: Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma. FINDINGS PRLP built strong relationships with key justice system stakeholders, including the OSCU, several pilot courts, leadership of the UAGO, the Parliament, and CSOs, including legal aid organizations, the nascent Independent Lawyers Association of Myanmar (ILAM), and several groups supporting the work of paralegals. According to interviewees, these solid relationships were forged through relationship-building and action-oriented offers of assistance by PRLP’s key personnel; by the deployment of foreign and local consultants who conducted assessments that were shared in translation with stakeholders; and by a number of accomplishments (described below)—from assistance to the OSCU in writing a three-year strategic plan,18 to initiation of a pilot court initiative that elicited interest from UAGO in a similar type of pilot program involving three local UAGO law offices, to a hands-on small grants program. While several of these relationships were developing in a positive direction by the beginning or middle of 2015, the 2015 elections results ensured a greater degree of transparency in communications between PRLP and certain state actors (e.g. the OSCU leadership and the arrival of a new Union Attorney General).19 Due to its work on the OSCU’s strategic plan as well as its analysis of civil and criminal case processing in three lower courts, PRLP secured the interest of the OSCU in undertaking a multi-year pilot court initiative in three township courts. This resulted in opening the court system to public scrutiny and opportunities to improve access to justice. PRLP developed a multi-dimensional plan with the OSCU to improve court processing (e.g. changes to several procedural and performance indicators designed to clear backlogs, reduce case postponements, and ensure witness attendance),20 public access (e.g. improved intake counters and installation of benches), and information dissemination (e.g. clearer posting of notices and basic information kiosks).21 An OSCU Case Management Committee is in charge of monitoring court performance and advising on judge training.22 Moreover, based on feedback from the pilot court sites, Coordinating Committees composed of representatives from the OSCU, High Courts, and other justice stakeholders (including law officers and practicing attorneys) were established to help problem-solve issues arising from implementation of the new procedures.23 A Phase II cohort of five additional pilot courts was added in the fall of 2016, and there is talk of expanding the number of courts in the future, with costs borne largely by the OSCU.24 The strategic plan and pilot court work opened opportunities for cooperation with the Court on skills training, public information, information technology (IT) modernization and data analysis, and consultation on the country’s new-but-unimplemented Legal Aid Law. PRLP’s positive experience with the strategic plan development process and continued relationship-building with the OSCU’s Chief Justice and Director General led to other important openings that could bear fruit during the life of the project. This includes skills training for judges (in some cases, related to the pilot court initiative), assistance to the OSCU in the development of a Public Information Unit (a function that the Court hopes to see created under a designated judge in the pilot court program), modernization of the court’s IT systems (in part to help provide a seamless connection for data and communications with the pilot courts), and possible data analysis work, although this last area has progressed slowly. 25 It also led to consultations between the OSCU and PRLP, and between OSCU and other justice stakeholders, on the new Legal Aid Law, which was adopted in Parliament and is now in the process of being revised to address organizational and management gaps. The latter consultations took place in July USAID/Burma Promoting Rule of Law Mid-term Evaluation 9 2016 in the form of an unprecedented two-day roundtable discussion featuring representatives from several ministries that would be affected by the legislation, CSOs, and the national Parliament. Nominees to the future Legal Aid Board also attended.26 The OSCU continues to work on general options for implementing the Legal Aid Legislation and established an OSCU Legal Aid Working Group last year.27 Results of the pilot court initiative are mixed, but progress made in the Phase I pilot courts is noteworthy, given the judiciary’s historical lack of transparency, undermining of citizens’ rights, and until very recently, distrust of most foreign consultants. 28 Early results from PRLP show modest to no improvement in three of the four efficiency indicators (backlog reduction was an exception), largely attributable to poor compliance by police and prosecutors, particularly in the Hlaing Thayar Township Court.29 According to interviewees and secondary literature, much of this poor compliance is a result of entrenched corruption within the judicial system, ranging from shared moneymaking schemes to routine police fabrication of evidence.30 All of this can fuel delays and the absence or disappearance of witnesses. 31 As for citizen satisfaction, it is difficult to know what court users think of their court experience, given information on legal literacy and expectations as well as access and fairness, which PRLP collects on court premises.32 But the initiative is still in its infancy—there are signs of improved citizen information and physical access33 as well as a willingness, at least in some locations (via High Court Coordinating Committees), to engage in stakeholder consultations. Indeed, there was optimism among some CSO interviewees that stakeholder dialogue on discrete issues, including attorney and family access to defendants in police custody, could result in tangible changes in practice and attitudes within the context of pilot court activities and heightened scrutiny.34 While transfers of judges to and from the pilot courts may compromise the initiative’s sustainability— disrupting learning and leadership at these sites—the OSCU indicated a willingness to suspend potential transfers for some period of time during the life of the initiative.35 PRLP made important inroads with the UAGO by helping with parts of the UAGO Five-Year Strategic Plan. Through the strength of the initial work done on the pilot court initiative with the OSCU, PRLP gained agreement from UAGO to begin an analogous case processing initiative with three pilot UAGO law offices.36 It also created a skills course for prosecutors on handling trafficking in persons (TIP) cases, including sensitive conduct when working with TIP victims/survivors and witnesses. PRLP based its curriculum on Association of Southeast Asian Nations (ASEAN) training for special TIP prosecutors that was endorsed by the ASEAN Senior Officials Meeting on Transnational Crime, making its acceptance and sustained use by UAGO much more likely.37 Finally, PRLP provided assistance to the UAGO Public Information Office in the form of media training for law officers and the development of Media Disclosure Guidelines.38 Engagement with anti-TIP initiatives broadened and deepened engagement with a range of other justice sector stakeholders. While PRLP cooperated with a wide range of justice system stakeholders since the beginning of the project, it developed new contacts and deepened ties with familiar stakeholders in connection with an expansion of anti-TIP activities in the wake of Burma’s reclassification to a Tier 3 country by the US Department of State.39 These new contacts include the United Nations Action for Cooperation against TIP, the International Organization for Migration, Save the Children, World Vision, CARE, and the Australia Asia Program to Combat TIP. Based on meetings and workshops with these organizations, PRLP initiated a two-pronged plan of action to (1) create a new grants program focused on prevention, protection, and legal support efforts related to anti-TIP work (the third grants round already mentioned), and (2) facilitate training programs for judges, prosecutors, and lawyers on anti-TIP issues, including the above preliminary efforts with the OSCU and UAGO.40 USAID/Burma Promoting Rule of Law Mid-term Evaluation 10 PRLP made inroads with the Union Parliament. The Union Parliament is a challenging institution with which to engage, based on its limited time in session, its small cadre of professional staff, and suspicion of both civil society actors and foreign donors/technical assistance implementers.41 Notwithstanding these impediments, PRLP partnered with the National Democratic Institute (NDI) to hold two important roundtables last quarter with members of parliament (MPs) on anti-TIP policy and legislative concerns, and on key access to justice issues, respectively. The former focused on discussing Burma’s compliance with the ASEAN Convention against TIP and Burma’s own anti-TIP law, which is scheduled to be amended this year to improve compliance.42 The latter, described in more detail in Evaluation Question 3 below, involved the presentation of research access to justice problems at another Parliamentary roundtable.43 In both cases, requests were made by MPs for more roundtables, and there is cautious optimism at PRLP and NDI that this can lead to regular formal hearings with robust and diverse CSO participation.44 PRLP’s engagement with a wide variety of CSOs with law-related interests generated a number of accomplishments. PRLPs work with CSOs under Objective 2, principally through training and grant making, leveraged limited resources to generate accomplishments and learning by many of the grantees that can benefit PRLP and CSOs in the coming years. The specifics of this work are detailed in Evaluation Question 3 below, but it is worth mentioning here that PRLP’s training and grant making initiatives had a multiplier effect related to engagement and capacity building, with grantees undertaking their own workshops, training-of-trainers, and public forums.45 CONCLUSIONS From a process standpoint, PRLP’s engagement with relevant justice sector stakeholders has been successful. PRLP forged relationships and collaborations in an effective, sequenced manner—utilizing available political openings and technical entry points. In light of the complex nature of the current transition and fragile government-civil society relations, PRLP accomplished this in a short amount of time. The pilot court initiative acquired momentum in a short period of time, but could benefit from the use of additional metrics to assess progress along various dimensions. The pilot court initiative shows a degree of political commitment by the judiciary. Still, the situation could benefit from a close analysis of the court efficiency data by PRLP and consideration of different explanations for results and trend (e.g. the possibility that backlog reduction is achieved by disposing of old cases by summary, arbitrary means) as well as an augmentation of qualitative standards to reflect procedural fairness, transparency, and equity concerns. At present, the user satisfaction surveys utilized by PRLP are potentially unreliable; while more appropriate survey methodologies are possible to explore and test.46 Whatever bundle of metrics is utilized, pilot court data need to be interpreted carefully and put it in their proper context. Case management efficiency improvements are only meaningful when they presuppose a just and respected judiciary—at best, a medium- to long-term goal given current realities in Burma. Given deep structural impediments in police and prosecutors’ difficulties in meeting the pilot courts’ performance improvement standards relating to witness attendance in court, 47 USAID/Burma and PRLP should calibrate expectations and look for improvement in areas where courts’ manageable interest is greater. That could include certain indicia of transparency, access, and equity (particularly regarding the treatment of women), even where overall standards of fairness may be adversely affected by the actions and behavior of police and law officers (and many judges). The strengths of PRLPs civil society support and emerging work on issues of gender-based violence (GBV) and TIP offers an opportunity to engage with the pilot court program in a more focused way going forward. As PRLP’s GBV and TIP work unfolds, PRLP’s cooperation with the OSCU and UAGO offers a geographic and technical basis to focus both the fairness and efficiency concerns of the project and engage civil society information-sharing, legal awareness, legal aid, media, and policy advocacy work. Equally important, this work offers an opportunity to engage with the pilot court program by USAID/Burma Promoting Rule of Law Mid-term Evaluation 11 promoting women’s empowerment generally with regard to female victims, witnesses, relatives, defendants, lawyers, and court staff. Evaluation Question 2: Was the Project able to respond to new opportunities to advance its stated objectives? FINDINGS PRLP’s successful relationship-building as well as the quality and responsiveness of its work created new opportunities for engaging with the OSCU and UAGO. PRLP’s rapid progress in building relationships with the OSCU, UAGO, and many different international NGOs and CSOs during the first year and a half of implementation48 gave PRLP the credibility and “agility to respond to emerging needs”.49 The early pivot to the OSCU demonstrated PRLP’s agility in shifting much of its attention away from UAGO—where, despite the SOW’s emphasis on capacity-building and support to a national legal reform strategy under Objective 1, the initial reception was sluggish and other donors appeared to not be making headway in collaboration—and toward a client more willing to engage.50 Collaboration with the OSCU on its Strategic Plan, in-depth work on a three-court case management study, and the agreement to launch a pilot court initiative whetted the appetite of UAGO to request collaboration with PRLP on a similar pilot law office initiative.51 Both engagements facilitated training opportunities in institutions (e.g. media relations training at UAGO, skills training at the OSCU), the hosting of an OSCU consultative roundtable on the Legal Aid Law with Parliamentary and various Ministry representatives; and later receptivity to collaborate on TIP-related training and capacity building in in both institutions, as discussed in the Findings for Evaluation Question 1 above. PRLP responded to many opportunities with other ROL actors during the base period. PRLP’s early and solid relationship-building with CSOs facilitated other opportunities to advance project objectives, including: support for the international NGO Namati’s work on land tenure issues and later support for creation of a nationwide paralegal association;52 organizational development assistance to fund research by, and later stand up, the advocacy coalition Access to Justice Initiative (A2JI);53 development of a Legal Aid Toolkit, widely distributed throughout the country, based on trainings provided to PRLP’s grantees and other partners;54 and the underwriting of certain organizational and continuing legal education costs associated with establishing a secretariat for ILAM.55 Many of these engagements, particularly the work of CSOs supported through the grants program, directly or indirectly helped reach female citizens through legal aid and awareness activities (discussed further in connection with Evaluation Question 3 below).56 These engagements led to, or strengthened, other ones that arose during the past year, such as the presentation of A2JI’s access to justice research to Parliament with NDI assistance, including a special grant to the international NGO Justice Base to facilitate A2JI court monitoring research; 57 work with the OSCU’s Legal Aid Working Group—enriched by CSO inputs;58 and TIP-related training and information￾dissemination support from civil society that will be channeled through the Round 3 small grants solicitation.59 See Findings for Evaluation Question 3 below for more detail. Programming overlaps involving the pilot court and UAGO initiatives as well as various PRLP￾supported CSO legal aid awareness activities in the same vicinity, opened up opportunities to advance project objectives. Thanks to emerging programmatic and geographic overlaps in diverse aspects of PRLP programming in Yangon, Mawlamyine, and Mandalay (various combinations of pilot courts, pilot law offices, and CSO access to justice and other activity), there are opportunities to further open up institutional dialogue and collaboration among these actors and regional parliaments interested in access to justice issues (there was clear interest expressed in Mawlamyine on behalf of the Mon Parliament), and PRLP staff and the USAID/Burma Democracy, Human Rights, and Governance (DRG) team contemplated such synergies.60 Such dialogue and collaboration are not only built into the concept of the local pilot Court Coordination Committees, but are utilized by the CSO Phoenix Association in Hlaing Thayar USAID/Burma Promoting Rule of Law Mid-term Evaluation 12 Township in West Yangon to anchor community stakeholder meetings to address community concerns about criminal justice issues.61 CONCLUSIONS PRLP responded effectively to new opportunities to meet partner needs and advance its objectives. The foregoing findings support the notion that PRLP seized new opportunities to engage with partners specifically envisioned in the PRLP SOW, and those falling within its broader mandate and array of expected results. The efforts were strategic and anchored by activities that enlarged organizational and individual connections. This is no small matter in a political environment guided by a decades-long intent to compartmentalize most institutions and civil society. PRLP’s work with Parliament, OSCU, and other stakeholders regarding Legal Aid Law amendments and the A2JI as well as consultation with the Parliament on the pilot court initiative can be conceived of as a ‘third generation’ multi-lateral relationship-building opportunity for the project. PRLP’s earlier first and second generation bilateral relationship-building and collaboration laid the groundwork for this opportunity. PRLP’s public consultation work is notable for Burma and provides a model for rational, inclusive policymaking, and the building of civic trust. Even where some new opportunities appear to be one-off interventions, PRLP’s capacity-building and ‘connector’ work with various institutions created the conditions for larger, more sustained development and collaboration in the future, particularly within the CSO community. A number of the new opportunities seized by PRLP may appear as one-off interventions, particularly at this point in the project. However, a deeper examination of these engagements shows than they constitute important building blocks in the construction of a new justice system infrastructure. For example, support to ILAM is a major step forward in creating a new, more open and encompassing platform for legal professionals, several of whose standing committees—including one on human rights—have the potential to advance reforms in the years to come. Similarly, the support to Namati for the creation of a paralegal association has similar potential to spur broader reform solidarity within an important segment of the new justice system in Burma.62 These steps are modest, but of great importance in a transition as fundamental and challenging as this one—something little understood by most of the general public and even many policymakers in Washington. Perhaps the most impressive opportunities seized by PRLP are those involving facilitation of public consultation, e.g. an open forum with Parliamentary and other GOB representatives at the OSCU to discuss Legal Aid Law amendments, and the A2JI coalition’s presentation to the Parliament of evidence regarding key access to justice shortcomings in Burma. Evaluation Question 3: To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? PRLP’s SOW envisioned a mix of supply- and demand-side activities, with the latter specifically focusing on 1) raising legal awareness and legal literacy among the population; 2) expanding access to justice; and 3) and enhancing public participation among marginalized populations in selected regions and/or states in the country.63 From the start, the design included a small grants program that would anchor the bulk of the demand-side portfolio. Eventually, PRLP issued small grants to 18 CSOs, including a number that featured paralegal work and some legal representation activities. These grants were generally made for one year, although a limited number of grants were made for a somewhat longer period of time or were later extended, based on the type and quality of work being done, and the capacity of the CSOs. At the same time, a few grants were terminated after eight or nine months because they were negotiated at a later date and ran up against uncertainties about whether the option period would be exercised. Ultimately, PRLP supported 19 organizations with $1.65 million in disbursed funding during the base period.64 USAID/Burma Promoting Rule of Law Mid-term Evaluation 13 A second round of grants, focused on legal aid and intended to reach marginalized populations in additional states and regions, including Kachin State, Northern Shan State and Mandalay, among others, was put out for bid in fall 2016 and grants were being negotiated with CSOs as the ET’s field work was conducted in January 2017. According to PRLP staff, the subject matter parameters of these grants were loosened to permit an emphasis on additional legal awareness work. A third grants solicitation was scheduled for February 2017, focusing substantively on issues related to TIP.65 The third round of grants will be limited to one-year agreements, unless PRLP is extended and additional funds are added.66 PRLP’s work with civil society—supporting access to justice, legal awareness raising, and public participation in policy formulation and oversight as well as building CSOs’ internal capacity—largely falls under Objective 2 of the project. PRLP supported a wide range of civil society activity in the justice sphere through the small grants program, targeted training and workshops, and facilitated and non-facilitated CSO engagement with public authorities. Although PRLP themes and activities encompassing access to justice, legal literacy/rights awareness, and other education, training, and advocacy work overlap and inform the work carried out by the project’s CSO grantees, it is useful to cluster the bulk of this work into a handful of categories for analytical purposes. Accordingly, the ET’s findings are arranged by the following subheadings: (A) Promoting Awareness of Individual Legal Rights; (B) Improving Access to Justice and Representation Skills of Lawyers and Paralegals; (C) Civil Society Policy Development and Advocacy; (D) Strengthening the Legal Profession as a Whole; and (E) Strengthening CSO Internal Capacity. FINDINGS Based on the work generated by grantees and trainings conducted by either PRLP, its grantees, or both— in areas ranging from community-based legal awareness trainings, to human rights and lesbian, gay, bisexual, and transgender (LGBT) rights training, PRLP conducted some 483 trainings and workshops that reached 15,067 participants, including 9005 men and 6063 women (see Annex IV for a summary of PRLP/Grantee Trainings and Workshops over the three-year span of the base period).67 A. Promoting Greater Awareness of Individual Legal Rights A wide range of grantee organizations were engaged in educating the public about legal rights and/or information with which citizens were largely unfamiliar previously. These ranged from information about where to seek legal and psychosocial services (for women confronting GBV, to guidance on what to say and not to say to police officers in particular situations (e.g. ordinary citizens who might find themselves involved in a vehicle accident or youth who might be unaware of the criminal legal consequences of selling one or two tablets of methamphetamine to make a small amount of money).68 Each of these grantees utilized their own outreach methods to tap community leaders/other ‘influencers’ who could spread information through pamphlets, community meetings, word-of-mouth, and other means.69 Such information dissemination was crucial in steering citizens clear of unethical ‘case brokers’ or village administrators who often coerce individuals into settlements of civil or criminal cases at a higher than necessary financial or emotional cost, and who may receive kickbacks in return.70 Several grantees and PRLP grants staff reported substantial community interest in these educational efforts as well as high attendance at certain CSO public forums—themselves a new phenomenon for many in Burma accustomed to earlier criminalization of such activity.71 In Mon State, legal aid groups provided legal information at the village level about key laws, including those addressing drugs, child protection, theft, fraud, and GBV.72 Several groups reported an increase in referrals to legal aid organizations and paralegals as a result of such awareness-raising work.73 In addition to information about legal rights and legal assistance referral information, grantees disseminated practical “Dos and Do nots” information, such as how to talk to police.74 All of this work reached large numbers of women and in most cases, provided them with important information they would otherwise not have received.75 USAID/Burma Promoting Rule of Law Mid-term Evaluation 14 Some of the most effective and influential activities involved community stakeholder dialogue and consultations where the public, police, law officers, lawyers, General Administration Department (GAD) officers, and even occasionally judges, came together to discuss matters of public concern. Several groups in Mon and Yangon, reported an increase in trust with the police (although not necessarily judges or law officers) as well as more referrals to legal aid organizations and paralegals as a result of such awareness￾raising work.76 Ultimately, 320 community legal awareness and legal education sessions were held in 40 townships across seven states and regions (Bago, Kayah, Kayin, Mon, Nay Pyi Taw, Shan, and Yangon), empowering 15,778 people with knowledge of their rights under the Burmese constitution; the penal, criminal procedure, and civil procedure codes; the Convention on the Elimination of all Forms of Discrimination Against Women (CEDAW); the Universal Declaration of Human Rights; the Convention on the Rights of the Child; the association law; and the child law.77 Trainings and legal awareness efforts were well-received by trainees, but beneficiary perceptions and impact are largely unknown. In Hpa-an, Mawlamyine, and Yangon, the ET interviewed individuals who participated in legal awareness trainings provided by PRLP and/or PRLP grantees—including paralegals, community activists, and ordinary citizens deemed to possess strong leadership and communications skills. All were positive about the importance of their PRLP-funded work and the quality of the training received as well as the numbers of other activists and citizens they reached in conducting training sessions for others or community forums. They did, however, acknowledge the difficulties of their work in terms of reaching citizens in certain villages as well as changing government personnel attitudes and legal practices, though several said they received good cooperation from police and local GAD officials.78 Aside from anecdotal evidence related by these KIIs and FGDs, it is difficult to ascertain the systemic impact that such activities had on citizens in the subject communities without more qualitative and quantitative data. B. Improving Access to Justice and Representation Skills of Lawyers and Paralegals PRLP training, workshops, and grants allowed legal aid/access to justice CSOs to make headway in representing clients more effectively in an evolving legal environment. Through its small grants program, PRLP supported the country’s two well-established legal aid centers in Yangon and Mawlamyine, with two years of grant support to the former’s Hlaing Tharyar Branch Office, which provided assistance in 1,761 cases in the busiest legal aid office in the country.79 This was the largest source of funding for the center, which provides a critical service for citizens of modest means in a densely populated West Yangon township suffering from large numbers of rape, theft, drug, and human trafficking cases. PRLP funding contributed to various legal aid lawyers providing free legal services in the form of court representation, legal counseling and advice, and mediation to 2,459 clients, with the majority of representation cases involving criminal charges of theft, weapons/arms, assault, narcotics, and prostitution.80 In addition, as reflected in Although it is difficult to assess the overall effectiveness of PRLP-assisted legal aid services given the short amount of time in which the project’s grants operated, there is some evidence that the justice centers obtain better outcomes than in the criminal justice system at large, and that PRLP grant support is vital to improving outreach, counseling, representation, and some case outcomes as well as improving relations with judges, law officers, and police. Anecdotally, legal awareness trainings prompted many citizens to seek free legal help that they would not have considered before. Table 2, a number of other services were also provided, such as help with legal referrals and the furnishing of shelters for women. Although it is difficult to assess the overall effectiveness of PRLP-assisted legal aid services given the short amount of time in which the project’s grants operated, there is some evidence that the justice centers obtain better outcomes than in the criminal justice system at large, and that PRLP grant support is vital to improving outreach, counseling, representation, and some case outcomes as well as improving relations USAID/Burma Promoting Rule of Law Mid-term Evaluation 15 with judges, law officers, and police. Anecdotally, legal awareness trainings prompted many citizens to seek free legal help that they would not have considered before. 81 Table 2: Legal Aid Services Provided By Grantees Cumulative Through 3-Year Base Period) Type of Services Cumulative Achievement (Years 1-3) Number of Cases Number of Beneficiaries Men Women Total Free legal representation 1004 907 365 1272 Referral 315 163 157 320 Legal advices 1299 681 634 1315 Safe house 63 0 63 63 Mediation 19 17 26 43 Total 2700 1768 1245 3013 Source: PRLP PRLP funded a range of trainings delivered to Burmese legal professionals—both legal aid lawyers and other independent legal practitioners—on skill-based topics including case analysis, witness interviewing, alternative dispute resolution, and trial preparation and presentation skills. PRLP offered trainings on legal organization management and working with vulnerable groups and judicial counterparts to legal aid attorneys. Training on representation of trafficked persons, including how to obtain survivor cooperation and respond to typical defenses, was also provided to such attorneys.82 All of the training participants interviewed praised these trainings as being of uniformly high quality, particularly the skills sessions concerned with witness interviewing and cross-examination. The skills courses will be rolled out to a variety of new sites in 2017, focusing on locations where Round 2 and 3 small grants will be issued. A significant accomplishment of the instructional activities and roundtables was the creation of a dual-language Legal Aid Toolkit for Burma, a self-education resource intended to educate and assist the country’s legal aid providers and more broadly, other lawyers in the country, in the development of case management and analysis skills.83 Thanks to PRLP funding, legal aid grantee partners (including the two centers above) trained and managed 278 paralegals, who can be an effective means of reaching underserved citizens. These paralegals provided community-based legal services that included counseling and advice, referrals to lawyers, hotline and outreach support, and documentation preparation to 2,303 clients in cases related to discrimination, harassment, and violence against LGBT individuals; arbitrary detention, intimidation, arbitrary detention, and violence against female sex workers; land disputes; labor complaints; domestic and GBV; and various criminal charges. PRLP also provided grant support to a special LGBT pilot ROL project run by Equality Myanmar/Colours Rainbow that furnished specialized training for 64 LGBT paralegals in Hlaing Tharyar, Pyay, and Mawylamyine.84 Finally, through a special purpose grant to Namati, PRLP supported the ongoing establishment of a national paralegal network in the country, focusing on issues of paralegal recognition/certification, networking and communications, professional development (including training-of-trainers and local skills training), sustainable funding, and organizational management. An expected launch of the network in 2017 may help solidify a network of skilled practitioners who are vital to public legal literacy and empowerment.85 PRLP did not provide significant support to informal and customary dispute resolution, even though the vast majority of disputes in Burma are handled informally. To some degree, this was due to simply to the nature of the organizations that applied for grants under the first round of grants, and whether they actively worked on informal dispute resolution as part of their regular portfolio of activities.86 Certainly USAID/Burma Promoting Rule of Law Mid-term Evaluation 16 the two justice centers engaged in some of this work. Still, this need to be attentive to informal and customary dispute resolution mechanisms and referral pathways came up as an important issue deserving of more attention from PRLP in meetings with several CSO representatives and legal aid lawyers and paralegals, who noted how provision of better information and representation can improve outcomes in such processes, especially for women. 87 In Burma, such informal means can include reliance on Literature and Cultural Associations in ethnic areas where there is strong religious leadership, mediation of land disputes via village land committees composed of village heads and village elders along with GAD officials, and ethnic armed organizations justice systems, particularly in conflict-affected areas, which may have feature both formal justice mechanisms and customary mediation.88 C. Civil Society Policy Development and Advocacy Government-civil society relations are weak, and PRLP support helped energize CSO research and advocacy activities. Through the small grants program, PRLP facilitated research by CSOs on extractives industry law and policy in Burma and on criminal justice system stakeholders in Bago.89 Certain legal research on criminal justice problems encountered in representing ordinary citizens was also conducted by the two PRLP-supported justice centers, representatives of which participated in consultations and lobbying activities with Parliament.90 At the state/regional level, a PRLP grant to a journalist group (Independent Mon News Agency) to report on regional issues concerning governance accountability and transparency—particularly in the energy, environmental, and foreign investment spheres (including issues of land grabbing)—was utilized to educate and lobby regional parliamentarians who posed questions to state government officials.91 Overall, the PRLP project monitoring and evaluation plan (PMEP) targeted 36 CSOs to receive training in policy development and advocacy, and through the end of the base period 61 CSOs were trained.92 The highest profile research and advocacy effort was conducted by a new PRLP coalition of some 40 organizations called A2JI, which came together in July 2015 to advocate for structural and policy reforms to improve access to justice in Burma. With PRLP financial and technical support—including assistance in creating a registered Burma CSO—the new unified civil society platform carried out three separate but interconnected research projects, which established baselines against which additional research could be developed and advocacy opportunities identified.93 In December 2016, with cooperation from NDI, A2JI representatives presented their research findings to 42 MPs at a formal roundtable opened by the Speaker of the Parliament’s Upper House. At the event, there were requests for additional such roundtables on access to justice issues.94 Despite this promising beginning, A2JI appears to still have a low profile —many CSO representatives had not heard of it or its work. According to several interviewees, however, this could change as it evolves and improves its work methods and quality standards. A bigger challenge concerns its capacity for unity of purpose and production of quality work products.95 One other gap is the lack of legal professionals and lawyer groups in the coalition; this may blunt A2JI’s credibility in the access to justice space relative to concrete legal reform proposals, including those tied to possible Legal Aid Law amendments.96 D. Strengthening the Legal Profession as a Whole PRLP focused on the legal profession as a whole through support to clinical legal education (CLE) for laws students and to the organizational development of the new ILAM. The former was supported through a grant to a CLE international NGO (Bridges Across Borders-South East Asia CLE Initiative) to address this subject in Burma, particularly with regard to the resurrected graduate legal degree program at Yangon University;97 ILAM, formally established in 2016, is receiving financial and technical support from PRLP via a grant to the International Bar Association (IBA). The latter is helping ILAM create a secretariat, rent and equip office space, establish pilot legal libraries, and deliver continuing legal education workshops (providing something of value for member dues), with the intent of developing USAID/Burma Promoting Rule of Law Mid-term Evaluation 17 financial administrative, and technical capacities to put ILAM on a more sustainable footing. PRLP also seeks to have ILAM advocate for the adoption of new laws to regulate the legal profession consistent with international standards and best ethical and other practices.98 E. Strengthening CSO Internal Capacity In addition to the foregoing civil society-driven activities supported by PRLP, PRLP also furnished internal capacity-building assistance to CSO partners. This consisted of organizational development work via assessment, mentoring, and training, with a particular focus on training in four areas: USAID regulatory compliance, financial management, technical reporting, and project cycle planning and management. Final grantee reports reviewed by the ET (one third of the total grantees) part reflected a high degree of care and technical sophistication.99 All partner interviewees praised PRLP’s hands-on assistance and partnership in guiding the grants work, the flexibility of PRLP’s grants team members. CSO interviewees praised all four of the skill training areas, with only two more experienced grantees mentioning that the project cycle training was too basic. A number of grantees noted that the reporting and project cycle management and planning courses were “highly relevant” to their practical work.100 Several grantees said the training and mentoring improved their definition of target issues and opportunities for innovation in their activity plans. Some grantees stated that they felt especially well supported on both the technical and financial/compliance fronts.101 Meanwhile, the PRLP civil society and grants teams noted the progress that most grantees made in all four areas of capacity-building. Financial and narrative reporting, financial compliance, and the use of milestones to drive activity/financial planning and reporting were reported by PRLP to have resulted in improvements in grantees’ organizational and technical capacity, particularly as reflected in reporting requirements as well as second-round grant applications for grantees who submitted new applications.102 This improvement was mirrored in the self-confidence expressed by several grantees who said they submitted second round grants. One PRLP civil society team member acknowledged that this is a novel kind of project in Burma and both PRLP and its grantees are ‘learning by doing.’ Several grants team members noted the need to be flexible in relaxing regulatory requirements around the margins to facilitate CSO partner activity feasibility and success. 103 Several CSOs—including two whose 12-month grants were reduced in length to eight or nine months due to delays and uncertainties surrounding the exercise of the contract option period and the PRLP’s need to begin winding down certain activities at the end of the base period—felt that grants should be made for longer periods of time to maximize momentum and technical accomplishment. They also noted that in many cases, longer grants would permit more evaluation and stock-taking to occur.104 PRLP grants team members agreed that longer grants should be encouraged in the future, but also noted the need for some initial grants to be of a shorter duration if this was a first USAID or other major donor grant and/or initial capacity assessment concerns were noted. Many Round 2 and most Round 3 grants cannot be much longer than 12 months due to the current expected end date of PRLP in September 2018. A summary of PRLP-supported CSO activities and their resulting accomplishments, in terms of both public services provided and internal capacity building, is presented in Annex VI. CONCLUSIONS PRLP was able to utilize a combination of grants funding and training to leverage a mix of access to justice, legal awareness, and research and policy advocacy activities on a large scale. Activities in all three areas offer promising directions for the future, and the potential to reach more vulnerable or marginalized citizens—including women who experience domestic abuse and human trafficking. Expanded court monitoring can assist the effectiveness of legal aid work and credibility of the pilot court initiative. In the case of A2JI, a coalition dedicated to quality research and policy advocacy may gain an important USAID/Burma Promoting Rule of Law Mid-term Evaluation 18 foothold in Parliament and particular ministries—modeling open, systematic dialogue that can gradually break down government-civil society distrust. 105 PRLP’s selection of grantees, based on both their technical and leadership potential and strong organizational and citizen networks, created a strong multiplier effect, which, if properly sustained by USAID and other donors, can change citizens’ understanding of their rights and their expectations of government. PRLPs partnership work with grantees on organizational capacity development is effective and was highly appreciated by PRLP grantees. From interviews with both PRLP and grantee staff, PRLP’s work with grantees exhibited a strong partnership ethos and concern for both technical rigor and sound financial management and compliance. This was highly appreciated by PRLP grantees. There is reason to believe PRLP’s work on the 2nd and 3rd round grants will continue in this direction, and that certain modest improvements can be incorporated—from further adapting certain training and mentoring to meet particular grantees’ needs and experience, to somewhat longer-duration 2nd round grants. Gaps in PRLP data collection hamper efforts to evaluate the immediate and intermediate results of PRLP and grantee access to justice and legal awareness work, particularly related to beneficiary satisfaction and understanding. While PRLP’s data collection efforts are strong, the project could do more to capture qualitative and quantitative information about beneficiary satisfaction with PRLP￾supported legal aid and legal awareness-related activities as well as perceptions of fairness and accessibility to these services and services provided in the courts.106 These gaps hamper PRLP’s ability to document the progress being made in a rich array of civil society-strengthening activities. As discussed below in Evaluation Question 4, PRLP is currently revising its AMEP to address many of these issues, while also capturing gender variations that can improve women’s access to justice. These AMEP revision efforts will be of great use to PRLP and the public alike. Some parts of PRLP’s legal aid and legal awareness activities reflect training and information dissemination modalities that deserve additional support and study/evaluation. The work of the two justice centers and the legal awareness work of certain grantees—particularly that of the Phoenix Association—show promise in terms of their outreach work, reliance on paralegals, and attention to both legal details and data in their own work and information dissemination. This can be an important source of learning for other groups, particularly if supplemented by additional qualitative evidence-gathering. It is especially important in seeking to better understand how various CSOs reach citizens, especially women, with their messaging and information, including referrals to legal, paralegal, and psychosocial services. The work of paralegals and their work in community dispute resolution in particular deserve closer study. The ET’s findings showed that paralegals are an especially cost-effective way to raise citizen legal awareness, provide important and accurate basic legal advice, furnish legal referrals, and ethically facilitate mediation and various other forms of community dispute resolution, particularly in ethnic minority communities. It is critical to recognize that the vast majority of disputes in Burma are settled informally in various ways, as indeed they are even in the most advanced legal systems.107 While the My Justice Project is dedicated to legal aid and the facilitation of community dispute resolution around the country—particularly in several outlying states heretofore underserved by international donors—PRLP can do more in these arenas. PRLP provided relatively little support to informal and customary dispute resolution, even though the vast majority of disputes in Burma are handled informally, and these processes can often be improved through better information and representation to participants, particularly women. PRLP grants are limited thus far in geographic reach, but more diversity is expected with future grants. PRLP’s Round 1 grants displayed a creditable degree of geographic diversity, with an emphasis on the initial target states of Mon and Kayin. In keeping with the contract’s call for more dispersion and outreach in the option period, more diversity can be expected going forward. This is critical in reaching additional vulnerable populations and disaffected minority groups while ensuring sustainability; as one CSO grantee in Yangon noted, truly locally-based CSOs must be prioritized: “They may lack some technical USAID/Burma Promoting Rule of Law Mid-term Evaluation 19 skills, but are full of local context.”108 In Hpa-an, for example, a non- PRLP grantee mentioned that he carried out his legal awareness work at a time of the community gathering in a monastic compound; given his limited funds, “we work at a festival time…because we can save money for food and the time and energy [otherwise] spent on invitations.”109 Moreover, depending on the region and coordination with My Justice, it is possible that PRLP could engage with various community dispute resolution systems to reach important segments of the population.110 PRLP’s legal reform and justice work in Burma included relatively little media support activities. The justice system lacked good journalistic attention—even with a free press and increasing CSO law￾related activity of the kind supported by PRLP. That work—including the reach of legal awareness training in some communities—lacked the kind of amplification that media coverage can provide. PRLP did support one media organization—the Independent Mon News Agency—and several citizen journalist activities via other grantees, but this was a relatively small area of activity in relation to societal needs. News stories, profiles, and feature articles are needed to stimulate popular interest and reinforce the impact of CSO law-oriented activities. This entails training on both sides of the equation—more instruction in media relations for CSOs, but different and more rigorous training on legal topics for journalists (something not done by USAID or other donors on media-focused projects). Evaluation Question 4: Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? Burma has a high number of women in the justice system, especially in the courts. According to the PRLP Gender Action Plan, this representation reflects the low status and pay of state legal positions in the country, and masks discrimination that occurs in both the state and traditional/informal justice institutions, whether as a matter of law and policy, or practice.111 Within the manageable parameters of the project, PRLP seeks to address these matters by calling attention to key gaps in the legal framework, heightening gender sensitivity among judges and law officers to issues such as GBV and TIP, and supporting activities that strengthen women’s needs for better legal and physical protection and access to legal information and services. FINDINGS A Gender Action Plan informs PRLPs technical work, including its activities with state institutions and CSO grantees. 112 While women are well represented or even over-represented in these activities, and PRLP collects and analyzes disaggregated gender data in all aspects of its programming, PRLP has sought to address issues of discrimination, harassment, and abuse in a targeted fashion, focusing on its interest and comparative advantage. Despite discrimination in government employment and the judicial system,113 this has meant more attention to demand-side information dissemination and awareness-raising, since the former is often tied to civil service issues that are difficult for a foreign organization to influence directly. Nevertheless, PRLP’s emphasis on four areas of awareness raising and information-sharing is accompanied by practical efforts to assist women with legal and other services, acknowledging the difficulties women have in obtaining such assistance, particularly when formal institutions are involved.114 This is a particular challenge given that women frequently appear in court as survivors of GBV or human trafficking, and that they may actually be brought to court as defendants under the gendered Race and Religion Protection Laws passed in 2015.115 Gaps in PRLP data collection and evaluation hamper efforts to analyze and act on issues affecting women’s understanding of, and satisfaction with, PRLP-supported access to justice and legal awareness activities. Although PRLP collects disaggregated gender data on all of its interventions, the types of data are mostly output-oriented and not probative of specific issues relating to women’s understanding of, and satisfaction with, PRLP-supported access to justice and legal awareness activities. Reportedly, the revised AMEP may utilize added metrics and/or analysis to generate insights into gender- USAID/Burma Promoting Rule of Law Mid-term Evaluation 20 based constraints to women’s access to justice and greater legal awareness.116 Specifically, the draft AMEP will include qualitative information on plans, policies, or procedures that may improve access or eliminate access constraints for women as well as address gender variations in: (1) the perception of court users related to their level of satisfaction with experiences in court; (2) the perception of the overall population regarding gender equality in criminal and civil trials; (3) the reach of project activities, including training and delivery of legal aid and survivor support services; and (4) the perception of the population regarding overall fairness and accessibility, including specific issues affecting female access, such as costs, language, domestic responsibilities, knowledge of rights, and human trafficking.117 The AMEP does not specifically address how legal aid and awareness activities are carried out through CSO grants, and what the intended and unintended impacts of those activities might be.118 During the base period, PRLP pursued four groups of interventions specifically related to gender and female beneficiaries. This involved (1) technical support to four CSOs in completion of Shadow Reports on key discriminatory laws and practices for submission to the Parliament and for consideration as part of Burma’s state review of compliance with the CEDAW in July 2016 in Geneva; (2) support for open and diverse consultations with CSOs on the draft Prevention and Protection from Violence Against Women Law (POVAW) (working closely with the United Nations Gender Theme Group and the Department of Social Welfare); (3) support for legal and psychosocial assistance/safe house provision for female survivors of GBV in Kayah State, furnished by CSO grantee Karenni National Women’s Organization; and (4) as a discrete subset of the grantee-led trainings discussed above in Evaluation Question 3, support for trainings by CSO grantee partners on GBV-related topics, encompassing 99 trainings and 3,200 community participants in 2016 alone (2,160 women and 1,040 men).119 These efforts may yield better outcome-oriented metrics once the new AMEP is fully operational.120 While the Gender Action Plan draws attention to problems with discrimination in land registration and ownership, PRLP did not address those issues during the base period. Land registration issues are a challenge for women; while the existing Township Farmland Law does not prioritize land registration by gender, cultural practices dictate that men do the registration and there is misinformation about the fact that registration forms require male registrants (they do not).121 When registered, however, land is not held jointly unless it is explicitly registered as such. This lack of legal registration leaves women, particularly those who have been widowed, at high risk of a land grab by the military and/or government. 122 The ET did not learn about any of the community-based legal awareness trainings discussed in Evaluation Question 3 above having addressed these issues, but the Action Plan notes that Namati, which has expertise in land tenure issues and a paralegal network, is best placed to continue to tackle such problems in the future.123 This may not, however, prevent PRLP or another USAID-funded project from coordinating with Namati or others to incorporate such issues in community legal awareness work in the future. CONCLUSIONS The Gender Action Plan’s strategy is sound and should be continued. The Plan’s strategic focus is sound, including its recommendations for future programming during the option period. This includes continuing assistance to CEDAW organizations (including for the purpose of eliminating or moderating laws that discriminate against women) and continued support, where feasible within grant parameters, for a range of legal awareness and empowerment activities in different parts of the country. At the same time, the Plan envisions fostering synergies with the pilot court program to address known problems with the lack of formal protective measures for women, weaknesses with judge and law officer insensitivity to GBV survivors, and inadequate referral pathways for pertinent social services for such survivors. The Plan further envisions the possibility of conducting women-only trainings for UAGO and OSCU staff, focusing on public outreach, and piloting a mobile/roving court system to make it easier to reach women closer to their homes. This would be a new approach for these institutions and uncover issues and dynamics not USAID/Burma Promoting Rule of Law Mid-term Evaluation 21 discussed in usual mixed-gender settings. PRLP is also considering supporting the establishment of a standing ILAM subcommittee for female legal professionals.124 In the absence of certain PRLP AMEP data and analysis—and given that PRLP has had several activities underway for less than two years—it is has not been possible to ascertain the impact on women of many of PRLP’s relevant activities, particularly those relating to legal aid and legal awareness work. Some of this problem will be remedied by a revised AMEP focusing more attention on qualitative data collection addressing women’s particular experiences in the judicial system and accessing legal aid and various support services, along with the specific obstacles thereto, from legal literacy to transportation.125 PRLP devoted few resources to supporting work on land rights issues, despite their significance to women, but this might be possible in subsequent grant funding rounds (or perhaps through collaborative activities with IPs—Namati or the International Commission of Jurists (ICJ)—based on their past work on such topics). Evaluation Question 5: How effective are the Project’s activities in coordinating with other USAID and other donors’ programs? The ET assessed intra- and/or inter-donor program coordination from three different perspectives: (1) conscious bilateral or multilateral stakeholder discussion/deliberation and forward strategic planning; (2) regular or semi-regular bilateral or multilateral donor/implementer information-sharing, without necessarily involving joint forward planning; and (3) informal, mostly bilateral information-sharing—if any— and ‘self-coordination.’ As discussed below, in the case of PRLP, the ET observed all three modes of coordination, and the results appear to be positive from the standpoint of donor management as well as GOB and nongovernmental beneficiaries. FINDINGS Donors and implementers working on ROL and justice-related topics regularly meet and coordinate. There are four major donors/implementers working on ROL and justice-related subject matters: USAID/PRLP; the European Union/British Council, the United Nations Development Program (UNDP), and the Japan International Cooperation Agency (JICA). A fifth donor, the Danish International Development Agency (DANIDA), is about to launch an ROL program led by the ICJ. Most of the directors of these programs (the person in charge of JICA’s work, which is principally with UAGO, is an exception), see each other frequently in formal and informal settings to share information. They also organize a number of different coordination meetings on a bilateral basis to clarify their respective objectives and areas of comparative advantage.126 However, many of these individuals did not have information on certain otherwise important current or planned activities of their counterparts.127 PRLP staff cooperated with a broad range of other IPs, although an exception is the lack of communication thus far between PRLP and the Open Myanmar Institute.128 Several mechanisms exist for PRLP coordination with other key donors and implementers, though these mechanisms are not always being used for systematic information-sharing among stakeholders. There is a standing semi-formal monthly ROL meeting hosted by UNDP. All parties are welcome, and normally up to 20 implementers, donors, and CSO representatives attend to hear presentations and share information, formally and informally. According to interviewees, the forum is not used for systematic information-sharing, brainstorming/problem-solving, or forward planning.129 There is also an informal bimonthly or quarterly-information-sharing meeting attended by leaders of the PRLP, My Justice, UNDP, and occasionally JICA, at which common challenges are discussed and some forward￾thinking is shared, including to avoid unnecessary program duplication or misunderstanding.130 Finally, there are still more informal, ad hoc, bilateral coordination meetings or phone/email communications USAID/Burma Promoting Rule of Law Mid-term Evaluation 22 between certain partners where there is a high degree of trust, potential for overlap/collaboration, or both.131 As for USAID project coordination and information-sharing, the Mission rarely convenes meetings that bring together project implementers to discuss key topics or brainstorm about common political or other transition challenges, although it occasionally holds all-IP meetings on a roughly quarterly basis to share project information.132 Regarding more informal coordination among USAID project implementers, this largely takes place in an informal, ad hoc manner among those implementers for whom immediate information is needed or future cooperation is a tangible possibility.133 As noted above, in the DRG space, PRLP sees quarterly real value in collaborating with NDI on more open evidence￾based policy forums on issues like TIP and access to justice; it is likely it will do so with other organizations interested in TIP or in amendments to the Legal Aid Law, including the two justice centers and even the OSCU’s Legal Aid Working group. Still broader collaboration with NDI and Parliament is foreseeable on a host of other major legislative initiatives, where PRLP could help mobilize a wide range of civil society and legal profession representatives as well as prominent legal experts.134 PRLP’s engagement and collaboration with other donors and implementers is substantial and productive. The results of this and other avenues of process cooperation and collaboration work are manifest, starting with USAID-funded collaboration with NDI and extending to the project’s cooperative work on the CEDAW Shadow Reports and paralegal work with Namati. PRLP engages with a variety of implementers and international organizations to develop its own TIP programming, while mindful of existing TIP initiatives. In the case of interaction with OSCU, PRLP coordinated well with UNDP on drafting of the Court’s Strategic Plan when offered the opportunity by the Court to accelerate the process; while PRLP became the main implementer, it kept UNDP apprised of its work and shared credit with UNDP for the final product. On amendments to the Legal Aid Law, PRLP and My Justice communicate with each other about any technical inputs to the Court’s Legal Aid Working Group. Other coordination and collaboration is evident with both the TIP and other legal skills training offered by PRLP (to which attorneys from other donor-supported CSOs were invited) and the use and distribution of the Legal Aid Toolkit.135 CONCLUSIONS Given the substantial project demands it faces, PRLP is doing a good job of coordinating and sharing￾information with other donors and implementers. An ‘inner circle’ of the main ROL project implementers—PRLP, My Justice, UNDP, and occasionally JICA—ensures that big picture coordination is clear and accessible, although this mechanism might be viewed as fragile given the extent to which it relies on the strong personal relationships among the first three IP representatives; the occasion of DANIDA’s project joining the mix could prompt the group to put their meeting on a stronger institutional footing via a formal rotation of hosting duties and/or selection of mutually compelling topics for discussion. PRLPs decentralized ad hoc communications with ROL-relevant implementers demonstrate their utility, as can be seen from the collaboration with NDI on open, evidence-based policy advocacy, and the coordination with other donors on the assistance provided to ILAM as well as access to the Aid Toolkit and associated trainings. PRLP staff interact minimally with the Open Myanmar Institute, despite that group’s significant work in the legal awareness arena. Current donor coordination works to avoid significant IP collisions, overlap, or confusion. Although multiple donors cooperate with OSCU and UAGO, PRLP’s funding streams, practical action-oriented programming at the district and township levels, and overall transparency carved out a place that is prominent and open to more scrutiny and information-sharing than many thought possible.136 While the more formal UNDP-led monthly information-sharing platform facilitates useful side-bar conversations for those international and local actors not able to stay in touch as frequently with each other as they would USAID/Burma Promoting Rule of Law Mid-term Evaluation 23 like, there seemed to be some IP interest in occasionally expanding the meeting agenda to include open discussion about a particular topic, rather than focusing on IP presentations of their work. While more forward-looking strategic planning with implementers (both from USAID and other donors) would be ideal, this is usually a luxury in fast-paced development work where daily and weekly cost-benefit calculations are continually made about time management. Ultimately, there must be mutual interest present for these mechanisms to work. There is room for USAID/Burma to convene forward-looking strategy sessions with its relevant implementers, something that may be more feasible as the CDCS process gets underway. USAID/Burma Promoting Rule of Law Mid-term Evaluation 24 VI. RECOMMENDATIONS Although PRLP recently entered the option period and only twenty months remain in the original contract period, there is still considerable time left to register more accomplishments, nurture synergies and stakeholder relationships between GOB and civil society actors, and assess political will and prospects for follow-on work. There are particular opportunities in this regard involving the pilot court initiative, revisions to the Legal Aid Law, and information-sharing and partnerships among a variety of CSOs, especially CSO grantees under PRLP Evaluation Question 1: Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma? 1.1 PRLP should consider expanding its work with CSOs and publicize it to a greater degree as a matter of (a) fulfilling the SOW’s vision of ‘balancing supply of and demand for just and equitable justice services’ and ensuring a ‘sustainable role for civil society’ and (b) ensuring a proper characterization of PRLP’s overall objectives during the transition. This can happen through existing programming—via Round 2 and 3 grant-making, sustained assistance to A2JI and ILAM (insofar as they can engage with other CSOs, Parliament and GOB), continued efforts to bring important legal reform issues to Parliament and relevant Ministries and other GOB bodies via CSO research and testimony, support as necessary to the new paralegal network, and coordination, oversight/monitoring, and stakeholder engagement on the pilot court and TIP initiatives (particularly where those two intersect). PRLP should be alert to other new ‘windows of opportunity.’ 1.2 Within the limitation of time and the budget, USAID and PRLP should invest in innovative ways to publicize the capacity growth and external impact of both individual CSO grantees and coalitions/networks. There are many stories to be told and accomplishments to be documented among these grantees, and it is worth the cost of hiring an international development professional with a legal background to undertake this task. 1.3 USAID and PRLP should consider commissioning an analysis of different Burmese PRLP CSO grantee approaches to legal awareness/empowerment work, using in-depth qualitative methods (e.g. comparative case studies). This type of analysis could give USAID, PRLP, and other donors/implementers a better understanding of how CSOs conceptualize and deploy their interventions as well as how effective these interventions are at achieving their objectives. PRLP and USAID could use the information to design future programs meant to strengthen CSOs’ legal awareness work. 1.4 USAID and PRLP should scrutinize the pilot court project’s performance data to ensure it is put in proper context, both with regard to the meaning of the data and its relation to other, more meaningful indicators of judicial reform progress. It is vital for the credibility and reputation of PRLP not to emphasize efficiency gains (e.g. backlog reductions and case disposition gains) without proper interpretation and an examination of the possibility that efficiencies may be obtained in ways that ignore fundamental fair trial standards and/or decision-making by judges. USAID/Burma and PRLP should calibrate expectations and look for improvement in areas where courts’ manageable interest is greater. That could include certain indicia of transparency, access, and equity (particularly regarding the treatment of women), even where overall standards of fairness may be otherwise be affected by the actions of certain judges, or especially police and law officers. Qualitative indicators and aggregated index indicators covering issues of physical access, informational transparency, and fair trial standards, disaggregated by gender, would be important to consider adding to the current performance data Moreover, PRLP should work with the OSCU to limit or suspend transfers of judges involved in the pilot court initiative during the remainder of the project. In any event, the Mission and PRLP need to be cautious about how, and to what extent, they showcase or expand the current program, and alert to signs of backsliding. USAID/Burma Promoting Rule of Law Mid-term Evaluation 25 1.5 One way to answer questions about both efficiency gains and adherence to basic fair trial standards, as well as to develop a better proxy for the court user’s interest in the initiative, is for PRLP to incorporate court monitoring work into the pilot program, or to encourage another donor and implementer to undertake such work separately and compare notes. Justice Base, a PRLP grantee that assisted the A2JI with court monitoring activities last year, is well placed to work with PRLP in weighing the pros and cons of each approach. 1.6 USAID and PRLP should use the project’s upcoming GBV and TIP work to anchor the pilot court initiative in a more focused way going forward. Given the US government’s and GOB’s mutual high-level interest in these topics, and given the investments that the project will already be making in grant-making, training, and mentoring in these subjects with CSOs, the OSCU, and UAGO, PRLP should consider adding a GBV and TIP focus to its work with pilot courts. This work could be broadened further to embrace a women’s empowerment agenda, embracing not only attention to equity concerns of different types of female court users, but also those of others in the judicial system, including judges, attorneys, and court staff. Evaluation Question 2: Was the Project able to respond to new opportunities to advance its stated objectives? 2.1 Insofar as PRLP has responded to new opportunities due to the ‘windows of opportunity’ clause built into its contract, USAID/Burma should continue to provide flexibility and support to PRLP in work plan/programmatic approvals in the option period, given the complexity of the present transition and the likelihood that new and important other opportunities will arise. 2.2 PRLP should look for new opportunities to expand networks and facilitate programmatic synergies among CSO actors by further strengthening CSO work involving stakeholder dialogue and acclimating CSO participants to the importance of using data in policy discussions, whether in the context of Parliamentary consultations or discussions around the pilot court initiative (e.g. by the High Court Coordinating Committees). 2.3 Taking a broader perspective, USAID and PRLP should work together to make sure that they are properly framing the project’s progress to Washington policymakers and the public in the context of a transition that is much more fraught and challenging than most outsiders can imagine. Even for those observers who remember how difficult—and uncertain and fragile—the transitions in the former Soviet Union were and continue to be, there is a failure to grasp the true depth of the simultaneous economic, cultural, social, and political challenges facing Burma at this moment. This is neither a cause for undue pessimism about setbacks nor excessive optimism about modest gains, but a call for balance and nuance in communications appropriate to the context. Evaluation Question 3: To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? 3.1 Given the lower profile of the Objective 2 work and the public’s legal literacy needs, PRLP should work with civil society partners to focus additional attention on PRLP’s civil society￾related activities and accomplishments, including additional evidence of project-driven synergies, e.g. between legal aid and public awareness raising; between legal aid or awareness raising and the pilot court initiative in relevant jurisdictions; and information-sharing, research, and/or policy advocacy among CSOs via the A2JI initiative and other cooperation or coalition efforts). To promote greater collective action and potentially higher-impact and higher profile results, PRLP should incentivize additional network/coalition-building among grantees and between grantees and other CSOs through special grants clauses and additional networking, training, and workshop opportunities where feasible. The above actions are important in reflecting the overall results of PRLP and garnering support for follow-on work USAID/Burma Promoting Rule of Law Mid-term Evaluation 26 3.2 PRLP should do more (via its revised AMEP) to capture qualitative and quantitative information about beneficiary access to, and satisfaction with, PRLP-supported legal aid and legal awareness-raising activities. 3.3 PRLP should look for opportunities through appropriate new grantees to promote better linkages to effective community dispute resolution via community leaders and trusted local groups. The focus on the formal system is important, but inadequate and not sensitive to the immediate needs and comfort level of much of the population. 3.4 Future grants should favor engagement with CSOs in new states and regions, especially Northern Kachin and Northern and Southern Shan—areas that coincidentally have high TIP statistics of informal and customary dispute resolution is critical for PRLP as it supports further community legal awareness and paralegal work through its grants program. Support for special trainings and workshops for paralegals should be prioritized as highly cost-effective wherever possible (collaborating with Namati where useful) on both skills development and substantive topics, especially TIP and GBV matters, and issues about gender sensitivity in interactions with women clients by male and female paralegals. 3.5 Legal aid and legal awareness work should be continued, utilizing subject matter anchors and entry points (including GBV and TIP work) with or outside of the pilot court initiative, as feasible and appropriate. Given the remaining needs and positive work already accomplished in these areas, PRLP should continue to support legal aid and awareness efforts, prioritizing a TIP and GBV focus where appropriate, or other important criminal cases with victims (e.g. theft, assault) that may, among other things, advance a stronger women’s empowerment agenda. 3.6 PRLP should explore ways to encourage promising journalists to cover justice system matters in more depth and with more insight, including via specialized training modules and workshops, since work in this arena currently lacks accuracy and perspective and fails to cover the impact of positive accomplishments by CSOs and government institutions alike. . Evaluation Question 4: Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? 4.1 Pursuant to the project’s Gender Action Plan, PRLP should foster synergies with the pilot court program to address known problems with the lack of formal protective measures for women, weaknesses with judge and law officer insensitivity to GBV victims, and inadequate referral pathways for pertinent social services for such victims. 4.2 Given the importance of legal awareness/access to justice activities to women’s agency and empowerment, PRLP should ensure that gender-sensitive changes to its AMEP are developed, approved, and put into use as soon as possible, so that PRLP, its grantees, and the general public can analyze and disseminate more useful information about potential improvements in women’s perception of, access to, comfort with, and results obtained from, legal awareness and access to justice activities. This should include information about women’s experience in using, or being hampered in using, both formal and informal pathways and dispute resolution mechanisms.137 4.3 Given the importance of land registration issues to women, PRLP should consider special grantee-funded assistance on that topic, coordinating with Namati and judging where best to target assistance geographically or otherwise on a case-by-case basis. More broadly, PRLP should also ensure that, notwithstanding A2JI’s new focus on anti-corruption issues in the coming year, the coalition continue to address important issues of access to justice issues from a gender perspective. In this vein, any future court monitoring activities by A2JI should explicitly incorporate observations of gender discrimination in the justice system. Evaluation Question 5: How effective are the Project’s activities in coordinating with other USAID and other donors’ programs? USAID/Burma Promoting Rule of Law Mid-term Evaluation 27 5.1 While PRLP has done a good job of coordinating with other donors and seeking to coordinate with other USAID implementers, USAID/Burma could facilitate more open discussion/brainstorming sessions with other IPs to explore possible synergies and share common challenges. This may be more feasible in connection with the impending CDCS process. 5.2 PRLP should urge UNDP to encourage more open discussion and problem-solving relating to common issues faced in the justice system at its monthly ROL sessions, adding balance to its usual agenda of formal presentations and round robin information-sharing. If that does not look feasible, PRLP should consider hosting or co-hosting such sessions to obtain more influence over the agenda and make such meetings more useful and interactive. USAID/Burma Promoting Rule of Law Mid-term Evaluation 28 ANNEXES USAID/Burma Promoting Rule of Law Mid-term Evaluation 29 ANNEX I: EVALUATION STATEMENT OF WORK I. Background This is a Statement of Work (SOW) for a mid‐term, external evaluation of the five‐year (October 2013 to September 2018), $12 million Promoting Rule of Law in Burma Project (the Project) implemented under Task Order number AID‐486‐TO‐13‐00008 by ARD Inc. (doing Business as Tetra Tech DPK). The Project supports the democratic development in Burma by promoting fair, equitable, and accessible justice and enables the public to participate in rule of law reform. Mr. Mark Silva is the Contracting Officer’s Representative for the Project (Project COR), and Mr. Michael Ronning is the Alternate COR for the Project. This external evaluation comes at the chronological mid‐point of the Project. It is a mid‐term, formative evaluation whose objectives are to help determine what components and project aspects are, and may not be, working well and why, and to make modifications and mid‐course corrections, if necessary, to guide the Project over its second half. The flexibility, adaptability, and outcomes of the Project should be examined, as typified by assistance offered to the Office of the Supreme Court of the Union, the Union Attorney General’s Office, and assistance to civil society, particularly assistance to legal aid providers. Particular attention should be given to, emerging yet unanticipated opportunities, such as the Access to Justice Initiative, the Independent Lawyers Association of Myanmar, and the support for networking among paralegals. The evaluation should provide pertinent information and analysis to assist USAID/Burma to identify, understand, and learn from what is being accomplished through this Project. In summary, the evaluation will help all involved better understand the initial results and contributions of the Project, and help re‐focus and strengthen it as needed. The mid‐term evaluation will occur during the final year of the base period. In April 2016, USAID Burma exercised the two-year option period, and the period of performance of this Task Order has been extended to September 30, 2018. The final evaluation report will be submitted to the Development Experience Clearinghouse in accordance with USAID policy. II. Background Decades of authoritarian rule has left the country with a justice system that lacks independence, is under￾resourced, absent public trust, and unable to serve the people of Burma. We have seen a demonstrable increase in access to justice due to an increase in legal aid, para legal, and community advocacy services. These changes are opening the justice system to public scrutiny. Building trust in the system and its core institutions, and expanding access to justice are indispensable elements of our strategy to promote democratic change and the rule of law. In 2013, USAID launched the five-year, $12 million activity that promotes the rule of law and access to justice in Burma as a means of supporting the democratic transition and bridging the center-periphery divide. To achieve his goal, the Project has two objectives: 1) Promote more effective, accountable and accessible justice sector institutions; and 2) Increase legal literacy, access to justice and participation of marginalized populations in target Regions/States. The following chart graphically represents the results framework for the Project. USAID/Burma Promoting Rule of Law Mid-term Evaluation 30 The previous Government of Burma was committed to developing a national rule of law strategy, and an inter-agency Legal Advisory Board was established to lead the development of a national strategy. As the inter-agency members did not work well together, the Board met sporadically over a three year period. Despite institutional inertia within the Government, the Promoting the Rule of Law Project assisted the Office of the Supreme Court of the Union (OSCU) to develop and implement its first strategic plan (2015- 2018), closely followed by the Union Attorney General’s Office (UAGO) which launched its first strategic plan (2016-2021) in January 2016. The UAGO plan was developed with assistance from the United Nations Development Program and the UAGO implementation committee meetings were facilitated by the Project’s Chief of Party and a UNDP technical advisor. The new Government may establish a senior national policy group on justice sector reform, and if established, these strategic plans may become part of a future national rule of law strategy. USAID and Project staff have encouraged the new Government and its advisors to establish a consultative process with a board rage of stakeholders. If the opportunity arises, the Project is well- positioned to support civil society participation in this process through the Access to Justice Initiative described below. USAID and Project staff are engaging with the new government's policy advisors, the OSCU, UAGO, legislators, and civil society leaders to identify additional opportunities to support justice sector reform efforts and the development of a national rule of law strategy. As part of the partnership with the OSCU, the Project is introducing new management tools to pilot transparency and efficiency within the justice system. In FY 2015, the Project launched the pilot court project in two Regions (Yangon and Bago) and one State (Kayin). The pilot court program introduces modern justice and court management initiatives designed to improve case management and processing, and increase accountability and transparency of the court system, which can be replicated in other courts, and eventually adopted nationally. Ultimately, the pilot courts will contribute to improved access to justice. In this fiscal year, five new pilot courts will be launched as part of an expansion of this activity. The Project has assisted the OSCU to train judges and adopt practices to improve transparency and case management, develop information materials for the public, train information officers at the pilot courts, and prepare architectural designs for model courts. Given the weaknesses in the system, small changes in procedure USAID/Burma Promoting Rule of Law Mid-term Evaluation 31 can have a dramatic impact. For example, the Project advocated for procedural reforms to compel police officers to testify and be cross-examined to improve transparency, attack the pervasive culture of impunity, and begin to strengthen due process in the legal system. In April 2015, the Chief Justice wrote to the Attorney-General and the Minister of Home Affairs informing them of this new procedure. The success of these solutions is evaluated by court‐user surveys to determine which efforts are proving useful to build public confidence, and these will be replicated in subsequent pilot courts. Additional judicial training focuses on media and public outreach; review of the Judicial Training Center’s curriculum; and improving judges’ case management skills. Working with the UAGO requires the Project to operate in a crowded field. Project staff has been able to provide ad hoc training and assistance to the UAGO’s administrators and Law Officers. In addition, Project senior management has established a collaborative relationship with counterparts from the United Nations Development Program and the Japanese International Cooperation Agency who have embedded advisors in the UAGO. These advisors meet monthly to share information, and where possible, collaborate with each other. A striking example of collaboration is the Project’s Chief of Party and the UNDP’s Rule of Law Specialist delivering technical assistance to improve the implementation of the UAGO strategic plan by co-facilitating UAGO implementation committee meetings. Assisting civil society is another principal focus for the Project. Technical and management capacity, cooperation, and networking among civil society organizations (CSO) have been perennial challenges in the sector, despite years of previous support from the donor community. The Project provided grants to 17 organizations supporting legal aid, legal awareness, and services for victims of gender based violence, building CSO networks, supporting journalists to report on corruption, and coordination between civil society and government officials. In addition to supporting these organizations’ technical programming, the Project provided capacity‐development trainings on USAID rules and regulations, financial management report writing, project cycle design and management, monitoring and evaluation, accompanied by ad hoc mentoring from Project staff on specific needs. In FY 2015, the Project brought together 40 plus CSOs to establish the Access to Justice Initiative (A2JI) to gather data, conduct research, and monitor court performance that will improve fact‐based advocacy, to prepare for engaging with a new government. A2JI will advocate for legal reforms to improve access to justice, and promote implementation of fact-based legal awareness campaigns. The Project is providing technical assistance to legal aid providers to improve service delivery, and will launch a Legal Aid Toolkit and a pilot legal aid database in quarter three of FY 2016 to improve efficiency and effectiveness in legal aid clinics. Once the option period is exercised, the Project will likely support the Independent Lawyers Association of Myanmar, and provide a grant to an international non‐governmental organization to support a national network of paralegals. III. Evaluation Rationale The purpose of the evaluation is to examine the extent to which the Project has achieved its goal; identify challenges encountered during implementation; suggest different technical approaches if necessary, and assess new opportunities for future engagement. Key domestic justice system stakeholders include, but are not limited to, the Office of the Supreme Court of the Union, the Union Attorney General’s Office, and CSOs. The following evaluation questions are listed in descending order of importance. 1. Identify the extent to which the Project has engaged successfully with justice sector stakeholder s in Burma? 2. Was the project able to respond to new opportunities to advance its stated objectives? 3. To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? 4. Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? 5. How effective are the project’s activities in coordinating with other USAID and other donors’ USAID/Burma Promoting Rule of Law Mid-term Evaluation 32 programs (see specific tasks below)? Specific Tasks The specific tasks to be undertaken by the evaluation team to prepare the deliverables include but are n ot limited to:  Meet with theJustice Htay Htar and Director General (Judiciary) of the Office of the Supreme Court (OSCU), the Permanent Secretary and members of the Strategic Plan Implementation Committee of the Union Attorney General’s Office (UAGO), the Director and/or staff, Judges Training Institute, OSCU, UAGO and civil society staff trained by the Project and the Project’s sub-grantees (Attachment 1).  Visit one or more phase-one and phase-two pilot courts. Possible sites would include phase one Township Courts in Hlaingthayar in Yangon Region, Taungoo Township Court, Bago Region, or Hpa-an in Kayin State, and to keep logistics simple, phase two Township and District Courts in Mawlamyine, Mon State and Pathein, Ayeyarwady Region.  Although most CSO sub-awards will be completed by July 31, 2016, the evaluation team should also visit selected CSO sub-grantees in states and regions.  Interview the CSO members of the Access to Justice Initiative to assess the potential achievement of this initiative.  Interview CSO legal aid providers that received funding and/or technical assistance from the Project.  Many donors and international nongovernment organizations support human rights training, legal awareness campaigns, and formal legal education activities in Burma. USAID’s Project has supported civil society-led legal awareness campaigns in target states and/or regions. While there is a need for increased public legal awareness, many donors and implementers are devoting resources to achieve this goal. The team should assess if the Project should invest additional resources to support legal awareness activities, and if so, can resources be targeted to complement other project objectives.  Interview donor agency and international nongovernmental organization technical specialists and/or the Team Leaders, My Justice Program implemented by the British Council, the Embassy of Denmark that is designing a $10 million Rule of Law program to be launched in early 2017, the Rule of Law Program Manager, United Nations Development Program, JICA advisors embedded in the UAGO, Justice Base, Namati, and the International Commission of Jurists. IV. Evaluation Design and Methodology This section outlines some of USAID/Burma’s initial expectations regarding the evaluation design and methodology. It is anticipated that the evaluation will have a non‐experimental design. The methodology will be based on a mixed method approach, comprised of the appropriate tools in response to the aforementioned research questions. These tools may include, and are not limited to, a combination of the following: A desk review of Project documentation; observation at events; focus group discussion with sub-grantees; surveys of beneficiaries, key informant interviews with individuals working in the rule of law sector (donor leads, U.S. Embassy and USAID staff, civil society leaders, and judges), and relevant constitutional provisions, laws, policies and strategies should also be identified and used to inform the evaluation. The evaluation team must start its work with a desk review of all relevant documents provided by the evaluation contracting officer’s representative (TO‐COR) before arriving to Burma. The valuation team will travel to Naypyitaw and one or more of the following regions and states to visit pilot courts and/or project grantees: Yangon, Bago, Ayeyarwaddy, Mon, and/or Kayin to view activities in different parts of the country. Detailed schedules for all the site visits and interviews will be developed by the evaluation team in coordination with the TO‐COR and the Promoting Rule of Project COR before arriving to Burma. USAID/Burma Promoting Rule of Law Mid-term Evaluation 33 Once in-country, the evaluation team will conduct an in‐briefing for USAID/Burma, carry out the evaluation, and conduct a PowerPoint presentation on preliminary findings in an out-brief to USAID/Burma before departing the country. The evaluation team may find it useful to consult a broad range of background documents apart from the Project’s documents provided by the TO-COR. These may include: 1. The Rule of Law in Myanmar: Challenges and Prospects December 2012 Report of the International Bar Association’s Human Rights Institute (IBAHRI) Supported by the IBAHRI Trust and the Open Society Foundation. www.ibanet.org 2. www.burmalibrary.org and www.biicl.org (the Bingham Center) are useful sources of primary documents and research or ‘think piece’ papers. 3. Myanmar Rule of Law Assessment 2013, New Perimeter DLA Piper and Jacob Balustein Institute forthe Advancement of Human Rights www.jb‐human rights, or www.newperimeter.org 4. The Project’s Rule of Law Update 2016 (Attachment 2) 5. Access to Justice and the Rule of Law http://www.fmreview.org/ 6. Women’s Empowerment in Sub‐national Government in Myanmar (2014) has useful background information on gender. http://asiafoundation.org/publications/pdf/1368 In addition to the above-mentioned assessments and reports, the Project COR may provide the evaluation team with additional reports, assessments, and technical materials that are relevant to the rule of law sector in Burma. V. Evaluation Deliverables and Reporting Requirements The Contractor must submit a detailed work plan (not to exceed 10 pages) before arrival in country to be approved (after the in‐brief at USAID/Burma) by the TO‐COR. The methodology, questions, timeline, and activities of the evaluation must be clearly articulated in the work plan. Adjustments to this work plan will be made based on comments and/or issues raised by the TO‐COR who will consult with the Project COR and Director, Office of Democracy and Governance, USAID/Burma. The work plan shall, at a minimum, include the following: 1) A detailed evaluation design matrix (including questions, methods and data sources used to address each question and the data analysis plan for each question); draft evaluation protocol; 2) A timeline for completing the evaluation, including a list of potential interviews, a draft schedule of targeted meetings and location of the meetings and list of potential interviewees; and, 3) Draft data collection instruments or their main features; identification of potential major constraints and limitations, and how they will be addressed or mitigated. The Contractor must submit a clear, informative, and credible evaluation report (ER) (up to 30 pages, excluding annexes and references) that reflects all relevant evaluation team findings, conclusions, and recommendations made in conjunction with the mid‐term performance evaluation of the Project. The ER must describe the evaluation design and the methods used to collect and process information requested in section III. The ER must disclose any limitations to the evaluation and, particularly, those associated with the evaluation methodology. The ER must be in line with relevant USAID ADS and USAID Evaluation Policy requirements and recommendations. Please see the attachments for USAID ER quality standards. In particular, the ER must include sufficient local and global contextual information so the external validity and relevance of the evaluation can be assessed. Evaluation findings must be based on facts, evidence, and data. Findings must be specific, concise and supported by reliable qualitative data. Conclusions must be supported by a specific set of findings. Recommendations must be practical, clear, action oriented, and supported by a specific set of findings, and conclusions. USAID/Burma Promoting Rule of Law Mid-term Evaluation 34 In addition to the required annexes, the ER must include the Evaluation SOW, a Burmese version of the Executive Summary, description of the Evaluation Team and its member qualifications, the final version of the Evaluation Work Plan (EWP), conflict of interest statements signed by all evaluation team members, tools used for conducting the evaluation, and pictures of a visited project site(s), sources of information, and a statement(s) of differences (if any) reported by the evaluation team members and/or the Mission and/or Project leadership. The ER must be submitted in electronic form using MS Word Times New Roman 12 or other legible font of similar size. Any data used to prepare the report will be presented in MS Office compatible format suitable for re‐analysis and submitted either by e‐mail or on a CD or a flash drive. The evaluation team must present major findings and preliminary conclusions and recommendations made in conjunction with the evaluation at a pre‐departure briefing for Mission management and staff. The draft ER will be due in 10 working days after that briefing. The draft ER must include all relevant evaluation team findings and conclusions made in conjunction with the evaluation and preliminary evaluation team recommendations. The Mission will have 15 working days to review the draft ER. The final ER will be due in 10 working days following the receipt of the Mission’s comments on a draft ER. The Contractor will use either a cover memorandum or similar format to explain how comments provided by the TO‐COR were addressed in the final ER. Please see the attachments for the format of the final ER. The evaluation team or the contractor must submit the final ER to the Development Experience Clearinghouse in accordance with USAID policy. VI. Evaluation Team Qualifications and Composition The evaluation team shall consist of three members: two technical specialists and an interpreter/administrative assistant. Both technical specialists must have extensive and documented experience in conducting performance evaluations. While Democracy, Human Rights and Governance (DRG) experience is required for both team members, one team member, ideally the senior team member, must have experience working on rule of law projects, especially in the rule of law and justice sector, and the other technical specialist must have experience in evaluating access to justice projects. USAID encourages participation of USAID staff on the evaluation team when his/her participation is considered beneficial for skill development, improving the evaluation quality, or for ensuring the use of evaluation results, and does not present a conflict of interest or a threat to the validity of the evaluation. It is anticipated that one or more USAID/Burma staff will accompany the evaluation team as observers and not as an evaluation team member. See Section F for Key Personnel. VII. Other Requirements To facilitate evaluation planning, USAID/Burma will make available to the Contractor information about previous USAID assistance in the rule of law sector, relevant portions of the contract including the statement of work, work plans for the past three years, and all available quarterly reports. The TO‐COR will email these documents to the evaluation team for preliminary review to enable the development of a realistic evaluation plan. When planning and conducting the evaluation, the Contractor will make every effort to reflect opinions and recommendations of all key stakeholders from the national and local governments, donors, civil society, and the private sector. The TO‐COR will provide the evaluation team with names and contact information for key individuals to be interviewed, including USAID/Burma and U.S. Embassy staff, the USAID/Burma Promoting Rule of Law Mid-term Evaluation 35 Project’s professional staff, the sub‐grantees, foreign donors based in Burma, and the Government of Burma and its affiliated agencies. The Contractor must submit an electronic version of the Evaluation Work Plan (EWP) to the TO‐COR within 15 working days following the award. The evaluation team will discuss any deviations from the EWP with the TO‐COR and seek USAID’s concurrence with the proposed changes in the EWP if those changes are significant, as determined by the TO‐COR. The evaluation team must invite the TO‐COR and other relevant USAID/Burma personnel to participate in all meetings and site visits planned in conjunction with the evaluation as soon as those events are on agenda. The evaluation team must conduct at least one weekly update for the TO‐COR and other relevant Mission personnel in order to keep them informed on the progress of the evaluation and any issues that may arise. The contractor is responsible for making all travel, transportation and lodging arrangements. The majority of potential informants from the justice sector institutions, the donor community, and leading CSOs are in Yangon and Naypyidaw , while the most conveniently located phase‐one and phase two pilot courts and/or project grantee activities are in Yangon, Bago, and Ayeyarwaddy Regions and Mon and Kayin States. Logistical support in country will be responsibility of the contractor. The evaluation team is required to submit all documents from the evaluation and all qualitative data collected by the evaluation team in an organized fashion and fully documented for use by those not familiar with the Project or evaluation. All qualitative data must be provided in an electronic file in machine‐readable, non‐proprietary formats. USAID will retain ownership of the survey and all datasets developed. USAID/Burma Promoting Rule of Law Mid-term Evaluation 36 ANNEX II: DATA COLLECTION PROTOCOLS AND QUESTIONS This document outlines the qualitative instruments for the mid-term performance evaluation of USAID/Burma’s PRLP. The contents of this document include: an informed consent statement, which must be presented to all respondent prior to beginning a KII or FGD; a KII guide and sample questions; and an FGD guide and sample questions. As not all respondents will have the same level of knowledge and experience with the PRLP, all respondents will not be able to answer all questions. The ET will document each interview and discussion, and periodically hold team meetings to analyze the data and establish initial findings. Major themes and conclusions drawn from qualitative data collected with these tools will be triangulated and verified against findings based on the desk review and SOs. The data collection questions will be finalized once the desk review is completed, any preliminary comments are received by the Mission based on this draft EWP, and additionally after the in-brief with the Mission has been conducted. This will allow the ET to refine the data collection tools for each key stakeholder. The ET will present the data collection tools to USAID at the in-briefing to solicit feedback and ensure that data collection is responsive to USAID and program needs. The final data collection tools will be included in the revised work plan and final report for USAID. INFORMED CONSENT STATEMENT The introduction and confidentiality protocol will be accurately translated and shared, yet adapted so the specific stakeholder in question understands its meaning. Introduction: Good morning/afternoon, and thank you for taking the time to speak with us today. As mentioned during our interview request, my name is [your name]. I’m working for Social Impact, which has been asked by USAID to do an evaluation of the Burma PRLP. We are independent evaluators and are not part of USAID or Tetra Tech. The results of the evaluation will be used by USAID and Tetra Tech to improve the remainder of the project, and help design similar projects in Burma in the future. Our ET has had the opportunity to review documents provided by USAID to get a better sense of the design and implementation of the PRLP. However, such documents can only tell us so much. We would like to speak with you today to hear about your experience, in your own words, in order to help us better understand various issues related to this project’s implementation, such as increasing certain aspects of justice system efficiency and transparency, and access to justice. Please note that, at any time, you may terminate the interview or decline to answer a specific question. You may also decline participation in this interview at any time. Procedures If you agree to participate, we will ask you a series of questions taking about [1-1.5 hours] of your time. We would like to record this discussion so that we can more easily remember what was said. With your permission, your answers will be audio recorded using a digital recorder. We can stop the recording at any time if there is something you would like to share off the record. Your name will not be included or attached to the recording in any way to protect your privacy. Risks/Benefits There is no large risk involved in your participation in this interview. The questions will not involve sensitive or personal information, and you can refuse to answer any question. The other participants in the group will be asked to keep what we talk about private, but this cannot be assured. Although this study may not benefit you personally, we hope that our results will add to the knowledge about strengthening rule of law in Burma. USAID/Burma Promoting Rule of Law Mid-term Evaluation 37 Confidentiality Protocol We will collect information on individuals’ organizations, and positions, but everything you tell us will be kept strictly confidential. A list of participants will be made available as an annex to the final ER, but those names and positions will not be associated to any particular findings or statements in the report. We may include quotes from respondents in the ER, but will not have any names associated with them; we may, however, associate quotations with an individual’s organizational affiliation, if you give us permission. We may also want to use a photograph of an individual or group. Do we have your permission to use a particular quote or photograph, without your name or other identifiable information in the report? Consent to use photograph or quotation (signature): ____________________ All information gathered will be used for the sole purposes of this evaluation, and will not be shared with other audiences or used for any other purpose. Once again, thank you for taking the time to speak with us today. Do you have any questions for us before we get started? Permission to Proceed Are you willing to join the interview/discussion?  Yes  No _______________________________ ________________ (Signature) (Date) Do we have your permission to record the interview?  Yes  No ______________________________ _________________ (Signature) (Date) USAID/Burma Promoting Rule of Law Mid-term Evaluation 38 KEY INFORMANT INTERVIEW GUIDE Interviewers provide an introduction that covers the following points:  Brief introduction of the ET members  Purpose of the evaluation and of the interview  Main topics of the interview  Confidentiality of responses  Request for and receipt of permission to interview Name(s): Partner Organization Name: Position(s): Location: Male ___ or Female ___ Date of Interview:___________________ Time of Interview:___________________ Name of Interviewer:___________________Name of Note-taker:___________________ Confidentiality explained: Yes_____ or No_____ Respondent(s) agreed to be interviewed: Yes_____ or No_____ Sample KII Questions OSCU Leadership and those working on issues of case management, IT/MIS strategic plan, and public information (individual questions to be adapted to particular staff/officers) 1. What have you found most useful about the assistance rendered to the OSCU by the PRLP? 2. How has the work of the OSCU and other judicial institutions changed as a result of the partnership with the PRLP? What can you point to as evidence of such change? 3. What have your judges found most useful about the practical skills training for new and sitting judges? For court staff? 4. What have been the tangible contributions of the Project to the OSCU’s strategic plan? 5. What are your expectations for the national case management strategy? What evidence can you point to that demonstrates progress with its implementation via the pilot court program? 6. Within the pilot court program thus far, which performance measures are most important to the judicial leadership? And what have you found most valuable about that program? 7. How can the pilot court program be improved and sustained? [probe efforts to better engage High Courts, judges (as peer trainers), lawyers, and other stakeholders] 8. Has the PRLP helped the judiciary better understand the value of, and utilize, data for management and policy purposes? What data and analysis has been most useful? 9. What are the biggest challenges facing implementation of the new Legal Aid Law? How can the PRLP be most useful in helping to address these challenges? [Continued assistance with stakeholder consultations? Research? Work on the implementation plan?] 10. What have been the biggest challenges encountered by your office in working with the project? 11. What have you found helpful about the project’s assistance in developing the Judicial Performance and Professional Development Strategy? USAID/Burma Promoting Rule of Law Mid-term Evaluation 39 12. What are some concrete ways in which the partnership or specific activities with PRLP could be modified or improved? Pilot Court Judges and Staff (individual questions may be adapted to particular judges and/or staff) 1. What aspects of the pilot court program have you found most useful and practical? How has the PRLP been of assistance? 2. How has the work of your court changed as a result of the partnership with the PRLP? What can you point to as evidence of such change? 3. Have the case management performance measures been helpful in making the work of your court more effective? What has been most valuable and satisfying, and why? [e.g. improved disposition rates, lower postponement rates, greater overall sense of control, learning new skills, etc.] 4. Have you been able to make use of data on court performance in your own work? Has this made a difference in your work? 5. Has it been helpful to learn about the results of the court users’ surveys? In what way? 6. What are the biggest challenges in implementing the performance measures? Why? 7. How could the pilot court program be improved? [probe issues of leadership, training, technology/automation, materials/libraries, cooperation from lawyers, more reflection and problem-solving] 8. What have you found most useful about the practical skills training for judges? [and staff?] How have you used these skills? How could the training be improved? 9. What have been the biggest challenges encountered by your office in working with the project? 10. What are some concrete ways in which the partnership or specific activities with PRLP could be modified or improved? UAGO Officials (to be adapted to different units/levels) 1. What has been the most valuable aspects of the PRLP assistance to your agency? Why? 2. Did you find the UAGO Law Office Assessment useful? What was most compelling and which recommendations might be prioritized for action in the next few years? [consolidating the case management system, improving pre-trial coordination, automation, specialized training] 3. What would you hope to achieve with a possible pilot law office program? What recommendations would be prioritized for implementation in these offices? 4. What expectations are there for a public information unit? What priorities exist? [improved website, media skills training, handling public information requests, etc.] 5. Has the PRLP been useful in contributing to the work of the working group drafting the law on violence against women? In what ways? How could this kind of engagement be improved? 6. How would you assess the usefulness of the legal skills training provided for UAGO law officers, including theory of change activities? How could this training be improved? 7. In what ways would you envision better linking improvements in the functioning of UAGO law offices to the work going on with the pilot court program in relevant jurisdictions? CSO Legal Aid/Other Grantees (to be adapted as needed to specific types of grantees) 1. How can the A2JI enhance the work of your organization? How can your unique skills and expertise be leveraged for maximum effect within the Initiative? [especially within key clusters]. 2. Do you believe the three clusters under the Initiative are the right ones, and are they organized appropriately? 3. Do you believe the Initiative is learning and adapting from lessons already learned regarding earlier advocacy and monitoring work? Is the Initiative as a whole being developed with the right organizational/collaborative approach? USAID/Burma Promoting Rule of Law Mid-term Evaluation 40 4. How would you assess your own organization’s work and policies in terms of gender-sensitivity? Why? 5. How would you assess the value of the grants you have received from the PRLP? What have you been able to do that you could not do previously? [focus not only on achievements—e.g. numbers of legal aid clients reached—but new modes of work, etc.] 6. How would you assess the value of the capacity-building assistance you have received from the project? What has been most useful? Has there been adequate mentoring? 7. How would you assess your capacity-building progress in this regard? [focus on new management methods, etc.]. What are the most challenging areas for the organizational development of your group? 8. What have been the most valuable skills training received by your staff? [probe as necessary: advocacy, capacity-building for lawyers, community-based awareness, legal and rights awareness building, paralegal training, etc.] 9. What new kinds of training might you require? [e.g. coalition and network-building skills, media and information skills, etc.]. Additional rounds of training in existing topics? 10. How would you envision the nascent Paralegal Network being used most effectively? 11. How does the work of your organization seek to reach and help certain vulnerable populations? How could the PRLP enhance those efforts? Parliamentary Committees 1. How would you assess the work of the PRLP generally, based on your knowledge of its work? 2. Where could the project add more value and direct more attention? 3. Could the project assist other union bodies in helping to bring more policy-relevant information and data to the attention of the committee? 4. Are there ways in which the project and its partners could better assist the committee in overseeing eventual implementation of the Legal Aid Law? 5. Has the committee found the ROL roundtables useful? How could they be made more valuable? PRLP Staff (will be utilized according to particular staff, including in small group interviews as well) 1. Which activities have proven most successful in advancing project objectives, and why? 2. Which activities have proven most successful in building trust with key government counterparts and why was this the case? With CSO partners? 3. What have been the biggest challenges faced by the project in gaining traction with the pilot court Program? The nascent A2JI? 4. What are the prospects now for the UAGO Pilot Law Office Program? What are the challenges? Will this result in better cooperation and action from the UAGO? 5. Which activities appear to have gained the most ‘local ownership’ and how has this been demonstrated? 6. Has the project conducted any kind of political economy analysis of the justice system, however informal? Was there any effort to do so after the elections or prior to the option period being exercised? Did this—or other reflective exercises—suggest a shift in emphasis in programming going into Year 4? 7. What are some of the key lessons learned from the project’s work thus far? 8. Which particular activities, including capacity-building assistance and training, components and activities have been most effective in strengthening CSO partner capacities, and how? What types of TA have appeared most effective? What metrics have been used to benchmark such progress (e.g. OEA, OCA methodologies)? Can a link be discerned between improved capacity and improved substantive results under Objective 2 components? USAID/Burma Promoting Rule of Law Mid-term Evaluation 41 9. Which kinds of collaborative work and capacity-building assistance have resulted in improved access to justice for various beneficiaries, including vulnerable groups? Which barriers to access to justice have proven least resistance to change? Most resistant? 10. What kinds of assistance and training have shown some success in linking to CSO grantee efforts to improve legal literacy among targeted communities/populations? 11. What synergies, if any, have there been between the CSO support work and the pilot court program in terms of legal literacy, citizen satisfaction, and improved access to justice? 12. Are there particular vulnerable groups that warrant more attention from the project in connection with its access to justice or legal literacy work? What kind of attention? 13. How has the project sought to integrate gender considerations into its various components and activities, particularly legal aid outreach and service delivery? Have these efforts followed the approaches/plans contained in the project’s Gender Action Plan? If not, why? 14. Which integration/sensitization activities/approaches have been most effective? Least effective? Why? 15. Are there other special focus initiatives/topical areas that could better assist women citizens as legal aid clients or otherwise? 16. Based on the work and reporting of some of the project’s grantees, can a link be discerned between these integration activities/methods and improved results in terms of gender equality in access to justice or legal literacy. If so, by what pathways might this have occurred? 17. How has project attempted to coordinate its work with that of other USAID and other donor￾funded IPs? How strategic is this coordination? 18. What has worked well and what has not with coordination? 19. Are there specific synergies with USAID and other donor programming that could be (better) exploited? 20. Are there notable lessons learned from other donor-funded projects about work with government or NGOs in the Burmese justice system? USAID/Burma Promoting Rule of Law Mid-term Evaluation 42 FOCUS GROUP DISCUSSION GUIDE Interviewers provide an introduction that covers the following points:  Brief introduction of the ET members  Purpose of the evaluation and of the FGD  Main topics of the of the FGD  Confidentiality of responses  Request for and receipt of permission to interview  Encouragement of full participation, varying opinions, and perspectives Name(s): Partner Organization Name: Position(s): Location: Male (#)___ or Female (#)___ Date of FGD:___________________ Time of FGD:___________________ Name of Interviewer:___________________ Name of Note-taker:___________________ Confidentiality explained: Yes_____ or No_____ All respondents agreed to be interviewed: Yes_____ or No_____ Sample FGD Questions FGD with Selected Legal Aid Beneficiaries in one locations (Most likely the Mawlamyine Justice Center) 1. What assistance have you received from the Center? 2. How did you find out about the service provided? 3. Why did you seek help from this organization? Why did you believe this organization could help? 4. What benefit did you receive from this provider? 5. What other kinds of recourse/help were available to you? 6. Did the assistance provided to you meet your needs? How? What did you like the most about the assistance you were provided? 7. How could services for people like you be improved in the future? Do you have any other suggestions? USAID/Burma Promoting Rule of Law Mid-term Evaluation 43 ANNEX III: LIST OF BACKGROUND AND DESK REVIEW DOCUMENTS USAID PRLP Reports - USAID PRLP Rule of Law Update - USAID PRLP Assessment of Court Administration - USAID PRLP Pilot Court Preliminary Evaluation - USAID PRLP Report on Access to Justice and Administrative Law - USAID PRLP UAGO Law Office Case Management Assessment - USAID PRLP Report on Strengthening Judicial Budgeting - USAID PRLP Pilot Court Preliminary Evaluation - USAID PRLP Quarterly Report - Apr-Jun 2014 - USAID PRLP Quarterly Report - Apr-Jun 2015 - USAID PRLP Quarterly Report - Apr-Jun 2016 - USAID PRLP Quarterly Report - Jan-Mar 2014 - USAID PRLP Quarterly Report - Jan-Mar 2015 - USAID PRLP Quarterly Report - Jan-Mar 2016 - USAID PRLP Quarterly Report - Jul-Sep 2014 - USAID PRLP Quarterly Report - Jul-Sep 2015 - USAID PRLP Quarterly Report - Jul-Sep 2016 - USAID PRLP Quarterly Report - Oct-Dec 2013 - USAID PRLP Quarterly Report - Oct-Dec 2014 - USAID PRLP Quarterly Report - Oct-Dec 2015 - USAID PRLP Quarterly Report - Oct-Dec 2016 Work Plans - USAID PRLP Year 1 Work Plan - USAID PRLP Year 2 Work Plan - USAID PRLP Year 3 Work Plan - PRLP Year 4 Work Plan Draft Other Project-Related Documents - PRLP Legal Aid Toolkit - PRLP Mid-Term Evaluation SOW - PRLP Contract - PRLP Gender Action Plan: Strategy for Integrating Gender in the USAID Promoting Rule of Law Project in Myanmar, 2014 - USAID PRLP Gender Action Plan: 2017-2018 - PRLP Activity Monitoring and Evaluation Plan, 2014 - PRLP A2JI Report on Access to Justice, 2016 - Various PRLP Grantee Milestone and Final Reports, 2015, 2016 - Final grantee reports from Independent Mon News Agency, Phoenix Association, Thwee Community Development Network, U Kyaw Myint Law Firm, Equality Myanmar Other Background Reports and Miscellaneous Documents Arnold, Matthew, Ye Thu Aung, Susanne Kempel, and Kyi Pyar Chit Saw. Municipal Governance in Myanmar: An Overview of Development Affairs Organizations. The Asia Foundation, Discussion Paper No. 7. July 2015. USAID/Burma Promoting Rule of Law Mid-term Evaluation 44 Brand, Marcus. Exercising Executive Power in Myanmar: Constitutional Options for Safeguarding the Rule of Law in Public Administration, Bingham Centre for the Rule of Law. November 2014. DLA Piper, Perseus Strategies, and Jacob Blaustein Institute for the Advancement of Human Rights, 2013. Myanmar Rule of Law Assessment. Washington: DLA Piper. Hook, David, Tin Maung Than, and Kim N. B. Ninh, Conceptualizing Public Sector Reform in Myanmar, Myanmar Development Resource Institute's Centre for Economic and Social Development (MDRI-CESD) and The Asia Foundation. June 2015. International Bar Association’s Human Rights Institute (IBAHRI). The Rule of Law in Myanmar: Challenges and Prospects. December 2012. International Commission of Jurists. Right to Counsel: The Independence of Lawyers in Myanmar. November 2013. Justice Base, 2016. Trial Observation Report, August, 2016 McCartan, Brian and Kim Jolliffe. Ethnic Armed Actors and Justice Provision in Myanmar, Asia Foundation, October 2016. Minoletti, P., 2014. Women’s Participation in the Subnational Governance of Myanmar. Yangon: Myanmar Development Resource Institute and Asia Foundation. Minoletti, Paul. Gender (in)Equality in the Governance of Myanmar: Past, Present, and Potential Strategies for Change. The Asia Foundation. April 2016. Myanmar Legal Aid Network (MLAW) and Enlightened Myanmar Research (EMR). Between Fear and Hope: Challenges and Opportunities for Strengthening Rule of Law and Access to Justice in Myanmar. September 2014. Office of the Supreme Court of the Union. Overview of the Judicial System in Myanmar, July 2016. Office of the Supreme Court of the Union. Strategic Plan 2015-2017 San Pe, Robert. Burma/Myanmar: Prospects for Democracy and Rule of Law, Bingham Centre for the Rule of Law. February 2014. UNDP, 2014. Rule of Law and Access to Justice in Myanmar (2013-2015). Fast Facts: UNDP Program Summary. New York: UNDP. Union Attorney General’s Office. Strategic Plan, 2015-2019 United States Institute of Peace, 2013. USIP Burma/Myanmar Rule of Law Trip Report, June 2013. Washington: USIP Rule of Law Center. Articles Araya, Y., 2013. “State Fragility, Displacement, and Development Interventions,” Forced Migration Review, No. 43. Oxford: Oxford Dep’t for International Development. Cheesman, N. and Kyaw Min Sa, 2014. “Not Just Defending: Advocating for Law in Myanmar,” Wisconsin International Law Journal 3(3), pp. 702-33. Cheesman, Nick, 2011. “How an Authoritarian Regime in Burma Used Special Courts to Defeat Judicial Independence. Law and Society Review. 45(4), pp. 801-830. USAID/Burma Promoting Rule of Law Mid-term Evaluation 45 Nardi, D., 2010. “Discipline-Flourishing Constitutional Review: A Legal and Political Analysis of Myanmar’s New Constitution. Australian Journal of Asian Law 12(1), pp. 1-34. Patanaik, B., 2006. “Perceptions, Responses, and Challenges in South Asia,” Forced Migration Review, No. 25. Oxford: Oxford Dep’t for International Development. Staples, K., 2013. “Fragile States, Collective Identities, and Forced Migration,” Forced Migration Review, No. 43. Oxford: Oxford Dep’t for International Development. Books and Book Chapters Cheesman, N., 2016. Opposing the Rule of Law: How Myanmar’s Courts Make Law and Order (Cambridge University Press, 2016). Cockett, R., 2015. Blood, Dreams and Gold: The Changing Face of Burma. New Haven: Yale University Press. Crouch, M, 2014. “The Common Law and Constitutional Writs: Prospects for Accountability in Myanmar,” in Crouch, M. and Lindsey, T., eds. 2014. Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. 141-157Harding, A., 2014. Crouch, M. and Lindsey, T., eds. 2014, Law, Society, and Transition in Myanmar. Oxford: Oxford University Press). “Law and Development in its Burmese Moment: Legal Reform in an Emerging Democracy,” in Crouch, M. and Lindsey, T., eds. 2014. Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. 377-399. Lindsey, T., 2014. “Unlike Any Land You Know About? Myanmar, Reform, and the Indonesia Model,” in Crouch, M. and Lindsey, T., eds. 2014. Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. 343-357. Nardi, D. and Moe, L., 2014. “Understanding the Myanmar Supreme court’s Docket: An Analysis of Case Topics from 2007 to 2011,” in Crouch, M. and Lindsey, T., eds. 2014. Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. 95-115. Selth, A., 2014. “Police Reform and the ‘Civilianisation’ of Security in Myanmar,” in Crouch, M. and Lindsey, T., eds. 2014. Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. 271-287 USAID/Burma Promoting Rule of Law Mid-term Evaluation 46 ANNEX IV: GRANTEE TRAININGS AND WORKSHOPS THROUGH BASE PERIOD Grantee Trainings and Workshops (through September 2016) Topic/Event Cumulative Achievement Trainings/Workshops Number of Participants M F Total Human Rights and LGBT Rights Trainings 13 193 138 331 Basic Legal Research Skills Training 1 8 10 18 Capacity Building Training or Junior lawyers and Paralegals 7 26 34 60 Community-Based Paralegal Trainings 17 163 257 420 Community-Based LegalAwareness Trainings 53 504 1019 1523 LGBT Rights and Paralegals Trainings 1 18 5 23 ParalegalAdvance Training 8 24 58 82 Referral Strengthening System Training 1 5 25 30 Community-Based LegalAwareness Workshops 272 3302 4764 8066 National Mock Trial Workshop 4 37 119 156 National Myanmar CLE Summer School Training Workshop 1 4 56 60 Campaign on GBV 35 1017 1664 2681 Training on GBV,CEDAW, Women Rights and Human RightsIssues 30 148 337 485 Communication, Presentation, Facilitation Skills Training 5 28 57 85 Case Management, Interviewing, Counseling Skills Trainings 12 110 164 274 Human Rights/Advocacy Trainings 3 14 47 61 Investigative Journalism Trainings 2 8 5 13 Citizen Journalismtrainings 2 14 8 22 Law Review Workshops 16 439 238 677 Total 483 6,062 9,005 15,067 USAID/Burma Promoting Rule of Law Mid-term Evaluation 47 ANNEX V: EVALUATION TEAM SI relied on a three-person ET composed of a TL/ROL and Justice Sector Expert, a mid-level Local Evaluation Expert, and a local interpreter/administrative assistant. Project Evaluation TL/ROL and Justice Sector Expert (Senior). Mr. Malcom L. Russell-Einhorn is a senior governance, ROL, and public administration specialist with 23 years of overseas development assistance experience including significant work in public administration, legal and regulatory reform, local government strengthening, legislative development, and other institutional capacity-building and governance systems strengthening programs. Currently, he is a Senior Fellow at the McCormack Graduate School of Public Policy and Global Studies at the University of Massachusetts, Boston. Mr. Russell-Einhorn has led several governance and ROL assessments for USAID, including developing governance vulnerability indicators for USAID projects, and has demonstrated the ability to adapt international best practices to complex local contexts. He recently worked as a TL for USAID DRG Cross-Sectoral Programming Assessment in Malawi, and a DRG performance evaluation for USAID/Zimbabwe. Project Evaluation Team Member. Mr. Aung Tun has over six years of professional experience in evaluating civil society access to justice programs in Burma transitional environments. He is a consultant for several development agencies in Burma in the area of social impact assessment. He has worked on the Mid-term Evaluation of USAID/Office of Transition Initiatives’ Kann Let program, focusing on five core support areas: peace process, intercommunal violence, media freedom, civil society engagement, and US foreign policy on Burma. Previously, he was a journalist working in Burma/Myanmar for several years. He is also involved in election and democracy projects, writing op-ed pieces and news analyses on Burma for various international media outlet including Asia Times online, The Diplomat, and local newspapers in both English and Burmese languages. Mr. Tun is presently affiliated with the Asian Development Bank as a liaison between the Bank and the government agencies which it funds. Interpreter/Administrative Assistant. Than Htike Oo has more than nine years of experience as a journalist—publishing articles in English and Burmese—and maintains extensive skills in administration and project management with international organizations. USAID/Burma Promoting Rule of Law Mid-term Evaluation 48 ANNEX VI: PRLP-SUPPORTED CSO WORK: AREAS OF ACCOMPLISHMENT AND CAPACITIES STRENGTHENED CSO Types Key Types of Activities Key Areas of Accomplishment and Capacities Strengthened Relevant Grantees and Other PRLP Partners Locations Topic: Promoting Greater Awareness of Individual Legal Rights/Legal aid provision *Basic human rights and LGBT rights training *Campaigns on GBV *low cost legal aid provision and paralegal work *Gender equality Training/workshops *Legal awareness *Psychosocial assistance for victims) *CEDAW awareness *GBV issues/awareness *Political-legal rights (Constitutional rights, citizenship rights, issues of federalism, as well as criminal, Penal, and Civil Procedure Codes, Convention on the Rights of the Child, Association Law, and the Child Law Improved knowledge on ROL, access to justice among stakeholders & relevant communities Improved outreach to women on many critical legal rights Improved partner CSOs’ capacity through training such as milestone reporting, financial reporting, project cycle management etc., plus improved ability to attract other donors. Unprecedented outreach to dozens of different townships re: info on critical individual rights LGBT Interest Groups, Justice Centers, leaders, NGO staff gender interest groups, social workers, lawyers, paralegals, legal aid groups, university students, communities at large Mon, Yangon, Pyay, Nay Phi Taw, Bago, Kayin, Mandalay Topic: Improving Access to Justice and Representation Skills of Lawyers and Paralegals *Free legal advice *Legal Presentation *Mediation and Counseling Cases *Paralegal training *Capacity Building for Junior Lawyers *Training on case management, interviewing, counseling, *Public consultations on access to justice/ROL *A2JIactivities (network development and access to justice and anti-corruption research and advocacy Improved access to justice via assisting significant numbers of criminal and civil cases Improved representation skills for lawyers and paralegals representing marginalized or vulnerable citizens Improved partner CSOs’ capacity such in narrative and milestone reporting, financial reporting, project cycle management etc., plus improved ability to attract other donors. Improved Interest group networks Legal aid groups, paralegals, legal clinics, lawyers, advocates, university students communities (including disadvantaged communities at large Yangon, Mon, Kayin, Pyay, Nay Pyi Taw, Bago USAID/Burma Promoting Rule of Law Mid-term Evaluation 49 CSO Types Key Types of Activities Key Areas of Accomplishment and Capacities Strengthened Relevant Grantees and Other PRLP Partners Locations *Support to ILAM *Capacity-building assistance to CSO partners *Legal Aid Law reform/amendments Topic: Civil Society Policy Development and Advocacy *Investigative and citizen journalism training *ROL reports *Land issues and women’s/minority rights *Policy advocacy (Local and Union Parliaments and governmental ministries/agencies) *Environmental issues and individual Rights Improved knowledge and access to justice, ROL Improved Advocacy/Legal Reform Improved partner CSOs’ capacity through training such as milestone reporting, financial reporting, project cycle management etc., plus improved ability to attract other donors. Improved Public Outreach and Public Participation Media groups; Legal research groups, CSO networks and individual CSOs Mon, Yangon, Kayin, Nay Pyi Taw, Mandalay, Bago Topic: Strengthening the Legal Professional as a whole *Continuing legal education activities *University legal education *Legal network activities/bar association activities *Legal ethics and discipline work *Networking between local and international legal/bar associations Improved interest group networks (stakeholder building) Improved Legal standards Improved legal education Improved legal professionalism Junior/senior lawyers/judges; law associations, legal practicing lawyers, law students and teachers Yangon, Mandalay, Nay Pyi Taw, Mon, Kayin USAID/Burma Promoting Rule of Law Mid-term Evaluation 50 ANNEX VII: PHOTOS OF PROJECT SITES FGD with lawyers and paralegals at the Mawlamyine Justice Center Front steps of the OSCU, Nay Pyi Taw USAID/Burma Promoting Rule of Law Mid-term Evaluation 51 Information kiosk at the Mawlamyine District Court (Phase II Pilot Court) USAID/Burma Promoting Rule of Law Mid-term Evaluation 52 Interview with judges at the Hpa-an Township Court (Phase I Pilot Court) Some of the law officer participants in the FGD on media relations training at the UAGO, Nay Pyi Taw. USAID/Burma Promoting Rule of Law Mid-term Evaluation 53 ANNEX VIII: EVALUATION WORK PLAN I. Introduction Since its launch in 2013, the Promoting Rule of Law in Burma project (PRLP) aims to promote more effective, accountable, and accessible justice sector institutions in the country, while also working to increase legal literacy and access to justice in several parts of the country—including through the participation of women and marginalized populations in several target states and regions. In conjunction with a wide range of host country partners, the project undertook a variety of legal reform activities with governmental and nongovernmental partners, assessing their capacity and commitment to achieve progress toward several intermediate project outcomes. These outcomes range from improvement of services provided by local courts, legal aid providers, and other justice system actors, to strengthened management and technical capacity of civil society organizations (CSOs) related to rule of law (ROL) reform, human rights, access to justice, and policy dialogue.138 Now, at roughly the mid-point of the project (in the final year of an extended base period), the United States Agency for International Development (USAID)/Burma is seeking an external mid-term performance evaluation to: (1) identify the extent to which the project has engaged successfully with justice sector stakeholders in Burma and responded to new opportunities to advance its stated objectives (including strengthening the capacity of key civil society partners to lower barriers to justice access); and (2) help inform USAID/Burma’s and PRLP’s strategic decision-making on potentially more focused programming choices for the remainder of the project. New opportunities for future engagement will also be explored. The evaluation will address key evaluation questions regarding the project’s success to date in engaging substantively with a number of critical justice system stakeholders in Burma; responding with agility to new opportunities to advance the broad objectives of the project; helping to build civil society capacity in legal literacy and access to justice, including for vulnerable groups and for women; and coordinating effectively with other donors and programs operating in the broader Burmese legal reform space. An important cross-cutting consideration is the degree to which the project as a whole utilized learning opportunities to adapt and shape technical and management approaches in the evolving local environment. A specific, related consideration is the extent to which political or managerial challenges encountered during the project’s implementation thus far were clearly discussed and thoughtfully considered, and how these challenges were addressed. In this sense, the purpose of the evaluation is not only to assess the IP’s performance as such; it is also to help the IP and USAID better understand how a developmental evaluation perspective—working innovatively in conditions of significant uncertainty and complexity—can further assist the project going forward. II. Project Background The PRLP is a five-year, $12 million activity—implemented by Tetra Tech—to promote the ROL and access to justice in Burma as a means of supporting the larger democratic transition and easing center￾periphery tensions. Under the project’s results framework—designed to promote a fair and just legal system and civil liberties—there are two principal objectives: (1) promoting more effective, accountable, and accessible justice sector institutions (helping promote better managerial capacity and legal system rationality, along with procedural transparency); and (2) increasing legal literacy, access to justice, and participation of marginalized populations in target regions/states. The project was designed at a time when Burma had just opened up to a significant degree to outside scrutiny and USAID was a significant, early aid provider in the legal reform environment. The project seeks to address these objectives while testing political will and capacity in a variety of institutions, particular apex institution such as the Office of the Supreme Court of the Union (OSCU), the Union Attorney General’s Office (UAGO), and the Judges Training Institute. PRLP’s scope was therefore USAID/Burma Promoting Rule of Law Mid-term Evaluation 54 very broad and flexible. The project has also sought to work with a government-supported pilot court program and participate in certain law reform activities aimed at legislative changes for which certain committees in the Parliament are responsible. Finally, the project has devoted substantial resources (including via grants and technical assistance [TA]/capacity-building support) to working with CSOs supportive of the project’s legal literacy, access to justice, and social inclusion objectives. These organizations are varied, ranging from legal education groups to community based legal aid providers to women’s organizations. In all of its work, the project incorporates significant legal policy reform and advocacy efforts, sometimes focused on key procedural or informational reforms. As the Mission has noted, “small changes in procedure can have a dramatic impact,”139 as illustrated by the change in April 2015 compelling police officers to testify and be subject to cross-examination in criminal cases. So too, court-user surveys in the pilot courts have provided valuable feedback to judicial authorities about critical problems affecting both efficiency and public satisfaction and perceptions of fairness. In general, the project’s base period witnessed the development of considerable trust-building and mapping of the justice system institutional terrain. A strong relationship was built with the OSCU and Attorney General’s Office, although the former has been significantly closed off from the international community and the latter has been weak on planning and capacity-building, despite significant attention having been lavished by other donors on this subject. To gain traction in this environment, PRLP did not emphasize issues of human rights or impunity directly, but rather made operational issues involving transparency and managerial rationality and accountability the leading edge of the project. At the mid-way point, several project activities are surfacing information about justice system stakeholder capacities and commitment to legal reform. With many other donors currently working, or anticipated to begin working in the same space, USAID/Burma appears interested in focusing and consolidating future activities in ways that will yield the greatest impact and play to PRLP’s comparative advantages relative to other donor-funded initiatives.140 There are two significant strategic developments pertaining to the second half of the project. The first is the bringing together of 40 CSOs under the umbrella of the Access to Justice Initiative (A2JI). This will consolidate and scale up a number of strands of the project’s work to gather data, conduct research, and monitor court performance—all with the goal of improving fact-based advocacy and helping reform￾minded justice sector stakeholders engage more effectively with the new government. The Initiative will also help anchor expanded TA to legal aid providers to improve service delivery (including via a Legal Aid Toolkit and a pilot legal aid database to enhance the work of legal aid clinics). Once the project’s option period is formally exercised, there are also plans to collaborate with the ILAM and provide a grant to an international nongovernmental organization (NGO) to help support a nationwide network of paralegals. A second development has to do with the UAGO plans to advance a prosecutorial law office program located in several of the many UAGO local offices throughout the country (with one such office in every one of the 330 townships). This program would address difficulties in training, adherence to the law, and issues relating to program management systems. PRLP is intending to deepen some of its capacity building activities with these offices, particularly with regard to data management and analysis (and the State Department’s Bureau for International Narcotics and Law Enforcement Affairs may be interested in funding some of this work as it relates to interaction with the police). Against this backdrop, USAID/Burma is interested in exploring the full range of opportunities and options open to them during PRLP’s remaining implementation period. Given the reportedly slow progress of some central capacity-building initiatives with the OSCU and UAGO (and significant existing or planned other donor work with the latter), there may be interest in devoting more attention to decentralized activities involving the pilot courts, local UAGO offices, and access to justice and legal literacy promotion work. III. Evaluation Purpose and Audience USAID/Burma Promoting Rule of Law Mid-term Evaluation 55 The evaluation is principally focused on assessing the project’s success to date in engaging substantively with various justice system stakeholders in Burma, including responding flexibly to new opportunities to advance the broad objectives of the project. However, the larger purpose of the evaluation is to help USAID/Burma and the PRLP—the principal audience for the exercise—more thoughtfully assess the program’s future trajectory. Such an assessment may include identification of future opportunities for engagement and how the scaling up of legal literacy and access to justice work can be shaped and sequenced, particularly through the A2JI and similar network-enhancing activities. The evaluation will also identify risks and assumptions underlying the project’s theory of change with regard to its second-half trajectory. In turn, it is understood from the evaluation team’s (ET’s) conference call with the Mission that USAID/Burma may rely on some of the evaluation’s conclusions and recommendations to help update the project’s overall statement of work (SOW) for the option period and influence and guide the finalization of a modified PRLP work plan. Also useful as an important cross-cutting consideration is the degree to which the project as a whole has utilized—or could better utilize in the future—learning opportunities (“Windows of Opportunity” as characterized in the project’s results framework) to adapt and shape technical and management approaches in the second half of the project. This can be assessed not only with reference to progress made on various activities, but political or managerial challenges encountered during the project’s implementation (and the extent to which the latter were clearly discussed and thoughtfully informed new management adaptations).141 Evaluation Questions For purposes of anchoring this mid-term evaluation and helping the Mission determine what PRLP has achieved thus far; what has worked well and where obstacles have been encountered; and above all, what new opportunities should (or might) be capitalized upon and what technical approaches might best be utilized and/or modified going forward, five evaluation questions were advanced in descending order of importance. They are: I. Identify the extent to which the Project has engaged successfully with justice sector stakeholders in Burma? II. Was the Project able to respond to new opportunities to advance its stated objectives? III. To what extent has the Project’s investment in civil society activities strengthened the capacity of civil society partners and reduced the barriers to access to justice for vulnerable groups? IV. Has the Project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality? V. How effective are the project’s activities in coordinating with other USAID and other donors’ programs? In accordance with the charge given to Social Impact, Inc. (SI) in the Mission’s SOW, the principal focus of the evaluation will be on questions one through three. At the same time, however, those questions cannot be fully and effectively answered without due attention to questions four and five. IV. Evaluation Timeline As confirmed with USAID subsequent to the initial conference calls with Mission personnel on November 7 and November 10, 2016, the full ET will meet in-country on January 8 to review the schedule of meetings and all issues requiring clarification from the Mission. The next day, the ET will hold an in-brief with the Mission to review the final comments on the evaluation work plan (EWP) and the detailed schedule for site visits and interviews. SI encourages USAID’s participation in all our evaluations, and the in-brief will provide an opportunity to confirm and work out the details of USAID participation in fieldwork, recognizing the benefits in terms of relationships, transparency, and context as well as tradeoffs related to potential response bias for certain respondents. USAID/Burma Promoting Rule of Law Mid-term Evaluation 56 Prior to departing Burma, the ET will hold a Mission out-brief, during which the ET will ask USAID to identify areas of particular interest and comment on the draft recommendations to improve their relevance, clarity, and actionable characteristics. After departing Burma, the ET will complete data analysis and finalize the draft evaluation report (ER) using USAID’s ER template. After receiving comments on the draft report, SI will incorporate them and provide a comment matrix to show how SI responded to each comment the Mission and/or IPs provided. The ET will also ensure the report addresses the gender issues outlined in the SOW and contains the evidence-based recommendation to improve gender equality. Upon Mission approval of the final report, SI will submit it to the Development Experience Clearinghouse. A specific timetable for both the fieldwork and analysis/report-writing is contained in Annex IV. The deliverables to be submitted are described in Annex V. V. Evaluation Details and Methodology The evaluation will use a mixed-methods approach that includes a desk review of documentary information (including considerable quantitative information gathered by the PRLP as well as other government information on the justice system relevant to its performance), key informant interviews (KIIs), site visits, and at least two focus group discussions (FGDs). To maximize participation in FGD’s, the ET will use an appreciative inquiry approach for some questions. As a strengths-based approach, appreciative inquiry focused on identifying what positive outcomes and experiences exist and how they were achieved.142 For example, appreciative inquiry will allow FGD participants in the legal aid context to identify methods or procedures that appeared to work well for them and other program participants. Appreciative inquiry questions may also be utilized in the context of some KIIs to help ascertain specific factors that appeared to generate project successes, and why—and also point to methods that could be used to improve other results. The ET is also examining the issue of gender disparities and sensitivities likely to be encountered during the evaluation and will be consulting with the SI’s Gender Specialist, Jennifer Mandel, to ensure that the evaluation design adequately takes into account evidence from gender analysis. Data Collection The ET’s evaluation design will incorporate a variety of targeted methods to address the evaluation questions by eliciting rich information as well as triangulating emergent trends and themes. Specific data collection methods will include: Desk Review of Key Documents and Other Secondary Materials As part of the desk review, the ET consulted a wide variety of background sources on the political and legal situation in Burma today as well as all of the PRLP documentation provided so far by the Mission. The ET will also review relevant non-project related documents such as peer-reviewed articles and other publications. Furthermore, the ET will collect other documentation in the field from key stakeholders, as available, particularly the PRLP team and a number of CSOs, including PRLP CSO grantees (certain interesting quantitative data may be available that was not obtained earlier). The findings from the desk review will inform the detailed evaluation design and will be used during data analysis and interpretation following fieldwork. Unstructured Site Observations The ET will conduct a limited number of unstructured site observations at a few pilot court sites and certain legal aid/justice centers (e.g. to see various protocols and intake procedures in action, evidence of public information dissemination features and use of MIS equipment, etc.). At present, it is anticipated that the ET will visit the dedicated legal aid center in Mon State (the Mawlamyaing Justice Center), the Hpa’an Phase One Pilot Court in Kayin State, the Phase One Hlaing Tharyar Township Pilot Court for the Yangon Region, and the Phase Two Township and District courts in Mawlamyine. These sites were selected via USAID/Burma Promoting Rule of Law Mid-term Evaluation 57 purposive sampling (based in large measure on the suggestions made by the Mission in the SOW) and in light of certain geographic/logistical constraints posed by the time parameters of the evaluation. In fact, the sample of pilot courts represents a large proportion of the existing sites within the Phase One and Phase Two programs. Key Informant Interviews The ET will conduct semi-structured interviews with a variety of key informants, including:  USAID/Burma Mission staff.  PRLP staff.  Key Burma legal system specialists located in and outside Burma (conducted remotely by phone)  GOB officials, including those knowledgeable about the project within the OSCU (including certain Hluttaw Committee members, those responsible for overseeing the pilot court initiative and MIS and case management work); Judicial Training Institute staff; Attorney General’s Office (including the Permanent Secretary and members of the Strategic Plan Implementation Committee as well as those responsible for legal aid implementation and the work of prosecutors in model offices); and pilot court staff and prosecutorial officials in selected sites.  PRLP grantees and network organizations, including those most centrally involved in legal literacy and legal aid-related activities (particularly the A2JI).  Project beneficiaries, including both legal aid-related groups and paralegals as well as principally male and female citizens with both resolved and pending cases.  Other donors and implementers working in the legal reform and especially access to justice space.  Other stakeholders, including international NGOs, journalists knowledgeable about legal reform issues, the Independent Lawyers Association of Myanmar, and certain legal community experts. A draft KII protocol can be found in Annex III. Focus Group Discussions The ET will seek to conduct an FGD with legal aid beneficiaries of the Mawlamyaing Justice Center in Mon State as well as judges from the Attorney General’s Office. This will provide additional perspectives on the client perspective of the work of a prominent legal aid provider and a member of the A2JI. A draft FGD protocol can be found in Annex III. Sampling: The KIIs and FGDs currently planned as part of the fieldwork represent a purposive sampling approach based on the suggestions stipulated in the Mission’s SOW for this evaluation, the ET’s desk review of relevant PRLP documentation, and natural time constraints posed by the geographic locations of PRLP activity and the manageable interest of the ET. In fact, the relative concentration of project activity and partners in the Yangon Region and in Mon and Kayin States, and the geographic proximity of the latter to Yangon makes this purposive sampling significantly probative of overall PRLP strategy and implementation. A draft interview schedule, with recommended sites for the fieldwork, can be found in Annex IV. As reflected in this schedule, the ET plans to travel to Naypyitaw, different sites in the Yangon Region, and a few different locations in Mon and Kayin States. Analysis Plan The ET will analyze data from across methodologies to triangulate findings in response to each evaluation question, providing more robust substantiation for conclusions. The ET will meet for several days in Burma at the end of the fieldwork period for purposes of preliminary analysis and development of the presentation of preliminary findings to present to USAID/Burma during the out-brief. All findings will be supported by primary and/or secondary data collected during the evaluation and will reference, where relevant and useful, representative examples of interviewee statements. USAID/Burma Promoting Rule of Law Mid-term Evaluation 58 Biases and Other Limitations There are several possible biases and other limitations that this evaluation will seek to minimize:  Selection bias may be a problem, either due to contacts for interviewees being given consciously or unconsciously by IPs or others due to the interviewees’ familiarity, convenience/proximity to the project, identification with favorable viewpoints, etc.  Response bias may be a challenge with both government and nongovernmental beneficiaries seeking to provide more favorable views due to a desire to receive continuing assistance or the like.  Recall bias is another problem. Respondents—particularly those interacting with a multiplicity of interlocutors or programs (which could be relevant for certain government and public beneficiary respondents alike)—may confuse one activity with another or have a tendency to blur or blend different experiences into a composite memory and response. These biases will be addressed through careful question articulation and follow-up/clarification questions, as well as by using multiple sources of data to triangulate answers to various inquiries. We will also use specific examples (‘anchoring responses’) to probe general responses in a more thorough way. Other biases may include: translation bias, which the ET will seek to correct by having the Local Evaluation Team Member check for translation errors or errors of interpretation; and gender bias (the ET’s all-male composition), which may cause some reticence in female interviewees, which the ET will address by putting the subjects at ease with preliminary informal conversation and/or by having a female colleague or CSO representative in the room at the same time. A broader limitation has to do with the duration of the fieldwork, which is just under three weeks. The condensed timeframe under which the ET will operate will limit the number of respondents. The same is true of geographic reach; the ET will be limited to project activity in Mon and Kayin when it comes to much of the access to justice and legal literacy work outside of Yangon. However, we are confident that in relation to the specific evaluation questions being asked of the ET by the Mission—which are less about the particulars of activity results and more about decisions of strategic focus and weighing program options—these limitations will be relatively minor in nature and highly unlikely to influence the strength and validity of the ET’s overall findings and recommendations. Gender and Social Sensitivity As noted above, SI will employ techniques that are gender-sensitive. This includes our interviewing of male and female KIIs to ensure as much diversity as possible within our purposive sampling constraints in a society like Burma. To ensure that both sexes are included in the sampling, the ET will conduct data collection activities at a location and time accessible to both women and men, and use gender appropriate data collection methods, including the questions to examine intended and unintended results affecting women and men, with consideration of confidentiality, privacy and cultural appropriation. SI will analyze data to assess if there are particular trends or themes that may be correlated with the gender of the respondent(s), and may determine that for purposes of eliciting better information and combating participant reticence, an all-women FGD may be utilized for project beneficiaries alongside a separate FGD for male beneficiaries. Questions will also be adapted based on local specialist input to ensure response bias and reticence are countered as effectively as possible. The use of a gender-sensitive approach will be crucial to ensuring valid responses to the evaluation questions that bear on successful engagement with certain stakeholders, including citizen beneficiaries of the legal aid and legal literacy activities. Quality Assurance Each evaluation product—from the EWP to the final report—undergoes a rigorous quality review, using checklists to ensure the delivery of usable, high-quality products that take into account gender USAID/Burma Promoting Rule of Law Mid-term Evaluation 59 considerations. SI has created an online dashboard which helps ensures SI’s delivery of efficient and expert services, allowing SI to track project deadlines and financial management while maintaining quality. Moreover, our Evaluation, Quality, Use, and Impact (EQUI) ™ trained team members and certified Team Leaders (TLs) use this tool to track the influence and value of EQUI™ evaluations while continuously improving the approach. Each checkpoint, the associated activities, and the ET member responsible is outlined below. See “Team Composition” section below for further information about the SI evaluation and program management team involved in the quality assurance (QA) process. 1. QA1: Work Plan and Evaluation Design. Following the EQUI™ QA criteria, the Project Director (PD) will review the feasibility and rigor of the proposed design, work plan, data collection tools and protocols. Special attention is given to ensuring that analytic tools are used to incorporate gender concerns and vulnerable groups into the evaluation design. 2. QA2: Preliminary Evaluation Findings. At the end of the data collection, the field teams will conduct meet to review and discuss their preliminary findings. Then the PD and the Project Manager (PM) will follow the EQUI™ QA criteria to assure that all evaluations questions are answered with empirical evidence and that any gaps have been addressed. 3. QA3: Presenting Interim Findings. The purpose of this checkpoint is to ensure that the ET is well prepared to present preliminary findings to USAID/Burma. Following the EQUI™ QA criteria, the PD will ensure that the SI presentation covers the main structural and presentational issues. 4. QA4: Draft ER. The PD will follow the EQUI™ QA criteria to ensure there is structure and logical linkages among the findings, conclusions, and recommendations, and that the draft complies with the reporting requirements for USAID/Burma. The PM will assist by verifying that data is accurately calculated and presented, and the Project Assistant (PA) will copy-edit and format the report. 5. QA5: Final ER. The PD and PM will use EQUI™ QA criteria to ensure quality of the executive summary, program and methodology description; adequacy of findings, analysis, conclusions, and final recommendations; compliance with USAID’s Evaluation Policy; and overall report presentation. Dissemination and Utilization SI is committed to using a utilization-focused approach to all of its evaluations. This means ensuring that this mid-term evaluation is fully directed toward, and aligned with, not only the Mission’s evaluation questions as such, but the larger program assessment and possible modification activities that are urgently required in the first and second quarters of 2017. In accordance with this utilization-focused orientation, SI will continue to actively engage with USAID/Burma and PRLP in the evaluation process. While the primary intended audience for this evaluation are the Mission and PRLP leadership, it is clear that other interested readers may include: USAID staff in Washington, DC and in the Regional Development Mission for Asia office in Bangkok; US Embassy staff in Yangon; and the United States Government Accountability Office (which we understand was conducting an overall assessment of United States Government assistance to Burma). The ET will engage USAID/Burma early on to gain a full understanding of primary and secondary intended users and also identify the range of potential evaluation uses. By doing so, SI can refine and implement an evaluation that specifically targets these purposes and creates a final report of maximum utility to the identified stakeholders. Team Composition SI will rely on a three-person ET composed of a TL/ROL and Justice Sector Expert, a mid-level Local Evaluation Expert, and a local interpreter/administrative assistance. The ET will also make use of the services of a Burmese Legal System Specialist, who will be available before, during, and after the fieldwork USAID/Burma Promoting Rule of Law Mid-term Evaluation 60 of the evaluation to provide information and perspectives on what the ET is learning. The Legal System Specialist may accompany the ET to fieldwork sites in Mon and Kayin to help interpret information received from KIIs and FGDs. Project Evaluation TL/ROL and Justice Sector Expert (Senior). Mr. Malcom L. Russell-Einhorn is a senior governance, ROL, and public administration specialist with 23 years of overseas development assistance experience including significant work in public administration, legal and regulatory reform, local government strengthening, legislative development, and other institutional capacity-building and governance systems strengthening programs. Currently, he is a Senior Fellow at the McCormack Graduate School of Public Policy and Global Studies at the University of Massachusetts, Boston. Mr. Russell-Einhorn has led several governance and ROL assessments for USAID, including developing governance vulnerability indicators for USAID projects, and has demonstrated the ability to adapt international best practices to complex local contexts. He recently worked as a TL for USAID democracy, human rights, and governance (DRG) Cross-Sectoral Programming Assessment in Malawi, and a DRG performance evaluation for USAID/Zimbabwe. Project Evaluation Team Member. Mr. Aung Tun has over six years of professional experience in evaluating civil society access to justice programs in Burma transitional environments. He is a consultant for several development agencies in Burma in the area of social impact assessment. He has worked on the Mid-term Evaluation of USAID/Office of Transition Initiatives’ Kann Let program, focusing on five core support areas: peace process, intercommunal violence, media freedom, civil society engagement, and US foreign policy on Burma. Previously, he was a journalist working in Burma/Myanmar for several years. He is also involved in election and democracy projects, writing op-ed pieces and news analyses on Burma for various international media outlet including Asia Times online, The Diplomat, and local newspapers in both English and Burmese languages. Mr. Tun is presently affiliated with the Asian Development Bank as a liaison between the Bank and the government agencies which it funds. Local Legal System Specialist. Mr. U Wai New Win holds over 40 years of experience working in the Burma legal system—serving as an advocate, judge, and legal clerk. Interpreter/Administrative Assistant. Than Htike Oo has more than nine years of experience as a journalist—publishing articles in English and Burmese—and maintains extensive skills in administration and project management with international organizations. Management and Responsiveness to USAID/Burma The TL is in charge of the day-to-day management of the ET and completion of all deliverables, including the EWP, data collection instruments, and the ER. The SI support team, comprised of the PD, PM, Gender Specialist, and PA, will play an instrumental role in managing the ET, collecting and synthesizing project background information, tracking project budgets, and providing QA on deliverables. The PD will conduct the final review of the evaluation deliverables and contribute overall technical guidance in the context of ongoing ET support. SI’s Gender Specialist will report to the PD and PM when conducting the gender review on technical deliverables. The PM will report to the PD and hold the primary responsibility for managing the ET and ensuring timely completion of contract deliverables. The PA will report regularly to the PM to ensure that administrative and logistics items are promptly arranged for the ET and support team. The PA will also review deliverables for contractual compliance and technical quality. Throughout the evaluation, both the TL and PM will be available to USAID/Burma technical and management staff. The PM will be responsible for working with the ET to respond to USAID/Burma’s requests as they may arise. If the request is of a more technical nature, the PM will delegate it to the TL or PD. In addition to as-needed requests, the PM will carry out systematic communications with USAID, including coordinating regular verbal check-ins with USAID to update the Mission on the status of the USAID/Burma Promoting Rule of Law Mid-term Evaluation 61 evaluation and gauge their satisfaction with the work. Before and after fieldwork, the PM will serve as the primary point of contact for USAID for management, administrative, and contractual matters. While the ET is in the field, the TL will be USAID’s primary point of contact for all matters, but the Contracting Officer’s Representative (COR) will have open communication with the entire ET via email and phone. USAID staff can also continue to contact the PM with any concerns or requests. In the event of any major issues, the PD will get involved immediately to ensure rapid resolution of the problem. USAID/Burma Promoting Rule of Law Mid-term Evaluation 62 ANNEX IX: EVALUATION DESIGN MATRIX Questions and Sub-Questions SI Sub-Questions to Guide Data Collection (i.e., general lines of inquiry consistent with the Mission’s Five Evaluation Questions and designed to inform data collection tools) Methodology and Instruments Data Source (1) Identify the extent to which the project has successfully engaged with justice sector stakeholders  Which activities have proven successful in building trust with key counterparts and in what specific ways has this trust or ‘buy-in’ been important to the project?  Which activities have proven successful in advancing project objectives substantively (relative to Objectives 1 or 2) and based on what evidence or metrics? (particularly with regard to various capacity-building activities). What key factors are responsible for this success?  Which activities appear to have gained some degree of local ‘ownership,’ demonstrated organizational initiative, and potential for sustainability, and how is this manifested?  Has the project conducted any kind of political economy analysis of the justice system? If not, has the project relied on any outside PEA? In either case, how has this type of analysis informed its work planning, including previous or currently contemplated shifts in program emphasis?  What shifts in strategy and/or programming are warranted in entering into the project’s option period, and why? Is there evidence to suggest specific modifications to any of the individual components under Objectives 1 or 2?  What are key lessons learned from the project’s work thus far? In particular, lessons  Document desk review  Unstructured SO  KIIs (semi-structured questionnaires)  FGDs  Desk review: project documents and CSO grantee documents may shed further light on the question  SO: Limited observation at pilot court sites, UAGO offices, and CSO legal aid sites  KIIs: PRLP staff; USAID personnel; local partner organizations; government counterparts (including OSCU and UAGO representatives); outside experts and other observers  FGDs: PRLP staff; program beneficiaries; project grantees; other ROL stakeholders USAID/Burma Promoting Rule of Law Mid-term Evaluation 63 Questions and Sub-Questions SI Sub-Questions to Guide Data Collection (i.e., general lines of inquiry consistent with the Mission’s Five Evaluation Questions and designed to inform data collection tools) Methodology and Instruments Data Source learned about the breadth of the PRLP SOW and the need to narrow it? (2) Was the Project able to respond to new opportunities to advance its stated objectives?  To what extent has the recent election affected the project’s work generally, and with respect to any of the answers given to the above questions?  What other events or developments afforded the project opportunities to advance component work under Objectives 1 and 2? Or opportunities to engage government or nongovernmental stakeholders in new ways?  How did the project respond to such opportunities, and how were these responses formulated? On the basis of what kind of internal analysis?  What if any managerial and staffing adaptations were entailed in responding to such opportunities?  What if any constraints (political, financial, managerial) prevented the project from responding to certain opportunities to advance its stated objectives?  How flexible and supportive was USAID/Burma in facilitating such responses? What kind of project modifications, if any, were needed to support such responses?  How has the project set aside time for learning and reflection to take stock of existing work or new opportunities? To analyze monitoring and evaluation (M&E) data? To  Document desk review  KIIs (semi-structured questionnaires)  FGDs  Desk review: project documents and CSO grantee documents may shed further light on the question  KIIs: PRLP staff; USAID personnel; local partner organizations; government counterparts; outside experts and other observers  FGDs: PRLP staff; program beneficiaries; project grantees; other ROL stakeholders USAID/Burma Promoting Rule of Law Mid-term Evaluation 64 Questions and Sub-Questions SI Sub-Questions to Guide Data Collection (i.e., general lines of inquiry consistent with the Mission’s Five Evaluation Questions and designed to inform data collection tools) Methodology and Instruments Data Source facilitate ‘bottom-up’ discussion and input from CSO partners? (3) To what extent has the project’s investment in civil society activities strengthened the capacity of civil society partners and reduced barriers to access to justice for vulnerable groups?  Which investments (components and activities) have strengthened CSO partner capacities, and how? What types of TA have appeared most effective? What metrics have been used to benchmark such progress (e.g. OEA, OCA methodologies)? Can a link be discerned between improved capacity and improved substantive results under Objective 2 components?  Which investments (components and activities) have resulted in improved access to justice for various beneficiaries, including vulnerable groups? Which barriers have proven least resistance to change? Most resistant?  What types of TA and CSO strategies have appeared most effective in PRLP’s access to justice work? Using what dispute resolution forums? What metrics have been used to benchmark such progress?  Which investments (components and activities) have resulted in improved legal literacy among targeted communities/populations?  Have there been any synergies between the CSO support work and the formal pilot court support work in terms of citizen satisfaction  Document desk review  Unstructured SO  KIIs (semi-structured questionnaires)  FGDs  Desk review: project documents and CSO grantee documents may shed further light on the question  SO: Limited observation at pilot court sites, UAGO offices, and CSO legal aid sites  KIIs: PRLP staff; USAID personnel; local partner organizations; government officials; project beneficiaries; certain knowledgeable experts in NGO management and civil society development in Burma  FGDs: PRLP staff; program beneficiaries; project grantees; other ROL stakeholders USAID/Burma Promoting Rule of Law Mid-term Evaluation 65 Questions and Sub-Questions SI Sub-Questions to Guide Data Collection (i.e., general lines of inquiry consistent with the Mission’s Five Evaluation Questions and designed to inform data collection tools) Methodology and Instruments Data Source and improved access to justice? What metrics might illuminate such synergies/progress?  Are there particular vulnerable groups that warrant more attention from the project in connection with its access to justice or legal literacy work? What kind of attention? (4) Has the project’s approach to integrating gender considerations into activities been effective in contributing to tangible improvements in gender equality?  How has the project sought to integrate gender considerations into its various components and activities? Have these efforts followed the approaches/plans contained in the project’s Gender Action Plan? If not, why?  Which integration/sensitization activities/approaches have been most effective? Least effective? Why?  Can a link be discerned between these integration activities/methods and improved results in terms of gender equality in any of the project’s main activity areas? If so, by what pathways has this occurred, and how can it be substantiated?  Document desk review  KIIs (semi-structured questionnaires)  Desk review: project documents and CSO grantee documents may shed further light on the question  KIIs: Senior PRLP staff; M&E staff; project gender specialist (5) How effective are the project’s activities in coordinating with other USAID and other donors’ programs?  How has the project attempted to coordinate its work with that of other USAID and other donor-funded IPs? How strategic is this coordination?  What has worked well, and what has not with coordination?  How has donor coordination or an analysis of comparative advantage and thematic emphasis informed PRLP programming shifts?  Are there specific synergies with USAID and other donor programming that could be  Document desk review  KIIs (semi-structured questionnaires)  Desk review: project documents and CSO grantee documents may shed further light on the question  KIIs: PRLP staff; local partners; M&E staff; observers of/in the development community USAID/Burma Promoting Rule of Law Mid-term Evaluation 66 Questions and Sub-Questions SI Sub-Questions to Guide Data Collection (i.e., general lines of inquiry consistent with the Mission’s Five Evaluation Questions and designed to inform data collection tools) Methodology and Instruments Data Source (better) exploited? Particularly sectoral programs and issues (e.g. via an administrative justice perspective) that could strengthen PRLP’s legal literacy and legal aid/access to justice issues?  Are there notable lessons learned from other donor-funded projects about work with government or NGOs in the Burmese justice system? USAID/Burma Promoting Rule of Law Mid-term Evaluation 67 ANNEX X: DISCLOSURE OF ANY CONFLICT OF INTEREST USAID/Burma Promoting Rule of Law Mid-term Evaluation 68 USAID/Burma Promoting Rule of Law Mid-term Evaluation 69 USAID/Burma Promoting Rule of Law Mid-term Evaluation 70 1 Crouch, M. and Lindsey, T., eds. 2014, Law, Society, and Transition in Myanmar. Oxford: Oxford University Press), pp. ___. 2 PRLP SOW, p. 5. 3 Id., Section C3.C, p. 6. 4 Id. 5 The project was designed with a base period of three years, which could be extended at USAID’s discretion via a two-year option period. At the end of the base period, which ended in September 2016, USAID/Burma exercised the option, extending the project through September 2018. 6 Interviews with USAID/Burma staff, November 10, 2016; January 9, 2017. 7 PRLP SOW, p. 8; Interviews with PRLP staff, January 9 and 10, 2017. 8 PRLP SOW, Section C.4., p. 7. 9 Although denominated a mid-term evaluation and originally intended to be conducted prior to the end of the base period, procurement of this evaluation was delayed by USAID/Burma and eventually launched contemporaneously with the exercise of the option period. 10 At the in-brief presentation and discussion for this evaluation held with the Mission on January 9, 2017, participants from the DRG team underscored the importance of this exercise for future programming judgments. 11 Though PRLP is categorized as an activity by ADS 201 definition, the evaluation questions posed by USAID/Burma refer to PRLP as “the Project.” 12 The data collection tools were gender-sensitive, including at least one question to capture gender differences relating to potential differential levels of results, issues of equality, and male/female empowerment. KIIs and FGDs were held at a location and time accessible to women and men. 13 Several FGD participants in Hpa-an came from surrounding, and in some cases relatively distant, townships. Expenses for travel, meals, and if needed, lodging, were provided. 14 Identities of certain participants in the FGDs were not recorded (e.g. subordinate judges and citizen beneficiaries), while in other instances, identities of interviewees were utilized for verification purposes during the writing up of the notes but then omitted from the final typed notes and the original handwritten notes destroyed. 15 Thanks to the large numbers of women represented in Burmese justice system institutions as well as IP, international NGO, and CSO communities, women outnumbered men in the ET’s interviews and FGDs. 16 The time limitations of the evaluation also made it difficult to interview many kinds of direct CSO grantee public beneficiaries (as opposed to those direct recipients of training or mentoring by grantees, such as CSO staff or volunteers). In many cases, these individuals (e.g. legal aid clients) may be difficult to locate at any given time, and/or may find it difficult to get away from work, family, or other competing obligations, thus necessitating advanced logistical scheduling not feasible under the final evaluation schedule. Still, others may be reluctant to participate in such a KII or FGD, may be reticent to share certain kinds of information, or be handicapped in evaluating some aspects of grantee activities (either due to their technical nature or the limited vantage point of the citizen). Nevertheless, had the evaluation not been under such tight time constraints, it is possible that at least one FGD of CSO citizen beneficiaries might have been feasible to organize (e.g. a certain group of similarly situated legal aid clients). 17 The ET was, however, able to interview the resident parliamentary strengthening adviser to the Parliament from NDI, along with PRLP staff and grantees knowledgeable about PRLPs efforts to engage with Parliament, providing a good sense of that dimension of the project’s work. 18 The OSCU also asked PRLP to facilitate meetings of their strategic plan implementation committee as they begin to develop a five-year strategic plan for 2018-2022. Interview with PRLP staff, January 23, 2017; Interview with USAID/Burma staff, January 23, 2017. 19 Interviews with PRLP staff, January 9, 10, 23, and 24; with USAID staff, January 9 and 23, 2017; with OSCU leadership, January 18, 2017; with CSO grantees, January 12, 13, 16, and 23, 2017. 20 There are four case processing indicators covering case clearance rates, case backlog reduction, lower postponement rates, and trial date certainty, some of which are tied to improved case scheduling and pledges by attorneys, police, and prosecutors to deliver witnesses to the court as planned. A fifth indicator addresses stakeholder overall user satisfaction with the court and its procedures among judges, court staff, lawyers, prosecutors, witnesses, and citizens/litigants. PRLP Preliminary Evaluation: Burma Pilot Court Project (May 2016), pp. 4-5. While the four efficiency indicators are unambiguous, the user satisfaction indicators are methodologically suspect in multiple ways, including susceptibility to a very significant risk of response bias (since the questionnaires are administered at the court and there may be reasons for all respondents, but especially citizens, to be reticent about their true sentiments). USAID/Burma Promoting Rule of Law Mid-term Evaluation 71 21 USAID PRLP Quarterly Report July-Sept. 2016 and Oct-Dec. 2016; Interviews with PRLP staff, January 9, 2017, Interviews and site visits to and interviews in Phase I courts in Hpa-an and Yangon, January 16 and 23, 2017 and Phase II court in Mawlamyine, January 16, 2016. 22 USAID PRLP Quarterly Report, Oct-Dec 2016, pp. 6-7; Interviews with OSCU leadership and staff, January 19, 2017. 23 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 7; Interviews with judges at Hpa-an Pilot Township Phase I Pilot Court, January 13, 2017; Interviews with judges at Mawlamyine Phase II Pilot District Court. 24 Interviews with PRLP staff, January 9 and 23, 2017. 25 Although court data is being collected by the pilot courts and reviewed by PRLP and the OSCU’s Case Management Committee, the OSCU has been slow to analyze the data in depth. Interviews with PRLP staff, January 10, 11, and 23, 2017. Interviews with PRLP staff, January 11 and 23, 2017. The OSCU has also been reluctant to utilize this data—or other existing data—for better planning, budgeting, or policy purposes. Interviews with OSCU judges, January 19, 2017. 26 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 13. 27 Under the law as passed, the OSCU has responsibility for its implementation. Interviews with OSCU leadership and the Legal Aid Working Group, January 19, 2017; Interviews with PRLP staff, January 10, 2017. 28 The willingness of the OSCU to quickly move forward on a reform effort of this magnitude with a USG-funded project, may appear to demonstrate the OSCU’s capacity to serve as a change agent. Interviews with PRLP staff, January 9, 10, and 23, 2017; Interviews with USAID staff, January 9, 23, and 25, 2017; Interview with donor representative, January 10, 2017. The pilot court initiative may appear to be a way for the judiciary to improve its abject reputation with the public and to get ahead of the reform curve in an unknown future where such reform gestures may win not only foreign and domestic praise but a higher bureaucratic profile relative to both the national budget and future foreign assistance funding. Some observers noted that the OSCU, with good salaries and working conditions in the capital, can afford to be reform-minded, and that resistance or incapacity to reform may come from the pilot courts. Interviews with PRLP staff, January 23, 2017; Interviews with CSO representatives, January 23, 2017. 29 PRLP Quarterly Report, July-Sept 2016; Interview with PRLP staff, January 9 and 10, 2017. 30 Interviews conducted with CSO/legal aid representatives in Mawlamyine on January 17, 2017; with CSO/legal aid representatives in Yangon on January 22 and 23, 2017, and with a current court clerk and retired judge in Yangon on January 22, 2017. Patterns of corruption are documented in a recent book by Australian scholar and Burma specialist Nick Cheesman, Opposing the Rule of Law: How Myanmar’s Courts Make Law and Order (Cambridge University Press, 2016). Cheesman documents written and oral testimony of such behavior over multiple decades, cites an OSCU letter to judges underscoring the need to avoid accepting ‘excessive’ payments, and documents moneymaking and fee sharing schemes in which case brokers, prosecutors, clerks, or relatives of a judge play key roles. Payments traditionally have a special legal slang—‘Section 870’—which is used euphemistically to mean paying a bribe. Cheesman, p. 175. Meanwhile, both the current court clerk and the retired judge interviewed by the ET on January 22, 2017 also cited as highly problematic a written instruction directing judges to sentence defendants in drug cases to the highest possible punishable prison term, even when the law as written permits a range of possible prison penalties and assumes judicial discretion. Interview on January 22, 2017. 31 Interviews with CSO/legal aid representatives, Yangon, January 23, 2017; Interviews with clerk and retired judge, Yangon, January 22, 2017. The retired judge and court clerk indicated that cases can be quickly disposed of with little regard for the interests of particular defendants, victims, or witnesses to provide better statistics for judicial superiors. 32 These are sensitive and nuanced topics in themselves for most citizens, and the fact that these matters are queried on court premises may potentially bias participant responses. 33 There are now information kiosks at each of the pilot courts, more approachable service counters, and benches for citizens awaiting appointments or court sessions. These were observed first-hand by the ET in Mawlamyine (Phase II), Hpa-an, and Hlaing Thayar (Phase II). Site visits on January 13, 16, and 23, 2017. Still, several practicing lawyers and legal aid practitioners interviewed said that as a behavioral and attitudinal matter, the pilot courts with which they are familiar are only marginally more welcoming to citizens in civil cases, and the efficiency and access reforms will have little bearing on the rampant corruption and intimidation in criminal cases. Interviews conducted in Yangon on January 22 and 23, 2017; Interviews conducted in Mawlamyine, January 16, 2017. Moreover, lawyer and CSO interviewees told the ET that ordinary citizens are still not free to observe proceedings in some of the pilot courts, as is otherwise advertised to the public by law and policy. Interviews conducted in Yangon on January 23, 2017. Still, there was optimism among some CSO interviewees that stakeholder dialogue on discrete issues, USAID/Burma Promoting Rule of Law Mid-term Evaluation 72 including attorney and family access to defendants in police custody, could result in tangible changes in practice and attitudes within the context of pilot court activities and heightened scrutiny. Id. 34 Interviews with CSO representatives, January 23, 2017. 35 Interview with PRLP staff, January 23, 2017. While most judges interviewed tended to dismiss such concerns, they were seen as significant problems by PRLP staff. Interviews with PRLP staff, January 9, 10, and 23, 2017; Interviews with judges, January 13 and 16, 2017. 36 USAID PRLP Quarterly Report, Oct-Dec 2016, pp. 11-12; Interviews with PRLP staff, January 10 and 11, 2017. The initiative was informed by a UAGO Law Office Case Management Assessment. That assessment also highlighted a very low conviction rate for simple felonies e.g. theft and assault—far below that associated with other crimes. It is possible that victims of these crimes are more assertive and sympathetic to judges, police, and prosecutors alike, but this highly unusual statistic may open the door to collaboration with UAGO on improved training for law officers in handling these kinds of cases (and examining more closely what might actually be transpiring in the criminal justice system in these areas). 37 PRLP Quarterly Report, Oct-Dec 2016, p. 11. Training is projected to begin this quarter. 38 PRLP Quarterly Report, Oct-Dec 2016, pp. 9-10; Interview with PRLP staff, January 11, 2017; Interview with UAGO officials, January 18, 2017; FGD with selected UAGO law officers having received media relations training, January 18, 2017. 39 The State Department’s Tier 3 designation (the lowest of three tiers) is defined as “[c]ountries whose governments do not fully comply with the minimum standards and are not making significant efforts to do so.” (Tier 1 countries comply with the minimum standards, while Tier 2 do not comply, but are making significant efforts to do so). https://www.state.gov/j/tip/rls/tiprpt/2014/226649.htm. 40 PRLP Quarterly Report, Oct-Dec 2016, pp. 7-9; Interviews with PRLP staff, January 11and 23, 2017. 41 Interview with PRLP staff, January 10; Interview with NDI representative, January 18, 2017. 42 PRLP Quarterly Report, Oct-Dec 2016, p. 9; Interview with NDI representative, January18, 2017. 43 Interview with PRLP staff, January 11, 2017; Interview with NDI representative, January 18, 2017. 44 Id. 45 Evidence of this multiplier effect, in terms of information disseminated and citizens reached, is discussed in the response to Evaluation Question 3. 46 A better approach, as discussed in the Recommendations section for Evaluation Question 1, would be to introduce court monitoring practices by activity staff, an outside organization, or both, as proxies for such surveys. Monitors could observe and record such matters as physical court access; courtesy to different types of court users, including according to gender, by different justice system participants, such as judges and court staff; explanations (or not) of law and procedure by judges; recognition (or not) of certain basic legal rights; and so forth. These data could be used for both qualitative and quantitative assessments as appropriate. 47 The OSCU Chief Justice wrote a letter to the Ministry of Home Affairs, the UAGO, and the President about the failure of police to ensure witnesses attend court on previously mutually agreed dates, but this did not result in any change in police/witness attendance. Interviews with PRLP staff, January 10, 2017; Interviews with OSCU judges, January 19, 2017; Interview with USAID/Burma staff, January 25, 2017. 48 Several other donor representatives and project counterparts/beneficiaries noted this in the course of various interviews. 49 PRLP SOW, p. 5. The contract provided a ‘window of opportunity’ clause that facilitated such flexibility, based on consultations with the PRLP COR. Id. 50 It helped that the Chief Justice was supported by a highly competent and dynamic administrator in the person of the Director General, who became the project’s key interlocutor. Interviews with PRLP staff, January 9 and 10, 2017; Interview with OSCU judges and staff, January 19, 2017. 51 Interviews with PRLP staff, January 9, 10, and 23, 2017. 52 Interview with Namati staff, January 10, 2017; Interview with PRLP staff, January 10, 2017. 53 Interview with A2JI leaders, January 10, 2017; Interviews with PRLP staff, January 10 and 23, 2017. 54 Interview with PRLP staff, January 10, 2017, FGD with Mawlamyine Justice Center attorneys and paralegals, January 17, 2017; Interviews with Yangon Justice Center attorneys, January 23, 2017. 55 Interview with PRLP staff, January 10 and 23, 2017; Interview with IBA representative, January 24, 2017. 56 Interviews with CSO representatives, January 13, 2017; FGD with CSO beneficiaries, January 13, 2013; Interviews with CSO/legal aid representatives, January 16, 2017; FGD with CSO/legal aid representatives, January 17, 2017. 57 Interview with PRLP staff, January 10 and 23, 2017; Interview with NDI representative, January 18, 2017; Interview with Justice Base representative, January 24, 2017. USAID/Burma Promoting Rule of Law Mid-term Evaluation 73 58 Interview with PRLP staff, January 10 and 23, 2017; Interview with OSCU leadership and judges, January 19, 2017. 59 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 8; Interviews with PRLP staff, January 10, 23, and 24, 2017. 60 Interview with Mon Parliamentary leadership representative, January 16, 2017; Interview with PRLP staff, January 23, 2017; Meeting with USAID DRG team, January 24, 2017. 61 Interview with CSO Representative, January 10, 2017. 62 The grant to Justice Base and support to A2JI provides a foundation on which to adapt court monitoring practices that can and should become more widespread as some courts open up to more outside scrutiny. 63 PRLP SOW, pp. 12-15. 64 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 17. Ultimately, 17 local grants had funds disbursed (one CSO grant was terminated early on), and two grants were also issued to Namati (to undertake establishment of a national paralegal association) and the IBA, respectively (IBA will help support the creation and initial operations of a secretariat for the new ILAM). 65 Although PRLP did not issue any grant solicitation specifically focused on public engagement in policymaking or the like, it was widely understood that PRLP’s access to justice and legal awareness themes significantly involved a number of public engagement and policy advocacy activities. At the same time, while PRLP did not specifically target grants or other activities in the aggregate at marginalized populations as such (which might be conflated by some with ethnic minorities), it was similarly understood that the vast bulk of CSO beneficiaries encompassed economically disadvantaged citizens and other vulnerable groups, including those at risk for GBV and TIP. 66 It was mentioned during the course of interviews with PRLP staff that PRLP already notified USAID/Burma that it reached its threshold of 80 percent expenditure of contract funding. Interviews with PRLP staff, January 10 and 24, 2017. 67 USAID PRLP Quarterly Report, Oct-Dec 2016, pp. 23-24. Totals are higher than might be expected on a project of this length and size based on training-of-trainers methodologies utilized by a number of PRLP CSO grantees with trainees who reached a large number of beneficiaries, particularly in the case of trainings on community legal awareness and GBV. For example, in the case of the former, some 325 trainings were conducted, with a total of 9,585 participants, nearly a third of the training totals for the base period. 68 FGD with public awareness trainees of the Kayin office of the Women’s Organization Network (WON) and the Kayin Women’s Empowerment Group (KWEG), January 13, 2017. 69 Interviews with representatives of KWEG and WON, Hpa-an, January 12 and 13, 2017, respectively; FGD with KWEG and WON public awareness trainees, January 13, 2017; interviews with representatives of Phoenix Association, Yangon, January 11 and 23, 2017, respectively; interview with 88 Generation community organizer, Hpa￾an, January 12, 2017. 70 FGD with legal awareness trainees, Hpa-an, January 13, 2017. 71 Interviews with CSO representatives, Hpa-an, January 13, 2017; FGD with legal awareness trainees, Hpa-an, January 13, 2017; Interviews with CSO representatives, Yangon, January 23, 2017; FGD with legal aid attorneys and paralegals, Mawylamyine, January 17, 2017. In addition to facing possible criminal penalties for public assembly deemed suspect or unlawful, distribution of certain printed material was regularly prosecuted under military rule, e.g. those distributing information about the United Nations Declaration of Human Rights were routinely arrested. 72 Interview with representatives of the Mawlamyine Justice Center, Mawlamyine, January 16, 2017; FGD discussion with Mawlamyine Justice Center attorneys and paralegals, Mawlamyine, January 17, 2017. 73 Interview with PRLP civil society team members, January 10, 2017; interviews with representatives of KWEG and WON, Hpa-an, January 12 and 13, 2017, respectively; interviews with representatives of Phoenix Association, Yangon, January 11 and 23, 2017, respectively; FGD with attorneys and paralegals at the Mawlamyine Justice Center, Mawlamyine, January 17, 2017. 74 Interviews with representatives of WON, Hpa-an, January 13, 2017; FGD with attorneys and paralegals at the Mawlamyine Justice Center, Mawlamyine, January 17, 2017. 75 Interviews with CSO representatives, January 13, 2017; FGD w/CSO beneficiaries, January 13, 2017; FGD with CSO/legal aid representatives, January 17, 2017; Interviews with CSO/legal aid representatives, January 23, 2017. 76 Interview with PRLP civil society team members, January 10, 2017; interviews with representatives of KWEG and WON, Hpa-an, January 12 and 13, 2017, respectively; interviews with representatives of Phoenix Association, Yangon, January 11 and 23, 2017, respectively; FGD with attorneys and paralegals at the Mawlamyine Justice Center, Mawlamyine, January 17, 2017. 77 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 17. 78 Interviews with PRLP civil society team members, January 10, 2017; interviews with representatives of KWEG and WON, Hpa-an, January 12 and 13, 2017, respectively; interviews with representatives of Phoenix Association, USAID/Burma Promoting Rule of Law Mid-term Evaluation 74 Yangon, January 11 and 23, 2017, respectively; FGD with attorneys and paralegals at the Mawlamyine Justice Center, Mawlamyine, January 17, 2017. 79 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 17; Interviews with staff lawyers from the Hlaing Tharyar Branch of the Yangon Justice Center, Yangon, January 23, 2017. 80 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 17. Men greatly outnumber women in seeking out legal assistance due to being charged with many more criminal violations; however, there is some evidence that more women have sought out legal help at the two PRLP-funded justice centers since their two respective grants were received, possibly reflecting better outreach efforts and women’s comfort level seeking assistance. 81 Interviews with justice center representatives, Mawlamyine and Yangon, January 16 and 23, 2017; FGD with Mawlamyine Justice Center attorneys and paralegals, January 17, 2017. Representation is more effective and outcomes are improved if crimes involve victims (e.g. theft, rape, assault cases). In other cases (e.g. weapons, gambling, drug cases), judges and law officers may be less motivated to act properly and planted evidence and coerced confessions and corrupt plea deals are exceedingly common. 82 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 19, interviews with Mawlamyine Justice Center representatives, January 16, 2017. 83 USAID PRLP Quarterly Report, July-Sept 2015; USAID PRLP Quarterly Report, Oct-Dec 2016, p. 18; Interview with Mawlamyine Justice Center representatives, January 16, 2017. The Toolkit’s launch event attracted over 200 Burmese lawyers, paralegals, law students, and CSO representatives. 84 USAID PRLP Quarterly Report, Oct-Dec 2016, p. 17. The grant also enabled the project to publish a report on the pilot, entitled From Victims to Agents of Change: Lives and Voices of LGBT Individuals as well as three short films on the experiences of selected paralegals and lawyers handling legal aid cases for transgender clients. 85 Interview with CSO representative, January 10, 2017; Interview with PRLP staff, January 10, 2017. 86 Interview with PRLP staff, January 23 and 24, 2017 87 Interviews with CSO representatives, January 12, 17, and 23, 2017. 88 Brian McMcartan and Kim Jolliffe, “Ethnic Armed Actors and Justice Provision in Myanmar,” The Asia Foundation, October 2016. 89 USAID PRLP Quarterly Report, July-Sept 2015 90 Interviews with Yangon and Mawlamyine Justice Center representatives on January 23, 2017 and January 16, 2017, respectively. 91 Interview with Independent Mon News Agency representative, Mawlamyine, January 17, 2017. 92 PRLP PMEP, Indicator No. 7. 93 PRLP Quarterly Report, Oct-Dec 2015; PRLP Quarterly Report, Oct-Dec 2016, p. 16. The research involved monitoring of criminal trials for access to justice problems, research on the effectiveness of CSO-led public advocacy campaigns, and research on ways in which 13 current laws of interest to the CSO community—including the Association Registration Law, the Legal Aid Law, and the Labor Organization Law—negatively impact access to justice. 94 Interviews with NDI representative in Nay Pyi Taw, January 18, 2017 and PRLP representatives, January 23 and 24, 2017, respectively. 95 The A2JI is presently expected to become a registered organization with a small staff that can access donor funding, potentially with assistance from PRLP, but doubts exist as to whether the individual member organizations can devote the time to A2JI that will allow it to thrive (while also acknowledging that some A2JI activities may even compete with, and potentially draw some donor funding away from, their organizations). Interviews with PRLP staff, January 23, 2017 and USAID staff, January 23, 2017. Meanwhile, even at present, the coalition’s report on access to justice issues was reportedly judged by many observers to be diffuse and superficial in many respects. A lot may depend on the skill and commitment with which the group addresses its research and advocacy activity for 2017, which concerns anti-corruption policies and law in Burma. A Catch-22 situation may exist, in which adequate funding may be needed to fund quality research by a dedicated group of A2JI designated researchers, but funding may depend on not only a registered entity existing, but reasonable evidence of capacity and commitment to deploy a dedicated research team. Interviews with PRLP staff and A2JI members, respectively, Yangon, January 10, 2017; Interview with PRLP staff, January 23, 2017. 96 Interview with PRLP staff, January 10, 2017, January 23, 2017. 97 Interview with faculty and staff at Yangon University Law Department, Yangon, January 11, 2017; Interview with PRLP staff, Yangon, January 24, 2017. 98 Interview with IBA representative, January 24, 2017; PRLP Quarterly Report, Oct-Dec 2016, p. 21. USAID/Burma Promoting Rule of Law Mid-term Evaluation 75 99 Final grantee reports from Independent Mon News Agency, Phoenix Association, Thwee Community Development Network, U Kyaw Myint Law Firm, Equality Myanmar. 100 CSOs partner interviews in Yangon Region, Mon State, and Kayin State, January 10-22, 2017. CSOs were especially gratified by PRLPs responsiveness to the presentation of initial ideas or concepts in terms of actual local needs and the CSOs actual capacity and skills to address those needs or concepts in terms of actual local needs and CSOs’ capacity and skills to address those needs. 101 Interviews with PRLP CSO grantees in Kayin and Mon States and Yangon Region, January 12, 13, 16, 17, 23, and 24, 2017. 102 Interview with members of the PRLP grants team, January 24, 2017. 103 Grantee interviewees in Kayin and Mon States and Yangon Region, January 12, 13, 16, 17, and 23, 2017. Several PRLP staff acknowledged that this is a very novel kind of project in Burma and both PRLP and its grantees are ‘learning by doing.’ 104 Interviews with CSO grantees, January 13 and 24, 2017; Interview with PRLP grants team, January 24, 2017. 105 On the other hand, some CSO leaders warned that the current National League for Democracy (NLD)-led government at times appeared even more closed to civil society than the previous government, and prone to excessive centralization. It is critical that A2JI and others gain access to lobby effectively for amendment of several laws that impinge on fundamental human rights, such as the infamous race and religion protection laws and Article 66(d) of the Telecommunications Law. 106 Currently, the project provides questionnaires on a limited number of legal knowledge topics to those receiving certain kinds of legal awareness and access to justice-related trainings or other educational forums. But these cohorts are not tracked subsequently for their views (whether or not they are exposed to any additional access to justice/legal awareness activities) and PRLP and other USAID implementers were discouraged by the Mission from attempting to undertake anything in the way of household surveys in Burma that would permit follow-up questioning. The new AMEP will reportedly include some such repeat surveys, at least with regard to follow-up with women beneficiaries of PRLP activities. Interview with PRLP staff, January 9 and 10, 2017; Gender Action Plan: 2017-2018, p. 16. 107 In Burma, this can include such informal means such as reliance on Literature and Cultural Associations in ethnic areas where there is strong religious leadership; mediation of land disputes via village land committees composed of village heads and village elders, along with GAD officials; and ethnic armed organizations’ justice systems, particularly in conflict-affected areas, which may include both formal justice mechanisms and customary mediation. Brian McCartan and Kim Jolliffe, Ethnic Armed Actors and Justice Provision in Myanmar, Asia Foundation, October 2016. 108 Interview with CSO leader in Yangon, January 10, 2017. 109 Interview with a non-grantee CSO in Hpa-an, Kayin State, January 14, 2017. 110 Interview with PRLP staff, January 23, 2017; Interview with USAID/Burma staff, January 23, 2017. 111 PRLP Gender Action Plan, pp. 4-7. 112 An initial plan was drafted in May 2014; it was succeeded by a revised plan in December 2016. PRLP Gender Action Plan: 2017-2018, pp. 4-5. 113 Under-representation of women in the current NLD-led government, for example, is egregious; women represent only five percent of Union ministers, 2.4 percent of Regional or State Ministers, none of the 330 township administrator positions, and only 40 of the 16,785 ward/village tract administrator positions (0.25 percent). PRLP Gender Action Plan: 2017-2018, p. 6. The absence of women serving at the lower levels of government is especially worrisome, as this is where women’s day-to-day needs are most effectively addressed and where political leadership experience is otherwise gained for advancement to higher levels of government. Discrimination is much less acute in the judicial system where women represent 52 percent of township and district courts nationwide, but the vast majority of chief judge positions are held by men. 114 PRLP’s research, though not using a representative sample, asserts that, 90 percent of women reported not using the formal justice sector (although this fundamentally reflects an overall distrust of the courts, regardless of gender) and 84 percent had not been to court previously. Some 16 percent of women indicated that someone opposed a decision to adjudicate a court case, such as a family member or spouse. Child care responsibilities, transport costs, and financial constraints severely limited women’s ability to access the courts, particularly in light of the fact that customs prevent married women from engaging in financial transactions or securing transport without permission of the male spouse. Other issues keeping women from court included shyness or shame, fear, and lack of education. PRLP Gender Action Plan: 2017-18, pp. 4, 12-13. 115 These include the Buddhist Women’s Special Marriage Law and the Health Care for Population Control Law. These controversial laws were pushed by Buddhist nationalist groups called MaBaTha ahead of the 2015 elections, USAID/Burma Promoting Rule of Law Mid-term Evaluation 76 provoking a clash between these groups and CSOs working on gender issues. There are potential clashes ahead, especially since the NLD is considering amendments to the laws and MaBaTha has strong connections to the previous Union Solidarity Development Party-led government. 116 PRLP Gender Action Plan: 2017-2018, p. 16; interview with PRLP staff, January 10, 2017. 117 PRLP Gender Action Plan: 2017-2018., p. 16. 118 This may be due to the complexity of these inquiries given the different methods used by various grantees to reach citizens, including women facing particular issues of legal literacy and security with respect to land ownership, GBV, and TIP. 119 The trainings included legal awareness for CEDAW, human rights and women’s rights, gender discrimination, referral services and systems for GBV survivors, domestic violence prevention, and the long-term needs of GBV survivors. PRLP Gender Action Plan: 2017-18, p. 17. 120 Interviews with PRLP staff, January 10 and 23, 2017. 121 PRLP Gender Action Plan, December 2016; Interview with PRLP staff, January 10, 2013; Interview with CSO representatives, January 10, 2017. 122 Even if seized land is returned, the community then decides how to distribute the collectively returned property, and without registration documents, women are even less likely to get the correct amount of land returned. See the report on land rights and tenure by Namati, https://namati.org/wp-content/uploads/2016/03/Namati-Gender-policy￾brief-FINAL-1.pdf. 123 PRLP Gender Action Plan: 2017-2018, p. 21. 124 PRLP Gender Action Plan: 2017-2018, pp. 20-21. If the POVAW bill passes, PRLP may be in a position to provide training to judges and law officers on the issuance of restraining orders. 125 However, the AMEP is focused primarily on court-centered activities in which PRLP is directly involved; insofar as there is much less attention paid to legal aid and awareness activities carried out indirectly by CSO grantees, that work ideally deserves more study. Indeed, it is precisely the importance of having a better understanding of these methods and pathways that argues for additional research and evaluation as discussed above in connection with Evaluation Question 3. 126 Interviews with several IP staff, January 9, 10, 23, and 24, 2017. 127 For example, most project directors or other senior staff interviewed for this evaluation were notably in the dark about the actual work and activity level of UNDP’s ROL resource and training centers. There is also a lack of awareness about many of the specific activities being undertaken by JICA with the OSCU and other counterparts. The entire international ROL implementing community in Burma was unaware of DANIDA’s four-year, $8.9 million ROL and Human Rights Project, put out for bid in August of 2016, until its award in late December 2016. Interviews with IP staff, Yangon, January 10 and 24, 2017 128 Interviews with PRLP staff, January 23 and 24, 2017. 129 Interviews with several IP staff, January 10, 23, and 24, 2017. 130 Trust of the parties is relatively high based on the current interpersonal relationships. According to some interviewees, it is likely that DANIDA representatives will be welcomed to the meeting to ensure coordination with a new program that has the potential to overlap with existing donor-funded ROL work, particularly regarding initiatives with OSCU, UAGO, and support for the organization and leadership of the legal profession. Interviews with IP staff, Yangon, January 10, 23, and 24, 2017. 131 Interviews with IP staff, January 10 and 23, 2017. My Justice and PRLP leaders communicated frequently since the My Justice program started up last year, motivated by their shared interest in ensuring as much work planning clarity as possible in the general legal aid/access to justice space, while avoiding confusion on the part of many CSO partners. This was relevant to PRLPs provisional decision to let My Justice fund most of the near-term needs of both the Yangon and Mawlamyine Justice Centers—in part to avoid such confusion, but also in recognition of My Justice’s superior financial resources at this stage of the PRLP project cycle. Other PRLP legal aid work with the centers may come with the development of additional satellite offices. Id. 132 Interviews with PRLP, USAID, and other USAID IP staff, January 10, 18, 23, and 24, 2017. 133 Interviews with PRLP and other USAID IP staff, January 10, 18, 23 and 24, 2017. 134 Interviews with PRLP staff, January 10 and 24, 2017. PRLP staff reported that they were in occasional touch with representatives from USAID’s main economic growth project, and discussed issues of intellectual property rights regulation and commercial dispute resolution (the OSCU indicated an interest in exploring the establishment of a separate commercial court system). Otherwise, at the present time there appeared to be few other pressing reasons for PRLP to interact with other USAID implementers, notwithstanding potential learning that could take place USAID/Burma Promoting Rule of Law Mid-term Evaluation 77 between PRLP and USAID’s Tenure and Global Climate Change Project or Civil Society and Media Project (although PRLP’s grants team has communicated with the latter about certain USAID and local compliance issues). 135 So too PRLP has been collaborative with several other donors (e.g. UNDP and DANIDA) in working with the IBA to build the organizational structure and capacity of ILAM. 136 Interviews with other donor IP representatives, January10 and 24, 2017; interviews with CSO representatives, January 10 and 23, 2017. Even as the DANIDA project comes online with potential programming with these two institutions, this general division of labor will continue: DANIDA will work on high-level policy roundtables and structural reforms, while PRLP will continue to refine and carefully calibrate the expansion of the respective pilot court and law office initiatives. This will also likely be the case with legal aid work, where PRLP and My Justice will continue to work with practitioners on supporting better legal awareness and representation work and paralegal expansion and professional certification, while others such as DANIDA focus principally on higher order issues of human rights protections and policies supportive of legal aid expansion. 137 As designed, PRLP is focused primarily on formal justice institutions under Objective 1—institutions that will take many years to reform to make more hospitable to women. Notwithstanding the attention paid to GBV and TIP cases that do end up in the formal system and should be handled in improved ways, it is important not to lose sight of the fact that many issues affecting women’s lives are likely to be addressed in informal ways that need to be better understood and prioritized for local and national policy attention. 138 The Project Results Framework lists these as two of the most salient among the eight intermediate outcomes under Objectives 1 (“Justice sector institutions are more effective, accountable, and accessible”) and 2 (“Legal Literacy, Access to Justice, and Participation of Marginalized Populations is Increased”). 139 PRLP Mid-term Evaluation SOW p. 6. 140 This was made generally known in a conference call with Mission staff on November 10, 2016. 141 If desired, the evaluation can help the implementer and USAID better understand how a developmental evaluation perspective—working innovatively in the current heightened conditions of uncertainty and complexity in Burma— can further assist the project going forward. 142 Appreciative inquiry centrally involves the practice of asking questions that strengthen an organization’s or group’s capacity to apprehend, anticipate, and heighten positive potential. See, e.g. Cooperrider, David L, et. al., 2000. Appreciative Inquiry: Rethinking Human Organization Toward a Positive Theory of Change, Stipes Publishing. USAID/Burma Promoting Rule of Law Mid-term Evaluation 78 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523