DISCLAIMER: The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. EVALUATION REPORT Justice for All Final Performance Evaluation Above: Participants in Justice For All Activity, Rajshahi, Bangladesh (Photo Credit: National Center for State Courts) USAID/Bangladesh AID–486–I–14-00001 (IDIQ); AID–388–TO–17–00001 (Task Order) May 11, 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mary Noel Pepys, Kyra Buchko, Saqeb Mahbub, and Shahidul Islam on behalf of Social Impact, Inc. Contact: Julia Rizvi Home Office Representative, Social Impact, Inc. Telephone: 703.465.1884 x248 2300 Clarendon Blvd., Suite 1000 Arlington, VA 22201 CONTENTS Acronyms........................................................................................................................................ i Executive Summary..................................................................................................................... iii I. Introduction................................................................................................................................ 1 II. The Development Problem and USAID’s Response............................................................. 2 III. Evaluation Purpose, Use, and Questions............................................................................... 5 IV. Evaluation Methodology ......................................................................................................... 6 V. Findings and Conclusions....................................................................................................... 10 VI. Recommendations ................................................................................................................ 36 VII. Lessons Learned................................................................................................................... 37 Annex A: Evaluation Scope of Work......................................................................................... 38 Annex B: Evaluation Team Member Profiles ........................................................................... 49 Annex C: Evaluation Design Matrix .......................................................................................... 50 Annex D: Data Collection Protocols......................................................................................... 55 Annex E: Mini-Survey Data and Findings.................................................................................. 75 Annex F: Documents Reviewed................................................................................................. 77 Annex G: List of Respondents.................................................................................................... 80 Annex H: Map of Evaluation Sites ............................................................................................. 88 Annex I: Disclosure of Any Conflicts of Interest ...................................................................... 89 TABLES AND FIGURES Table 1: Key Informants by Stakeholder Group and Gender.................................................................6 Table 2: FGD Respondents by Stakeholder Group and Gender...........................................................7 Table 3: Most Effective Task for Creating Public Demand for Legal Aid.......................................11 Table 4: Tasks That Should Continue in the Future for Awareness-Raising................................11 Table 5: Implementation of Recommendations from the Mid-Term Evaluation: EQ 2.........17 Table 6: Implementation of Recommendations from the Mid-Term Evaluation: EQ 4.........26 Table 7: Implementation of Recommendations from the Mid-Term Evaluation: EQ 6.........32 Figure 1: Mini-Survey Satisfaction with Partnership Between NCSC and Sub-Grantees .....31 i | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ACRONYMS ADR Alternative Dispute Resolution AOR Agreement Officer's Representative BCTIP Bangladesh Counter-Trafficking in Persons Activity BLAST Bangladesh Legal Aid Services Trust BNWLA Bangladesh National Women Lawyers Association BWJA Bangladesh Women Judges Association CD Compact Disc CLAC Chowki Legal Aid Committee COR Contracting Officer’s Representative CrRO Criminal Rules and Orders CVE Countering Violent Extremism DEC Development Experience Clearinghouse DFID Department for International Development DG Democracy and Governance DLAC District Legal Aid Committee DV Domestic Violence EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion GIZ Gesellschaft für Internationale Zusammenarbeit GO Government Organization GOB Government of Bangladesh IAWJ International Association of Women Judges ICT Information and Communications Technology IFCE International Framework for Court Excellence IR Intermediate Result JATI Judicial Administration Training Institute JFA Justice for All USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | ii KII Key Informant Interview LAC Legal Aid Committee LASA Legal Aid Services Act M&E Monitoring and Evaluation MOLJPA Ministry of Law, Justice, and Parliamentary Affairs MSI Management Systems International NCSC National Center for State Courts NGO Non-Governmental Organization NLASO National Legal Aid Services Organization PC Project Coordinator PE Performance Evaluation PIRS Performance Indicator Reference Sheets PM Project Manager SI Social Impact, Inc. SOW Scope of Work TIP Trafficking-In-Persons UNDP United Nations Development Programme UPLAC Union Parishad Legal Aid Committee USAID United States Agency for International Development UZLAC Upazila Legal Aid Committee VAW Violence Against Women iii | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV EXECUTIVE SUMMARY INTRODUCTION Through the Democracy and Governance (DG) Programs’ Monitoring and Evaluation (M&E) Support Activity task order, United States Agency for International Development (USAID)/Bangladesh engaged Social Impact, Inc. (SI) to conduct a final performance evaluation (PE) of the Justice for All (JFA) Activity. Implemented by the National Center for State Courts (NCSC) in a total of 15 districts with a performance period of six years (2012-2018), the overarching goal of the JFA Activity is to improve access to justice for all Bangladeshi citizens by 1) direct intervention in the provision of legal aid services and 2) indirect support to the judicial branch to enhance its self-governance. JFA’s major target group is Bangladeshi citizens who not only have limited access to justice but also have a limited understanding of their rights in the justice system. The other major target group is judges, who are responsible for asserting their self-governance and improving their court administration and case management to enhance judicial services. Women judges are specifically identified within the second target group to ensure they have equal opportunity within the judicial branch. EVALUATION PURPOSE The purpose of this final PE was to assess the technical and programmatic validity and effectiveness of the JFA Activity, assess NCSC’s performance in achieving actual results against targeted results, assess sustainability aspects of the Activity, and make recommendations to USAID/Bangladesh concerning future programming considerations. The key intended users of the evaluation are the USAID/Bangladesh DG Office, NCSC, and bilateral and multilateral donors engaged in justice sector interventions in Bangladesh. EVALUATION METHODOLOGY To answer the seven Evaluation Questions (EQs) listed below, a four-person Evaluation Team (ET) conducted an extensive desk review of program documents and secondary resources, Key Informant Interviews (KIIs) with 65 individuals, 20 Focus Group Discussions (FGDs), 8 Direct Observations (DOs), and a mini-survey. Fieldwork took place in seven pilot districts—Dhaka, Jessore, Khulna, Rajshahi, Cox’s Bazar, Rangpur, and Kurigram—over four weeks in February 2018. In total, the ET collected data from more than 250 individuals representing the Government of Bangladesh (GOB), USAID, NCSC, JFA sub￾grantees, legal aid services organizations, external stakeholders, legal services clients, community members, and legal professionals. FINDINGS AND CONCLUSIONS EQ1: HOW EFFECTIVE HAS THE ACTIVITY BEEN IN STIMULATING DEMAND AMONG BANGLADESHI CITIZENRY FOR LEGAL SERVICES THROUGH THE DISTRICT LEGAL AID COMMITTEES? FINDINGS JFA focused on the demand side of legal aid through thousands of awareness-raising tasks including courtyard meetings, public hearings, radio programs, video shows, and information dissemination. JFA sub-grantee partners consider courtyard meetings to be the most effective and important tasks to continue. Community members and legal aid clients across seven districts received their first ever training about legal aid from JFA. Legal aid applications in JFA pilot districts increased by 125 percent over three years (2014-17). As of the end of Year 5, monthly legal aid case intake had increased by 95 percent nationally, with a 117 percent increase in JFA pilot districts over the same period, demonstrating a higher demand/need for legal services in pilot districts relative to non-pilot areas. Non-pilot districts also reported that they have USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | iv been very active in organizing and implementing awareness-raising tasks, and that the commitment of district judges and District Legal Aid Officers (DLAOs) has resulted in a notable increase in legal aid applications over the last few years. CONCLUSIONS • JFA substantially contributed to increased awareness about GOB-sponsored legal aid services. • JFA awareness-raising was highly effective in communicating with potential clients and a wide range of stakeholders in pilot districts. • JFA’s support has been effective in stimulating demand for legal services in pilot districts. • The increased awareness and demand for legal services observed in pilot districts is in line with national trends. EQ2: DO THE LEGAL SERVICES PROVIDED MEET ESTABLISHED STANDARDS THAT ARE RESPONSIVE TO CLIENT NEEDS? FINDINGS The Legal Aid Services Act and its Legal Aid Services Regulations provide for the qualifications and appointment of panel lawyers to represent clients, and the National Legal Aid Services Organization (NLASO) provides guidelines on panel lawyers’ duties and responsibilities. JFA strengthened the capacity of District Legal Aid Committees (DLACs) with informational and skills-based training programs, support for coordination meetings with stakeholders, and site visits for capacity building and monitoring—all of which enabled DLAC staff to better coordinate their work and meet client needs and expectations. Legal aid clients are largely satisfied with panel lawyers, due to lawyers’ responsiveness and to favorable decisions regarding their cases. Panel lawyer rosters are periodically reviewed to ensure quality, and judges may remove lawyers due to poor performance or client complaints. Upazila Legal Aid Committees (UZLACs) and Union Parishad Legal Aid Committees (UPLACs) have been activated to conduct bi-monthly meetings and provide a local legal aid referral mechanism for citizens. However, communication between the DLACs and UZLACs/UPLACs is limited. CONCLUSIONS • JFA support and capacity building tasks with DLACs have improved the quality and delivery of legal aid services to meet the diverse needs of clients. • Most legal aid clients are satisfied with panel lawyers, so long as they receive favorable rulings on their cases. • Limitations on effective monitoring and coordination between DLACs and UZLACs/UPLACs undermine the relative importance of the lower-level committees in referring legal aid cases to the DLACs. EQ3: HOW EFFECTIVE HAS THE ACTIVITY BEEN IN PROMOTING EQUAL OPPORTUNITIES FOR FEMALE LEGAL PROFESSIONALS? FINDINGS While female judges are treated equally and without bias by other judges and lawyers, their representation in leadership positions is disproportionately small to that of male judges. JFA assistance brought the Bangladesh Women Judges Association (BWJA) from a largely inactive association to one that is registered with its own constitution and training opportunities for its members. Female lawyers reported that they do not operate in a supportive or secure environment and that they suffer from professional and societal sexual discrimination. JFA attempted to support the Bangladesh v | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV National Women Lawyers Association (BNWLA) but found a lack of interest by the association’s leadership. Consequently, JFA provided training to active female lawyers in several districts and conducted tasks with female law students. CONCLUSIONS • JFA appropriately targeted tasks to promote equal opportunities for female legal professionals. • JFA was critical in developing BWJA as a self-sustaining organization that advances an equal role for women in the judiciary. • While JFA increased awareness of female lawyers to advocate for equal opportunity within the legal profession, social and cultural obstacles continue to thwart access to equal opportunities. EQ4: HAS JFA CONTRIBUTED TOWARD CAPACITY IMPROVEMENT OF THE JUDICIARY WITH SPECIAL FOCUS ON CASE MANAGEMENT AND ADMINISTRATION? FINDINGS JFA tasks with the Supreme Court and the Ministry of Law, Justice, and Parliamentary Affairs (MOLJPA) focused on best practices for self-governance, integrity, and self-examination—and specificially supported modernization of the Criminal Rules and Orders (CrRO) and the Civil Rules and Orders, and the creation of the Judicial Evaluation Policy and the Judicial Case Management Policy. JFA collaborated with the Judicial Administration Training Institute (JATI) to develop training modules on court excellence and judicial governance and to enhance JATI’s visibilty and outreach. Evidence suggests that the Court Excellence Bench Book, which JFA and JATI collaboratively produced and distributed, is not systematically used by subordinate court judges. The quality and impact of district-based court administration and case management trainings for subordinate court judges are subject to the competency and commitment of District Judges. No evidence was presented of case backlog reduction as a result of the trainings. Subordinate court judges cited stricter enforcement of current procedural laws as an essential reform to reduce case backlog. CONCLUSIONS • JFA made serious attempts to support the Supreme Court in improving the legal framework for judicial self-governance. • Collaboration with JFA enhanced JATI’s organizational capacity and outreach. • Subordinate court judges are not motivated to use the Bench Book, as they believe it does not contain references to Bangladeshi laws and rules. Additionally, the Bench Book has not been sanctioned by the Supreme Court and the MOLJPA. • There is no evidence of case backlog reduction as a result of JFA case management trainings, nor did judges effectively advocate for improvements in the legal and regulatory framework. EQ5: TO WHAT EXTENT ARE THE ACTIVITY OBJECTIVES VIABLE WITHIN THE CURRENT JUDICIAL AND POLITICAL CONTEXT IN BANGLADESH? FINDINGS Capacity of the judiciary to assert self-governance is questionable in the current judicial and political context, given the GOB’s considerable control over the highly-politicized judicial appointment process, the strained relationship between the executive and judicial branches, and the MOLJPA’s control over the subordinate courts. Yet, the Supreme Court and MOLJPA may have the political will to address case backlog and enhance information and communications technology (ICT). USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | vi Government-sponsored legal aid has taken root in Bangladesh. However, effective coordination between GOB and non-governmental organization (NGO) legal aid services providers is extremely limited, which can negatively impact viability. Awareness-raising about access to legal aid is non￾controversial and essential to delivery of services. Though opinions may differ about what types of awareness-raising tasks are most effective, who is best able to implement them, and how, the issue of their viability is not in question. While discrimination of female judges and law students is not prevalent, considerable challenges and barriers exist for female lawyers. Access to leadership positions within the judiciary and bar is a challenge for both female judges and lawyers. CONCLUSIONS • Several JFA Activity elements remain viable in the current judicial and political context: o Legal aid frameworks and structures to deliver GOB-sponsored legal aid services o Enhanced collaboration between the Supreme Court and the MOLJPA to reduce case backlogs and develop ICT o Increasing the participation of female judges and lawyers and their professional capacity to advocate for equal opportunity within the judiciary and bar o Increased awareness about GOB-sponsored legal aid for the poor and marginalized • Promoting women’s equal access to leadership positions within the judiciary and bar is not viable. • Legal aid services delivery can be enhanced through greater government organization (GO)/NGO collaboration. EQ6: HOW SUCCESSFUL WAS THE IMPLEMENTING PARTNER IN MANAGING SUB-GRANTS TO ACHIEVE THE ACTIVITY OBJECTIVES? FINDINGS JFA sub-grantees reported high satisfaction with NCSC management, both in terms of program implementation and financial oversight. They found particularly useful the development of awareness￾raising materials, in-person visits by JFA leadership and staff, guidance on budgetary matters, and JFA Sub-grantee Quarterly Coordination Meetings. In response to heightened security concerns and travel restrictions during 2015-16, NCSC adjusted operations and practices, adopting flexible solutions to minimize interruptions in communications and tasks with sub-grantees. JFA sub-grantees indicated that guidance and management provided by NCSC strengthened overall capacity of their organizations. CONCLUSION • NCSC was highly successful in managing sub-grants, which enabled local partners to effectively contribute to achieving JFA Activity objectives. EQ7: TO WHAT EXTENT HAS JFA ACHIEVED PROGRESS TOWARD ESTABLISHING SUSTAINABILITY MECHANISMS? FINDINGS JFA implemented extensive training to support DLAOs’ administrative capacity, but no handover mechanisms exist to ensure a smooth transition from one DLAO to the next. JFA extended case management trainings to the districts but relied upon District Judges to conduct the trainings without oversight. JFA’s training of female lawyers and tasks for female law students were geared toward the individual rather than through institutions, such as the BNWLA and university law departments. vii | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV JFA’s interventions in enhancing JATI’s organizational capacity and increasing its visibility were geared toward enhancing sustainability. Likewise, JFA continuously assisted the BWJA to develop it from a fledging association to one that is legally registered with a constitution and strategic plan. CONCLUSIONS • While several DLAOs (particularly permanent staff members) reported increased skills and capacity, JFA training was not implemented in a way that ensured transfer of knowledge and experience from DLAOs to their successors. • JFA’s support for JATI included development of critical organizational infrastructure and capacity, increasing the likelihood of sustainable gains for the organzation. • JFA contributed to the professionalization of the BWJA, gains that are likely sustainable. • JFA’s work with female law students may be sustainable at an individual level; however, this work is unlikely to continue due to the approach used. RECOMMENDATIONS RECOMMENDATIONS FOR USAID 1. Support the expansion of DLAC support, LAC activation, and GOB-sponsored legal aid awareness-raising tasks to the 49 non-JFA pilot districts to grow demand for legal aid services throughout Bangladesh. 2. Support a comprehensive mapping of GOB-sponsored legal aid services and resources to determine the needs of DLACs/DLAOs in the 49 non-JFA pilot districts. This should be implemented in close coordination with NLASO, GOB stakeholders, as well as other justice sector donors and relevant NGOs. 3. Develop a future program that focuses on the Supreme Court’s and MOLJPA’s policy guidance to reduce case backlog, by revising substantive law and procedural rules and with the imposition of strict compliance of adjournment applications filed by lawyers. RECOMMENDATIONS FOR NATIONAL CENTER FOR STATE COURTS 4. In consultation with JFA sub-grantees, develop tailored sustainability plans for continued awareness-raising to be conducted by UZLACs/UPLACs, including legal literacy and community sensitization events, as well as UZLAC/UPLAC activation and capacity-building. 5. Gather and consolidate client feedback on legal aid services to share with panel lawyers and district judges for improved service delivery that is responsive and tailored to the needs of female and male clients. 6. Prior to JFA Activity close-out, transfer awareness-raising materials and implementation resources to Winrock International, Plan International, MSI, GIZ, and other implementing partners to disseminate in districts where donor-funded interventions are active or may resume in the future. 7. Continue collaboration with the Supreme Court and MOLJPA to implement the Judicial Evaluation Policy, Judicial Case Management Policy, CrRO, Civil Rules and Orders, and Bench Book (when approved) to help reduce case backlog and ensure that JFA’s key achievements are sustainable. 8. In collaboration with JATI, significantly increase quality control over district-based case management trainings to ensure that the trainings are substantively meaningful, taught by skilled trainers, impact participants’ behavior and understanding of the issues, and remain sustainable. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | viii 9. Revise the Court Excellence Bench Book in consultation with JATI and the Supreme Court to better incorporate relevant references to Bangladeshi laws. In addition, establish plans for printing and distributing the hard copy written version of the revised Bench Book in Bangla. LESSONS LEARNED 1. Encourage implementers to incorporate, during project design, establishment of formal networks or communities of practice among sub-grantee partners. This will facilitate knowledge sharing and mutual support among partners, which will build capacity and promote local ownership. 2. Invest time and resources in the appropriate individuals, at every level, to support their potential to serve as champions of reform and change agents, particularly in Bangladesh, given the volatility of judicial leadership. Provide skills training, professional development, and other support to such champions to amplify their influence and impact. 3. Manage programs and people flexibly and creatively. Where security risks are high and variable, maximize the use of ICT or other mechanisms to keep communication channels open and maintain cohesion among team members. 4. Develop standardized content and professional quality, user-friendly materials and visual aids. Apply the most effective elements of localized awareness-raising campaigns to create materials and strategies that will communicate consistent, clear messages to all beneficiaries. 5. Maintain several entry points at various levels within the Supreme Court as well as with District Judges and subordinate court judges, some of whom will, undoubtedly, become thought leaders in positions of power. 1 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV I. INTRODUCTION Since the adoption of the Constitution of Bangladesh (1972), which promotes a society in which the rule of law, fundamental human rights and freedom, and equality and justice will be secured for all citizens, the judicial branch has been seeking its independence from the executive branch to ensure that these constitutional values are upheld. While the judiciary has achieved meaningful success in asserting its autonomy, there is a continuing struggle between the two branches of government. To improve access to justice for all Bangladeshi citizens, the United States Agency for International Development (USAID)/Bangladesh supported the Justice for All (JFA) Activity, implemented by the National Center for State Courts (NCSC). The JFA Activity had a performance period of six years (October 2012—September 2018) and a total estimated cost of $7,997,747. NCSC initially launched the JFA Activity in five districts, added eight districts, and then added two more districts—for a total of 15 districts. The overarching goal of the JFA Activity is to improve access to justice for all Bangladeshi citizens by 1) direct intervention in the provision of legal aid services and 2) indirect support to the judicial branch to enhance its self-governance. The Activity’s major target group is Bangladeshi citizens who not only have limited access to justice but also have a limited understanding of their rights in the justice system. The other major target group is judges who are responsible for asserting their self-governance and improving their court administration and case management to enhance judicial services. Women judges are specifically identified as JFA stakeholders within the second target group to ensure they have equal opportunity within the judicial branch. This report details the results of a final performance evaluation (PE) of the JFA Activity conducted by Social Impact, Inc. (SI) for USAID/Bangladesh. The Evaluation Team (ET) collected data in seven JFA implementation site districts during February 2018. In what follows, the ET lays out the development problem and USAID’s response; evaluation methodology and limitations; findings and conclusions, organized by Evaluation Question (EQ); recommendations for USAID and NCSC going forward; and lessons learned. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 2 II. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE THE DEVELOPMENT PROBLEM Historically, the executive branch in the Government of Bangladesh (GOB) relies on its traditional role and imposes its will over governance of the judiciary, while judges—although accustomed to the overbearing presence of the executive branch—continue to assert their roles as independent decision￾makers. Political clashes between the two branches of government (particularly between the Chief Justice of the Supreme Court and the Prime Minister) over actions taken by the courts, the parliament, and the GOB have resulted in rifts that thwart steady progress in strengthening judicial independence. As the primary executive branch ministry that interfaces with the judicial branch, the Ministry of Law, Justice, and Parliamentary Affairs (MOLJPA) continues to have significant involvement in areas that affect the judiciary, such as court administration, criminal prosecution, and access to legal services. Its Judicial Administration Training Institute (JATI) has authority over judicial trainees and their completion of the program, which is the first step of executive branch influence over the judiciary—followed by executive power over the appointment and promotion of judges. Female judges, while reportedly treated equally by male judges as well as male lawyers, face considerable challenges in obtaining leadership positions on the bench, either by being appointed in the subordinate courts as a District Judge or as a Supreme Court judge. Female lawyers confront significant challenges in being treated as equal professionals to male lawyers, which impacts their opportunities to assume leadership positions on the Bar Council and on District Bar Associations. USAID’S RESPONSE Within the context described above, USAID conducted a Bangladesh Rule of Law Assessment (2011) to assess primary opportunities and constraints affecting the development of rule of law and to identify priority areas that could benefit from USAID’s assistance. The Assessment concluded that USAID’s efforts may be best directed toward enhanced judicial self-governance and independence as well as support for expanded access to justice, especially for the poor. Informed by the Rule of Law Assessment, USAID created JFA, its first rule of law program in Bangladesh, in 2012. The JFA Activity is structured around three complementary objectives designed to improve access to legal aid services, increase citizens’ awareness of their legal rights, and enhance judicial self￾governance to better serve the public. The three complementary objectives are each listed as Intermediate Results (IRs) below. IR 1: IMPROVED DELIVERY OF LEGAL AID IN THE FORMAL JUSTICE SECTOR In order to improve the effectiveness and use of legal aid services to ensure equal access to justice for all citizens, the JFA Activity assisted MOLJPA and the National Legal Aid Services Organization (NLASO). As a result of JFA’s assistance, the counterparts of MOLJPA and NLASO at the national, district, and local levels are to have increased their capacity, tools, and systems to deliver more effective and responsive legal aid services to citizens. The following sub-IRs have been JFA’s focus in making progress toward IR 1. SUB IR 1.1 The legal and operation framework for legal aid will be improved. The legal framework regulating the organization and administration of legal aid services is improved. Legal aid services operate uniformly and the District Legal Aid Committees (DLACs) fulfill their obligations under the legal framework. 3 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV SUB IR 1.2 Citizen knowledge about legal aid services increased. NLASO and DLACs, Union Parishad Legal Aid Committees (UPLACs), and Upazila Legal Aid Committees (UZLACs) are more visible, and target beneficiaries of legal aid services are aware of such services and how to access them. SUB IR 1.3 DLAC services effectively delivered. Legal aid services are delivered more holistically and tailored to the needs of applicants. The delivery of services meets uniform standards in compliance with the legal framework and the aspirations set in the NLASO strategic plan. IR 2: INCREASED SELF-GOVERNANCE OF THE JUDICIARY TO BETTER SERVE THE PUBLIC In order to increase self-governance of the judiciary to better serve the public, the JFA Activity assisted lower levels of the courts to improve operational efficiency and strengthen integrity. JFA engaged judicial leadership and lower court judges to increase their understanding and acceptance of best practices and resources on self-governance, integrity, and self-examination. The following sub-IRs have been JFA’s focus in making progress toward IR 2. SUB IR 2.1 Understanding of judges of judicial self-governance improved. Judges understand the parameters and benefits of judicial self-government in the context of improved access to justice and service delivery for citizens. Judges effectively advocate for improvements in the legal and regulatory framework governing the judiciary to achieve self￾governance. SUB IR 2.2 Legal framework for judicial self-governance improved. Improvements to the legal framework governing the judiciary introduced to support increased self-governance that leads to access to justice and quality service delivery for citizens. SUB IR 2.3 Equal opportunity for women within the judiciary promoted. Women judges demonstrate increased professional capacity and the ability to advocate for equal opportunity within the judiciary. Women’s participation in the judiciary increases and access to leadership positions reflects equal opportunity. IR 3: INCREASED CITIZEN AWARENESS OF LEGAL RIGHTS AND RESPONSIBILITIES ON SELECTED LEGAL ISSUES In order to increase citizen awareness of legal rights and responsibilities on select legal issues, the JFA Activity worked with MOLJPA, civil society, and professional associations to transfer critical legal knowledge and tools for concrete action to target communities. As a result of JFA, citizens are to have increased awareness of their legal rights and of the mechanisms to exercise them in the formal justice sector. Additionally, an enabling environment is to have been created to ensure that citizens can effectively exercise those rights. The following sub-IRs have been JFA’s focus in making progress toward IR 3. SUB IR 3.1 Availability of legal information increased. Legal information is developed and disseminated using multi-media mechanisms. Legal information is updated to meet the information needs of target populations. The availability of legal information is maintained and progressively expanded. SUB IR 3.2 Legal literacy of citizens increased. Target beneficiaries of legal aid services have actual understanding of their rights and remedies and how to exercise them. Also, willingness to exercise rights and remedies in the formal justice sector is increased. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 4 SUB IR 3.3 Community sensitization on exercise of legal rights increased. Pro-poor, pro-women, and pro-youth sensitivity is increased among target communities. Community leaders and members demonstrate acceptance of the exercise of legal rights and remedies in the formal justice sector. JFA has been implemented in a total of 15 pilot districts located across the country, in seven of Bangladesh’s eight administrative divisions.1 Geographic coverage was extended in four phases from the initial core group of districts that are also “divisional towns” to add neighboring districts. To date, JFA tasks have been implemented in 120 out of 491 Upazilas (sub-districts), in 1,120 out of 4,553 Union Parishads (rural councils), and in 7 Chowkis (rural administrative units) covering nearly one-quarter (23 percent) of the 64 districts in the country. As of February 2018, JFA was actively programming under IR 1 and IR 3 in 12 pilot districts. JFA closed offices and ended tasks in three of the 15 pilot districts as of the end of 2017.2 DEVELOPMENT HYPOTHESIS The development hypothesis for JFA is that gains in efficiency and responsiveness of legal aid services and in performance and governance of the judiciary will create the enabling environment for citizens to access effective legal remedies within the formal justice sector. Support for legal literacy will increase the citizens’ awareness of their rights and capacity to make meaningful choices to access and utilize legal aid services and formal justice institutions. Increased use of improved services will in turn lead to concrete outcomes that directly benefit the lives of disenfranchised communities while increasing citizen confidence in institutions, which can be used by institutions as a catalyst for continued reform. CRITICAL ASSUMPTIONS The following assumptions are held to be critical to achieving the objectives of the Activity: • GOB counterparts will remain engaged and supportive of JFA objectives throughout the life of the Activity. • Statistical and numerical data will be made available to the Activity in a timely manner and will be complete and error-free. • Grantee-generated data will be consistent in terms of definitions, data collection methodologies, and data validity. • Perception data will not be skewed to negative or positive extremes. • Sensitization tasks will lead to understanding and acceptance. • External actors and/or factors will not excessively interfere with the completion of tasks. 1 Out of 15 pilot districts, three are in the northwest (Rangpur Division); three are in the southwest (Khulna Division); three are in central Bangladesh (Dhaka Division); one is in central/north (Mymensingh Division); one is in the central/east (Sylhet Division); one is in the southeast (Chittagong Division); and three are in the central/west (Rajshahi Division). 2 JFA active pilot districts include: Bagherat, Cox’s Bazar, Dhaka, Jessore, Khulna, Kurigram, Maulvibazar, Munshiganj, Mymensingh, Pabna, Rajshahi, and Rangpur. In 2017, activities closed in Bogra, Dinajpur, and Tangail. 5 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV III. EVALUATION PURPOSE, USE, AND QUESTIONS The purpose of this final PE was to assess the technical and programmatic validity and effectiveness of the JFA Activity, assess NCSC’s performance in achieving actual results against targeted results, assess sustainability aspects of the Activity, and make recommendations to USAID/Bangladesh concerning future programming considerations. The key intended users of the PE are the USAID/Bangladesh Office of Democracy and Governance, NCSC, bilateral and multilateral donors engaged in justice sector interventions in Bangladesh, USAID/Washington, and leaders of USAID Forward. The PE will be used to assess whether future programming in the justice sector is relevant and viable in Bangladesh, whether simultaneous interventions are underway by other donors, and whether potential changes in judicial reform priorities for the GOB would impact such programming. USAID also requested that the ET identify any steps that should be taken by NCSC or USAID during the remainder of the JFA period of performance to promote sustainability of Activity achievements. This final PE focused on the EQs below, which were offered by USAID/Bangladesh and revised in consultation with SI. When answering these EQs, the ET considered unintended results and lessons learned. EFFECTIVENESS 1. How effective has the Activity been in stimulating demand among Bangladeshi citizenry for legal services through the DLACs? 2. Do the legal services provided meet established standards that are responsive to client needs? 3. How effective has the Activity been in promoting equal opportunities for female legal professionals? 4. Has JFA contributed toward capacity improvement of the judiciary with special focus on case management and administration? RELEVANCE 5. To what extent are the Activity objectives viable within the current judicial and political context in Bangladesh? MANAGEMENT 6. How successful was the implementing partner in managing sub-grants to achieve the Activity objectives? SUSTAINABILITY 7. To what extent has JFA achieved progress toward establishing sustainability mechanisms?3 3 JFA devised six sustainability mechanisms: a) Ensure counterpart ownership and commitment from the planning stages; b) Develop local capacity to act and participate; c) Facilitate knowledge transfers; d) Institutionalize tools and systems; e) Create a demand for accountability; and f) Demonstrate impact. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 6 IV. EVALUATION METHODOLOGY During the evaluation planning phase, SI conducted kick-off meetings with USAID/Bangladesh on January 22 and with NCSC on January 17 to discuss evaluation use, sampling, and measures to protect vulnerable program beneficiaries during data collection. The ET conducted fieldwork over four weeks during February 2018. Consisting of four members (two international and two Bangladeshi nationals), the ET maintained gender balance of two males and two females. (See Annex B: Evaluation Team Member Profiles). All team members conducted data collection in Dhaka, and the ET split into two gender-balanced teams—each composed of international and national team members—to collect data in cities outside Dhaka. DATA COLLECTION METHODS This evaluation used qualitative methods consisting of document review, Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), a mini-survey, and direct observation (DO) of JFA tasks. DOCUMENT REVIEW The ET conducted a review of documents produced by NCSC and relevant to JFA to better understand JFA Activity design and implementation and to inform data collection protocol development so that instruments appropriately supplemented or cross-checked information in the background documents. See Annex F, Documents Reviewed for a full list of documents reviewed by the ET. KEY INFORMANT INTERVIEWS Information gathered from KIIs contributed to the ET’s responses to all EQs. KIIs were conducted one￾on-one or in small groups, as appropriate, with representatives of the following target groups: USAID, NCSC, JFA sub-grantees, GOB, legal aid services organizations, and external stakeholders. The ET conducted KIIs with 65 stakeholders at the national and district levels (Table 1), including representatives from the Supreme Court, MOLJPA, NLASO, JATI, Bangladesh Women Judges Association (BWJA), District Legal Aid Committees (DLACs), university law departments, international donors, and other USAID implementing partners. KIIs were also conducted with JFA sub-grantee organizations: Association for Development—Comilla (AID-Comilla), Bangladesh Legal Aid Services Trust (BLAST), Bangladesh National Women Lawyers Association (BNWLA), DemocracyWatch, Light House, Madaripur Legal Aid Association (MLAA), Rupantar, Sachetan, Sylhet Jubo Academy (SJA), and the Women’s Job Creation Center (WJCC). Each KII was guided by an interview protocol adjusted for the different types of respondents. See Annex D, Data Collection Protocols. TABLE 1: KEY INFORMANTS BY STAKEHOLDER GROUP AND GENDER Stakeholder # Interviews # Males # Females Total People USG 2 2 2 4 NCSC 2 - 3 3 JFA Sub-Grantees 13 11 2 13 GOB 14 12 3 15 Legal Aid Services Organizations 13 10 3 13 External Stakeholders 15 8 9 17 Total 59 43 22 65 7 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV FOCUS GROUP DISCUSSIONS Information gathered from FGDs contributed to the ET’s responses to all EQs. The ET conducted 20 FGDs with a total of 200 people (Table 2) who represented each of the target groups below: 1. JFA Activity participants (legal aid committee members, judges, panel lawyers) 2. Legal services clients and community members 3. Female legal professionals FGDs with legal aid committee members included participants from Upazila (sub-district) Legal Aid Committees (UZLACs), Union Parishad (rural councils/lowest tier of local government) Legal Aid Committees (UPLACs), and seven Chowki (rural administrative units/Upazila level judicial administration) Legal Aid Committees (CLACs). Other JFA Activity participants included groups of district-level judges and panel lawyers serving in JFA pilot district courts. The ET worked with NCSC to select locations that were convenient and safe for FGD participants. Most FGDs, except those held with female legal professionals, consisted of both men and women. Female legal aid clients were given the opportunity to choose whether to discuss their legal cases, even those dealing with domestic abuse, in front of others. Despite cultural norms and sex inequality in Bangladesh, female participants did not hesitate to speak freely and frankly in the group setting. On occasions when some individuals dominated the discussion, the FGD facilitator solicited the opinions of all participants. Each FGD was guided by a protocol adjusted for the different groups; however, to provide for a comparable analysis, some questions within protocols were identical. See Annex D, Data Collection Protocols. TABLE 2: FGD RESPONDENTS BY STAKEHOLDER GROUP AND GENDER Stakeholder # FGDs # Males # Females Total People JFA Task Participants 10 71 38 109 Legal Services Clients & Community Members 5 9 41 50 Female Legal Professionals 5 - 41 41 Total 20 80 120 200 DIRECT OBSERVATIONS The ET conducted DOs of eight JFA tasks held in Khulna, Jessore, Rajshahi, and Dhaka. The ET observed a judicial case management training session, a pot-song, a gambhira, courtyard meetings, and a high school debate on effectiveness of the law on legal aid. These DOs informed the ET about the impact of JFA’s outreach tasks as they relate to EQs 1, 2, and 6. The ET documented the profile of participants (both male and female) and the degree of participation in the tasks. Each DO was guided by a list of questions capturing what the ET observed. See Annex D, Data Collection Protocols. MINI-SURVEY The ET conducted a mini-survey with 12 project managers (PMs) representing the 11 JFA sub-grantees managing JFA Activity offices in the 15 pilot districts to gather their perspectives on the effectiveness of JFA tasks, as well as lessons learned to consider for future programming. The mini-survey was also designed to obtain sub-grantee opinions about NCSC’s management. The mini-survey protocol consisted of three closed-ended questions and one open-ended question, and it was administered via phone or in person. See Annex E, Mini-Survey Data and Findings, for the survey and complete findings. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 8 SAMPLING The ET considered several factors when determining the seven pilot districts it visited for in-person data collection, including: whether the established pilot district was fully functioning and implementing tasks related to all three of JFA’s complementary objectives; whether JFA tasks could be observed during evaluation fieldwork; geographic diversity; and travel/logistical issues. In consideration of these factors and guidance provided by USAID and NCSC, the ET visited: Dhaka, Jessore, Khulna, Rangpur, Rajshahi, Kurigram, and Cox’s Bazaar. While both sub-teams conducted KIIs and FGDs in Dkaha and Khulna, one sub-team visited Jessore, Rangpur, Kurigram, and Cox’s Bazar while the other sub-team visited Rajshahi. The ET used a purposive sampling approach to identify candidates for KIIs using criteria such as involvement, even tangentially, with one of JFA’s major components, or knowledge of justice reform in Bangladesh. Additionally, the ET sought the recommendations of USAID and NCSC, particularly for KIs and FGD respondents actively involved with the JFA Activity. Mini-survey respondents were identified on the basis of their positions as Project Managers (PMs) or Project Coordinators (PCs) with JFA sub￾grantees. DATA ANALYSIS ET members transcribed KII, FGD, mini-survey, and DO notes and produced electronic summaries throughout fieldwork. The ET met when both sub-teams were in Dhaka to digest and analyze data and information gathered. Team members discussed variations in the information provided by the different stakeholders. The ET also discussed patterns and discrepancies that helped answer the EQs. For qualitative data in which some of the evidence was anecdotal or inferred, the ET used triangulation to identify inconsistencies, ensure reliability, and reduce the potential response bias described below. The ET developed parallel data collection protocols with the same or similar questions across its KIIs and FGDs, which enabled greater data triangulation because each method addressed sub-sets of the same EQs. Findings were validated or refuted by the other techniques. Methodological triangulation also enabled the ET to strengthen the potential linkages and accuracy of its data when the results obtained through one method were less conclusive than those obtained through another method. The ET analyzed data with regard to gender and focused on developing findings and conclusions that represented the perspectives and experiences of both male and female respondents, as well as any differences or similarities between them. All data collected through KIIs and FGDs were disaggregated by sex, when feasible. LIMITATIONS SELECTION BIAS Selection bias is an inherent risk when implementing partners help to facilitate contact with project beneficiaries. There is also a possibility of selection bias given that key informants who decline to be interviewed might differ from those who willingly participate in that their attitudes and perceptions, government affiliation, socio-demographic characteristics, and experience may indirectly impact the data that is gathered. While NCSC was helpful to the ET in identifying groups of individuals to meet in the seven pilot districts, there did not appear to be a selection bias because the ET heard a variety of opinions during KIIs and FGDs covering all EQs and JFA tasks. RECALL BIAS Recall bias is a challenge for evaluations that cover an extensive period of performance. Since the scope of this final PE covered JFA Activity tasks and events that occurred as early as 2012, some respondents had difficulty recalling their experience and reaction to tasks that took place years ago. The ET was able to locate and meet with key informants who could recall their involvement in the early years of the JFA Activity, but there were few such respondents. 9 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV RESPONSE BIAS Response bias is a risk because key informants may be motivated to provide responses that would be considered either socially desirable or influential in ensuring their continued participation in the JFA Activity. For instance, the ET mitigated potential response bias by asking case management training participants about the impact of training; despite providing positive comments about the training, participants demonstrated a lack of knowledge about the duration or content of the curriculum. Equally important is the concern by respondents that, despite promises of confidentiality and non-attribution, their comments may not be anonymous. Response bias can also be connected to cultural and social norms and affected by gender and social ranking. For instance, if the first person who speaks in a FGD is the most senior person in the group, other participants might take their cues from this person and only echo his/her responses. The ET witnessed this type of response bias while engaging in a DO of a case management training of subordinate court judges, one of whom challenged the trainer and was reprimanded. While the participant was not responding to a question from the ET, the social ranking of the trainer reduced the impact of the case management trainings because participants were not encouraged to discuss issues or concerns relating to their case backlogs, particularly if their issues or concerns were at odds with those of the trainer. UNAVAILABILITY OF GOVERNEMENT PERSONNEL The ET was limited in its ability to obtain firsthand opinions of JFA’s work with the Supreme Court due to the exodus in October 2017 of the former, reform-minded Chief Justice and judicial officers in the Supreme Court Office of the Registrar General and Supreme Court Research Unit with whom JFA worked closely to improve the legal framework for judicial self-governance. The ET attempted, unsuccessfully, to meet with the current Chief Justice of the Supreme Court. Ultimately, the ET was able to meet with only a few Supreme Court officials who had knowledge of JFA’s tasks. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 10 V. FINDINGS AND CONCLUSIONS EQ1: HOW EFFECTIVE HAS THE ACTIVITY BEEN IN STIMULATING DEMAND AMONG BANGLADESHI CITIZENRY FOR LEGAL SERVICES THROUGH THE DISTRICT LEGAL AID COMMITTEES? The JFA Activity focused on the demand side of the legal aid services supply/demand equation primarily through tasks under Objective 3, which were designed to increase citizen awareness and use of GOB￾sponsored legal aid through the DLACs in 15 pilot districts. Under Objective 1, JFA’s activation of UZLACs/UPLACs to receive legal services client inquiries also included support for committee-led awareness-raising tasks. FINDINGS Awareness-Raising in 15 Districts In KIIs with 12 JFA PMs and PCs who represent the 15 pilot districts (including districts where the JFA Activity has ended), respondents reported that they implemented the following awareness-building tasks (as part of either legal literacy campaigns or community sensitization events): courtyard meetings, public hearings, radio programs, pot-song (folk song), street drama/theater, gambhira (song/dance on social themes), school debates, video shows/projection, “mikings” (mobile messaging via loudspeaker), installation of billboards, wall writings, National Legal Aid Day celebrations, and distribution of posters, leaflets, brochures, stickers, and information cards. In KIIs with JFA sub-grantees and DLAOs, and in FGDs with UZLACs/UPLACs, community members or legal aid clients, participants from the seven pilot districts confirmed that in their district, JFA organized and/or directly conducted courtyard meetings, public hearings, mikings, and distributed informational/promotional materials. In addition, KIs and FGD participants cited one or more of the other awareness-raising tasks listed above. According to JFA reports, following an evaluation of tasks being implemented by sub-grantees in a first design phase (2013-14), JFA standardized a core set of awareness-raising tasks and accompanying materials to ensure consistency in implementation and messaging through multiple partners across districts. Since 2014, grant-funded programming has been aligned in this way. According to JFA reports, as of the end of Year 5, nearly 10,000 awareness-raising tasks reaching more than 517,000 citizens had been conducted in JFA implementation sites. JFA’s intended strategy was to create a critical mass of reform by having several districts all implementing the same activation and awareness-raising interventions, thereby motivating other districts to adopt best practices. KIs and FGD participants in all districts visited by the ET reported that while JFA and other organizations have made great strides in expanding awareness and understanding, there are many more people in pilot and non￾pilot districts who still remain unaware of GOB legal aid, or of the right to legal aid generally. Effectiveness of Legal Aid Awareness Tasks In DOs of several awareness-raising tasks, including a courtyard meeting, gambhira, and a school debate, the ET noted several themes: • Participants/audience members were highly engaged; • Women/girls’ made up at least 50 percent (school debate), 65-70 percent (gambhira), and 90 percent (courtyard meeting) of participants, and many brought along their young children; • Presenters/performers/facilitators were well prepared, energetic, and knowledgeable; • Standardized written materials (e.g., leaflets, info cards, handouts) were distributed; • Dowry, violence against women (VAW), child marriage, and other family law issues of relevance to women were discussed; and, • UZLAC/UPLAC representatives organized the event and/or actively participated. 11 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Results of the mini-survey4 conducted with 12 JFA PMs/PCs about the quality and value of JFA tasks revealed that the courtyard meeting is considered the most effective awareness-raising task, and the single most important task to continue following the end of the JFA Activity. In addition, according to a JFA summary of program achievements by pilot district (based on sub-grantee self-assessment), 13 of 14 representatives of JFA partner Non-Governmental Organizations (NGOs) cited the implementation and/or impact of one or more legal aid awareness-raising tasks in their list of most important or impactful Activity accomplishments. Tasks related to activation of the various local￾level Legal Aid Committees (LACs), and subsequent assumption of responsibility for raising public awareness directly were listed in the program achievements of 10 of 14 sub-grantees. Evidence of Increased Demand In FGDs with community members and legal aid clients, and in KIIs with JFA sub-grantees and DLAOs, most participants reported that JFA was the main or only source of information about legal aid, either for themselves or for clients who reported where they first learned about legal aid. Moreover, almost all surveyed respondents said that they were aware of NGOs and other organizations in their districts that distribute information about legal aid and routinely provide client legal representation and other services. However, FGD participants in one district reported that only JFA was engaged in legal aid awareness-raising and thus could be considered the sole catalyst for the increased demand in their community. That increased demand, in part, is demonstrated by increases in legal aid applications. According to JFA’s Analysis of Legal Aid Services in JFA Pilot Districts: Applications and Dispositions in 2014-17, applications for legal aid in pilot districts increased overall by 125 percent over three years. The ET collected data from DLAOs in several districts on numbers of applications filed over various time periods, each of which substantiated steady increases, reflecting growing demand.5 Moreover, JFA reporting indicates that, as of 4The survey also included a question about NCSC’s management of sub-grantees, and a question about future JFA tasks, responses to which are detailed below in EQ 6 and EQ 7, respectively. 5In Khulna, for example, applications for legal aid increased from 495 (124 male applicants; 371 female) in 2013 to 673 (199 male; 371 female; three minors) in 2016. Similarly, in Kurigram, where the JFA Activity started in 2016, applications increased from 320 (104 male; 216 female) in 2015 to 639 (224 male; 415 female) in 2017. In Rangpur, legal aid applications increased from 363 in 2013 to 891 in 2017. In Jessore, applications increased from 598 (196 male; 400 female; 2 minors) in 2016 to 688 (199 male; 478 female; 11 minors) in 2017. TABLE 3: MOST EFFECTIVE TASK FOR CREATING PUBLIC DEMAND FOR LEGAL AID n = 12 TABLE :ACTIVITIES THAT SHOULD CONTINUE IN THE FUTURE FOR AWARENESS-RAISING Gn =PUBLIC 12 DEMAND FOR LEGAL AID TABLE 4: TASKS THAT SHOULD CONTINUE IN THE FUTURE FOR AWARENESS-RAISING TABLE : ACTIVITIES THAT SHOULD CONTINUE IN THE FUTURE FOR AWARENESS-RAISING 50% 17% 17% 8% 8% 0% 10% 20% 30% 40% 50% 60% Courtyard Meeting Public Hearing Video Show School Debate Leaflet/Poster Distribution 33% 25% 25% 8% 8% 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% Courtyard meeting Public hearing Street Drama School debate Folk/Pot Song USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 12 the end of Year 5 of the program (September 2017), monthly legal aid case intake had increased by 95 percent nationally, with a 117 percent increase in JFA pilot districts over the same time period, demonstrating a higher demand/need for legal services in pilot areas relative to non-pilot areas. In FGDs with UPLACs and UZLACs in all seven districts visited by the ET, participants report that, with JFA’s organizational and financial support, they have conducted bi-monthly meetings, organized and/or participated in awareness-raising tasks, and assisted clients to complete their legal aid applications to the DLAC. JFA reporting confirms this, citing increases in the number of referrals from UZLACs and UPLACs to DLACs as evidence of higher citizen demand. Though the number of referrals doubled between Year 2 and Year 5, JFA’s own data analysis shows that as a source of applications, LAC referrals peaked at four percent in 2016, and declined to two percent of all applications filed in 2017. Viewed another way, applying JFA’s reported LAC activation rate of 89 percent (as of December 2017), it would appear that, on average, each activated UZLAC or UPLAC (1103 out of 1240) referred only three cases that went to the DLAC since the JFA Activity began.6 These statistics are consistent with comments from KIs (external stakeholders, sub-grantees, and DLAOs) that UZLACs/UPLACs may try to assist community members to resolve issues informally through counseling or by utilizing the village court mechanism. If these steps are unsuccessful, the client usually decides to go to the DLAC directly, whether or not they received a referral at the local level.7 Non-Pilot Districts In KIIs with DLAC Chairs and DLAOs in two non-pilot districts, respondents reported that they have been active in organizing and implementing awareness-raising tasks since being posted in their respective districts between 1-2 years ago. DLAC initiatives include conducting informational meetings in villages, sponsoring legal aid fairs, miking campaigns, etc. Even without a budget for such events, the district judge has managed to identify resources and raise funds to cover minimal costs. In addition, the district judges report that they actively engage with the UZLACs and UPLACs in their districts, making visits and encouraging them to hold informational sessions and publicize GOB legal aid. One KI stated that a different group of UZLAC and UPLAC Chairs are invited personally to attend the DLAC monthly meetings. All KIs agreed that their ‘activism’ and commitment has resulted in a substantial increase in legal aid applications filed in their districts from 2016-2017. In Magura, legal aid applications increased from 201 to 448, and in Narayangonj, the number increased from 678 to 985 during the same period. CONCLUSIONS JFA substantially contributed to increased awareness about GOB-sponsored legal aid services in 15 out of 64 districts. By implementing thousands of awareness-raising tasks in geographically diverse areas of the country, the JFA Activity provided hundreds of thousands of Bangladeshi citizens with essential information about the right to free legal aid and how to access services. JFA awareness-raising was highly effective in communicating with potential clients and a wide range of stakeholders in pilot districts. By designing a core set of tailored tasks and 6 JFA reports list the total number of referrals recorded by UZLACs/UPLACs annually: 0 (Y1); 526 (Y2); 883 (Y3); 668 (Y4); 1061 (Y5). 7 JFA reports that referrals are complex because of the length of time between when a justice seeker first hears about legal aid and the time when the justice seeker actually visits the DLAC to seek assistance. This is due to a multiplicity of factors. The UPLAC/UZLAC is one source of referrals, and the justice seeker may hear similar recommendations from others. By the time the justice seeker goes to the DLAC and reports who made the referral, they may not necessarily name the UZLAC/UPLAC specifically. UZLAC/UPLAC records always show much higher numbers of referrals than those of the DLAC as a result. 13 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV developing standardized informational and sensitization materials, JFA promoted consistent messaging through its sub-grantee partners across stakeholder groups. While the message of GOB-sponsored legal aid has yet to reach most districts and their citizens, JFA’s efforts have established a model and solid foundation on which to build awareness nationally. JFA’s support has been effective in stimulating demand for legal services in pilot districts. During the project period, a greater increase in the number of legal aid applications filed with DLACs was observed in pilot districts relative to non-pilot districts. The role of UZLACs and UPLACs as a source of applications that ultimately reach the DLAC, however, appears to be minimal. Whether it is due to external obstacles, such as lack of transport funds or cultural or other barriers, or due to the LACs failing to fulfill their reporting responsibilities, there remains a gap in optimal functioning of the DLAC/UZLAC/UPLAC referral system as a mechanism to increase demand. The increased awareness and demand for legal services observed in pilot districts is in line with national trends. As noted in national statistics and discussions with non-pilot district respondents, awareness-raising about GOB-sponsored legal aid is happening throughout the country, with the number of legal aid applications and demand for services increasing as a result. While the rate of increase in applications in pilot districts is approximately 20 percent above the national average, reflecting in part the positive and impressive impact of JFA interventions, the key to growing a robust GOB-led legal aid system lies in the quality and commitment of the district judge, supported by an active DLAO. District Judges and DLAOs who gain reputations for activism and reform will lead efforts to improve citizen awareness, thereby stimulating demand, and can influence fellow judges and motivate other LAC actors. EQ2: DO THE LEGAL SERVICES PROVIDED MEET ESTABLISHED STANDARDS THAT ARE RESPONSIVE TO CLIENT NEEDS? JFA focused on the supply side of legal aid through tasks under Objective 1, supporting GOB institutions and actors mandated to deliver legal aid services. Working with the GOB legal aid system at all levels, JFA provided resources, training, and technical assistance to assist legal aid services organizations to meet the diverse needs of growing numbers of legal aid clients, male and female, in 15 pilot districts. FINDINGS Standards for Legal Services The Legal Aid Services Act and its Legal Aid Services Regulations provide for the appointment and assignment of panel lawyers to provide free legal representation to eligible clients. Panel lawyers appearing before the SC must have at least seven years of experience, while DLACs require a minimum of five years’ practice. KIIs with DLAOs reveal that DLACs receive recommendations for panel lawyers from a variety of sources, including bar associations, universities, judges and local government officials. Every panel list must include at least one female lawyer, and clients may choose their own lawyer to be enlisted in the panel to represent them. KIs report that the SCLAC and DLACs maintain lists of lawyers with various years of experience in criminal and/or civil matters. Panel lawyers are required to submit status reports to the relevant court every three months and may be removed from the panel at the discretion of the SCLAC/DLAC. JFA reports indicate that the NLASO Guideline for Panel Lawyers provide direction on issues such as: panel lawyers duties and responsibilities; do’s and don’ts of conducting legal aid cases; relationship and communication between panel lawyers and clients; and submission of reports to the DLAO. However, according to JFA documents, DLAOs report that compliance varies greatly among offices, with most pilot districts indicating limited implementation of the guidelines. In addition, Alternative Dispute Resolution (ADR) is considered a form of legal aid services, and all DLAOs are directed to conduct ADR, although there are no prescribed procedures for ADR sessions that the DLAO must follow. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 14 Measures of Client Needs According to JFA documents, legal aid services being delivered in pilot districts are primarily to indigent criminal defendants and poor women seeking remedies in family disputes. In 14 out of 15 pilot districts, the majority of legal aid clients are women. In six of seven districts visited, KIs confirmed that during their tenures, the DLAC received more inquiries and legal aid applications from female than male clients. In Dhaka District, however, a KI estimated that currently two-thirds of legal aid clients are men, many of whom are convicted prisoners or long-held detainees. JFA data analysis reveals that in 2017, for the first time, most applicants countrywide were men (55 percent), reflecting the upward trend in jail referrals. DLAOs, UPLAC members, legal aid clients and community members report that legal aid cases brought by female clients tend to focus on family disputes, dowry and maintenance issues, as well as domestic violence (DV), while the majority of cases brought by male clients focus on land disputes and other interpersonal conflicts. In FGDs with UPLACs and community members, respondents in three districts indicated that child marriage was a problem in their community that could be addressed by legal aid services, and trafficking in persons (TIP) was identified as an issue in two districts. Legal aid clients face specific obstacles when seeking legal aid services. In KIIs with DLAOs, and in FGDs with UZLACs/UPLACs and legal aid clients, barriers to client access to services cited include: cost and/or lack of transport; loss of wages due to missed work; and lack of confidence/knowledge about processes. At an FGD with legal aid clients, five out of eight respondents mentioned that transport expenses are a major obstacle in accessing justice, despite free legal representation. KIIs and FGDs revealed that NGOs sometimes cover transport costs for clients, while LACs do not. Women who are concerned about safety and/or gender-based cultural attitudes related to travel face additional obstacles. In this context, JFA tasks sought to increase client access to relevant information and resources, secure competent legal representation appropriate for the case, and provide gender-sensitive services to female clients. District Level Service Delivery JFA reports detail consistent engagement of district-level counterparts to improve legal aid service delivery, including a variety of informational and skills-based training programs for DLAOs and DLA staff, support for coordination meetings with stakeholders, and regular site visits for capacity building and monitoring. In KIIs with DLAOs in six districts, respondents reported that JFA trained their legal aid office staff on all aspects of office administration and case management, including compliance with NLASO documentation and reporting requirements. Training also focused on improving customer service, i.e., providing client assistance on preparing legal aid applications, creating a client-friendly environment, and how to engage with lawyers, taking into account the different needs of women and men. However, two KIs indicate that DLAO monitoring of panel lawyers and reporting to NLASO needs to improve in order to ensure better client services. In KIIs with NCSC, DLAOs and JFA sub-grantees, respondents reported that currently only 27 out of 64 districts have full-time DLAOs; in 37 districts (including some JFA pilot districts), full-time judges are carrying out responsibilities of a DLAO on a part-time basis. Four KIs reported that they were handling DLAO duties on top of their regular work, and that having only two staff was insufficient to provide services in a timely, professional manner. In addition, two KIs reported that they had not attended any JFA DLAO training. In KIIs with DLAOs in five districts, respondents indicated that JFA’s support in organizing and conducting client consultation meetings and panel lawyer meetings on a quarterly basis has enabled them to better coordinate their work and meet client needs and expectations. KIIs with GOB and external stakeholders, including former and current NLASO directors, reveal that JFA provided basic management skills training to all DLAOs which, according to one district judge, is not part of their legal training. In DOs of two client consultation meetings, the ET noted that the DLAOs conducted the 15 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV meeting with legal aid clients in a professional and respectful manner. Each client was asked about the status of their case and provided an opportunity to request information or intervention by the DLAO. According to JFA documents, NCSC prepared and distributed the NLASO-endorsed Legal Aid Best Practices Manual, which sets forth performance goals for DLAOs, including the monitoring of legal aid services utilization and client satisfaction. DLAOs also participated in a formal self-assessment to develop DLAC improvement plans, which include reporting on compliance with the NLASO Guidelines for Panel Lawyers. A KII with a DLAO confirmed that legal aid cases receive a priority designation, per NLASO requirements, and judges and court staff treat these cases with more urgency. Despite follow up to facilitate legal aid case processing, DLAOs report that most clients greatly underestimate the time required for a case to go through the formal system. In addition, to address case backlogs, DLAOs encourage parties to use ADR or counseling to solve disputes before proceeding with court cases. In KIIs with DLAOs, respondents reported that they conduct ADR sessions with legal aid clients as much as possible, and a DLAO reflected that she is particularly effective in using ADR to handle family law matters with female clients. She also has been praised by the district judges for her speedy case disposal rate. Another district judge opined that mediation and ADR are underutilized, and that judges need more skills training to be effective. Panel Lawyer Performance Each district court in Bangladesh maintains a standing list, or panel, of lawyers who have agreed to provide legal representation and services to legal aid clients. The DLAC selects and manages the “panel lawyers” and assigns lawyers to particular clients and cases. In FGDs with legal aid clients in three districts, the vast majority of respondents expressed satisfaction with their panel lawyers due to the responsiveness of their advocate, and the fact that the case was decided in their favor. FGD participants also noted that female clients are matched with a female panel lawyer in cases where they may be uncomfortable speaking with a male lawyer. In one FGD, however, a legal aid client had changed lawyers several times, and was dissatisfied because of the long delays in moving his cases forward. In a DO of a client consultation meeting, three clients out of 12 expressed dissatisfaction with their panel lawyer; in another, two out of nine were not satisfied. According to KIIs with DLAOs and in FGDs with UZLACs and UPLACs, reasons cited by those dissatisfied with their lawyers include: demand for money/bribes; lack of communication or presence in court; or failure to obtain a favorable judgment. KIIs with DLAOs revealed that panel lawyer rosters are periodically reviewed to ensure quality, and that judges may remove individual lawyers for poor performance or client complaints. However, even high￾performing panel lawyers who serve their clients well earn very modest fees for each case they handle, and lawyers frequently experience delays in receiving payments. In FGDs with panel lawyers, the ET engaged with a broad range of practitioners in terms of years and areas of experience. In two FGDs, the panel lawyers were all very senior, some with decades of criminal defense or family law experience. Another FGD included panel lawyers with expertise in representing victims of domestic violence and trafficking. While several KIs questioned the skills and motivations of panel lawyers relative to their NGO colleagues, many FGD panel lawyer participants articulated their sincere commitment to the poor, and all expressed interest in trainings and professional development tailored to their needs. Activation of UZLACs and UPLACs According to JFA documents, more than 1,100 UZLACs and UPLACs have been ‘activated’ with JFA support to conduct bi-monthly meetings and fulfill their mandate to provide a referral mechanism for citizens to file legal aid applications with the DLAC. Part of the role LACs have tried to play in ensuring legal services meet client needs is listening to issues and complaints and connecting clients to appropriate legal or other resources. In FGDs with UPLACs, respondents report that a wide variety of issues come before the committees including problems of domestic violence and even child marriage, USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 16 with members who are working on these issues in their regular jobs providing information to community members on prevention and their rights in a situation, or contacts for further information. KIIs with DLAOs and FGDs with legal aid clients and UPLACs confirm that “focal points” are designated in some LACs to provide assistance with applications to female clients in particular and accompany them to the district court. In FGDs with leaders of multiple UPLACs, participants indicated that legal aid clients initially seek to resolve their disputes informally at the local level, even if the district court is the most appropriate venue. In addition, dissatisfied clients who complain to UZLACs or UPLACs about their panel lawyers return to seek assistance from LACs to resolve the dispute, straining limited resources. Moreover, most participants in an FGD with UPLAC Chairmen/Member Secretaries from 13 UPs in Jessore and an FGD with UZLAC leadership indicated that they frequently feel ill-equipped to deal with many of the disputes that arise in their jurisdictions. They also mentioned that they would like training on topics such as basic legal literacy and conflict resolution skills. In addition, DLAOs in four districts revealed that many UZLACs/UPLACs are not regularly submitting obligatory progress reports to their DLACs. It is unclear whether this is due to a low level of execution of the particular task or a lack of bi-monthly meetings, or whether in fact the opposite is true and administrative and reporting responsibilities are neglected as a result. Some DLAOs follow up with LACs to inquire about reports; others indicate they do not have the resources to do so. Four DLAOs indicated that better monitoring of UZLACs and UPLACs was needed. CONCLUSIONS Legal Aid Service Regulations and NLASO guidelines establish basic standards for panel lawyers as the key providers of legal aid services. The SCLAC and DLACs appoint and maintain lists of eligible panel lawyers who must meet minimum requirements in terms of years of practice, and committees try to balance their lawyer rosters with criminal and civil practitioners. While the NLASO Guidelines address panel lawyer standards of practice for carrying out their responsibilities to legal aid clients, it is not clear to what extent panel lawyers in the pilot districts are following these instructions. Women represent the majority of legal aid clients in 14 of 15 pilot districts. Women bring cases focused on family disputes, dowry and maintenance issues, as well as domestic violence (DV), while the majority of cases brought by male clients focus on land disputes and interpersonal conflicts. Access to services remains a challenge, due to barriers such as transport costs, lost work time, and need for accompaniment of female clients to court. Where NGOs facilitate transport, cover court costs, or meet other needs, they may be considered more responsive to clients than GOB legal services organizations. JFA support and capacity building of DLACs have improved the quality and delivery of legal aid services to meet the diverse needs of clients. The GOB-sponsored system handles increasing numbers of legal aid applications on a wide range of criminal and civil law issues. JFA training has assisted DLACs, through the DLAOs, to respond effectively to legal aid clients by practicing good office management, connecting them with appropriate panel lawyers, conducting client consultation meetings, and providing customer-friendly services that account for the differences in needs of female versus male clients. The current level of staffing of DLAOs, both in terms of the number of full-time judicial officers and administrative staff, however, is insufficient to meet the growing demand, both in pilot and non-pilot districts and threatens to compromise the speed and quality of legal aid services to all clients. Most legal aid clients are satisfied with their panel lawyers, so long as they obtain a favorable result in their case. DLAC panel lawyers vary greatly in terms of substantive knowledge and experience, as well as client communications and professional ethics. Many panel lawyers would benefit from professional development training on a variety of substantive criminal and civil law topics, 17 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV TABLE 5: IMPLEMENTATION OF RECOMMENDATIONS FROM THE MID-TERM EVALUATION: EQ 2 Note that numbers below match the recommendation number from the Mid-Term Evaluation Report 2. The JFA Chief of Party, together with the NLASO Director or Deputy Director, should present the short-term action plan, updated Pilot Court Progress Report, and improvement plans to DLACs, UPLACs, and ULACs as a first step in improving communication and collaboration through the four levels of the legal aid system. JFA reported that NLASO did not seek a short-term action plan, but did submit quarterly district activity reports to NLASO beginning in Q14. The ET found that LAC coordination meetings were organized in several pilot districts, and that NLASO collaborated with JFA on the Chowki orientation workshop in November 2017. 3. JFA should support DLAO skills training and experience sharing meetings at the division level to strengthen relationships among districts and ease the burden of traveling to Dhaka for more distant DLACs.6. JFA should explore building upon lessons learned from JUST’s judicial leadership program with the Supreme Court, which focused on leadership development, change management, code of conduct, disciplinary procedures and strategic planning, to introduce similar leadership skills components into JATI training of district judges. The ET found that JFA considered the suggestion but determined that Dhaka remained the most appropriate location for the trainings and lessons learned meetings, based on the location and for the convenience of most of the participating DLAOs. In addition, NLASO prefers to hold meetings in Dhaka. 4. In the event that UNDP discontinues training on Alternative Dispute Resolution (ADR) for DLAOs, JFA and USAID should consider whether JFA should organize practical skills training on ADR and client counseling for DLAOs to build their capacity to perform the additional duties resulting from the LASA amendment. According to JFA documents, ADR is addressed as a cross-cutting topic in all management training. Women judges have been trained in mediation processes, and JFA reports that a session was organized on ADR at the Chowki LAC management training in February 2018, although the ET did not see any training materials or the topic listed on the program agenda. Now that UNDP is no longer providing training, JFA or its successor program should revisit this issue, as DLAOs specifically requested skills training on ADR. 8. JFA should work directly and through sub-grantees to identify and provide ongoing support to selected female members of ULACs and UPLACs who have expressed interest in, or taken initiatives to, raise awareness about legal aid, assist legal aid clients to access the system, and mentor other women to do the same. USAID should ensure that these women are included in its upcoming human rights/gender assessment. JFA reports that sub-grantees are using women UZLAC and UPLAC members as focal points for referrals and are encouraged to become active, with some districts formally designating female focal points. The ET notes that support should include training in basic legal literacy and on general district court procedures, so that they can assist clients more knowledgeably. and practical skills such as interviewing witnesses and cross-examination, as well as lawyer-client relations. Such offerings can serve to reward and motivate practitioners when serving as lawyers to the poor and marginalized. Limitations on effective monitoring and coordination between DLACs and UZLACs/ UPLACs undermine the relative importance of the lower-level committees in referring legal aid cases to the DLAC. The DLAOs have identified a communications and coordination problem in the legal aid referral system. This issue may be tied to the very low percentage of legal aid applications filed with the DLAC that are reported to be sourced from an UZLAC or UPLAC. Though they may be engaged in awareness-raising tasks and bi-monthly meetings, non-reporting breaks a link in the referral system. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 18 EQ3: HOW EFFECTIVE HAS THE ACTIVITY BEEN IN PROMOTING EQUAL OPPORTUNITIES FOR FEMALE LEGAL PROFESSIONALS? The ET engaged in extensive document review about JFA tasks related to promoting equal opportunities for female legal professionals and that support the objective of Sub IR 2: promotion of equal opportunity for women within the judiciary. During the ET’s fieldwork, many JFA tasks associated with Sub IR 2 were substantiated through KIIs and FGDs. These tasks included capacity building of the BWJA, collaboration with BNWLA on ad hoc trainings for female lawyers, strategic dialogues between female lawyers and law students, court visits, judgment readings, and internship opportunities for female law students. FINDINGS ‘Finding Our Voices Gender Mapping’ as a Foundational Document for Programming The JFA Activity began with a directive to promote equal opportunity for women in the judiciary; however, it was expanded in 2014 to include all women in the legal profession. Indeed, an indicator referring to the level of professional capacity of women judges was retired to reflect the expansion of JFA’s tasks to promote equal opportunities for female law students, lawyers, and judges.8 The expansion was generated, in part, due to the needs of female lawyers and judges set forth in the Finding Our Voices Gender Mapping, a survey conducted by JFA of a cross-section of female legal professionals as well as male lawyers and judges.9 General Mapping is the annual survey conducted for the purpose of informing JFA and USAID on tasks that support female legal professionals in their professional development and empower them to become active in promoting equal opportunities within their profession. The first mapping survey was conducted in 2014. This first survey provided a baseline against which all future General Mappings were compared. In reviewing the questionnaire, the ET found that each annual general mapping consisted of the same survey questions, which allow comparison of responses across years. The 11-page Gender Mapping survey is comprehensive and encompasses all aspects of women’s experience in law school and in their legal profession, including women’s motivation to study law, training and education environment, work environment, and future employment prospects. Annual reports of the Gender Mapping identified challenges and barriers for female judges and female lawyers to enter and advance within their respective professions. Throughout all KIIs and FGDs during which the challenges and barriers for female lawyers and judges were discussed (including data collectin with 46 female law students, lawyers, and judges; and five male judges), the respondents’ comments were similar, if not identical, to the findings presented in the General Mapping annual reports, and most recently the annual report of 2017. The uniformity of responses provided the ET a solid foundation upon which to evaluate current JFA tasks, as well as provide programmatic recommendations to JFA and USAID. As a result of the Gender Mapping findings related to challenges and barriers for female legal professionals, JFA developed tasks that supported female legal professionals in their professional development and empowered them to become active in promoting equal opportunities within their profession according to each category of female legal professional. 8 USAID modified the cooperative agreement in September 2013 to revise Result 2.3 to reflect the expansion of all legal professionals. 9 USAID’s Justice for All Program Finding Our Voices: Mapping Opportunities for Women in the Judiciary and Legal Professions, 2017 can be found in JFA’s Quarterly Report 18. 19 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Female Judges and the Bangladesh Women Judges Association (BWJA) The ET found that respondents (10 female judges and five male judges located in Rajshahi, Dhaka, and Khulna) believe female judges are treated equally and without bias by other judges and lawyers. Female judges are respected for their competency and are not subject to professional limitations and sexual biases that confront female lawyers (to be discussed below). While there is a disparate representation of female to male judges (six female High Court judges to 74 male High Court judges and 435 female subordinate court judges to 1,238 male subordinate court judges), as well as a disproportionate number of female to male judges in leadership positions (three female District Court judges to 61 male District Court judges), all respondents attributed the disparity to historical gender bias. One respondent added that female applicants to the judiciary have only recently increased, noting that this explains the minimal number of female judges in total and in leadership positions. As a result of document review and a KII with JFA respondents, the ET found that JFA’s work supporting female judges was initially supplemented by a two-year sub-grant with BLAST, one of the largest legal services NGOs in Bangladesh (operating in over 19 districts) to “implement professional development and outreach tasks designed to mobilize female judges and legal professionals and conduct outreach to promote gender equality in the judiciary and legal professions.” After the sub-grant with BLAST ended in 2016, JFA shifted its attention to direct work with the BWJA. In early 2016, JFA documents revealed that the BWJA was still largely inactive in providing services to its members. As a result of the Gender Mapping recommendations, JFA sponsored a delegation of four BWJA representatives to participate in the International Association of Women Judges (IAWJ) Conference in Washington, DC after which the delegation visited state and federal courts and met with representatives of the U.S. Department of Justice. The ET conducted KIIs with two IAWJ Conference attendees who praised the interaction with other female judges from around the world as being beneficial in raising their consciousness about the role of female judges and in increasing their networking opportunities. According to an FGD with eight BWJA representatives, the trip was instrumental in inspiring them to focus more efforts in developing the Association. Throughout 2016 and 2017, the BWJA developed into a “strong and active association as a result of JFA’s assistance,” as stated by one of its founders in a KII. JFA helped the BWJA become legally registered with its own constitution. JFA also helped the Association draft a five-year Strategic Plan to strengthen its institutional capacity and to provide capacity-building opportunities to female judges. The eight BWJA respondents expressed praise for JFA’s assistance which allowed the BWJA to publish its constitution, organizational brochure, pocket calendar, address book of all members, Women in Justice Success Story booklets, and yearly magazine, “Shoronika.” According to BWJA respondents, because of JFA’s assistance, training and other opportunities have been provided to its members. Respondents singled out trainings on mediation, ethics and the code of conduct of judicial officers, and the celebration of International Women’s Day as assistance that was most impactful. However, according to KIIs with JFA and the FGD with BWJA members, much remains to be done to develop the capacity of the BWJA as a self-sustaining organization. The BWJA does not have an office, nor staff; its budget is derived from membership fees; it does not receive donor funding, other than JFA support; and its members are full-time judges whose time is already stretched by their duties to their profession and family. Nevertheless, the leadership of the BWJA were enthusiastic in an FGD with the ET to grow the BWJA and were willing to put forth the necessary effort to achieve this growth. Female Lawyers and the Bangladesh National Women Lawyers Association (BNWLA) The ET found an entirely different scenario for female lawyers, unanimously supported by the KIIs and FGDs, in which female lawyers reported that they did not operate in a safe and secure environment. Female lawyers are marginalized; are discriminated against by male lawyers and by clients; are not mentored by senior lawyers like their male counterparts; are not provided with the necessary working USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 20 conditions in bar association offices to conduct their law practice; are not provided with sufficient facilities, such as accessible toilets and childcare; are sexually harassed by their male counterparts particularly at night in lawyers’ offices; are not safe traveling at night; and are not supported by their families due to the time required to complete legal work (days at the courthouse and nights in the office). Women reported not being supported by their families as a result of issues such as their duty to care for children and the home. As well, male family members prohibit women from working with male lawyers at night—which is the preferred timeframe for male lawyers to maintain their honor by leaving home. The ET found that self-identified successful female lawyers attributed their success primarily to the support they received from male members of their families, be it a father, brother, or husband (suggesting that without this support, the female lawyer would not have achieved success). After JFA’s sub-grant with BLAST to provide assistance to women lawyers from 2014–2016, JFA’s assistance to female lawyers was geared toward supporting individual lawyers rather than developing the organizational capacity of the BNWLA. JFA maintains contact with BNWLA by providing its members the opportunity to be trainers at JFA’s trainings for female lawyers. In the KII with the President of the BNWLA, the ET found that although BNWLA has a sporadic relationship with JFA, the President was unaware of and seemingly disinterested in JFA tasks. The ET learned during this KII that the BNWLA is an exclusive organization whose membership is regulated and based on recommendations by other female lawyers. Interestingly, according to separate KIIs with JFA and BNWLA, successful female lawyers do not mentor younger lawyers and instead have a “pull yourself up by your bootstraps” attitude. Consequently, “young female lawyers are shunned not only by senior male lawyers but also by senior female lawyers for mentorship,” as stated a key informant. BLAST was also the recipient of a sub-grant to support female lawyers through 2016. JFA continued trainings initiated through the BLAST sub-grant after it ended, conducting a series of professional development trainings on legal drafting and court procedures for active female lawyers in five pilot districts where female lawyers, judges, and law students are located in sufficient numbers to warrant JFA tasks (Dhaka, Rajshahi, Rangpur, Chittagong, and Sylhet). To date, according to JFA’s records and confirmed in a KII with JFA, JFA has conducted 15 trainings on legal drafting and court procedures to 343 female lawyers. JFA’s records indicate a self-recorded 60 percent increase in professional capacity by female lawyers who participated in JFA’s trainings, a result that was confirmed during FGDs with 25 female lawyers who commented that “we learned how to draft a complaint and how to take a case to trial.” While this may seem elementary, experiential training is not included in law school curriculum and the absence of senior lawyers as mentors for new female lawyers necessitate such trainings. One respondent confirmed the need for JFA’s trainings, stating that “in order to increase equal opportunities for female legal professionals, their capacity must be equal to that of male legal professionals.” Interestingly, while the FGDs with female lawyers were conducted in Rangpur, Dhaka, and Rajshahi, the geographic diversity of the female lawyers did not impact the uniformity of their opinion that being trained by and only with other women is preferable to being trained alongside men. In their opinion, men often dominate the discussion and they feel more at ease expressing their opinions and answering all the questions when among women. Additionally, those who participated in the training suggested the duration be longer than one day, with some suggesting one day and others one week. Respondents also suggested trainings be held on weekends. JFA recently conducted two strategic dialogues between female lawyers and bar representatives to address the challenges encountered by female lawyers and mechanisms to mitigate them. According to a KII, this task is too recent to ascertain its usefulness and impact, but JFA is giving serious consideration to continuing this work. 21 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Female Law Students In contrast to the experiences of female lawyers and judges, the ET found in its FGD with eight female law students (studying in law departments from three universities in Dhaka) that they do not generally feel discriminated against due to their sex, and that their challenges as law students are similar to those faced by male law students. Interestingly, only four of the eight female students willingly attended law school because they had always wanted to be lawyers or judges. One admitted she wanted to be a judge because “judges always have the last say.” The remaining four cited family pressure to study law—one law student wanted to be “anything but a lawyer” while another wanted to be an astronaut. While JFA initiated its work with female law students in 2014 on an ad hoc basis, according to document review and KIIs with JFA, JFA has since developed a robust set of tasks supporting female law students, such as women-to-women dialogue sessions between female law students and young female lawyers to discuss the practical steps for entering the legal profession as well as the discriminatory barriers to advancement; court visits and judgment analysis sessions to provide law students exposure to the work of judges and lawyers; and internships to promote career development and to familiarize female law students with various offices of the government legal aid services. To date, JFA has engaged with 2,090 female law students through its tasks. All eight law students in the FGD reported participating in one or more JFA tasks with enthusiasm. Those who sat with the DLAO in Dhaka for a judgment analysis session found it very useful. “It was the first time we had such practical experience, and we learned not only about differentiating between facts and opinions, but also about the legal aid system.” Others attended a JFA-sponsored workshop on access to justice which inspired them to organize their own legal aid forum. A few students attended a women-to-women dialogue which “opened our eyes” to the legal profession. None of the respondents, however, had been interns. Although JFA initiated its work through university law departments, the law students who participated in JFA tasks proactively organized their own activities in collaboration with JFA, for which they were often provided with JFA’s documentary video that highlights the profiles, aspirations, and career paths of female lawyers to encourage law students to enter the legal profession. Because of students’ proactive engagement, JFA began to develop its own database of active female law students which diminished the need for coordination with law departments. All eight law students in the FGD reported that their current JFA tasks are the result of word-of-mouth invitations. JFA’s major task affecting all female legal professionals is its mobilization of female judges, lawyers, and law students in celebrating International Women’s Day, not only on the day of the celebration, March 8, but also on preceding days, specifically to encourage female law students to embrace careers in the judiciary and legal profession. This year, in celebration of International Women’s Day, JFA organized for female law students a hands-on visit to the Dhaka subordinate court, a law career development session with the Supreme Court Legal Aid Committee, and a group study session to analyze a family law judgment. As with previous celebrations, JFA released the Finding our Voices Gender Mapping Report on March 8. CONCLUSIONS The project’s expansion of tasks that promote equal opportunities from female judges to all female legal professionals was appropriate. The expansion was informed by the Finding Our Voices Gender Mapping survey. The willingness of USAID and JFA to collect and consider data regarding the underlying challenges to equal opportunity, and the flexibility of both to adjust the focus of the Activity led to an effective strategy/approach that ultimately benefited considerably more female legal professionals. Additionally, the General Mapping survey was found to be extremely useful in recommending the next steps for JFA to take in order to resolve the challenges and barriers facing women legal professionals in their pursuit of equal opportunities. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 22 JFA was critical in developing BWJA as a self-sustaining organization that advances an equal role for women in the judiciary. JFA’s support of and assistance to the BWJA helped the Association increase its organizational capacity as well as inspire its members to proactively engage in the development of the BWJA. While JFA increased awareness of women lawyers to advocate for equal opportunity within the legal profession, social and cultural obstacles continue to thwart access to equal opportunities. JFA’s assistance to individual female lawyers was well-received and reportedly enhanced their professional capacity while stimulating their interest in pursuing advancement within the legal profession. Female lawyers, however, continue to face discrimination in the workplace and are pressured to support the family in the home. JFA’s assistance to female law students was enthusiastically received and provided students with information on challenges facing female legal professionals, practical benefits of being a judge or lawyer, and issues surrounding access to justice. JFA provided numerous activities for female law students through the legal aid forum, the women-to-women dialogue sessions, court visits, judgment analysis sessions, and internships. JFA’s tasks inspired female law students to take proactive steps to organize their own activities supporting and empowering female law students. EQ4: HAS JFA CONTRIBUTED TOWARD CAPACITY IMPROVEMENT OF THE JUDICIARY WITH SPECIAL FOCUS ON CASE MANAGEMENT AND ADMINISTRATION? FINDINGS Support to the Supreme Court and MOLJPA JFA supported tasks that focused on an increased understanding and acceptance of best practices on self-governance, integrity and self-examination. In this regard, JFA worked directly with the Supreme Court and MOLJPA to improve the legal framework governing the judiciary that would lead to access to justice and quality service delivery for citizens. A review of JFA documents indicate that JFA encountered reluctance from the Chief Justice and Minister of Law to tackle issues relating to independence. After several failed attempts to engage these counterparts during 2013, JFA introduced the International Framework for Court Excellence (IFCE) as a toolkit to improve self-governance, integrity and self-examination. To reinforce the principles of IFCE, JFA sponsored a high-level Bangladeshi delegation, including Supreme Court judges, to Singapore to attend the Asia-Pacific Judicial Reform Forum, which was followed by two court excellence workshops for subordinate court judges. While IFCE did not gain traction with the Supreme Court, JFA narrowed its focus to a mutually agreed upon endeavor - improving court administration and case management at the policy level with the Supreme Court, and at a practical level at subordinate courts. Legal Framework for Judicial Self-Governance As a means to improve the legal framework for judicial self-governance and to provide a basis for reform advocacy and policy dialogue with the Supreme Court, JFA conducted a comprehensive analysis of the law, rules, directives, and other acts regulating judicial self-governance and thereafter provided policy recommendations to the Supreme Court that would enhance judicial self-governance and court excellence standards. Over a period of numerous months, JFA consulted frequently with the Supreme Court Office of the Registrar General and other stakeholders to develop relevant court policies and procedural rules. Criminal Rules and Orders and Civil Rules and Orders JFA also worked over numerous months with the Supreme Court Office of Registrar General to modernize the Criminal Rules and Orders (CrRO) and Civil Rules and Orders. While improvement of the Civil Rules and Orders has languished, amendments to the CrRO were approved by the Supreme 23 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Court in June 2017 and forwarded to the MOJLPA for vetting where they remain today. In a GOB KII, however, the ET learned there is no sign of progress with the CrRO at the MOJLPA. Judicial Case Management Policy In November 2016, after providing the Registrar General an analytical report of eight countries and five states in the U.S. concerning case management procedures, JFA began assisting the Supreme Court in drafting the Judicial Case Management Policy, the purpose of which is to provide guidance to subordinate court judges to achieve greater efficiency in case processing. Two joint JFA-Supreme Court workshops chaired by the Chief Justice were held in January 2017 during which the concepts of judicial case management were reviewed. According to a GOB KII, the Supreme Court was dissatisfied with the draft of the policy and the Supreme Court Research Unit made significant revisions to improve its quality and usability. As a result of the departure of the Chief Justice in October 2017, the Judicial Case Management Policy is still pending adoption at the Supreme Court. The GOB key informant speculated that the new Chief Justice has no reason to ignore the Judicial Case Management Policy; however, he may have his own vision for reform, which would necessitate revisions to and therefore delay adoption of the Judicial Case Management Policy. Judicial Evaluation Policy In July 2017, the Supreme Court implemented the Judicial Evaluation Policy for which JFA provided assistance. The goal of the Policy was to evaluate subordinate court judges in 12 test districts. For the evaluation, District Judges were asked to report in a custom-made database on the case management performance of the subordinate court judges. In a KII with a District Judge and subordinate court judges, the concept of evaluating judicial performance was warmly received, yet implementation by daily record￾keeping for the evaluation was cumbersome and time-consuming. Subsequently, the Supreme Court, on its own initiative, built an online platform to streamline the process. Because of glitches in the software as well as changes in personnel at the Supreme Court, reporting from the 12 test districts has halted. In October 2017, the Chief Justice was removed and key judicial officers within the Supreme Court were “posted out,” thus all ongoing reform efforts supported by JFA were suspended. ICT Training for Court Staff In response to the Chief Justice’s Modernization Agenda for which JFA co-organized (with the Supreme Court) a two-day workshop in April 2016 titled “Strengthening Judicial Performance Evaluation and Establishing Criteria for Measuring Success,” JFA agreed to assist the Chief Justice with one of his main goals to computerize the judiciary. In the Fall 2017, JFA initiated an Information and Communications Technology (ICT) training to subordinate court staff in two pilot districts to build their capacity to work within a modernized judiciary. JFA has continued such training in other pilot districts. Respondents from multiple KIIs expressed satisfaction about the ICT training, one stating that as there is no provision of ICT training of newly-appointed court staff. JFA provided training is therefore essential. Collaboration with Judicial Administration Training Institute (JATI) Joint JFA-JATI Trainings A review of JFA’s documents indicated that JFA began collaborating with JATI in late 2014, while concurrently developing training modules that focused on court excellence and judicial governance issues including accessibility, efficiency, transparency, and accountability. In March 2015, JFA jointly conducted with JATI a one-day training of the trainer workshop and a two-day training on court administration and case management, which, according to a KII, were well-received. These trainings were followed by a similar joint training in 2016 during which the same resources generated in the first training were utilized. A KII with JATI revealed their regret that the court administration and case management trainings did not continue. In addition to other subject matter trainings at JATI, JFA USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 24 sponsored mediation training for Supreme Court women judges in 2017 and an ICT training of court staff at JATI. KIIs with JATI and JFA staff, along with a review of JFA documents, indicate that JFA underwent a re￾examination of exclusive training with JATI after its initial trainings due to rotational postings of key judges. Additionally, the unavailability of time slots in the training schedule; the lack of JATI’s logistical capacity; the involvement of MOLJPA’s bureaucracy in selecting trainees (particularly those who did not work in the pilot districts); as well as quality of trainers (some of whom were not well-versed in court excellence principles) were significant obstacles for continuation. A KII with an external stakeholder indicated that JFA staff introduced the Bangladesh Counter￾Trafficking in Persons (BCTIP) Activity to JATI, which facilitated their work with district-level and special court judges to enhance understanding about TIP. Through JATI, BCTIP has provided training to 40 district judges, and numerous subordinate judges on TIP law, procedure, and related case management topics. Though the KI had not personally observed any JFA-organized trainings, JATI staff were positive about JFA’s work. The KI, however, expressed frustration about the quality of trainings delivered by senior judges through JATI and suggested that BCTIP could increase collaboration with JFA on curriculum development and training delivery for judges, particularly those serving in districts where the tasks overlap. Technical Assistance to JATI In addition to trainings with JATI, JFA funded the attendance of JATI’s Director of Administration at a court administration conference in Sydney, Australia in 2014 and a court excellence induction program in Singapore in 2016. JFA also worked with JATI’s management to widen its visibility by upgrading course curricula and training plans, creating a training calendar, increasing trainer resources and capacity, printing the JATI brochure, and developing a new logo. JFA has also provided financial and logistical support to JATI, including the donation of projectors. In an extensive KII with JATI leadership during which each of JFA’s tasks was discussed, JATI highly praised JFA for its collaborative and respectful relationship with JATI and attributed its growth as a training institute in part to JFA. Court Excellence Bench Book: Core Competencies and Integrity Also, in collaboration with JATI, JFA began the development of the “Court Excellence Bench Book: Core Competencies and Integrity” in late 2015. The Bench Book is designed to provide judges, judicial officers, and court officials in the subordinate courts a resource that includes information, tools, and techniques to achieve core values of court excellence and judicial governance. According to a KII with JFA, the Book also contains best practices from Bangladeshi courts addressing backlog and case processing delays. In early 2017, JFA initiated dissemination of the Bench Book to JATI as well as to the pilot districts; however, since it has yet to be approved by the Supreme Court, the Book has not been systematically distributed to subordinate court judges in all 64 district courts, which was JFA’s intention. KIIs with subordinate court judges indicated very limited use, understanding, and ownership of the Bench Book. One judge could not find his copy, and another had only heard about it. According to four KIs, all of whom were men, the usefulness of the Bench Book is questionable, given that the hard copy written version is distributed in English only and therefore is not helpful to court staff. A Bangla translation of the Bench Book is available on compact disc (CD). The Bench Book also does not have relevant references to Bangladeshi laws and rules. Furthermore, given that neither the Supreme Court nor the MOLJPA has sanctioned the Bench Book, judges are not motivated to use it. Case Management Training at the Subordinate Courts For the reasons stated above and to directly assist judges in improving their operational efficiency, JFA began conducting court administration and case management trainings in late 2015. These trainings also 25 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV integrated IFCE principles at the subordinate courts. KIIs revealed that the District Judge generally conducts the trainings, facilitated by a JFA representative, and uses training materials provided by JFA. The format of the training involves a PowerPoint presentation on court excellence principles followed by practical lessons ex-tempore. It is unclear whether District Judges are the recipients of JFA trainings before conducting their own trainings. In KIIs with six judges, of whom two were women who participated in one or more JFA trainings on court administration and case management in Khulna or Rajshahi, the ET heard a variety of comments about the duration and impact of the trainings. While some judges had difficulty recalling the training, others recalled participating in 2-3 trainings. They reported differently on training duration; one judge stated that the training lasted 6-7 hours, while other judges noted that the training lasted 1-2 hours. One judge complained that the trainings were held at inopportune times, particularly in late afternoons after court hours. Judges were tired, and it was difficult to concentrate on the course content. As for the impact of the training, respondents praised the training because it helped them to “more effectively reduce their caseload inventory.” However, when probed for specific impacts, most respondents could neither reiterate lessons learned nor refer to actions taken as a result of the training. One respondent stated that because of the training, he reviews easy cases along with difficult ones so that the number of resolved cases is greater, while another respondent prioritizes older cases for resolution. The ET heard repeatedly that case backlog is increasing rather than decreasing. In Rajshahi, the ET observed a case management training conducted by the acting District Judge for 18 judges (13 male; five female) of the court. The training lasted for 30 minutes and consisted of a short slide presentation during which the acting District Judge read out the five features of a credible justice system and the seven areas of court excellence. Following the slide presentation, one judge questioned the District Judge and was criticized for disagreeing with him. There was no discussion of the IFCE booklet or the NCSC performance measurement tools that were distributed to the participants. And while the DLAO reminded the acting District Judge to engage the participants in discussion, the ET observed that junior judges were cautious about speaking up and expressing strong opinions as the culture of court hierarchy is strong. While this one training may be an outlier, the ET found that it pointed to the need for quality control. Although field-based trainings are laudable, the quality of trainings can be fickle as they are dependent upon an individual and his/her competency and commitment, rather than an institution. In its numerous KIIs and FGDs with 18 judges, of whom 13 were women, representing the Supreme Court, District Court and subordinate court judges, the ET asked respondents for their solutions to resolving case backlog. Stating numerous and severe challenges to improving case management and reducing case backlog, the respondents suggested the following reforms: • Judges should comply with the strict requirements regarding adjournment applications filed by lawyers (considered the most severe barrier); • The GOB should significantly increase the number of judges and courtrooms; • Criminal and Civil procedural laws and rules should be modernized; • Witnesses should be compelled to testify to reduce case delay; • Substantive laws, many of which date back to the 1800s, relating to certain financial transactions, such as “check dishonor” cases, should be revised to bypass the need for court review; • Sudden transfers of judges should be reduced; • Political and arbitrary promotion of judges should be eliminated; and • Executive and political influence on judicial decision-making should be rejected. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 26 TABLE 6: IMPLEMENTATION OF RECOMMENDATIONS FROM THE MID-TERM EVALUATION: EQ 4 Note that numbers below match the recommendation number from the Mid-Term Evaluation Report 5. JFA should continue court administration and case management training at JATI as well as introduction of the core competencies curriculum. In doing so, JFA should consult the JATI needs assessment prepared by the UNDP Justice Sector Facility (JSF) project and any materials or lessons learned from training provided by UNDP’s JUST project. The team found that while well-intentioned, district-based case management trainings have the potential of not achieving the same results as JATI trainings in that district-based trainings are dependent upon the commitment and competency of the District Judge while JATI trainings are system-based. The Team is recommending that ad hoc district-based case management trainings either be reduced or quality control oversight be significantly increased. 6. JFA should explore building upon lessons learned from JUST’s judicial leadership program with the Supreme Court, which focused on leadership development, change management, code of conduct, disciplinary procedures and strategic planning, to introduce similar leadership skills components into JATI training of district judges. The team found that UNDP continued to support the Supreme Court in the above-stated areas until recently. To distinguish its assistance, JFA focused on developing the Judicial Evaluation Policy, the Judicial Case Management Policy, which is currently being reviewed by the Supreme Court for adoption, as well as the Criminal Rules and Orders, which is currently being vetted by MOLJPA. 13. JFA should conduct any strategic (re) introduction of the IFCE model carefully and in collaboration with the Chief Justice as well as other judges that are familiar with the IFCE as a result of participation in regional conferences supported by JFA. The team found that IFCE principles are integrated in the District-based case management trainings, yet there remains a lack of understanding and ownership of the principles. Judges in KIIs stated that the principles contain lessons learned which are not applicable to Bangladeshi subordinate courts 14. JFA and USAID should continue to engage new high-level GOB counterparts and leverage existing relationships (former NLASO Director, now Registrar General of the Supreme Court; Chief Justice who was the recipient of the Judicial Self-Governance Award) to create or strengthen necessary buy-in for continuation and expansion of activities. The team found that JFA excelled in engaging with its GOB counterparts, particularly the Supreme Court Office of Registrar General and the Supreme Court Research Unit, in supporting their efforts for the Judicial Evaluation Policy, the Judicial Case Management Policy, the Criminal Rules and Order, and the ICT training of court staff. CONCLUSIONS JFA made serious attempts to support the Supreme Court in improving the legal framework for judicial self-governance. However, the turnover of the Chief Justice and Registrar General during the duration of the JFA Activity significantly hindered JFA’s progress. Also, as a result of the disruptions in the judicial leadership, JFA was unable to create a critical mass for judicial self￾governance reforms. JFA’s collaboration with JATI enhanced JATI’s organizational capacity and outreach. JFA provided JATI with technical assistance and support, and as a result of a productive and strong working relationship, JATI attributes its growth as a training institute in part to JFA. JATI found JFA technical assistance relating to training materials and joint trainings to be most helpful. Subordinate court judges are not motivated to use the Bench Book, as it does not contain references to Bangladeshi laws and rules, has been distributed in hard copy in written English only (though available in Bangla on CD), and has not been sanctioned by the Supreme Court and the MOLJPA. 27 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV There was no evidence of case backlog reduction as a result of JFA case management trainings, nor did judges effectively advocate for improvements in the legal and regulatory framework to achieve self-governance. However, awareness of the need to significantly reduce case backlog as well as the need for increased self-governance was evident among the District Judges and subordinate court judges which, in part, can be attributed to the JFA case management trainings. The solutions mentioned by respondents for reducing case backlog pertained to revisions of outdated laws and procedural rules, as well as the strict imposition of certain procedural rules. Additionally, district-based case management trainings may not warrant continuation because such trainings fail to ensure ownership by the stakeholders and do not demonstrate a meaningful impact on the capacity of judges to reduce case backlog. EQ5: TO WHAT EXTENT ARE THE ACTIVITY OBJECTIVES VIABLE WITHIN THE CURRENT JUDICIAL AND POLITICAL CONTEXT IN BANGLADESH? FINDINGS IR 1 Improved Delivery of Legal Aid in the Formal Justice Sector GOB support of legal aid firmly established With the Legal Aid Services Act (LASA) of 2000, subsequent amendments to LASA and the Legal Aid Policy in 2013-2014, and adoption of the LASA Regulation in 2015, government-sponsored legal aid for the poor and marginalized is firmly rooted in law and continues to grow in practice. NLASO, the statutory body established to implement the GOB scheme across the country, is actively engaged in developing—with JFA support—its new 2018-2021 Strategic Plan. In KIIs with external stakeholders, GOB, current and former NLASO directors, and JFA sub-grantees, as well as FGDs with UZLAC/UPLACs and panel lawyers, the vast majority of respondents indicated that GOB legal aid, delivered through the SCLAC and DLACs, is here to stay. KIs commented that since legal aid is mandated by law, every administration, no matter which political party is in power, is bound to ensure this service for its poorest citizens; no administration will move to discontinue or reverse the progress made. However, two KIs stated that while the current Prime Minister and Law Minister strongly support legal aid, reform initiatives are often influenced by many decision makers within the justice sector, and support for legal aid does not necessarily equate to endorsement of judicial reforms, for example. Government—Non-governmental organization (GO/NGO) collaboration In the Bangladesh context, GO/NGO collaboration refers to public and private sector actors working together based on common goals, objectives, and strategies within the confines of available resources. KIIs with six external stakeholders confirmed that currently coordination between GOB and NGO legal aid services providers is extremely limited. This is due to a lack of mutual respect, trust, and recognition of the others’ strengths, values, and comparative advantages. Without meaningful collaboration, viability could be compromised. In KIIs with four external stakeholders and four legal aid service organization officials, respondents agreed that legitimate GO/NGO collaboration would be able to exert positive influence on the current legal aid scheme, mitigating the inherent weaknesses of the government due to bureaucracy and resource limitations of NGOs. Coordination with other Donors and USAID Implementers on Legal Aid Delivery In KIIs with five external stakeholders, all respondents indicated that despite the ongoing challenges of capacity building and sustainability, the demand for legal aid services continues to grow, and they are optimistic about the continuing viability of legal aid in the formal sector. KIs described some of their current work and plans in the context of future viability. One KI from the Bangladesh Counter￾Trafficking in Persons (BCTIP) Activity reported that Winrock International has signed a Memorandum of Understanding with NLASO to work with panel lawyers on handling TIP and related labor cases. A KI reported that United Nations Development Programme (UNDP) is going to be working with NLASO to USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 28 open a pilot district where they will focus on community outreach and supporting UPLACs/UZLACs. Another KI from the United Kingdom’s Department for International Development (DFID) reported that it is designing a new project that will have a Violence Against Women component, including support for NGO-provided legal aid. DFID continues to work with prison populations, and the KI recognizes that coordination with SCLAC on jail appeals and reducing prison overcrowding is mutually beneficial. Other KIIs mentioned the need for better information sharing and use of technology at the local level to enhance services delivery, including coordination with the GOB Union Information Centers (UIC) to make provider lists more readily available. IR 2 Increased Self-Governance of the Judiciary to Better Serve the Public Conflict between the Executive and Judicial branches In multiple KIIs, external stakeholders expressed concern regarding the capacity of the judiciary as a whole to assert self-governance in the current political climate. In addition to the forced resignation of the Chief Justice, as previously referenced, the Supreme Court Registrar General, Syed Aminul Islam, a highly respected senior leader in the judiciary, was transferred out of the Supreme Court. A KI suggested the dismissals of the Chief Justice and Registrar General, as well as the continuous politicization of appointments, promotions, and transfers, has shaken the confidence of judges. “They can no longer trust the system, since it is so politicized,” a respondent noted. Another key external stakeholder went so far as to state that the independence of the judiciary in Bangladesh has reached a record low. A KII with UNDP revealed that the strained relationship between the executive and judicial branches and their unwillingness to work together led to two UNDP rule of law projects in 2012 - one with the Supreme Court and the other with the MOLJPA as the implementing partner. A KII with DFID indicated its waning interest in working with the MOLJPA, as opportunities for real reform are limited due to political constraints. The same key external stakeholder who lamented the low level of judicial independence in Bangladesh stressed the importance of continuing engagement with the Supreme Court and subordinate courts, as many reform-minded judges want to pursue excellence, transparency, and fairness in the courts and need external incentives to do so. Challenges to judicial self-governance In addition to the lack of judicial control over the appointment, promotion, and transfer of judges, often without notice, KIIs with subordinate court judges also indicated serious issues concerning the power of the purse for the judiciary. The judiciary and subordinate courts do not have control over the budget, nor do they have the authority to submit proposed budgets to the GOB. Alarmingly, the most recent budget allocated by the GOB to the judiciary is a smaller percentage of the national budget than previous allocations. One judge stated that the judiciary is no longer a priority sector for the government, stating “the government is more willing to allocate resources to the police and the military” than the judiciary. Support for certain judicial reforms As previously stated in EQ 4, there was some progress made during JFA towards developing a legal framework for judicial self-governance with the passage of the Judicial Evaluation Policy and the pending Judicial Case Management Policy and the CrCO, both of which have a chance of adoption should they become a priority of the Supreme Court and the MOLJPA. Interestingly, KIIs with representatives of the Supreme Court and MOLJPA suggested political will to address the issue of case backlog. The issue appears prominent both in the Supreme Court’s Strategic Plan 2017-2022 and the current MOLJPA’s Sustainable Development Goals Work Plan, that focuses on ADR and case-coordination mechanisms to tackle inefficiency in the judiciary. 29 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV A KII with a Supreme Court representative (who spoke on behalf of the newly-appointed Chief Justice) provided the ET with the Chief Justice’s four priority reform areas: • Building capacity of the justice system; • Developing a witness management system; • Developing infrastructure at the district level; and, • Training of judges Additionally, the ET learned that the Supreme Court is gearing its efforts toward the development of an “e-judiciary” which is identified in the Supreme Court Strategic Plan. This was confirmed in a KII with USG representatives who stated that the development in e-governance through Information and Communications Technology (ICT) is a priority of the GOB under its “Digital Bangladesh” agenda. Role of the Female Legal Professional According to all relevant KIIs and FGDs, the viability of Sub IR 2.3 is dependent upon the role of the female legal professional. While equal opportunity for female judges is promoted among both female and male judges, equal opportunity for female attorneys, while viable, is fraught with challenges and barriers due to gender bias, both professional and societal as previously described. Among female law students, equal opportunity exists at university law departments. Access to leadership positions As for equal access to leadership positions for female judges and lawyers, the viability is questionable according to respondents in relevant KIIs and FGDs. There are three major barriers to an increase in leadership positions for female judges and lawyers: the first is the gender bias that exists within the political appointment process; the second is the fact that female judges and lawyers themselves are uncomfortable using political means to obtain a leadership position; and the third is the reality that the small number of female judges with seniority is not sufficient to accede to leadership positions. According to KIIs, the judicial leadership appointment process as well as the bar leadership election process are fundamentally political. While judicial appointments to the judiciary are based on competency, appointments to the Supreme Court and to the subordinate courts as a District Judge are often influenced by the political parties. Election to the Bar Council and leadership positions with the District Bar Associations are also based on political party support. To be considered a viable candidate for the Bar Council or for a leadership position with a District Bar Association, the candidate must receive a nomination by a political party; only then will that candidate be seriously considered by the overwhelmingly male members. KIIs have revealed that female legal professionals are uncomfortable kowtowing to political parties and have neither experience nor interest in seeking political support to obtain leadership positions within the judiciary and bar. IR 3. Increased Citizen Awareness of Legal Rights and Responsibilities on Selected Legal Issues In KIIs with external stakeholders, GOB, current and former NLASO directors, and JFA sub-grantees, as well as FGDs with UZLAC/UPLACs and panel lawyers, the vast majority of respondents indicated that GOB-sponsored legal aid has now been solidly established and will remain viable. Awareness-raising among potential clients and all citizens about the right to legal aid and how to obtain it is non￾controversial and essential to its continued expansion across the country. Though KIs and FGD participants differ about what types of tasks are most effective, who is best able to implement tasks and how, the issues of viability and necessity are not in question. One KI with MSI, who was not familiar with JFA’s work, indicated that the new Countering Violent Extremism (CVE) programming through the Obirodh—Road to Tolerance Activity will be centered around awareness-raising and engagement of diverse segments of the community, including poor and marginalized populations. The KI considers dissemination of information about access to justice and legal USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 30 75% 17% 8% Excellent Very good Good FIGURE 1: MINI-SURVEY SATISFACTION WITH PARTNERSHIP BETWEEN NCSC AND SUB-GRANTEES aspects of issues that give rise to extremism in communities to be a potential CVE activity that could be implemented by reputable local NGO partners with capacity and experience in legal aid programming. CONCLUSIONS Several JFA Activity elements remain viable in the current judicial and political context: • Legal aid frameworks and structures that have been put in place because of the delivery of GOB-sponsored legal aid services, as political and financial support, in addition to donor investments. • Collaboration between the Supreme Court and the MOLJPA to reduce case backlogs and develop information and communications technology. • Increasing the participation of female judges and lawyers and their professional capacity to advocate for equal opportunity within the judiciary and bar. • Increased awareness about GOB-sponsored legal aid for the poor and marginalized, as expanding citizen knowledge on this rights issue is non-controversial, even apolitical, and will continue formally and organically through a variety of means. Promoting women’s equal access to leadership positions within the judiciary and bar is not viable, given that the appointment and election process to leadership positions is political and dependent upon circumstances that are not favorable to women. The viability of legal aid services delivery can be enhanced through greater GO/NGO collaboration. While the GOB and NGOs have common goals and vision with respect to legal aid services delivery, the basic institutional approach to addressing access to justice differs. Effective collaboration should recognize these differences and build based on the comparative advantages of GOB mechanisms and NGO approaches to legal aid. EQ6: HOW SUCCESSFUL WAS THE IMPLEMENTING PARTNER IN MANAGING SUB-GRANTS TO ACHIEVE THE ACTIVITY OBJECTIVES? FINDINGS Satisfaction of Individual Sub-Grantees In KIIs with 12 JFA sub-grantees (11 from districts where JFA is currently active; one from a ‘closed’ pilot district), all KIs indicated that they were very satisfied with NCSC’s management of their sub-grant. Respondents mentioned both management support for program implementation as well as financial oversight. When asked about specific types of support they considered most helpful, the majority of KIs cited one or more of the following: • Development and provision of awareness-raising materials, such as billboards, posters, leaflets, stickers, info cards, brochures, etc. • In-person visits by NCSC project office staff, including members of the Grants Team, Legal Team, as well as Communications and M&E staff. • Guidance and support on financial/budgetary matters, including compliance with USAID procurement rules, review of financial statements, and assistance with administrative procedures. • JFA’s Quarterly Sub-grantee Coordination Meetings and various trainings held in Dhaka for sub-grantee staff members, detailed further below. 31 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV KIs also reported that JFA consistently disbursed operational and programming funds to sub-grantees in a timely manner. As one KI stated, “JFA provided everything—materials and advice—that I needed to do my job daily.” Moreover, 75 percent of mini-survey respondents rated their level of satisfaction with NCSC as Excellent, as shown in Figure 1. Coordination meetings and skills trainings According to KIIs, JFA’s Sub-Grantee Quarterly Coordination Meetings were one of the helpful management tools JFA used to share important, practical information on a variety of issues related to program implementation. JFA reports indicate that, as of January 1, 2018, a total of 14 coordination meetings have been held with sub-grantee PDs/PCs and finance staff since program inception.10 Topics addressed during the meetings include: achievements and challenges faced by sub-grantees during the previous quarter; activity implementation strategies; event-specific planning, e.g., National Legal Aid Day; and post-event debriefs. In addition, JFA provided information and/or instruction on essential sub￾grantee skills and competencies, such as report writing, conducting courtyard meetings, procurement processes, and budgeting, among other topics. All sub-grantee KIs reported participating in Quarterly Coordination Meetings, and they uniformly appreciated the opportunity to discuss achievements, challenges, and activity implementation strategies with JFA staff and leadership, and to share experiences with fellow sub-grantees. Challenges to operating environment According to KIIs with external stakeholders, the environment for development assistance in Bangladesh deteriorated significantly during 2015-2016 due to heightened political tensions and a marked increase in violence and lethal crime. 11 JFA reports provide a general assessment of the political and security context and track the operating environment over the course of the project. A countrywide blockade during January-March 2015 halted intercity travel, and subsequent killings of both foreign civilians and Bangladeshi nationals constituted a threat of terrorism. NCSC confirmed that the high-risk security situation, which only began to ease in the second half of 2017, impacted the JFA Activity in terms of communications and coordination with sub-grantees and the numbers of events and tasks that could be conducted. NCSC management indicated that, in response to these challenges, the JFA team pursued a variety of strategies to make up for the restrictions on their travel to districts, including maximizing the use of technology, internet, and alternative communications channels. KIIs with sub-grantees in several districts confirmed that JFA management utilized a combination of communication methods, particularly during times of heightened concerns about security, and were responsive and flexible, advising on event modifications and rescheduling as necessary. Partner capacity building Several KIs reported that the guidance and oversight provided by JFA leadership and staff has enabled their organizations to analyze and enhance internal systems so that their own capacity for managing the NGO overall has improved. One KI highlighted the increased capacity of staff to interact effectively with judicial officials as a result of support from the JFA legal team. In addition, six KIs indicated that they regularly consult with more experienced JFA program managers for guidance and advice on program implementation or other operational matters and may continue to coordinate after JFA tasks end. 10 The 15th Sub-Grantee Coordination Meeting was held on February 14-15, 2018, during the period of ET fieldwork, and the ET took advantage of the opportunity to conduct KIIs with eight PMs/PCs from various districts while they were in Dhaka. 11 See also Bloody Blockade Cripples Bangladesh Economy, https://www.aljazeera.com/indepth/features/2015/03/bloody-blockade￾cripples-bangladesh-economy-150311112641085.html (accessed 3/7/18). USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 32 TABLE 7: IMPLEMENTATION OF RECOMMENDATIONS FROM THE MID-TERM EVALUATION: EQ 6 Note that numbers below match the recommendation number from the Mid-Term Evaluation Report 11. JFA should develop a sustainability plan for current sub-grantee awareness-raising, sensitization, and ULAC/UPLAC capacity-building activities to continue following the end of the project. The ET found that while JFA instructed sub-grantees to write letters to UZLACs and UPLACs encouraging them to conduct their own activities with their own funds to ensure sustainability of awareness raising, JFA did not develop formal sustainability plans nor draft sustainability guidelines, per NCSC management response. JFA field data and FGDs confirm that some UPLACs and UZALACs are securing their own funds and conducting awareness-raising activities independently. JFA also reported that it advocated for Chowki LACs to receive GOB funds to cover awareness activities. 12. JFA senior management should consult with each sub-grantee individually during quarterly coordination meetings to determine if there are any particular challenges or sensitive issues that have emerged with DLAC, ULAC, or UPLAC officials that should be addressed by a field visit by a JFA senior manager. The ET found that sub-grantees are highly satisfied with the support and guidance they receive from JFA Management at quarterly coordination meetings, as well as during site visits by various JFA Team Members from the project office in Dhaka. Although sub-grantee capacity building is not specifically set forth as part of the three objectives of the JFA Activity, NCSC reported that it intended for its management practices to strengthen the overall capacity of JFA partners and adjusted its accompaniment and oversight role as sub-grantees gained experience and became more independent. JFA sub-grantees indicate that the guidance and management provided by JFA leadership and staff has strengthened their overall capacity for management of their own organizations. As a sub-grantee noted, “We are very appreciative, even when JFA asks us to redo the work, as it helps us learn and enhance our capacity. The capacity of [our organization] as well as individual staff member has increased due to JFA’s support. We feel that we can design a budget properly and with confidence.” However, while JFA leadership recognizes the importance of organizational development to sustainability of both sub-grantees and the legal aid awareness and service delivery tasks in which they are engaged, sustainability issues should have been addressed more directly and earlier in the life of project, as discussed in more detail in EQ 7, below. CONCLUSION NCSC was highly successful in managing sub-grants, which enabled local partners to effectively contribute to achieving JFA Activity objectives. NCSC’s hands-on, individualized attention, combined with a variety of responsive and positive management practices, enhanced the ability of partners to implement tasks under Objectives 1 and 3, and generated a high level of satisfaction among all sub-grantees. In the face of heightened security concerns and travel restrictions during 2015- 2016, NCSC adjusted operations and management practices, adopting flexible solutions to minimize interruptions in its communications and tasks with sub-grantees. Regular coordination meetings and trainings encouraged team building and partner networking opportunities, which also strengthened JFA partners’ organizational capacities, enabling more experienced JFA Activity implementers to serve as models to newer partners. 33 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV EQ7: TO WHAT EXTENT HAS JFA ACHIEVED PROGRESS TOWARD ESTABLISHING SUSTAINABILITY MECHANISMS? FINDINGS IR I: Improved Delivery of Legal Aid in the Formal Justice Sector Sustainability of DLAO Training According to JFA documents and KIIs with DLAOs, JFA provides training to new DLAOs at regular intervals and conducts training for Legal Aid office staff. One KI indicated that the experience she gained as a DLAO was extremely valuable and will have a positive impact on how she approaches her next position. However, another KI reported that he had never attended a DLAO training and would be leaving his post early. In KIIs with an external stakeholder and a DLAO, respondents explained that while the maximum term for a DLAO is three years, many DLAOs are transferred to another posting and/or promoted well before the end of their term. There is no handover mechanism to ensure a smooth transition from one DLAO to the next, and departing DLAOs do not generally provide any information or insight from their experience to their successor. According to KIIs with DLAOs, sub-grantees, and external stakeholders, the legal aid referral work of the UZLAC is hampered in some sub-districts due to interference with or lack of support for the UZLAC Chairman’s leadership. While UZLACs are headed by the publicly elected Upazila Chairman, the UZLAC includes representatives from both the government and NGO sectors. The Upazila Nirbahi (Executive) Officer (UNO) is the UZLAC Member Secretary. While the UZLAC Chair is aligned with the Ministry of Local Government, Rural Development and Co-operatives, the UNO is a government official and holds the top executive position at the Upazila level. In KIIs with a sub-grantee and a DLAO, and in an FGD with panel lawyers, respondents report that tensions due to personality conflicts and power dynamics between the UP Chairman and UNO undermine the ability of many UZLACs to fulfill their mandates. IR 2: Increased Self-Governance of the Judiciary to Better Serve the Public Sustainability of Legal Framework for Judicial Self-Governance JFA continually engaged in efforts, particularly in the last two years, to support the Supreme Court’s and the MOLJPA’s legal framework for judicial self-governance by providing expert analysis and policy recommendations for reform of applicable laws and criminal and civil procedural rules, and specifically the Judicial Case Management Policy, the Judicial Evaluation Policy and the CrRO and Civil Rules and Orders. With periodic consultations with the Supreme Court Office of the Registrar General and other stakeholders, JFA continued its engagement in pursuing the adoption of the Judicial Evaluation Policy and the Judicial Case Management Policy by the Supreme Court and the CrRO by the MOLJPA. While the Judicial Evaluation Policy was adopted and implemented by the Supreme Court (which by virtue of being operational is sustainable), the Judicial Case Management Policy and the CrRO are still languishing as a result of the change of key personnel in the Supreme Court and MOLJPA. Sustainability of Tasks supporting JATI JFA’s interventions in enhancing JATI’s organizational capacity and increasing its visibility—such as upgrading course curricula and training plans, creating a training calendar, printing the JATI brochure, and developing a new logo—are sustainable in that JATI continues to use the products. JFA’s collaboration with JATI on the creation of the Court Excellence Bench Book has the potential of being sustainable as it has been distributed to JATI as a resource for its trainings as well as to the subordinate courts in pilot districts for use in district-based case management trainings. However, KIIs revealed that the Bench Book was not utilized by judges because the references to Bangladeshi laws and rules were considered to be not relevant; nor was it used by court staff because the hard copy written version of the Bench Book was distributed only in English (though a Bangla version is available on CD). While the USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 34 joint trainings with JATI helped to transfer knowledge to the trainers, and while resources provided by JFA to JATI for the joint trainings were acknowledged as being useful, the sustainability of these efforts is unclear as the joint trainings are ad hoc only. Sustainability of Case Management Trainings at District Courts While there have been numerous district-based case management trainings throughout the JFA Activity, KIIs with subordinate court judges and District Judges, particularly in Khulna and Rajshahi, as well as the DO of the case management training in Rajshahi, revealed minimal impact, at best. It was evident that a lack of quality control over district-based case management trainings resulted in disparate impacts of the trainings, depending upon the District Judge as the trainer. Without continued JFA support, there is no evidence that the trainings will continue and are, thus, unsustainable. Female judges and Sustainability of the BWJA JFA, in just two years from 2016-2018, assisted BWJA to advance from a fledging association to one that is registered and provides services to all female judges. The BWJA now has a constitution, recently￾developed strategic plan for future services, directory of members, and trainings for its members, along with a public relations brochure to encourage support. The leadership of the BWJA told the ET that they are keenly interested in growing the Association. Female Lawyers and Law Students With regards to female lawyers and law students, JFA worked primarily with individuals, rather than with institutions; thus, sustainability of its tasks is tenuous. As the BNWLA represented only a percentage of all female legal professionals, JFA was compelled to seek active female lawyers with whom to provide legal drafting and court procedure trainings. KIs who participated in the trainings attested to increased knowledge and professional capacity, which resulted in being more confident in the courtroom. Female law students engaged in a variety of JFA tasks ranging from women-to-women dialogues, court visits, judgment readings, and internships, the impact of which was demonstrated by the law students’ inspiration and enthusiasm to be leaders in pursuing equal opportunity for female legal professionals. IR 3: Increased Citizen Awareness of Legal Rights and Responsibilities on Selected Legal Issues Sustainability of legal aid awareness-raising UZLAC and UPLAC members represent various segments of the community, including local government, NGOs, health and welfare organizations, women’s groups, educators, journalists, etc. In each FGD with UZLAC or UPLAC members, two or more participants reported that they regularly disseminate information provided by JFA about GOB-sponsored legal aid services, frequently in connection with their full-time jobs, and will continue this practice. Moreover, FGD participants reported that some UPs created a budget line item in their work plans or otherwise secured funds for public hearings and other awareness-raising events, and other UPs are considering doing the same. When asked about future awareness-raising in their districts, FGD participants (community members and UPLACs) in every surveyed district reported that, on an informal basis, they voluntarily share information they have learned through JFA tasks about GOB-sponsored legal aid services to people in their communities and will continue to do so. The ET found that, in preparation for project close-out, JFA instructed sub-grantees to write letters to UZLACs and UPLACs encouraging them to conduct awareness-raising tasks with their own funds to ensure sustainability. However, according to JFA staff, JFA did not develop formal sustainability plans nor draft guidelines to assist sub-grantees in applying any of sustainability mechanisms JFA has identified. 35 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV CONCLUSIONS While several DLAOs reported increased skills capacity, particularly among permanent staff members, JFA training was not implemented in a way that ensured transfer of knowledge and experience from DLAOs to their successors. The ET did not find evidence of planning for sustainability of trainings or a mechanism to ensure knowledge transfer when DLAOs rotate in and out of posts. This means knowledge remains with one individual only and not with the District Legal Aid office. It is also unlikely that the trainings for DLAOs will continue without donor support. While JFA achieved progress with the Supreme Court and the MOLJPA in developing policies supporting the legal framework for self-governance, adoption and implementation of the policies has in most cases yet to be realized. Though the Judicial Evaluation Policy was adopted and implemented by the Supreme Court, the Judicial Case Management Policy and the CrRO remain under consideration. These latter policies could be sustainable if current leadership in the Supreme Court and MOLJPA agree to adopt and implement them. JFA’s support for JATI included development of critical organizational infrastructure and capacity, increasing the likelihood of sustainable gains for the organzation. JFA assisted JATI with upgrading course curricula and training plans, developing a new logo, etc. Developing this type of organiational infrastucture helps to ensure the organization is visable, organized, and relevant. JFA’s district-level case management trainings have resulted in varied results, at times dependent on the District Judge leading the training. It was difficult for JFA to provide quality control over these trainings and the District Judges that led them, and it is unlikely they will continue without JFA support. JFA contributed to the professionalization of the BWJA, gains that are likely sustainable. In just two years, BWJA developed into a fully-functioning association representing all female judges. With leadership reporting continued interest in strengthening and growing the Association, sustainability of these recent gains is likely. JFA’s work with female law students may be sustainable at an individual level (knowledge gained by each student engaged with JFA); however, this work is unlikely to continue due to the approach used. JFA, while initially working through university departments at law schools in Bangladesh, shifted to working and coordinating directly with law students on campuses. This quickly increased the reach of JFA tasks (as students were able to organize and share information more readily than departments) but did not help with establishing a mechanism for tasks to continue post-JFA. There is evidence that awareness-raising tasks will continue due to personal commitment or strategically secured funds. UZLAC or UPLAC members reported their commitment to continuing to share information about legal aid services. Greater indication of sustainability, however, is with UPs that have created a budget line item in their work plans for public hearings and other awareness-raising events. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 36 VI. RECOMMENDATIONS RECOMMENDATIONS FOR USAID 1. Support the expansion of DLAC support, LAC activation, and GOB-sponsored legal aid awareness-raising tasks to the 49 non-JFA pilot districts to grow demand for legal aid services throughout Bangladesh. 2. Support a comprehensive mapping of GOB-sponsored legal aid services and resources to determine the needs of DLACs/DLAOs in the 49 non-JFA pilot districts. This should be implemented in close coordination with NLASO, GOB stakeholders, as well as other justice sector donors and relevant NGOs. 3. Develop a future program that focuses on the Supreme Court’s and MOLJPA’s policy guidance to reduce case backlog, by revising substantive law and procedural rules and with the imposition of strict compliance of adjournment applications filed by lawyers. RECOMMENDATIONS FOR NATIONAL CENTER FOR STATE COURTS 4. In consultation with JFA sub-grantees, develop tailored sustainability plans for continued awareness-raising to be conducted by UZLACs/UPLACs, including legal literacy and community sensitization events, as well as UZLAC/UPLAC activation and capacity-building. 5. Gather and consolidate client feedback on legal aid services to share with panel lawyers and district judges for improved service delivery that is responsive and tailored to the needs of female and male clients. 6. Prior to JFA Activity close-out, transfer awareness-raising materials and implementation resources to Winrock International, Plan International, MSI, GIZ, and other implementers to disseminate in districts where donor-funded interventions are active or may resume in the future. 7. Continue collaboration with the Supreme Court and MOLJPA to implement the Judicial Evaluation Policy, Judicial Case Management Policy, CrRO, Civil Rules and Orders, and Bench Book (when approved) to help reduce case backlog and ensure that JFA’s key achievements are sustainable. 8. In collaboration with JATI, significantly increase quality control over district-based case management trainings to ensure that the trainings are substantively meaningful, taught by skilled trainers, impact participants’ behavior and understanding of the issues, and remain sustainable. 9. Revise the Court Excellence Bench Book in consultation with JATI and the Supreme Court to better incorporate relevant references to Bangladeshi laws. In addition, establish plans for printing and distributing the hard copy written version of the revised Bench Book in Bangla. 37 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV VII. LESSONS LEARNED 1. Encourage implementers to incorporate, during project design, the establishment of formal networks or communities of practice among sub-grantee partners, in order to facilitate knowledge sharing and mutual support among partners, which will build capacity and promote local ownership. 2. From the grassroots to the national stage, invest time and resources in the appropriate individuals, at every level, to support their potential to serve as champions of reform and change agents, particularly in Bangladesh, given the volatility of judicial leadership. In cases when the entrenched hierarchy makes it difficult to ‘manage up’ or influence those in higher positions of authority, equip partners and stakeholders with the tools and strategies to model reform-minded behavior for their colleagues and peers, and to inspire those who look to them for leadership. Provide skills training, professional development, and other support to such champions to amplify their influence and impact. 3. Manage programs and people flexibly and creatively. Adopt a flexible approach to programming, both in terms of scheduling and content, while preserving partner and stakeholder engagement. In Bangladesh, where security risks are high and variable, proactive, clear-sighted management of in-country staff and local partners is essential. During times when travel around the country is restricted or impeded, implementers should maximize the use of ICT or other mechanisms to keep communication channels open and maintain cohesion among team members. 4. Local and regional information and awareness-raising campaigns may benefit greatly from the development of standardized content and professional quality, user￾friendly materials and visual aids. When future program tasks are implemented in various, often diverse, regions around Bangladesh, it can be tempting to tailor messaging to the local context, sometimes too narrowly. Apply the most effective elements of localized awareness-raising campaigns to create materials and strategies that will communicate consistent, clear messages to all beneficiaries. 5. Given the volatile dynamics between the MOLJPA and the Supreme Court, as well as the ease with which leadership of the Supreme Court can be removed, it is wise to maintain several entry points at various levels within the Supreme Court as well as with District Judges and subordinate court judges, some of whom will, undoubtedly, become thought leaders in positions of power. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 38 ANNEX A: EVALUATION SCOPE OF WORK Scope of Work for the Justice for All (JFA) External Final Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Activity/Project Name: Justice for All (JFA) Contractor: National Center for State Courts (NCSC) Agreement Officer Technical Representative (AOR): Habiba Akhter Cooperative Agreement/contract number: AID-388-A-12-00004 Total Estimated Cost (TEC): $7,997,747 Life of Project/Activity: Start Date: 10/01/2012 End Date: 9/30/2018 Activity Geographic Regions: Dhaka, Mymensingh, Khulna, Rangpur, Rajshahi, Bogra, Dinajpur, Tangail, Munshiganj, Jessore, Cox’s Bazar, Moulvibazar, Pabna, Kurigram and Bagerhat. Mission Development Objective (DO): MO-Bangladesh, a knowledge-based, healthy, food secured and climate resilient middle-income democracy; DO-Citizen confidence in governance institutions increase I. Background The Constitution of Bangladesh guarantees an independent judiciary and equality under the law for all citizens. After more than four decades of its adoption, the rights guaranteed by the Constitution continue to be unclear despite some gains in independence at the highest levels of the judiciary in this country. The same goes with the efforts of a vibrant civil society to bring legal services to all citizens. The Government of Bangladesh (GOB) has undertaken an ambitious Vision 2021 including establishing judicial independence, judicial accountability, gender equality, and protection of the rule of law.12 The judiciary faces many challenges, the first being its continued dependence upon the executive branch. Higher courts have gained some degree of independence and self-governance, but the executive remains in charge of all career and administrative governance decisions affecting lower court judges and operations. Also, the judiciary is perceived in multiple surveys as corrupt and slow. Large case backlogs are causing long delays for citizens in both civil and criminal cases, leading to high levels of pretrial detention and general dissatisfaction with the financial and time costs of justice. In the prevailing context, access to justice is a challenge for most citizens. Formal justice institutions are complex and fraught with delays, resulting in low levels of acceptance especially within rural communities. Widespread poverty hinders access as citizens cannot afford the costs of litigation or even physically shuttling to the court. Thus, access to justice is reserved for the privileged and powerful. To ensure legal rights and access to justice requires citizen awareness first. An aware citizenry can make use of the available legal aid services. Above all, an enabling environment would be that in which all 12 Vision 2021 was the political manifesto of the Awami League political party before winning the National Elections of 2008. It stands as a political vision of Bangladesh for the year 2021. 39 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV citizens pursue legal options as and when needed, without any hindrances such as accessibility, cost, or other. USAID’s Justice for All (JFA) Activity is a response to ease the legal options for the common citizens. Development Hypothesis Gains in efficiency and responsiveness of legal aid services and in performance and governance of the judiciary will create the enabling environment for citizens to access effective legal remedies within the formal justice sector. Support for legal literacy will increase the awareness of citizens of their rights and their capacity to make meaningful choices to access and utilize legal aid services and formal justice institutions. Increased use of improved services will in turn lead to concrete outcomes that directly benefit the lives of disenfranchised communities while increasing citizen confidence in institutions which can be used by institutions as a catalyst for continued reform. Critical Assumptions Assumption 1: GOB counterparts will remain engaged and supporting of JFA objectives throughout the life of the Activity. Programmatic impact and successful data gathering requires counterpart cooperation and buy-in. JFA performance targets are contingent upon continued participation of GOB counterparts. Assumption 2: Statistical and numerical data will be made available to the Activity in a timely manner and will be complete and error-free. Statistical and numerical data is generated by GOB counterparts internally. Data validity cannot be guaranteed and substituting data generation would be both costly and against JFA’s capacity building objectives. In addition, data is collected and reported according to set timetables that may or may not correspond to JFA Activity quarterly and annual work flow. JFA will mitigate these risks by working with National Legal Aid Services Organization (NLASO) to strengthen internal data recording and reporting mechanisms. Assumption 3: Grantee-generated data will be consistent in terms of definitions, data collection methodologies, and data validity. JFA will take mitigation measures to ensure grantee-generated data is collected and submitted in a uniform, consistent manner. All JFA grantees will receive an M&E orientation as part of their induction training. The orientation will include discussion of Performance Indicator Reference Sheets (PIRS) and training on the proper use of standard data collection tools. Regular monitoring by the M&E Specialist will ensure that the tools are applied consistently. Assumption 4: Perception data will not be skewed to negative or positive extremes. JFA has elected to combine sampling of legal aid service users with sampling of the general public to capture data that provides a focal point as opposed to using a broad survey that measures public opinion generally. Measuring the views of legal aid service users rather than simply collecting perceptions will mitigate risks of overly negative or overly positive perceptions that are disconnected from the reality of service responsiveness, accessibility, quality, etc. Assumption 5: Sensitization tasks will lead to understanding and acceptance. Certain tasks are geared toward building sensitivity to key concepts, practices, or behaviors. Achieving project goals will require that the topics of the sensitization tasks be first disseminated, then understood, and finally accepted. Assumption 6: External actors and/or factors will not excessively interfere with the completion of tasks. Certain indicators are contingent upon factors or decisions by actors external to the Activity such as the effective adoption of proposed laws and the effective promotion of women within the judiciary. While these indicators are important measures of activity impact, demonstrating progress is partially dependent upon institutions that are not direct activity counterparts. Result Framework: Attached as Annex-A Objectives of the Activity USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 40 USAID’s Justice for All Activity’s overarching goal is to improve access to justice for all Bangladeshi citizens. The Activity is structured around three complementary objectives designed to lead to increases in access to legal aid services and formal justice sector remedies. The information outlined in this section is drawn from the Year 1 to 6 Work Plans, as approved by USAID. IR 1: Improved delivery of legal aid in the formal justice sector. Over the life of the Activity, NCSC will assist the Ministry of Law, Justice and Parliamentary Affairs (MOLJPA) and NLASO in improving the effectiveness and use of legal aid services to ensure equal access to justice for all citizens. As a result of program tasks, counterparts at the national, district, and local level will have increased capacity, tools, and systems to deliver more effective and responsive legal aid services to citizens. Specific tasks will demonstrate progress toward IR 1 by achieving the following sub￾IRs: Sub IR1.1: Legal and operational framework for legal aid improved. The legal framework regulating the organization and administration of legal aid services is improved. Legal aid services operate uniformly and the District Legal Aid Committees fulfill their obligations under the legal framework. Sub IR1.2: Citizen knowledge about legal aid services increased. NLASO and its network of District Legal Aid Committees (DLACs), Union Legal Aid Committees (ULACs), and Upazila Legal Aid Committees (UPLACs) are more visible and target beneficiaries of legal aid services are aware of such services and how to access them. Sub IR1.3: DLAC Services effectively delivered. Legal aid services are delivered more holistically and tailored to the needs of applicants. The delivery of services meets uniform standards in compliance with the legal framework and the aspirations set in the NLASO strategic plan. IR 2: Increased self-governance of the judiciary to better serve the public. Over the life of the Activity, NCSC will assist judges within the lower judiciary to improve operational efficiency and strengthen integrity. NCSC will also assist women judges to obtain equal consideration for promotion and judicial leadership positions. As a result of program tasks, judicial leadership and lower judiciary judges will have increased understanding and acceptance of best practices and resources on self-governance, integrity, and self-examination, creating a critical mass for judicial self-governance reforms. Specific tasks will demonstrate progress toward IR 2 by achieving the following sub-IRs: Sub IR 2.1: Understanding of judges of judicial self-governance improved Judges understand the parameters and benefits of judicial self-government in the context of improved access to justice and service delivery for citizens. Judges effectively advocate for improvements in the legal and regulatory framework governing the judiciary to achieve self-governance. Sub IR 2.2: Legal framework for judicial self-governance improved. Improvements to the legal framework governing the judiciary introduced to support increased self￾governance that leads to access to justice and quality service delivery for citizens. Sub IR 2.3: Equal opportunity for women within the judiciary promoted. Women judges demonstrate increased professional capacity and the ability to advocate for equal opportunity within the judiciary. Women participation in the judiciary increases and access to leadership positions reflects equal opportunity. IR 3: Increased citizen awareness of legal rights and responsibility on selected legal 41 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV issues. Over the life of the Activity, NCSC will work with the MOLJPA, civil society, and professional associations to transfer critical legal knowledge and the tools for concrete action to target communities. As a result of program tasks, citizens will have increased awareness of their legal rights and of the mechanisms to exercise them in the formal justice sector, and the enabling environment will be created to ensure they can effectively exercise those rights. Specific tasks will demonstrate progress toward IR 3 by achieving the following sub-IRs: Sub IR 3.1: Availability of legal information increased. Legal information is developed and disseminated using multi-media mechanisms. Legal information is updated to meet the information needs of target populations. The availability of legal information is maintained and progressively expanded. Sub IR 3.2: Legal literacy of citizens increased. Target beneficiaries of legal aid services have actual understanding of their rights and remedies and how to exercise them. Also, willingness to exercise rights and remedies in the formal justice sector is increased. Sub IR 3.3: Community sensitization on exercise of legal rights increased. Pro-poor, pro-women, and pro-youth sensitivity is increased among target communities. Community leaders and members demonstrate acceptance of the exercise of legal rights and remedies in the formal justice sector. Gender JFA Activity pursues gains in gender equality by targeting women as beneficiaries and partners. The Activity also promotes an enabling environment within which women professionals have increased access to opportunities and women beneficiaries can make meaningful choices to exercise their legal rights. Sustainability mechanisms The JFA Activity has devised six sustainability mechanisms so that the achievements from its efforts do not get lost once the project comes to an end with no immediate follow-up actions. These are: a. Ensure counterpart ownership and commitment from the planning stages b. Develop local capacity to act and participate c. Facilitate knowledge transfers d. Institutionalize tools and systems e. Create a demand for accountability f. Demonstrate impact In Year 6, JFA will focus on institutionalizing gains and methodologies within MOLJPA, NLASO, Judicial Administration Training Institute (JATI), and the Supreme Court’s Registrar General’s Office and Research Unit. II. Objectives of the Evaluation The objective of this final performance evaluation is to assess the technical and programmatic validity, effectiveness, assess JFA implementers’ performance in achieving actual results against targeted results, assess sustainability aspects of the activity, and make recommendations to USAID/Bangladesh concerning future programming considerations. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 42 The audience for this evaluation is USAID/Bangladesh, USAID/Washington, leaders of USAID Forward, National Center for State Courts (NCSC), and bi-lateral and multi-lateral donors to Bangladesh. III. Evaluation Questions This Scope of Work (SOW) is for a final evaluation of JFA program implementation from October 2012 to January 2018. Considering the objective of the performance evaluation, the evaluators will review, analyze, and evaluate the JFA activity using the following prioritized questions. Evaluation Questions may be refined during the work plan development process for this particular evaluation. Effectiveness: 1. How effective has the activity been in stimulating demand among Bangladeshi citizenry for legal services through the District Legal Aid Committees? 2. Do the legal services provided meet established standards that are responsive to client needs? 3. How effective has the activity been in promoting equal opportunities for female judges? 4. Has JFA contributed toward capacity improvement of the judiciary with special focus on case management and administration? Relevance: 5. To what extent are the activity objectives relevant to the current judicial and political circumstances in Bangladesh? Management: 6. How successful was the implementing partner in managing sub-grants to achieve the activity objectives? Sustainability: 7. To what extent has JFA achieved progress toward establishing sustainability mechanisms? IV. Proposed Evaluation Methodology The Evaluation Team is encouraged to use a mixed methods approach that would include the use of qualitative and quantitative tools to tease out qualitative and quantitative data, and suggest alternative approaches during the planning stage. The evaluation methodology may include the following tools; however, the Evaluation Team is encouraged to propose new methods of data collection and analysis in the work plan. 1. Document Review The Evaluation Team shall review relevant USAID and sector-specific documents, as well as key documents from USAID’s implementing partners and outside sources. A suggested list of documents is included in Annex B. The Evaluation Team will use this literature to develop an initial response to the questions listed above, and to set forth hypothesized cause-effect relationships that can be tested through field research and interviews. The Evaluation Team will also use the information from the desk review to design tools for conducting KIIs and FGDs. 2. Key Informant Interviews The Evaluation Team will conduct interviews with key informants from USAID/Bangladesh, relevant USAID/Washington staff, JFA in Washington and Dhaka, local sub-contractors and sub-grantees, relevant 43 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV GOB representatives, civil society representatives, the media, donors, stakeholders, and other relevant beneficiaries. These KIIs will help the Evaluation Team to collect opinions from the stakeholders to analyze the project achievements and challenges. For this purpose, the Team should establish a standard data collection format. 3. Focus Group Discussions The evaluation should include focus group discussions (FGDs) with different stakeholders to secure detailed information about personal and group feelings, perceptions, and opinions on overall project performance. In addition, these FGDs will help to get in-depth understanding on project impact especially focused on the citizens who received legal aid services from DLAC, UPLAC, ULAC in the JFA working areas. Existing Sources of Information USAID/Bangladesh DG Office will provide documents for the desk review that are not available from other sources and contact information for relevant interviewees. The list of documents is presented in Annex B. Understandably, the list is not exhaustive and the Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. V. Deliverables All deliverables are internal to USAID and the Evaluation Team unless otherwise instructed by USAID. Evaluation deliverables include: Evaluation Team Planning Meeting (s) – essential in organizing the team’s efforts. During the meeting (s), the team will review and discuss the SOW in its entirety, clarify team members’ roles and responsibilities, work plan, develop data collection methods and instruments, review and clarify any logistical and administrative procedures for the assignment, and prepare for the in-brief with USAID/Bangladesh. Evaluation Design and Work Plan – Complete a detailed work plan (including task timeline, methodology outlining approach to be used to answer each Evaluation Question, and describe in detail the team responsibilities, and the data analysis plan): the draft work plan will be submitted within 10 working days after commencement of the evaluation; the final work plan will be submitted within 3 days after the international team members’ arrival in Bangladesh. Evaluation Design Matrix – A table will be prepared that lists each Evaluation Question and the corresponding information sought, information sources, data collection sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation fieldwork starts. It should also be included as an annex in the evaluation report. Data Collection Instruments – Data collection instruments will be developed and submitted to USAID/Bangladesh during the evaluation design phase prior to the commencement of the evaluation fieldwork. The completed evaluation report should also include the data collection tools, instruments and list of people interviewed as an annex in the evaluation report. In-brief Meeting – An in-brief meeting with USAID/Bangladesh will be held within 2 working days of international team members’ arrival in Bangladesh. Mid-term briefing and interim meetings – The Evaluation Team Leader (or his/her delegate) will brief the DG M&E program Contracting Officer’s Representative (COR) on progress with the evaluation on a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 44 Debriefing with USAID – A PowerPoint presentation of initial findings, conclusions and preliminary recommendations will be made to USAID/Bangladesh (DG, PRO, PHNE, FDHA other teams who has worked on preventing Domestic Violence (DV) before the international team members depart from Bangladesh. Debriefing with Partners – The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and tasks only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider partner comments and incorporate them appropriately in drafting the final evaluation report. Draft Evaluation Report – The Evaluation Team will analyze all data collected during the evaluation to prepare a draft Performance Evaluation Report and submit the report within 15 working days after the departure of international team members from Bangladesh. The draft report must be of high quality with well-constructed sentences, and no grammatical errors or typos. The report should answer ALL the Evaluation Questions and the structure of the report should make it clear how the Evaluation Questions were answered. The draft report must meet the criteria set forth under the final report section below. USAID will provide comments on the draft report within 10 working days of submission. Final Report – The Evaluation Team will submit a final Performance Evaluation Report that incorporates Mission comments and suggestions no later than 10 working days after USAID/Bangladesh provides written comments on the draft Performance Evaluation Report. The report will be submitted in English, electronically. The format of the final report is provided below. The final report must meet the following criteria to ensure the quality of the report: The evaluation report must represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not, and why. Evaluation reports shall address all Evaluation Questions included in the SOW. The evaluation report should include the SOW as an annex. All modifications to the SOW, whether in technical requirements, Evaluation Questions, Evaluation Team composition, methodology or timeline need to be agreed upon in writing by the technical officer. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. Evaluation findings will assess outcomes and impact on males and females. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. Sources of information need to be properly identified and listed in an annex. Recommendations need to be supported by a specific set of findings. Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. Format of the final performance evaluation report should strike a balance between depth and length. The report will include a cover sheet, table of contents, table of figures and tables (as appropriate), 45 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV glossary of terms (acronyms), executive summary, introduction, purpose of the evaluation, scope and methodology, findings, conclusions, lessons learned and recommendations. Where appropriate, the evaluation should utilize tables and graphs to link with data and other relevant information. The report should include, in the annex, any “Statement of Differences” by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. The report will be submitted in English, electronically. in both Word and PDF formats. The report will be disseminated within USAID. USAID will submit (also electronically, in English) this report excluding any potentially procurement-sensitive information to Development Experience Clearinghouse (DEC) for dissemination among implementing partners, stakeholders, and the general public. The DEC submission must be within three months of USAID’s approval of the final report. All quantitative data, if gathered, must be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A CD with all the data could be provided to the COR. The final report will be edited/formatted by DG M&E activity implementing partner and provided to USAID/Bangladesh 15 working days after the Mission has reviewed the content and approved the final revised version of the report. VI. Team Composition/ Technical Qualifications and Experience Requirements for the Evaluation Team This Evaluation Team will be composed of five key personnel: a team leader (US/TCN Evaluation Specialist), three team members (one US/TCN Sectoral Specialist and two Local Sectoral Specialists), and one evaluation assistant. The qualifications of the team lead and team members will be consistent with the Indefinite Delivery, Indefinite Quantity (IDIQ) RDMA/Bangkok requirement. Team Leader (International): Senior-level Evaluation Specialist with knowledge in different research methodologies and at least ten (10) years of practical experience in Rule of Law, Legal aid service related program or program evaluation, with Bangladesh or Asia experience preferred; LLB or JD qualifications from a recognized university; and graduate qualifications in a related field. Experience conducting USAID evaluations in the referred sector below is preferred. Specific technical experience in the field of justice sector reform and access to justice in developing countries, whether through primary research, policy analysis, program design and management, program evaluation, or sector assessments. Proven experience as Team Leader on multi-person teams conducting assessment or evaluation assignments. Demonstrated ability to work in a politically sensitive or conflict environment; and ability and willingness to travel to multiple areas within Bangladesh. Team leader will provide leadership for the Team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from Team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. Team Member (International): This position requires a Mid-level (or Senior-Level) Evaluation Specialist with knowledge in different research methodologies and at least six (6) years of practical experience in research or program evaluation in the justice sector, LLB or JD qualifications from a recognized university; and graduate qualifications in a related field; specific technical experience in the field of justice sector reform or access to justice issues in developing countries, whether through primary research, policy analysis, program design and management, program evaluation, or sector assessments. The team member will participate in team meetings, key informant interviews, group meetings, site visits, and draft the sections of the report relevant to his/her expertise and role in the team. S/he will also participate in presenting the report to USAID or other stakeholders and be USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 46 responsible for addressing pertinent comments provided by USAID/Bangladesh or other stakeholders. Ability to conduct interviews and discussion and write well in English is essential. Team Members (National): At least six (6) to eight (8) years of in-depth knowledge and experience of issues relating to judicial reform, and/or legal aid provision in Bangladesh is required. Experience in design, management monitoring and evaluation of justice sector reform, access to justice or similar governance programs Bangladesh is preferred. Prior experience and ability to conduct evaluations, and to write well in English is required. S/he will work with the Evaluation Team members to manage Focus Group Discussions (FGD) and other data collection methods, analyze findings and draft inputs for the evaluation report. Overall the team will need expertise in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, innovations and challenges in promotion of public sector transparency and accountability; and USAID practices and requirements in program performance measurement. VII. Illustrative Budgeted Level of Effort (LOE) Below is the illustrative budgeted LOE per position for this activity. Illustrative LOE may be modified based on the complexity of the activity and in accordance with the approved work plan. The Contractor will monitor labor costs across all task order activities to ensure that the approved budget for Labor costs is not exceeded. Illustrative Budgeted LOE Team Leader Team Member Local Sectoral Specialist (1) Local Sectoral Specialist (2) Document Review and Work Plan 7 5 3 3 Fieldwork, Data Analysis, and Presentation 24 24 20 20 Draft Evaluation Report 10 7 2 2 Final Evaluation Report 3 2 0 0 Total LOE 44 38 25 25 VIII. Conflict of Interest All Evaluation Team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID/Bangladesh will provide the conflict of interest forms. 47 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV IX. SCHEDULING AND LOGISTICS Work is to be carried out over a period beginning from January 10, 2018, with fieldwork completed in February-March and final report and close out concluding o/a April 23, 2018. Schedule Task / Deliverable Proposed Dates (2018) Review background documents and preparation work January 8 - 26 Draft work plan submitted to USAID/Bangladesh January 29 Travel to Bangladesh by expat team members February 4/5 Team Planning Meeting February 6 In-brief with USAID/Bangladesh February 7 Meet with Plan International staff February 7 Submit Final Work Plan to USAID February 9 Data collection February 8 - 26 Data analysis in-country Evaluation Team submits draft presentation for USAID/Bangladesh DG Team review; data analysis continues after submission February 25 USAID provides comments (as needed) on draft presentation; team continues fieldwork February 26 Presentation and debrief with DG Team and USAID/Bangladesh February 28 Debrief meetings with key stakeholders February 28 Expat team members depart Bangladesh March 1 Produce draft report March 5 – March 22 Submit Draft Evaluation Report to USAID/Bangladesh March 23 USAID and partners review draft and provide comments April 5 Review draft comments, edit, finalize report April 6 – April 18 Submit Final Evaluation Report to USAID/Bangladesh April 19 USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 48 Funding and Logistical Support The proposed evaluation will be funded and implemented through the DG PROGRAMS’ M&E SUPPORT ACTIVITY project. Social Impact will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, translation services, international and local travel, hotel bookings, working/office spaces, computers, printing, photocopying, arranging field visits, local travel, hotel, and appointments with stakeholders. The Evaluation Team should be able to make all logistical arrangements including the vehicle arrangements for travel within and outside Dhaka and should not expect any logistical support from the Mission. The team should also make their own arrangements regarding space for team meetings and equipment support for producing the report. X. REPORTING REQUIREMENTS The total pages, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary - concisely state the project purpose and background, key Evaluation Questions, methods, most salient findings and recommendations (2-3 pp.); Introduction – country context, including a summary of any relevant history, demography, socio￾economic status, etc. (1 pp.); The Development Problem and USAID’s Response - brief overview of the development problem and USAID’s strategic response, including design and implementation of the program and any previous USAID activities implemented in response to the problem (2-3 pp.); Purpose of the Evaluation - purpose, audience, and synopsis of task (1 pp.); Evaluation Methodology - describe evaluation methods, including strengths, constraints and gaps (1 pp.); Findings and Conclusions - describe and analyze findings for each Evaluation Question using graphs, figures and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes (12-15 pp.); Lessons Learned - provide a brief of key technical and/or administrative lessons on what has worked, what has not worked, and why for future project or relevant program designs (2-3 pp.); Recommendations – prioritized and numbered for each key question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp). Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, focus group discussions, surveys, and tables. Annexes should be succinct, pertinent and readable. Should also include, if necessary, a statement of differences regarding significant unresolved difference of opinion by funders, implementers, or members of the Evaluation Team on any of the findings or recommendations. The report format should be restricted to Microsoft products and 11-point type font should be used throughout the body of the report, with page margins one-inch top/bottom and l 49 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ANNEX B: EVALUATION TEAM MEMBER PROFILES Team Leader: Ms. Mary Noel Pepys has over 20 years of professional experience in the rule of law, specifically international legal and judicial reform. Ms. Pepys worked with USAID, U.S. Department of State, The World Bank, UNDP, and numerous U.S. contractors to provide legal and technical assistance to national governments, judiciary, bar, and law faculties. She has extensive knowledge of common law and civil law principles, international justice and rule of law programs, and experience in managing and monitoring programs pertaining to the enhancement of criminal justice systems and the reduction of judicial corruption. She also has significant experience in applying leadership traits and transferring leadership skills to ensure the sustainability of rule of law programs. International Expert: Ms. Kyra Buchko is a lawyer and conflict resolution expert with 20 years of experience in international development program design, management, and evaluation within the justice sector. Ms. Buchko is an exceptional manager of people, projects, and processes. From 1992 to 1999, Ms. Buchko served as the Program Director for the American Bar Association’s Rule of Law Assistance Programs—leading the strategic planning and project development for judicial reform, legal education, and legal association development in 10 countries. From 2011 to 2013, she served as a Research Team Leader for the Caritas Development Institute in Bangladesh, designing data collection instruments for an evaluation of conflict resolution practices. Ms. Buchko has applied her technical skills to many training and evaluation activities, including the mid-term PE of USAID/Bangladesh’s JFA Activity. National Specialist: Mr. Saqeb Mahbub is a lawyer who has served as an evaluator or legal advisor for over a dozen development projects in the past seven years. Mr. Mahbub applies his legal knowledge to promote access to the justice sector and legal aid for the most vulnerable populations. Mr. Mahbub has also specialized in access to justice and legal aid for women, recently serving as team leader for an evaluation of a UNHCR-funded project to combat violence against women. He is not only a justice expert, but also a well-experienced evaluation and research team member who is able to conduct interviews and FGDs, analyze data, and produce evaluation reports. National Specialist: Mr. Shahidul Islam is an M&E specialist with over 15 years of experience in the development sector. Mr. Islam has specialized in justice sector and legal aid reform, working for six years as the M&E Officer for two large UNDP funded projects, Access to Justice (A2J) and Justice Sector Facility (JSF). He is intimately familiar with best practices for evaluation design and data collection, as well as the key risks and lessons learned from projects in the justice reform and legal aid sectors. Mr. Islam brings over a decade of dedicated M&E research experience, having designed and implemented data collection tools for dozens of evaluations and studies, drawing on mixed-methods, including KIIs, FGDs, and a variety of surveys. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 50 ANNEX C: EVALUATION DESIGN MATRIX EFFECTIVENESS: 1. How effective has the activity been in stimulating demand among Bangladeshi citizenry for legal services though the District Legal Aid Committees? Document Review Direct Observation 1. NCSC/JFA partner activities KIIs (up to 1.5 hours) • USAID • NCSC • JFA Sub-grantees • Legal Aid Services Organizations FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid Committee members, Judges, Panel Lawyers) • Legal Services Clients and Community Members • Women Judges and Other Female Legal Professionals • Description of participation in, involvement with, or knowledge of JFA Activity • Changes in demand for legal services among male and female Bangladeshi citizens • Men’s and women’s perception of inhibiting and facilitating factors for stimulating demand for legal services • Men’s and women’s perception of the quality and relevance of programmatic interventions • Identification of successful and unsuccessful program elements and impacts on men and women • Identification of best practices and lessons learned • Suggestions for how interventions might be improved or changed in future programming in this area • Content analysis • Trend analysis • Gap analysis • Comparative analysis • Gender analysis EFFECTIVENESS: 2. Do the legal services provided meet established standards that are responsive to client needs? Document Review Direct Observation • NCSC/JFA partner activities KIIs (up to 1.5 hours) • USAID • Description of participation in, involvement with, or knowledge of JFA Activity • Description of current established standards for legal services • Changes in standards for legal services over the life of the project/since • Content analysis • Trend analysis • Gap analysis • Comparative analysis 51 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV • NCSC • JFA Sub-grantees • Legal Aid Services Organizations FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid Committee members, Judges, Panel Lawyers) • Legal Services Clients and Community Members • Women Judges and Other Female Legal Professionals respondent became aware of JFA • Men’s and women’s perception of the quality and responsiveness of legal services • Men’s and women’s perception of inhibiting and facilitating factors for meeting established legal aid services standards and responding to client needs • Identification of successful and unsuccessful program elements • Identification of best practices and lessons learned • Suggestions for how interventions might be improved or changed in future programs in this area • Gender analysis EFFECTIVENESS: 3. How effective has the activity been in promoting equal opportunities for female legal professionals? Document Review Direct Observation • NCSC/JFA partner activities KIIs (up to 1.5 hours) • USAID • NCSC • GOB • JFA Sub-grantees • External Stakeholders FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid • Description of participation in, involvement with, or knowledge of JFA • Changes in opportunities for women legal professionals • Men’s and women’s perception of the quality and relevance of programmatic interventions • Identification of successful and unsuccessful program elements focused on women legal professionals • Identification of best practices and lessons learned • Suggestions for how interventions might be improved or changed in future • Content analysis • Trend analysis • Gap analysis • Comparative analysis • Gender analysis USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 52 Committee members, Judges, Panel Lawyers) • Women judges and other female legal professionals programming in this area EFFECTIVENESS: 4. Has JFA contributed toward capacity improvement of the judiciary with special focus on case management and administration? Document Review Direct Observation • NCSC/JFA partner activities KIIs (up to 1.5 hours) • USAID • NCSC • JFA Sub-grantees • Legal Aid Services Organizations • External Stakeholders FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid Committee members, Judges, Panel Lawyers) • Women Judges and Other Female Legal Professionals • Description of participation in, involvement with, or knowledge of JFA Activity • Changes in capacity of judiciary, and among male and female judges, particularly regarding case management and administration • Men’s and women’s perception of inhibiting and facilitating factors for building capacity in judicial administration and case management • Men’s and women’s perception of the quality and relevance of programmatic interventions • Identification of successful and unsuccessful program elements • Identification of best practices and lessons learned • Suggestions for how interventions might be improved or changed in future programming in this area. • Content analysis • Trend analysis • Gap analysis • Comparative analysis • Gender analysis RELEVANCE: 5. To what extent are the activity objectives viable within the current judicial and political context in Document Review KIIs (up to 1.5 hours) • USAID • NCSC • Description of participation in, involvement with, or knowledge of JFA Activity • Men’s and women’s perception of current judicial and political climate in Bangladesh as it affects JFA activities • Content analysis • Gap analysis • Comparative analysis • Gender analysis 53 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Bangladesh? • JFA Sub-grantees • Legal Aid Services Organizations • External Stakeholders FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid Committee members, Judges, Panel Lawyers) • Women Judges and Other Female Legal Professionals • Men’s and women’s perception of the viability of future programmatic interventions, given the current judicial and political context • Identification of best practices and lessons learned for promoting viability of activity objectives • Men’s and women’s suggestions for how interventions might be improved or changed in light of current (and future) political and judicial context MANAGEMENT: 6. How successful was the implementing partner in managing sub-grants to achieve the activity objectives? Document Review KIIs (up to 1.5 hours) • USAID • NCSC • JFA Sub-grantees • Description of JFA sub-grant management policies/practice • Identification of successful and unsuccessful Activity elements • Perception of inhibiting and facilitating factors for successfully managing sub￾grants to achieve objectives • Suggestions for how interventions might be improved or changed in future sub￾grant management • Content analysis • Comparative analysis SUSTAINABILITY: 7. To what extent has JFA achieved progress toward establishing sustainability mechanisms? Document Review Direct Observation • NCSC/JFA partner activities KIIs (up to 1.5 hours) • USAID • NCSC • Perception of progress toward establishing mechanisms for sustainability • Perception of the quality and relevance of sustainability-related programmatic interventions • Identification of successful and unsuccessful actions to create environment for • Content analysis • Gap analysis • Comparative analysis • Gender analysis USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 54 • GOB • JFA Sub-grantees FGDs (up to 2 hours) • JFA Training/Activity Participants (e.g., Legal Aid Committee members, Judges, Panel Lawyers • Women Judges and Other Female Legal Professionals sustainability • Perception of inhibiting and facilitating factors for sustainability • Identification of lessons learned and good practices • Suggestions for how interventions might be improved or changed to strengthen sustainability 55 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ANNEX D: DATA COLLECTION PROTOCOLS INFORMED CONSENT AGREEMENT - KII Evaluators must review this form in detail with all informants before the interview and be sure that they understand it clearly before obtaining their signature. If the informant is illiterate or expresses discomfort signing the form but verbally consents to proceeding with the interview, the evaluator may sign the form to indicate that they received verbal consent. Purpose: Assalamualaikum/Adab. Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from an organization called Social Impact, a company that is based in the United States. Our team is in Bangladesh to conduct a study about the work of the National Center for State Courts (NCSC) delivered through a project known as Justice for All, or JFA for short. JFA provides assistance and training to District Legal Aid Committees (DLACs) and other GOB and non￾governmental organizations involved with awareness-raising and delivery of legal aid services to poor and marginalized populations. You have been asked to participate today so that we can learn more about the support you received from [JFA, or grantee/partner XX] because of your [special expertise in XXX ]. We are speaking with about 50 individuals who are justice sector experts or involved in leadership roles in JFA activities, representing the different types and areas of JFA activities. We would like your honest impressions, opinions and thoughts about various issues related to this activity’s implementation. We are independent consultants who have no affiliation with those who provided you with assistance. In addition, we are not affiliated with any other JFA sub-grantee, nor do we represent the government judiciary or legal profession in Bangladesh. Procedures: If you agree to participate, we ask you to discuss your experience and opinion of the activities and services implemented under the JFA program. The interview will take about 1 hour of your time. Although we will publish our findings in a public report, all of your answers will be kept confidential. Nothing you tell us will be attributed to any individual person. Rather the report will include only a composite of all of the answers received by all of the individuals we interview. Although we may use quotes, none of the individuals interviewed will be named in the report. Voluntary Participation: Participation in this interview is completely voluntary. You do not have to agree to be in this study. You are free to end the interview at any time or to decline to answer any question which you do not wish to answer. If you decline to participate in the interview, no one will be informed of this. It will in no way affect your access to services provided by the project or its partners. If you would like to have someone present in the interview with you, you are welcome to bring someone for support. Do you have any questions at this time? [Interviewer should answer any questions] Permission to Proceed I understand the purpose of the interview as outlined above and understand that I can withdraw from the interview at any time and for any reason. I agree to participate in the interview (Evaluator records). Yes € No € Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 56 KII PROTOCOL – USAID AND NCSC A. Demographic Info: ID: [write name in the coding sheet and enter the ID code on this interview form]: Interviewer Name: Primary Notetaker Name: Sex of respondent: Function/Job of the individual: B. KII Questions 1. Has NCSC/JFA contributed to increasing Bangladeshi citizen demand for legal services through DLACs? If yes, how so? If not, why not? Which activities do you consider particularly successful/unsuccessful? Why? (EQ1) 2. Does demand for legal services differ among men versus women? If yes, how? Has JFA accounted for these differences? (EQ1, CC Gender) 3. How has the JFA Activity promoted equal opportunities for female legal professionals? Please provide examples. (EQ3, CC Gender) 4. To what extent and how has NCSC supported capacity improvements within the Bangladeshi judiciary? Which activities do you consider particularly successful/unsuccessful? Why? (EQ4) 5. What were the main challenges to building capacity within the Bangladeshi judiciary in the areas of case management and administration? (EQ4) 6. Were capacity improvements equally observed among men and women in the judiciary, or were there differences? (EQ4, CC Gender) 7. What external factors are facilitating JFA’s achievement of the activity goals and objectives? Which factors constrain JFA’s achievement of the activity goals and objectives? (EQs 1-5) 8. How does the current judicial and political context in Bangladesh affect the viability of JFA activities under each Component? Please give examples. (EQ5) 9. What changes (if any) to existing strategies and activities do you recommend for future programming in this area? (EQs 1-5) 10. To what extent did NCSC/JFA successfully manage sub-grants with local partners to achieve activity objectives? How could NCSC/JFA improve its management in future programming? What are some best practices in this area? (EQ6) 11. How has JFA built the capacity of its local partners (with active and closed sub-grants) to sustain activities beyond the conclusion of JFA? (EQ7) 57 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 12. To what extent has JFA ensured GOB and local ownership from JFA Activity planning stages? Has JFA enhanced GOB and local capacity to support and extend legal aid services once the JFA Activity ends? If so, how?(EQ7) 13. To what extent has JFA facilitated knowledge transfers and/or supported the institutionalization of tools and systems to sustain the results of the JFA Activity? Please give examples. (EQ7) 14. What unintended outcomes from the JFA Activity have you seen thus far, if any? 15. What (if any) have been the good practices used by NCSC JFA that you think might be useful to other organizations working in the area of access to justice / legal aid services delivery? 16. Do you have any other feedback that you would like to share that was not covered by these questions? KII PROTOCOL – GOVERNMENT OF BANGLADESH (GOB) Demographic Info: ID: [write name in the coding sheet and enter the ID code on this interview form]: Interviewer Name: Primary Notetaker Name: Function/job of the Individual Organization Sex of respondent (observed): District: KII Questions 1. Please describe the role/mission of your Department and its relationship to the delivery of legal aid and/or access to justice. 2. Please describe your involvement (if any) with NCSC/JFA. Please characterize how informed you are about NCSC/JFA activities. • Probe: Have you participated in any of the activities i.e. training, meeting, workshop, or events organized by JFA? Do you interact with NCSC or its sub-grantee partners in the course of your duties? 3. What are the main factors enabling/constraining improvements in delivery of government-sponsored legal aid services for poor and marginalized populations in Bangladesh? (EQ1, EQ4) • Probe: Any difference between men and women in need of legal aid? 4. What is your perception of the work done by NCSC/JFA to increase demand for legal services provided through the DLACs? (EQ1) 5. What is your perception of the quality of legal aid services provided to Bangladeshi citizens through the DLACs, and whether such services are meeting the needs of both male and female citizens/clients? (EQ2) USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 58 • Probe: Do you think the type of work performed by NCSC is relevant to the needs of poor and marginalized community members (male/female)? 6. What are the main factors enabling/constraining the promotion of equal opportunities for female legal professionals? (EQ3) 7. How has the JFA Activity promoted equal opportunities for female legal professionals? Please provide examples. (EQ3, CC Gender) 8. What are the main factors enabling/constraining the building of capacity within the judiciary, particularly in the areas of case management and administration? (EQ4) 9. To what extent and how has NCSC supported capacity improvements within the Bangladeshi judiciary? Which activities do you consider particularly successful/unsuccessful? Why? (EQ4) 10. To what extent has JFA ensured GOB and local ownership from JFA Activity planning stages? Has JFA enhanced GOB and local capacity to support and extend legal aid services once the JFA Activity ends? If so, how?(EQ7) 11. To what extent has JFA facilitated knowledge transfers and/or supported the institutionalization of tools and systems to sustain the results of the JFA Activity? Please give examples. (EQ7) 12. What (if any) have been the good practices used by NCSC JFA that you think might be useful to other organizations working in the area of access to justice / legal aid services delivery? 13. What changes (if any) to existing strategies and activities do you recommend for future programming in this area? (EQs 1-5) 14. Do you have any other feedback that you would like to share that was not covered by these questions? KII PROTOCOL – JFA SUB-GRANTEES Demographic Info: ID: [write name in the coding sheet and enter the ID code on this interview form]: Interviewer Name: Primary Notetaker Name: Sex of respondent (observed): District: KII Questions 1. Please describe your partnership with NCSC. Please describe the types of support you received from JFA as part of this partnership. (EQs 1-4) 2. In which JFA Activity areas do you consider you have made the most progress? In which areas have you made the least progress? Please explain. (EQs 1-4) 59 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 3. To what extent has your partnership with JFA contributed to stimulating demand among Bangladeshis for legal services through the DLACs? Were there other factors that helped with this besides JFA? What factors have enabled/constrained progress in this area? (EQ1) 4. Have outcomes for male and female clients and legal aid recipients been the same or have they differed? (EQ1) 5. To what extent has assistance provided by JFA to your organization contributed to your capacity to increase awareness and knowledge among citizens about legal aid services? What factors have enabled/constrained progress in this area? (EQ2) 6. Do the services provided through your organization or others you are familiar with respond to the needs of both male and female legal aid services clients? Please provide examples. (EQ2) 7. To what extent has your partnership with NCSC helped to promote equal opportunities for women judges and other female legal professionals? What factors have enabled/constrained progress in this area? Have outcomes for male and female judges been the same or have they differed? (EQ3) 8. To what extent has JFA directly provided or supported your organization’s contribution to improvements in the judiciary, specifically in the areas of case management and administration? Please provide examples. (EQ4) 9. Please describe the training and support provided by NCSC in the area of grant management. What suggestions for improvement do you have for future USAID programs implemented through sub￾grants? (EQ6) 10. How has the evolving political and judicial context in Bangladesh affected the viability of JFA activities? Do JFA-supported activities have the same/higher/lower level of importance or impact as they did when you began working with JFA? (EQ5) 11. To what extent do you think that the results of your partnership with NCSC/JFA will be sustained after the close of JFA? (EQ7) 12. Are you receiving technical or financial support from other international organizations or NGOs? If you feel comfortable sharing, which other organizations provide funding and how have they supported you? How does the work of these other organizations complement or fill gaps in the support you receive from NCSC? (EQ7) 13. What (if any) have been the good practices used by NCSC JFA that you think might be useful to other organizations working in the area of access to justice / legal aid services delivery? 14. What changes (if any) to existing strategies and activities do you recommend for future programming in this area? (EQs 1-5) 15. Do you have any other feedback that you would like to share that was not covered by these questions? USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 60 KII PROTOCOL – LEGAL AID SERVICES ORGANIZATIONS Demographic Info: ID: [write name in the coding sheet and enter the ID code on this interview form]: Interviewer Name: Primary Notetaker Name: Function/job of the individual Sex of respondent (observed): District: KII Questions 1. Please describe your partnership with JFA and its sub-grantee XXX. Please describe the types of support you received from JFA as part of this partnership. (EQs 1, 2, 4) 2. To what extent has your partnership with JFA contributed to stimulating demand among Bangladeshis for legal services through the DLACs? Were there other factors that helped with this besides JFA? What factors have enabled/constrained progress in this area? (EQ1) 3. Have outcomes for male and female clients and legal aid recipients been the same or have they differed? (EQ1) 4. To what extent has assistance provided by JFA to your organization/committee contributed to your capacity to increase awareness and knowledge among citizens about legal aid services? What factors have enabled/constrained progress in this area? (EQ2) 5. Do the services provided through your organization or others you are familiar with respond to the needs of both male and female legal aid services clients? Please provide examples. (EQ2) 6. How have JFA activities in the areas of judicial case management and administration affected your work or your ability to carry out your responsibilities? Please provide examples. (EQ4) 7. How has the evolving political and judicial context in Bangladesh affected the viability of JFA activities? Do JFA-supported activities have the same/higher/lower level of importance or impact as they did when you began working with JFA? (EQ5) 8. To what extent has JFA taken steps to ensure that its efforts and JFA activities, specifically the delivery of legal aid services through DLACs, will continue and be sustainable after the end of the project? Please provide examples. (EQ7) 9. What (if any) have been the good practices used by NCSC JFA that you think might be useful to other organizations working in the area of access to justice / legal aid services delivery? 10. What changes (if any) to existing strategies and activities do you recommend for future programming in this area? (EQs 1-5) 61 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 11. Do you have any other feedback that you would like to share that was not covered by these questions? USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 62 Protocol for Direct Observation of JFA Activities ID: [write name in the coding sheet and enter the ID code on this interview form]: Date Primary Notetaker Name: Activity Name: Facility District: Number of Participants _____ Sex of participants ___male ___ females Sex of facilitator(s) ___male _____female 1. What were the topics/issues addressed by the activity? 2. To what extent might this activity contribute to one of the JFA Activity objectives? Which one? How? 3. To what extent did the facilitator engage both men and women participants in the topics discussed? Please describe. 4. Were any measures taken to enable female participation in activity? (childcare, hours/day of activity, transportation) If so, please describe: 5. To what extent has the organizer considered needs/opportunity for follow-up on this activity? Notes by Observer PROTOCOL FOR KIIS WITH EXTERNAL STAKEHOLDERS (INTERNATIONAL DONORS, BAR ASSOCIATIONS, UNIVERSITY/LAW FACULTIES) Demographic Info: ID: [write name in the coding sheet and enter the ID code on this interview form]: Interviewer Name: Primary Notetaker Name: Function/job of the Individual Organization Sex of respondent (observed): KII Protocol for External Stakeholders 1. Please describe the role/mission of your organization and your main activities in the areas of legal aid and/or access to justice. 63 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 2. Please describe your involvement (if any) with NCSC/JFA. Please characterize how informed you are about NCSC/JFA activities. Probe: Have you participated in any of the activities i.e. training, meeting, workshop, or events organized by JFA? Do you interact with NCSC or its sub-grantee partners in the course of your duties? 3. What do you see as the main factors enabling/constraining improvements in delivery of government￾sponsored legal aid services for poor and marginalized populations in Bangladesh? (EQ1, EQ4) 4. What is your perception of the work done by NCSC/JFA to increase demand for legal services provided through the DLACs? (EQ1) 5. What is your perception of the quality of legal aid services provided to Bangladeshi citizens through the DLACs? (EQ2) 6. Are there differences in the legal aid needs of male versus female citizens/clients? To what extent have JFA interventions responded to those differences? (EQ 2, CC Gender) 7. What do you believe are the main factors enabling/constraining the promotion of equal opportunities for female judges and other women legal professionals? (EQ 3) 8. What are the main factors enabling/constraining the building of capacity within the judiciary, particularly in the areas of case management and administration? (EQ4) 9. Have you observed any differences in the impact of capacity building on men versus women in the judiciary? Please provide examples (EQ4, CC Gender) 10. To what extent does the JFA Activity remain viable going forward, within the current political and judicial context in Bangladesh? (EQ 5) 11. To what extent has NCSC/JFA created mechanisms to ensure that its efforts and JFA activities, including the delivery of legal aid services, will be sustainable after the end of the project? Please provide examples. (EQ7) 12. Is there any feedback you would like to provide on an issue that we have not already covered? USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 64 INFORMED CONSENT AGREEMENT - FGD Evaluators must review this form in detail with all informants before the start of the FGD and be sure that they understand it clearly before obtaining their signature. If the informant is illiterate or expresses discomfort signing the form but verbally consents to proceeding with the interview, the evaluator may sign the form to indicate that they received verbal consent. Purpose: Assalamualaikum/Adab. Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from an organization called Social Impact, a company that is based in the United States. Our team is in Bangladesh to conduct a study about the work of the National Center for State Courts (NCSC) delivered through a project known as Justice for All, or JFA for short. JFA provides assistance and training to District Legal Aid Committees (DLACs) and other GOB and non￾governmental organizations involved with awareness-raising and delivery of legal aid services to poor and marginalized populations. You have been asked to participate today so that we can learn more about the support you received from [JFA, or grantee/partner XX] because of your [special expertise in XXX ]. We are speaking with about 50 individuals who are justice sector experts or involved in leadership roles in JFA activities, representing the different types and areas of JFA activities. We would like your honest impressions, opinions and thoughts about various issues related to this activity’s implementation. We are independent consultants who have no affiliation with those who provided you with assistance. In addition, we are not affiliated with any other JFA sub-grantee, nor do we represent the government judiciary or legal profession in Bangladesh. Procedures: If you agree to participate, we ask you to discuss your experience and opinion of the activities and services implemented under the JFA program. The interview will take about 1 hour of your time. Although we will publish our findings in a public report, all of your answers will be kept confidential. Nothing you tell us will be attributed to any individual person. Rather the report will include only a composite of all of the answers received by all of the individuals we interview. Although we may use quotes, none of the individuals interviewed will be named in the report. Voluntary Participation: Participation in this interview is completely voluntary. You do not have to agree to be in this study. You are free to leave the FGD at any time or to decline to answer any question which you do not wish to answer. If you decline to participate, no one will be informed of this. It will in no way affect your access to services provided by the project or its partners. Do you have any questions at this time? [Interviewer should answer any questions] Permission to Proceed I understand the purpose of the interview as outlined above and understand that I can withdraw from the interview at any time and for any reason. I agree to participate in the interview (Evaluator records). Yes € No € Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ 65 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV JFA Focus Group Discussion (FGD) Protocol - Instructions for Facilitator I. Introduction: The moderator should first introduce herself or himself, then welcome and thank all of the focus group discussion (FGD) participants for being there. The moderator should then explain the purpose of the FGD, how participants were selected, any potential benefits or risks to participating in the FGD, and how long it will take. II. Ground Rules: While the ground rules will vary depending on the FGD, they will generally include the following: Everyone is encouraged to share their ideas, and the FGD is stronger if everyone participates. There are no wrong answers, and everyone’s perspective is equally valued. The ideas shared during the FGD should not be shared outside the FGD with non-participants in order to respect participants’ privacy. Disagreements about ideas can be valuable and productive, but personal attacks will not be tolerated. After establishing these ground rules, the moderator should ask if there are any questions or concerns participants have, and these issues should be addressed and consensus reached as a group before moving on. III. Confidentiality: The moderator should clearly describe how the data collected will be used, including with whom it will be shared, and crucially, whether names or other personal or identifying information will be included with the data. The moderator must be honest about how the data will be used, but should also reassure the participants that the data will be treated sensitively and that their privacy will be respected to the greatest degree possible given the needs and purposes of the evaluation. For the purposes of the JFA Evaluation, data obtained through FGDs will be kept private and anonymous. Quantitative data results will be aggregated to demonstrate overall numbers – e.g., X% of FGD participants were women, Y% of participants had received legal aid services over the last 5 years, etc. The ET will use collected qualitative data to help identify and support evidence of program successes (or failures), and may use non-attributed – i.e., anonymous – quotes from its FGDs and KIIs as part of this process. After providing this information, and to ensure the data collected are reliable, the moderator will explain that participation in the FGD is entirely voluntary and that there are no consequences for declining to participate. After informing participants of this information, the moderator should ask each member to confirm that they consent to participate. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 66 FGD PROTOCOL: JFA TRAINING/ACTIVITY PARTICIPANTS (LEGAL AID COMMITTEE MEMBERS, JUDGES, PANEL LAWYERS) a) FGD ID: [write name in the coding sheet and enter the ID code on this interview form] Interviewer Name: Primary Notetaker Name: Stakeholder type: District: Upazila: Union: Respondent #: 1 2 3 4 5 6 7 8 9 10 Sex of respondent: EQ# Discussion Questions Answer 1 Theme: Effectiveness 1. In what ways did the training/activity organized by JFA (through its Sub-grantee XXX) contribute to increasing citizen demand for DLAC legal services? Probes: a) In what ways, if any, did the training/activity increase your capacity to address the issue of citizen demand? b) Was the training designed in a way to increase demand among both men and women? c) What new information or new skills did you learn? 67 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 1 Theme: Effectiveness 2. Why do you think the trainings were/were not effective in enabling you to contribute to the objective? Probes: a) Please provide examples of how you have or have not been able to apply lessons learned. b) Were there factors in your community, the legal/political/social environment that helped or prevented you from applying knowledge gained? 2 Theme: Effectiveness 3. In what ways did the training/activity contribute to the delivery of legal services that respond to the needs of citizens/clients? Probes: a) What are the established standards for legal services provided through DLAC? b) How are legal services meeting the needs of clients? What needs, if any, are not being met? Note: If FGD group is directly involved in providing legal services, integrate Questions 2 & 3 from above here 3 Theme: Effectiveness 4. In what ways did the training/activity contribute to expanding opportunities for female judges (and other legal professionals)? Probes: a) Were female judges/legal professionals involved in the activity? b) Did the activity incorporate specific needs of male and female participants? c) Were there differences in the results/impact of the activity on males versus females? Please provide examples. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 68 4 Theme: Effectiveness 5. In what ways did the training/activity contribute to improving the capacity of the judiciary in the areas of case management and judicial administration? Probes: a) What are the main challenges faced by the courts in case management and administration? b) What aspects of the training/activity helped address those challenges through judicial capacity building? How? Note: If FGD group is directly involved in case management and administration, integrate Questions 2 & 3 from above here 5 Theme: Relevance 6. To what extent is the type of JFA training/activity in which you participated still relevant or useful in the current political/judicial climate? Probes: a) Are there any aspects of JFA training/activity that would be difficult or unwise to continue, given national/local politics or other circumstances in your community? b) What aspects of the training/activity should be continued? Why? 7 Theme: Sustainability 7. To what extent has JFA created conditions for you to continue your work and sustain gains over the long term (after the end of the project)? Probes: a) What recommendations do you have for JFA to help support you in the future to address legal aid services delivery, judicial capacity building and/or citizen awareness-raising? 69 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV EQ# Discussion Questions Answers 1 Theme: Effectiveness 1. In what ways did the training/activity organized by JFA (through its Sub￾grantee XXX) contribute to increasing male and female citizen demand for DLAC legal services? Probes: a) What are the main obstacles to accessing legal services in your community? b) In what ways, if any, did the training/activity increase your capacity to address these obstacles? c) What new information did you learn? d) How did the training address the different needs of male versus female legal services clients? 1 Theme: Effectiveness 2. Why do you think the training/activity was/was not effective in increasing citizen demand for legal services? Probes: a) Was the activity designed to meet a gap in your awareness/ knowledge? b) Were the presenters knowledgeable about the subject matter? c) Were there factors in your community, the legal/political/social environment that helped or prevented you from applying knowledge gained? 2 Theme: Effectiveness 3. In what ways did the training/activity contribute to the delivery of legal services that respond to the needs of both male and female citizens/clients? Probes: a) How are legal services meeting the needs of male and female clients? Any observable differences between these groups? b) If you received or sought legal services as a result of JFA activity, were you satisfied with the services? Why or why not? c) What citizen legal services needs, if any, are not being met? USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 70 FGD PROTOCOL: LEGAL AID SERVICES CLIENTS AND COMMUNITY MEMBERS a) FGD ID: [write name in the coding sheet and enter the ID code on this interview form] Interviewer Name: Primary Notetaker Name: Stakeholder type: District: Upazila: Union: 5 Theme: Relevance 4. To what extent is the type of JFA training/activity in which you participated still relevant or useful in the current political/judicial climate? Probes: a) Are there any aspects of JFA training/activity that would be difficult or unwise to continue, given national/local politics or other circumstances in your community? b) What aspects of the training/activity should be continued? Why? 7 Theme: Sustainability 5. To what extent has JFA created conditions for legal services and information to be accessible to you over the long term (after the end of the project)? Probes: a) What more can be done to ensure continued and expanded access to DLAC legal aid? b) What other recommendations do you have for JFA to help support you in the future to address legal aid services delivery, judicial capacity building and/or citizen awareness-raising? 71 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Respondent #: 1 2 3 4 5 6 7 8 9 10 Sex of respondent: USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 72 FGD PROTOCOL: WOMEN JUDGES AND OTHER FEMALE LEGAL PROFESSIONALS FGD ID: [write name in the coding sheet and enter the ID code on this interview form] Interviewer Name: Primary Notetaker Name: District: Upazila: Union: Respondent #: 1 2 3 4 5 6 7 8 9 10 Sex of respondent: EQ# Discussion Questions Answers 1 Theme: Effectiveness 1. In what ways did the training/activity organized by JFA (through its Sub-grantee XXX) contribute to increasing male and female citizen demand for DLAC legal services? Probes: a) What are the main obstacles to accessing legal services in your community? b) Are these obstacles different for men than for women? c) In what ways, if any, did the training/activity increase your capacity to address these obstacles? d) What, if any, were the gaps/shortcomings of the training? e) What new information did you learn? 73 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV 1 Theme: Effectiveness 2. What have you been able to apply from the training/activity in your work? Probes: a) Was the activity designed to meet a gap in your awareness/ knowledge? b) Please provide examples of how you have or have not been able to apply lessons learned. c) Were there factors in your community, the legal/political/social environment that helped or prevented you from applying knowledge gained? 2 Theme: Effectiveness 3. In what ways did the training/activity contribute to the delivery of legal services that respond to the needs of citizens/clients? Probes: a) What are the established standards for legal services provided through DLAC? b) How are legal services meeting the needs of clients? What needs, if any, are not being met? 3 Theme: Effectiveness 4. In what ways did the training/activity contribute to expanding opportunities for women judges and other female legal professionals)? Probes: a) As a female judge (or other legal professional), have your professional opportunities improved or expanded as a result of JFA-related activity or interventions? Please provide examples. b) Did the activity incorporate different needs of male and female participants? c) Were there differences in the results/impact of the activity on males versus females? Please provide examples. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 74 4 Theme: Effectiveness 5. In what ways did the training/activity contribute to improving the capacity of the judiciary in the areas of case management and judicial administration? Probes: a) What are the main challenges faced by the courts in case management and administration? b) What aspects of the training/activity helped address those challenges through judicial capacity building? How? Note: If FGD group is directly involved in case management and administration, integrate Questions 2 & 3 from above here 5 Theme: Relevance 6. To what extent is the type of JFA training/activity in which you participated still relevant or useful in Bangladesh today? Probes: a) Are there any aspects of JFA training/activity that would be difficult or unwise to continue, given national/local politics or other circumstances in your community? b) What aspects of the training/activity should be continued? Why? 6 Theme: Sustainability 7. To what extent has JFA created conditions for you to continue your work and sustain gains over the long term (after the end of the project)? Probes: a) What other recommendations do you have for JFA to help support you in the future to address legal aid services delivery, judicial capacity building and/or citizen awareness-raising? 75 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ANNEX E: MINI-SURVEY DATA AND FINDINGS The ET conducted a mini-survey with sub-grantees to gather their perspectives on the effectiveness of JFA activities, and learnings to be considered for future programming. The survey also included questions about effective management of sub-grantees by NCSC. A total of 12 Project Coordinators/ Project Managers were interviewed with a structured questionnaire through phone call. The survey included the following questions: 1. What is the most effective action to create public demand for legal aid? Choose one. 2. What is the most potential activity that needs be continued in future? Choose one. 3. What are the new activities should be considered for future programming? 4. How was the partnership between NCSC and sub-grantees? Choose one. Findings: 1. What is the most effective action to create public demand for legal aid? Activities Number % Courtyard meeting 4 33.4 Public hearing 3 25.0 Street drama 3 25.0 Folk/pot song 1 8.3 School debate 1 8.3 Total 12 100.0 2. What is the most potential activity that needs be continued in future? Activities Number % Courtyard meeting 6 50.0 Public hearing 2 16.7 Video show 2 16.7 School debate 1 8.3 Leaflet/poster 1 8.3 Total 12 100.0 USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 76 3. What are the new activities should be considered for future programming? • Coordination and networking with the local level NGOs for awareness building; • Operating mobile legal aid advice and information desk; • Motivating senior lawyers through Bar Association to support legal aid cases; • Support the ward meeting organized by Union Parishad for awareness building; • Strengthening capacity of DLAC to monitor UPLAC/UZLAC performances; • Providing support to UZLACs to establish help desk at their office premises; • Providing training to UPLAC/UZLAC members; • Providing support to DLAC to organize clients and panel lawyers meetings; • Organizing spouse fair where they will be oriented about the situation to seek legal aid. 4. How was the partnership between NCSC and sub-grantees? Level of satisfaction Number % Excellent 9 75.0% Very good 2 16.7% Good 1 8.3% Total 12 100.0 77 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ANNEX F: DOCUMENTS REVIEWED "Cooperative Agreement No. AID-388-A-12-00004 [and all subsequent modifications]” between USAID/Bangladesh and National Center for State Courts (NCSC) 2425 Wilson Boulevard, Suite 350 Arlington, VA 22201-3320 USA. September 20, 2012 – June 29, 2017. National Center for State Courts. Justice for All Program Performance Management Report Quarter 2: January 1, 2013 - March 31, 2013. Report. 2013. National Center for State Courts. Justice for All Program Performance Management Report Quarter 3: April 1, 2013 - June 30, 2013. Report. 2013. National Center for State Courts. Justice for All Program Performance Management Report Quarter 4: July 1, 2013 - September 30, 2013. Report. 2013. National Center for State Courts. Justice for All Program Performance Management Report Quarter 6: January 1, 2014 - March 31, 2014. Report. 2014. National Center for State Courts. Justice for All Program Performance Management Report Quarter 7: April 1, 2014 - June 30, 2014. Report. 2014. National Center for State Courts. Justice for All Program Performance Management Report Quarter 8: July 1, 2014 - September 30, 2014. Report. 2014. National Center for State Courts. Justice for All Program Performance Management Report Quarter 10: January 1, 2015 - March 31, 2015. Report. 2015. National Center for State Courts. Justice for All Program Performance Management Report Quarter 11: April 1, 2015 - June 30, 2015. Report. 2015. National Center for State Courts. Justice for All Program Performance Management Report Quarter 12: July 1, 2015 - September 30, 2015. Report. 2015. National Center for State Courts. Justice for All Program Performance Management Report Quarter 13: October 1, 2015 - December 31, 2015. Report. 2015. National Center for State Courts. Justice for All Program Performance Management Report Quarter 14: January 1, 2016 - March 31, 2016. Report. 2016. National Center for State Courts. Justice for All Program Performance Management Report Quarter 15: April 1, 2016 - June 30, 2016. Report. 2016. National Center for State Courts. Justice for All Program Performance Management Report Quarter 16: July 1, 2016 - September 30, 2016. Report. 2016. National Center for State Courts. Justice for All Program Performance Management Report Quarter 17: October 1, 2016 - December 31, 2016. Report. 2016. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 78 National Center for State Courts. Justice for All Program Performance Management Report Quarter 18: January 1, 2017 - March 31, 2017. Report. 2017. National Center for State Courts. Justice for All Program Performance Management Report Quarter 19: April 1, 2017 - June 30, 2017. Report. 2017. National Center for State Courts. Justice for All Program Performance Management Report Quarter 20: July 1, 2017 - September 30, 2017. Report. 2017. National Center for State Courts. Justice for All Program Workplan: October 1, 2015 - September 30, 2016. Report. 2015. National Center for State Courts. Justice for All Program Workplan: October 1, 2016 - September 30, 2017. Report. 2016. National Center for State Courts. Justice for All Program Workplan: October 1, 2017 - September 30, 2018. Report. 2017. National Center for State Courts. Justice for All Program – Coop. Agr. 388-A-12-00004 – Monitoring and Evaluation Plan Year 1 Reporting: October 1, 2012 - September 30, 2013. Report. 2013. National Center for State Courts. Justice for All Program – Coop. Agr. 388-A-12-00004 – Monitoring and Evaluation Plan Year 2 Reporting: October 1, 2013 - September 30, 2014. Report. 2014. National Center for State Courts. Justice for All Program – Coop. Agr. 388-A-12-00004 – Monitoring and Evaluation Plan Year 3 Reporting: October 1, 2014 - September 30, 2015. Report. 2015. National Center for State Courts. Justice for All Program – Coop. Agr. 388-A-12-00004 – Monitoring and Evaluation Plan Year 4 Reporting: October 1, 2015 - September 30, 2016. Report. 2016. National Center for State Courts. Justice for All Program – Coop. Agr. 388-A-12-00004 – Monitoring and Evaluation Plan Year 5 Reporting: October 1, 2016 - September 30, 2017. Report. 2017. National Center for State Courts. Justice for All Program Monitoring and Evaluation Plan: October 1, 2012 - September 30, 2017. Report. 2013. National Center for State Courts. Justice for All Program Performance Management Report Quarter 1: October 1, 2012 - December 31, 2012. Report. 2012. National Center for State Courts. Justice for All Program Performance Management Report Quarter 5: October 1, 2013 - December 31, 2013. Report. 2013. National Center for State Courts. Justice for All Program Performance Management Report Quarter 9: October 1, 2014 - December 31, 2014. Report. 2014. National Center for State Courts. Justice for All Program Workplan: October 1, 2012 - September 30, 2013. Report. 2012. National Center for State Courts. Justice for All Program Workplan: October 1, 2013 - September 30, 2014. Report. 2013. 79 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV National Center for State Courts. Justice for All Program Workplan: October 1, 2014 - September 30, 2015. Report. 2014. Social Impact, Inc. Mid-Term Performance Evaluation of the Justice for All Program. Report. 2015. The Nielsen Company (Bangladesh) Ltd. Baseline Survey on Public Perceptions of Government Legal Aid Services. Report. 2014. The Nielsen Company (Bangladesh) Ltd. Baseline Survey on Public Perceptions of Government Legal Aid Services. Report. 2015. The Nielsen Company (Bangladesh) Ltd. Baseline Survey on Public Perceptions of Government Legal Aid Services. Report. 2016. The Nielsen Company (Bangladesh) Ltd. Baseline Survey on Public Perceptions of Government Legal Aid Services. Report. 2017. United States Agency for International Development. Office of Democracy and Governance. Scope of Work for the Justice for All (JFA) External Final Performance Evaluation. By USAID/Bangladesh. 2018. United States Agency for International Development. Request for Applications (RFA) Number RFA-388-12- 000002; RFA Title: Justice for All. By USAID/Bangladesh. 2012. United States Agency for International Development. USAID Response to BESA Evaluation Report Recommendations. By Technical Team of USAID/Bangladesh. 2016. USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 80 ANNEX G: LIST OF RESPONDENTS Position Associated Org. Target/Respondent Group Sex District/Division Key Informant Interview (KII) Governance Advisor DFID External Stakeholders M Dhaka Project Coordinator Rupantar JFA Sub-grantee F Khulna District Legal Aid Officer DLAC Legal Aid Services Organization M Khulna District Legal Aid Officer DLAC Legal Aid Services Organization M Jessore Project Coordinator Rupantar JFA Sub-grantee M Jessore Former NLASO Director NLASO Legal Aid Services Organization M Dhaka District Legal Aid Officer DLAC Legal Aid Services Organization M Dhaka Operations Director, Rule of Law Programme GIZ External Stakeholders F Dhaka Project Coordinator Light House JFA Sub-grantee M Pabna Project Coordinator Democracy Watch JFA Sub-grantee M Mymensingh Project Coordinator Light House JFA Sub-grantee M Bogra Project Coordinator AID - Comilla JFA Sub-grantee M Dinajpur Head-Urban Programme Plan International External Stakeholders F Dhaka Chief of Party Winrock International External Stakeholders F Dhaka Chief of Party MSI External Stakeholders M Dhaka President Bar Association External Stakeholders M Tangail President Bar Association External Stakeholders M Mymensingh District Legal Aid Officer DLAC Legal Aid Services Organization F Cox's Bazar Project Coordinator BNWLA JFA Sub-grantee M Cox's Bazar District and Sessions Judge Judiciary External Stakeholders M Dhaka Former Director NLASO External Stakeholders M Dhaka Hon. Executive Director BLAST External Stakeholders F Dhaka District Legal Aid Officer NLASO Legal Aid Services Organization M Kurigram Project Coordinator AID - Comilla JFA Sub-grantee M Rangpur Chairman Supreme Court LAC Legal Aid Services Organization M Dhaka 81 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV District Judge and DLAC Chair Magura District Court Legal Aid Services Organization M Magura District Legal Aid Officer DLAC Legal Aid Services Organization F Magura Chairman CLAC Legal Aid Services Organization M Cox's Bazar District Legal Aid Officer DLAC Legal Aid Services Organization M Narayangonj Joint Metro Session Judge District Court, Khulna GOB M Khulna Project Coordinator WJCC JFA Sub-grantee M Khulna District Judge District Court, Khulna GOB F Khulna Director (Admin) JATI GOB M Dhaka Judge Supreme Court/BWJA GOB F Dhaka Judge District Court, Dhaka GOB M Dhaka Judge District Court, Dhaka GOB M Dhaka District Judge District Court, Dhaka GOB M Dhaka Project Coordinator MLAA JFA Sub-grantee M Dhaka Project Coordinator SJA JFA Sub-grantee M Dhaka Project Coordinator BNWLA JFA Sub-grantee M Dhaka Additional District Judge District Court, Rajshahi GOB M Rajshahi Assistant Judge Rajshahi District Court, Rajshahi GOB M Rajshahi DLAO District Court, Rajshahi Legal Aid Service Provider M Rajshahi Sr. Judicial Magistrate Magistrate Court Rajshahi GOB M Rajshahi Acting Project Coordinator BLAST Legal Aid Service Provider F Rajshahi Dean Faculty of Law, Rajshahi University External Stakeholder M Rajshahi Executive Director Centre for Peace and Justice External Stakeholder M Dhaka Executive Director BLAST External Stakeholder F Dhaka Resident Legal Adviser US Embassy, Dhaka & USAID USG M Dhaka Resident Legal Officer US Embassy, Dhaka & USAID USG M Dhaka Deputy Director BLAST External Stakeholder/Former Sub￾grantee F Dhaka USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 82 Co-ordinator BLAST External Stakeholder/Former Sub￾grantee F Dhaka Senior Deputy Director BLAST External Stakeholder/Former Sub￾grantee F Dhaka President BNWLA JFA Sub-grantee F Dhaka Joint-Secretary MOLJPA GOB F Dhaka Director NLASO GOB M Dhaka Special Officer, High Court Division Supreme Court GOB M Dhaka Registrar, High Court Division Supreme Court GOB M Dhaka Project Officer, Responsive Justice IP UNDP External Stakeholder F Dhaka Research and Reference Officer Supreme Court GOB M Dhaka Chief of Party NCSC USG F Dhaka Deputy Chief of Party NCSC USG F Dhaka Program Manager NCSC USG F Dhaka Office of D&G USAID USG F Dhaka Office of D&G USAID USG F Dhaka Focus Group Discussion (FGD) Chairman Aicegati UPLAC JFA Activity Participants M Khulna Chairman Rudaghara UPLAC JFA Activity Participants M Khulna Chairman Damodar UPLAC JFA Activity Participants M Khulna Member Surkhali UPLAC JFA Activity Participants M Khulna Secretary Rangpur UPLAC JFA Activity Participants M Khulna Secretary Sagladah UPLAC JFA Activity Participants M Khulna Secretary Gagirhat UPLAC JFA Activity Participants M Khulna Secretary Damodar UPLAC JFA Activity Participants M Khulna Secretary Dumuria UPLAC JFA Activity Participants M Khulna Member Ultra Gilatala UPLAC JFA Activity Participants M Khulna Chairman Azagora UPLAC JFA Activity Participants M Khulna Chriman Srifaltala UPLAC JFA Activity Participants M Khulna Member Ittara UPLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna 83 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants F Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants M Khulna Legal Aid Panel Lawyer Khulna DLAC JFA Activity Participants F Khulna Chairman Narendrapur ULAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Secretary Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants F Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Member Narendrapur UPLAC JFA Activity Participants M Jessore Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Women Lawyer Dhaka District Court Female Legal Professional F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants M Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants M Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants M Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants M Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants M Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 84 Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Legal Aid Panel Lawyer Dhaka DLAC JFA Activity Participants F Dhaka Teacher Ukhia International School JFA Activity Participants M Cox's Bazar Teacher Ukhia International School JFA Activity Participants F Cox's Bazar Teacher Ukhia International School JFA Activity Participants F Cox's Bazar Teacher Ukhia International School JFA Activity Participants F Cox's Bazar Teacher Ukhia International School JFA Activity Participants M Cox's Bazar Teacher Ukhia International School JFA Activity Participants M Cox's Bazar Teacher Ukhia International School JFA Activity Participants M Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Housewife Cox's Bazar DLAO Legal Services Clients F Cox's Bazar Farmer Cox's Bazar DLAO Legal Services Clients M Cox's Bazar Farmer Cox's Bazar DLAO Legal Services Clients M Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife BNWLA Community Member F Cox's Bazar Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Housewife AID-Comilla Community Member F Rangpur Chairman UPLAC, Damdarpur UP JFA Activity Participants M Rangpur 85 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants F Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants F Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants F Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants F Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants M Rangpur Member UPLAC, Damdarpur UP JFA Activity Participants F Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants F Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants M Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants F Rangpur Panel Lawyer Rangpur DLAC JFA Activity Participants F Rangpur Woman Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Women Lawyer Rangpur District Court Female Legal Professional F Rangpur Legal Aid Clients DLAO Office LSC & CM M Khulna Legal Aid Clients DLAO Office LSC & CM M Khulna Legal Aid Clients DLAO Office LSC & CM F Khulna USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 86 Legal Aid Clients DLAO Office LSC & CM F Khulna Legal Aid Clients DLAO Office LSC & CM F Khulna Legal Aid Clients DLAO Office LSC & CM F Khulna Legal Aid Clients DLAO Office LSC & CM F Khulna Legal Aid Clients DLAO Office LSC & CM F Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant M Khulna UZLAC UZLAC Office JFA Activity Participant F Khulna UZLAC UZLAC Office JFA Activity Participant F Khulna UZLAC UZLAC Office JFA Activity Participant F Khulna UZLAC UZLAC Office JFA Activity Participant F Khulna UZLAC UZLAC Office JFA Activity Participant F Khulna Community Member Courtyard LSC & CM M Rajshahi Community Member Courtyard LSC & CM M Rajshahi Community Member Courtyard LSC & CM M Rajshahi Community Member Courtyard LSC & CM M Rajshahi Community Member Courtyard LSC & CM M Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi Community Member Courtyard LSC & CM F Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant M Rajshahi UPLAC UPLAC Office JFA Activity Participant F Rajshahi UPLAC UPLAC Office JFA Activity Participant F Rajshahi UPLAC UPLAC Office JFA Activity Participant F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi 87 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Lawyers Bangladesh Bar Association Female Legal Prof. F Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant M Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant M Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant M Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant M Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant M Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant F Rajshahi Panel Attorneys Associated with DLAO Rajshahi JFA Activity Participant F Rajshahi Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Judges BWJA Female Legal Prof. F Dhaka Law Students Jagannath University Female Legal Prof F Dhaka Law Students Jagannath University Female Legal Prof F Dhaka Law Students Jagannath University Female Legal Prof F Dhaka Law Students Eastern University Female Legal Prof F Dhaka Law Students Eastern University Female Legal Prof F Dhaka Law Students Eastern University Female Legal Prof F Dhaka Law Students Daffodil International University Female Legal Prof F Dhaka Law Students Daffodil International University Female Legal Prof F Dhaka 88 | USAID/Bangladesh Workers’ Empowerment Program Mid-term Evaluation USAID.GOV ANNEX H: MAP OF EVALUATION SITES 89 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV ANNEX I: DISCLOSURE OF ANY CONFLICTS OF INTEREST USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 90 91 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV USAID.GOV USAID/Bangladesh Justice For All Final Performance Evaluation | 92 93 | USAID/Bangladesh Justice For All Final Performance Evaluation USAID.GOV United States Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20004