UZBEKISTAN LEGAL REFORM PROGRAM EVALUATION REPORT Prepared under Contract No.: GS-10F-0033M / 7200AA18M00016, Tasking N078 DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY II UZBEKISTAN LEGAL REFORM PROGRAM EVALUATION REPORT JULY 2023 Prepared under Contract No.: GS-10F-0033M /7200AA18M00016, Tasking N078 Submitted to: Matthew Baker, USAID COR Submitted by: Zumrat Salmorbekova Catherine Caligan Seara Grundhoefer Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301- 634-5444; E-mail: Parry-Matthew@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | LER II N078 UZBEKISTAN LRP EVALUATION REPORT USAID.GOV TABLE OF CONTENTS TABLE OF TABLES II TABLE OF FIGURES II ACRONYMS III EXECUTIVE SUMMARY IV EVALUATION BACKGROUND IV EVALUATION OBJECTIVES, USE, EVALUATION QUESTIONS AND METHODOLOGY IV FINDINGS IV RECOMMENDATIONS VII 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS 1 EVALUATION OBJECTIVES 1 EVALUATION QUESTIONS 1 2. LRP PROGRAM BACKGROUND 1 3. METHODS AND LIMITATIONS 2 4. FINDINGS AND CONCLUSIONS 7 EVALUATION QUESTION 1 7 EVALUATION QUESTION 1A 11 EVALUATION QUESTION 2 12 EVALUATION QUESTION 3 14 EVALUATION QUESTION 4 19 EVALUATION QUESTION 4A 21 5. RECOMMENDATIONS 23 RECOMMENDATIONS FOR USAID/UZBEKISTAN 23 RECOMMENDATIONS TO THE IMPLEMENTING PARTNER OF LRP, TETRA TECH DPK 24 ANNEX A. EVALUATION STATEMENT OF WORK 26 ANNEX B. EVALUATION DESIGN MATRIX 34 ANNEX C. PROGRAM BACKGROUND 38 ANNEX D. EVALUATION FINDINGS ON LRP APPROACHES 40 ANNEX E. EVALUATION METHODS AND LIMITATIONS 45 ANNEX F. DATA COLLECTION INSTRUMENTS 50 ANNEX G. SOURCES OF INFORMATION 83 ii | LER II N078 UZBEKISTAN LRP EVALUATION REPORT USAID.GOV TABLE OF TABLES Table 1: KIIs by Respondent Group.......................................................................................................................4 Table 2: FGD Respondents by Gender (left) and Physical Location (right) .................................................5 Table 3: Progress in Gender Legislation ...............................................................................................................7 Table 4: Deliverable Schedule................................................................................................................................32 Table 5: Deliverable Table......................................................................................................................................33 Table 6: Evaluation Design Matrix ........................................................................................................................34 Table 7: Activities and Approaches by LRP Objective.....................................................................................38 Table 8: LRP Strengths and Examples..................................................................................................................41 Table 9: LRP Limitations and Examples...............................................................................................................43 Table 10: Free Legal Aid Pre-FGD Information ................................................................................................74 Table 11: Free Legal Aid FGD Demographics and Service Received...........................................................74 Table 12: Free Legal Aid FGD Questions and Probes.....................................................................................77 Table 13: Training Participant Pre-FGD Information.......................................................................................79 Table 14: Training Participant FGD Demographics and Training Received................................................79 Table 15: Training Participant FGD Questions and Probes...........................................................................81 Table 16: Sources of Information .........................................................................................................................83 TABLE OF FIGURES Figure 1: Map of FGD Locations...........................................................................................................................47 Figure 2: Team Lead Conflict of Interest Statement.............................Error! Bookmark not defined. Figure 3: Project Manager Conflict of Interest Statement...................Error! Bookmark not defined. Figure 4: Project Assistant Conflict of Interest Statement..................Error! Bookmark not defined. iii | LER II N078 UZBEKISTAN LRP EVALUATION REPORT USAID.GOV ACRONYMS BIT Bilateral Investment Treaty CSO Civil Society Organization ET Evaluation Team EQ Evaluation Question FCR Findings, Conclusions, and Recommendations FGD Focus Group Discussion GBV Gender-Based Violence GoU Government of Uzbekistan ICNL International Center for Not-for-Profit Law IP Implementing Partner KII Key Informant Interview MoI Ministry of Investment MoJ Ministry of Justice NBT Nazar Business & Technology NGO Non-Governmental Organization LRP Legal Reform Program SOW Scope of Work ToT Training of Trainers TT DPK Tetra Tech DPK US United States USAID United States Agency for International Development USG United States Government WG Working Group GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | iv EXECUTIVE SUMMARY EVALUATION BACKGROUND As part of the DRG Learning, Evaluation, and Research II Activity (DRG-LER II), the United States Agency for International Development (USAID)/Uzbekistan commissioned NORC consortium partner Social Impact (SI) to conduct a performance evaluation of the Legal Reform Program (LRP) in September 2022, implemented by Tetra Tech DPK (TT DPK). LRP is a $19,000,000, six-year initiative that started in October 2018 to support the Government of Uzbekistan’s (GoU) efforts in implementing an ambitious and comprehensive set of economic and social reforms aimed at boosting growth, modernizing its legal framework, promoting transparency and accountability, and improving services for citizens. LRP is a mechanism through which USAID responds to requests for technical assistance from the GoU. Specifically, it provides support to the GoU’s Pillar II, Rule of Law, of the Five Pillar Action Strategy for 2017-2021 in addition to the GoU’s goals surrounding civil society development. EVALUATION OBJECTIVES, USE, EVALUATION QUESTIONS AND METHODOLOGY The objectives of the evaluation are to: 1) assess the performance of the activities and the effectiveness of the mechanism and approaches to improve legal reform and rule of law development; and 2) identify actionable recommendations that can inform any course corrections for LRP’s remaining years or activity designs for USAID/Uzbekistan. The evaluation findings will be utilized by USAID/Uzbekistan to foster learning and guide their current and future programmatic strategies. This evaluation answers four evaluation questions (EQs) that focus on performance, effectiveness, and recommendations. These EQs are listed under the Findings section below. Data collection methods aimed to generate the highest-quality and most credible evidence, taking into consideration time, budget, and other limiting factors. The evaluation team (ET) primarily employed qualitative methods, including desk review, consultations and co-creation meetings with primary users, key informant interviews (KIIs) and focus group discussions (FGDs) involving recipients of free legal aid and training participants. In total, the ET collected data from 35 key informants and 45 beneficiaries. For more detailed information, please refer to pages 4-5. FINDINGS The discussion below presents key findings for each EQ. Please refer to pages 6-22 of this report for detailed findings and conclusions for each EQ. EQ1: TO WHAT EXTENT HAS LRP SUPPORTED THE DEVELOPMENT OF RULE OF LAW AND THE PROVISION OF FREE LEGAL AID BY ACTIVITY-SUPPORTED NGOS/CSOS? SUB-EQ1.1: WHAT ARE STRENGTHS AND WEAKNESSES OF LRP’S ACTIVITIES/APPROACHES IN ACHIEVING ITS OBJECTIVES ON THE DEVELOPMENT OF RULE OF LAW AND THE PROVISION OF FREE LEGAL AID BY ACTIVITY-SUPPORTED NGOS/CSOS? ● Government officials, NGO partners, and experts acknowledge LRP's significant contributions to legislative development in various areas. While progress has been made, the implementation of these laws still faces challenges, except for gender-related legislation, which has shown promise due to strong interest from the President. However, the full impact of these legal GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | v reforms is yet to be observed, as it is influenced by the government's interests and political will in advancing reforms. ● National partners and LRP grantees recognized the positive contribution of LRP in legislation protecting individuals with special needs, leading to changes in relevant laws. However, challenges in implementation were highlighted, including the absence of sign language interpreters in court hearings, limiting the participation of individuals with hearing issues. ● Frequent administrative changes within government institutions pose challenges for stable and effective project implementation, requiring LRP to invest additional efforts in building relationships and conducting capacity-building activities for new leadership and staff, as reported by IP staff and experts. ● LRP has made significant strides in offering free legal support to vulnerable groups through its 18 grantees and NGO Madad, thereby expanding the reach of free legal aid and positively impacting societal attitudes. As of June 30, 2023, LRP’s efforts have assisted 423,667 individuals. Moreover, lawyer training sessions have contributed to improving the quality of legal services provided. ● LRP's grantees have significantly improved the quality of life for beneficiaries of free legal aid, with recipients expressing satisfaction with the services and praising the expertise of the lawyers. ● The IP and LRP grantees have observed noticeable changes in the behavior of legal aid beneficiaries, including increased confidence, seeking of legal assistance, and asserting their rights, resulting in more detailed appeals and improved legal literacy. ● LRP's effective awareness-raising activities and outreach efforts have contributed to increased legal literacy among disadvantaged populations, fostering a stronger legal culture and confidence in the legal system, as noted by key informants and LRP grantees. ● LRP's collaboration with other NGOs, such as Madat, has significantly contributed to the development and improvement of legal aid service provision standards, with LRP providing valuable technical assistance and professional development support to regional offices. ● LRP grantees encounter obstacles in delivering free legal aid, primarily due to the pending status of the governing law, which hampers necessary modifications and the establishment of standardized procedures. Challenges also arise from licensing restrictions, court fees, and the need for regulation of pre-trial activities. Limited access to remote regions and resource constraints further impedes the provision of legal assistance to individuals in remote areas. ● LRP's strengths include engaging qualified experts, creating discussion platforms, conducting capacity-building activities, providing free legal aid to vulnerable groups, geographic reach, research and training materials, and collaboration with local entities. The program's flexibility and adaptability were also recognized as valuable attributes. ● The evaluation identified gaps and limitations in LRP's approaches, including challenges in capacity-building activities, mixed views on study tours, limitations in tracking legislative outcomes and beneficiary data, and limited coordination with other United States government (USG)-funded programs in Uzbekistan. EQ2: HOW EFFECTIVE HAS LRP SUPPORT BEEN TO GOU REQUESTS FOR AMENDMENTS TO THE CURRENT LEGISLATION AND/OR ADOPTION OF A NEW LEGISLATION? ● LRP's support in enhancing legislative activity is recognized by national partners, who appreciate its alignment with the country's development strategy, flexibility, engagement of qualified experts, and capacity to generate new ideas and foster cooperation with government stakeholders. ● A Ministry of Justice (MoJ) representative praised LRP's technical assistance in conducting legal assessments, formulating regulatory statutes, and addressing anti-corruption legislation, while the Business Ombudsperson office highlighted the project's contribution in identifying and addressing issues faced by entrepreneurs through analysis and suggestions for legislation. ● The success of LRP's efforts to address government requests for legislative amendments and new legislation is influenced by the political will and priorities of the GoU, with some laws GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | vi progressing smoothly while others face limited support and low priority, as highlighted by key informants. ● LRP's support to government requests for legislative amendments and adoption has been broad rather than deep. On the one hand, this approach allowed flexibility to adapt to the GoU’s changing legislative priorities and requests across various areas such as economy, environment, diplomacy, public administration, technology, gender, and human rights. However, on the other hand, many of LRP’s legislative efforts yielded limited outcomes, as the suggested legislative amendments by the project were not adopted due to the frequent shifts in the GoU’s legislative priorities and interests. ● Respondents noted that the traditional operating methods and hierarchy within the government pose a challenge for LRP in meeting shifting legislative priorities, as national partners lack skills in planning and anticipating changes. This was exemplified by the diversion of attention towards constitutional reform and the de-prioritization of diplomatic law, hindering effective planning and implementation of recommendations. EQ3: TO WHAT EXTENT DID THE GRANTS PROGRAM CONTRIBUTE TO THE DEVELOPMENT OF THE LEGAL-ENABLING ENVIRONMENT AND STRENGTHEN ORGANIZATIONAL CAPACITY FOR CSOS/NGOS AS PLANNED? ● LRP's support in developing a Presidential Decree on regulatory impact assessment is seen as a significant step towards establishing mechanisms for civil society to monitor state bodies and enhance accountability. The decree's adoption will mandate consultation with NGOs, raising legislation standards and promoting public oversight of the government. ● KII data shows positive changes in state institutions' collaboration with NGOs, increasing the reputation of LRP grantees. However, challenges persist, with some government agencies reluctant to engage in open dialogues, particularly regarding regulations on foreign funding. ● The evaluation found that LRP grantees have made significant contributions to the development of legislative amendments related to civil society monitoring, social partnership, and the rights of citizens with special needs. ● Discussions with the implementing partner, experts, and USAID highlighted persistent challenges, including limited NGO participation in law amendment processes, lack of understanding of policy and law-making processes, restricted government oversight capability, a small number of NGOs available for engagement, and the impact of historical and current government restrictions on NGOs. ● The lack of transparency and limited interest of the GoU in receiving support from LRP for advancing the legal framework on NGOs is recognized as the main obstacle affecting the drafting process of the Code on NGOs, which was never finalized and adopted. A contrasting trend exists between the President's support for NGOs and the caution and tightening control exhibited by the GoU. The access of NGOs to foreign funding has been restricted by Decree #328 (of June 2022), creating uncertainties regarding the outcome of appeals made by LRP grantees. ● All LRP grantees unanimously expressed appreciation for the project's training sessions, which greatly enhanced their organizational and professional capacity in various areas. The use of interactive methods, practical case studies, and specialized training tailored to their needs were particularly valued. The financial sustainability session was highlighted as particularly impactful. ● Participation in LRP has fostered increased networking and collaboration among grantees and other NGOs, leading to enhanced knowledge exchange and synergy. This has facilitated effective addressing of legal issues faced by vulnerable groups through referrals and consultations, resulting in improved provision of legal aid. ● The sustainability of legal aid and service provision by LRP grantees is a concern, and efforts are being made to establish partnerships with government organizations and promote self￾sustainability. Some grantees have continued to provide pro-bono services through collaborations, but retaining lawyers without a sustainable income is challenging. The IP is GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | vii working to train NGOs for self-sustainability, although the lack of an established pro-bono culture and bureaucratic obstacles present obstacles to progress. EQ4: GOING FORWARD, GIVEN THE GOU’S AGENDA OF ONGOING LEGAL REFORMS IN UZBEKISTAN, IN WHICH AREAS SHOULD USAID-FUNDED ACTIVITIES BE TARGETED TO ENABLE THE HOST COUNTRY TO BUILD A MORE SUSTAINABLE AND DEMOCRATIC RULE OF LAW SYSTEM/CULTURE? SUB-EQ 4.1: WHAT TYPE OF TECHNICAL PROGRAMMING, INNOVATIONS FOR DESIGN AND FUNDING (I.E., PILOTING, CO-CREATION, STAKEHOLDER INPUT, ETC.) OR PARTNERING ARRANGEMENTS SHOULD USAID PURSUE? ● Key stakeholders identified seven areas that require targeted USAID-funded activities to support the host country in building a more sustainable and democratic rule of law. These areas highlight the importance of capacity building, local engagement, collaboration, access to legal aid, support for vulnerable groups, NGO capacity strengthening, and legislative implementation to promote a sustainable and democratic rule of law. ● The programming suggestions for promoting democratic rule of law and fostering a culture of the rule of law aim to leverage technology, facilitate knowledge exchange, and improve access to information, ultimately strengthening the rule of law and enhancing the effectiveness of legal processes. RECOMMENDATIONS Based on the findings and conclusions described in the previous section, the ET proposes the following recommendations to assist primary users in fostering learning and informing current and future USAID/Uzbekistan programmatic strategies. Please see Section 5: Recommendations below for more elaboration on recommendations. SUMMARY OF RECOMMENDATIONS FOR USAID/UZBEKISTAN 1. Limit the scope of legislative development support by focusing on specific areas, such as improving legal instruments and enforcing legislation in administrative and regulatory law. 2. Enhance legal literacy and qualifications of government employees, including decision-makers and leaders. 3. Support local capacity building for grassroots implementation of the rule of law. 4. Strengthen communication and collaboration between NGOs, regional/local government bodies, and other stakeholders. 5. Continue supporting capacity building of NGOs, particularly in human rights, women's rights, children's rights, and trade unions. 6. Prioritize the people-to-people approach and fund grants to empower local NGOs and establish partnerships. 7. Mandate the collection of outcome-level data to assess the effectiveness of USAID-funded activities. 8. Enhance coordination and information sharing among USAID programs and other USG-funded initiatives. SUMMARY OF RECOMMENDATIONS TO THE IMPLEMENTING PARTNER (TETRA TECH DPK) OF LRP: 1. Strengthen the long-term institutional capabilities of grantees through training, mentorship, and resources to ensure their sustainability. 2. Support grantees in diversifying funding sources and enhancing financial sustainability beyond donor funding. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | viii 3. Develop a clear action plan for the post-LRP period in collaboration with stakeholders to ensure continuity and sustained progress. 4. Maintain a critical evaluation of government legislative requests to focus on areas aligned with LRP objectives. 5. Address language barriers and enhance accessibility to capacity-building activities for participants from different regions of Uzbekistan. 6. Refine outcome-level indicators and track the adoption of suggested regulations for effective monitoring and evaluation. 7. Collect and analyze data on barriers faced by beneficiaries of free legal aid to inform necessary bureaucratic and legislative changes. 8. Facilitate dialogues and discussions between government officials and civil society representatives to foster a shared understanding of the rule of law and promote collaboration. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 1 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this study is to help USAID better understand the performance of its investments in legal reform and rule of law development in Uzbekistan through LRP. EVALUATION OBJECTIVES The objectives of the evaluation are to: a) assess the performance of the activities and the effectiveness of the mechanism and approaches to improve legal reform and rule of law development; and b) identify actionable recommendations that can inform any course corrections for LRP’s remaining years or activity designs for USAID/Uzbekistan. The evaluation’s primary user is USAID/Uzbekistan. The evaluation findings provide USAID/Uzbekistan and its implementing partner (IP) learning on delivering rule of law programming to inform current and future programmatic strategies. EVALUATION QUESTIONS The set of evaluation questions (EQs) this evaluation addressed evolved from the original set of EQs stated in the statement of work (SOW) (see Annex A. Evaluation Statement of Work). After conducting the desk review and co-creation meetings with USAID and the IP, the ET proposed edits which were approved by USAID in late December 2022. The final EQs that the evaluation addressed include the following four questions: EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Sub-EQ1.1: What are strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity￾supported NGOs/CSOs? EQ2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or adoption of a new legislation? EQ3: To what extent did the grants program contribute to the development of the legal-enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? EQ4: Going forward, given the GoU’s agenda of ongoing legal reforms in Uzbekistan, in which areas should USAID-funded activities be targeted to enable the host country to build a more sustainable and democratic rule of law system/culture? Sub-EQ4.1: What type of technical programming, innovations for design and funding (i.e., piloting, co-creation, stakeholder input, etc.) or partnering arrangements should USAID pursue? 2. LRP PROGRAM BACKGROUND Since the election of President Mirziyoyev in 2016, Uzbekistan has launched a series of wide ranging economic and political reforms. The Action Strategy for 2017-2021 highlighted five pillars of the GoU for development: 1) Governance, 2) Rule of law, 3) Economic development and liberalization, 4) GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 2 Infrastructure, health, education, and employment, and 5) Security and harmony/foreign policy. Pillar II reflects a focus on developing rule of law, enhancing the role of civil society, and improving the legal and judicial systems in Uzbekistan. Numerous reform measures under this pillar have been established and enforced through Presidential Decree, such as the 2018 Decree “Measures to Drastically Increase the Role of Civil Society Institutions in the Process of the Country’s Democratic Renewal.” Parliament has been another avenue for legal reform, passing laws on Public Control, Mediation, and Social Partnership, among other areas. In addition to laws and decrees, the GoU has made a concerted effort to engage international partners to learn legal and judicial best practices. Notable reforms to Uzbekistan’s legal framework include, but are not limited to, establishing a new judicial structure, creating an independent governmental body tasked with combating corruption, amending the law “On Citizenship of the Republic of Uzbekistan” to allow citizenship to stateless individuals, and granting citizens the ability to direct complaints and appeals to the President. These reforms demonstrate Uzbekistan’s continued commitment to developing rule of law and improving legal and judicial systems. To obtain support for the broader legal reform efforts, the GoU’s Ministry of Justice (MoJ) approached USAID/Uzbekistan in 2018 with a series of requests. As a result, LRP became a mechanism through which USAID responds to requests for technical assistance from the GoU. Specifically, it provides support to the GoU’s Pillar II, Rule of Law, of the Five Pillar Action Strategy for 2017-2021 in addition to the GoU’s goals surrounding civil society development. USAID/Uzbekistan contracted TT DPK to implement LRP, which has three overarching objectives: 1. Supporting Rule of Law Development in Uzbekistan; 2. Development of the Legal Profession; and 3. Developing a Legal-Enabling Environment for Civil Society Organizations (CSOs)/Non￾Governmental Organizations (NGOs). All three objectives, in addition to the cross-cutting goal of Promoting Gender Equality and Protection of Women’s Rights, contribute to the GoU’s broader legal reform effort. LRP also supports CSOs working on legal aid, legal training, judicial reform, and public control functions through a small grants program. A snapshot of LRP’s main activities, theory of change, and planned achievements for project years 1-3 can be found in Annex C. Program Background. 3. METHODS AND LIMITATIONS The evaluation design consisted of the following three phases, across which the ET employed qualitative methods to deliver evidence-based answers to the EQs: 1. Evaluation design and consultations which included introductory calls, co-creation meetings with USAID/Uzbekistan and TT DPK, a desk review of project and other strategic documents, and developing the evaluation design report, including data collection tools and preparation for the data collection phase. 2. Data collection which focused on scheduling and conducting in-person and remote Key Informant Interviews (KIIs), and Focus Group Discussions (FGDs), and conducting rolling data analysis. 3. Analysis and reporting which concentrated on coding, synthesizing, analyzing and triangulating across data sources, drafting and submitting evaluation findings report, and delivering a virtual findings presentation. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 3 The evaluation design was guided by an overall analytical framework presented in the Evaluation Design Matrix (see Annex B. Evaluation Design Matrix). In the sections below, we elaborate on each study component, organized by three phases: evaluation design and consultations, data collection, and analysis and reporting. PHASE 1: EVALUATION DESIGN AND CONSULTATIONS DESK REVIEW The desk review process included reviewing documents collected from USAID/Uzbekistan and the IP. The evaluation received over 124 files made up of documents such as USAID Uzbekistan strategy, LRP SOW, annual and quarterly reports, workplans, activity, monitoring, evaluation and learning plans, indicator progress reports, lists of LRP sub-grantees and LRP-assisted legislation, and LRP-supported assessment reports and analytical and educational products, which were attached to the narrative reports. The ET captured a subset of information into a desk review matrix designed to organize the information collected per draft EQ as outlined in the SOW (see Annex A. Evaluation Statement of Work). Preliminary desk review results were used in the preparation for the co-creation meetings, drafting of the data collection tools, and developing a preliminary set of sampled participants. Relevant project documents were reviewed and summarized in advance of KIIs to provide context and nuance to discussions with individual stakeholders. In Phase II the ET used LRP documents to triangulate the findings from the primary data collected through KIIs and FGDs. CONSULTATIONS During Phase I, the ET conducted consultations with the USAID/Uzbekistan DGO team. The team used these framing conversations to refine the EQs and learn more about the current context and evaluation utilization needs. CO-CREATION MEETINGS The second step in the inception phase involved two co-creation meetings. The first co-creation meeting with the IP and part of the USAID/Uzbekistan DGO team was conducted on November 29, 2022. The meeting introduced the ET to the IP, informed the evaluation objectives and principles, and ensured the engagement of the intended users from the start of the evaluation process. The second co-creation meeting was conducted with the DG office in USAID/Uzbekistan on December 21, 2022 which confirmed the final set of EQs; provided an understanding of USAID’s learning priorities; explored USAID’s expected outcomes, assumptions, and perceptions of LRP’s success; and defined key stakeholders to engage and the best methods to engage key stakeholders. The data sources and data collection methods are presented in the Evaluation Design Matrix (see Annex B. Evaluation Design Matrix). SAMPLING APPROACH The ET used criteria to identify the focal objectives for the evaluation, which narrowed down the focus of the evaluation to LRP’s Objective 1 and Objective 3. The ET also narrowed down the focus of the evaluation to the following priority legislation topics, based on the level of perceived success and learning priorities: entrepreneurship (large success), intellectual property rights (moderate success), and legal framework for NGOs (limited success). Gender legislation was a final priority focus highlighted by the Mission, as it resulted in some changes in the practical application of two gender equality laws and became a mandatory regulation in laws and procedures, on local and national levels. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 4 Therefore, within LRP’s Objective 1, legislation activities were focused on gender, intellectual property rights, entrepreneurship, and NGO legal frameworks. Finally, in consultation with USAID/Uzbekistan and the IP, combined with the ET’s review of key project documents, the ET focused on the Tashkent, Fergana, and Samarkand regions for primary data collection. More details on the sampling approach, LRP legislation focus selection and data collection site selection is included in Annex E. Evaluation Methods and Limitations. PHASE II: DATA COLLECTION KEY INFORMANT INTERVIEWS The KII candidates were elicited through the desk review and co-creation meetings and included the following key stakeholder groups: donor, IP key staff, relevant government and civil society partners, grantees, internal and external experts, lawmakers, and representatives of other rule of law sector programs implemented in Uzbekistan. The team used purposeful sampling and then chose key informants capable of answering questions randomly within the purposive sampling.1 LRP grantees were selected based on their geographic location and focus of their grant activity, particularly those who provide both free legal aid to vulnerable groups and contribute to improving the legal-enabling environment for NGOs and interaction between state and civil society. The KIIs were conducted both remotely and in person. While the ET conducted remote interviews with the donor, IP staff, sectoral experts, and other English-speaking informants, the local data collection firm Nazar Business & Technology (NBT) Consulting conducted in-person interviews with relevant government officials, lawmakers, civil society partners, and grantees and experts in Tashkent, Fergana, and Samarkand. The ET conducted 35 KIIs, as listed in Table 1. The ET developed five semi￾structured, open-ended interview protocols, each tailored to a KII agency (see Annex F. Data Collection Instruments) which provided flexibility to capture each respondent’s insights yet maintain comparability. Table 1: KIIs by Respondent Group Respondent Group Number USAID/Uzbekistan 3 Implementing Partner 7 GoU Stakeholders 9 NGO Partners 5 Technical Experts 3 LRP Grantees 6 Other 2 TOTAL 35 FOCUS GROUP DISCUSSIONS NBT conducted FGDs with two types of stakeholders, a) recipients of free legal aid and b) participants of capacity-building activities. FGDs with female and male free legal aid recipients in Tashkent, Fergana, 1 This method helped to select competent key informants who possess knowledge, observations, and experiences about the evaluated project. It also helped to capture various views, opinions, observations, and lessons learned, as well as to receive rich information, insights, and an in-depth understanding of the issues at focus. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 5 and Samarkand were critical to assess the project’s effectiveness of service provision to vulnerable groups,2 as well as to identify specific components that are perceived as most and least beneficial. FGDs with participants of LRP-supported various training courses provided the evaluation with participants’ perspectives about the successes and shortcomings of the capacity-building activities and usefulness in improving the development, adoption, and implementation of legislation and provision of legal consultations and legal aid services to citizens. The following categories of capacity-building beneficiaries were targeted: ● Government employees, specialists, and legislators who took part in training events to implement gender legislation and prevent gender-based violence. ● Lawyers/attorneys and judges who attended training courses on intellectual property rights. ● Grantees’ legal clinic experts, lawyers, and advisors who participated in training courses to improve their legal aid service provision and consultation skills, including Madad’s consultation centers legal advisors. FGDs were segregated by topics of training and type of respondents to facilitate open dialogue and generate the highest-quality data possible. Simple random sampling was applied in the selection of FGD respondents. The team held six FGDs with free legal aid recipients and two with capacity￾building event attendees. Table 2: FGD Respondents by Gender (left) and Physical Location (right) Gender Legal Aid Recipients (37) Training Participants (8) Location Legal Aid Recipients (37) Training Participants FGDs (8) Male 14 2 Tashkent 8 4 Female 23 6 Fergana 12 4 Samarkand 17 0 PHASE III: ANALYSIS AND REPORTING The ET analyzed data through an iterative process during and after the data collection, employing various data analysis techniques to support the development of findings, conclusions, and recommendations. They included rolling analysis; content and comparative analysis; gap analysis; and data triangulation. More information on how these were conducted is described in Annex E. Evaluation Methods and Limitations. LIMITATIONS The evaluation’s most important limitations and the ET’s mitigation measures are presented below: a. Data availability and data quality. There are two main challenges associated with remote and in-person data collection, namely a) availability/willingness of state officials and NGOs to participate, and b) complications due to time differences, connectivity, and reception issues. To mitigate both of these challenges, the ET followed careful data collection preparation including formal government protocols and provided clear communication to respondents 2 Vulnerable groups included labor migrants, victims of domestic and gender-based violence, low-income families, people with disabilities, those who are unemployed, and populations living in cross-border and rural areas. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 6 on the evaluation well in advance of fieldwork starting. The ET built in flexibility to the schedule and utilized multiple methods to ensure the participation of key informants. b. Selection bias. A core aspect of an evaluation’s validity is the extent to which it can generalize findings beyond the data collection sample. The ET did not collect in-person primary data in all regions of LRP activity implementation due to budgetary constraints. As specific operational contexts may vary across regions, the locations selected for primary data collection do not necessarily reflect the entirety of LRP’s programming experience. Second, there was a fair degree of non-response. These challenges were mitigated through a combination of purposive sampling to ensure representation from key cross-sections of respondent populations, flexibility in scheduling interviews, and the identification of replacements in the sample selection. c. Recall bias. The evaluation covers a study period of over four years with many activities that address three objectives, implemented within a complex operating environment. As with any study, the further back the ET asked respondents to recall, the more likely and significant was the possibility of errors in memory. These challenges were mitigated by triangulating findings against secondary sources as well as from other interviews. Furthermore, when interviewers were asking questions from a relatively long time ago, they used historical anchors to help respondents remember. d. Response bias. Every evaluation must contend with certain respondents not being fully forthright in their answers or misreporting their responses to questions, resulting in findings being based on the subjective perceptions of key informants. The challenge of respondents misrepresenting or omitting information is particularly acute in studies that deal with sensitive topics and/or with sensitive populations. Finally, there is often a perverse incentive on the part of beneficiaries to present a positive reflection of programming in the hope of continued support (whether through current or future programs). These challenges were mitigated through the use of local researchers familiar with the context and content; detailed consent forms that explained the role of the independent evaluator and the anonymity of respondents vis-à-vis evaluation findings; and triangulation of data (as noted above). e. Social desirability bias. This refers to the tendency of respondents to overreport more desirable attributes. The team mitigated this type of bias by emphasizing to participants the desire to both hear about what has worked well in addition to what the challenges/weaknesses have been. The ET was clear in the informed consent that there are no direct benefits to respondents from participating in the evaluation and that there will be no retaliation against them for their responses. f. Halo effect: As mentioned above, there is a tendency among respondents to under-report socially undesirable answers and adjust their answers to approximate what they perceive as the social norm – this is called halo bias/effect. In the case of this evaluation, halo effect may have manifested if respondents were not willing to reveal information that might reflect negatively on the implementer and/or government. However, this limitation did not have significant impact as the ET obtained information from a variety of stakeholders and ensured anonymity in reporting their responses. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 7 4. FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: TO WHAT EXTENT HAS LRP SUPPORTED THE DEVELOPMENT OF THE RULE OF LAW AND THE PROVISION OF FREE LEGAL AID BY ACTIVITY-SUPPORTED NGOS/CSOS? RULE OF LAW DEVELOPMENT Interviews with IP staff, USAID representatives, government officials, NGO partners and experts indicate that the Legal Reform Program (LRP) has made noteworthy contributions and efforts in the development of legislation. However, the immediate impact of these efforts is yet to be fully observed, as progress in the rule of law sector is strongly influenced by the interests and political will of the GoU to further advance legal reforms, as highlighted by the respondents. LRP played an important role in enhancing and improving various aspects of legislation and legislative development. As shared by key informants, the legislation improvements included environmental, human rights, and gender-related laws, civil and administrative regulations, alternative dispute resolution, trade regulations, entrepreneurship, and other business-related legal frameworks. Key informants, including NGO partners, government representatives, and experts, also highlighted the critical role of the LRP in advancing legislative development. This encompasses progress in various legislative areas such as economic rights, the entrepreneurship code, alternative dispute resolution, amendments to the law on mediation and arbitration, the passage of the gender equality law, and amendments to criminal and administrative codes to address gender-based violence (GBV). As a result, these efforts have contributed and produced a number of Presidential decrees and resolutions, brought in millions of dollars to Uzbekistan, and are expected to bring entrepreneurship development and growth of the economy.3 Two GoU representatives also highlighted the improved global ranking of the country. As one national partner said: “Support of projects funded by USAID helps Uzbekistan improve its rating on the world stage.” The World Justice Project Rule of Law Index indeed shows a 0.5 increase in Uzbekistan’s overall index score.4 Table 3: Progress in Gender Legislation Progress in Gender Highlighted by KII/FGD respondents ● 13/35 respondents emphasized the project’s achievements in gender-related legislative activities, particularly the development of the first-ever law on “Guarantees of Equal Rights and Opportunities for Women and Men,” which helps NGOs to operate shelters for survivors of GBV and illegal human trafficking in a more legally friendly environment, facing fewer obstacles from the GoU. ● Assistance to GoU Working Group (WG) in drafting the new law on Protection of Women from Harassment and Violence, which was subsequently accepted by the Legislative Chamber and approved by the Senate of Uzbekistan in September 2019 ● Achievements in the gender equality legal framework set the stage for further progress, culminating in the recent passage of significant amendments to the Criminal and Administrative Codes in April 2023. Passage of these amendments represents a significant step forward in closing loopholes that could enable more violence against women and children, thereby addressing these critical issues. 3 The State Committee of The Republic of Uzbekistan of Statistics: Production of Gross Domestic Product of the Republic of Uzbekistan for 2020; https://stat.uz/images/uploads/dos/vvp_dek_2020_en.pdf. Uzbekistan's Economy and Society, Statista Report, 2021: https://www.statista.com/study/48500/uzbekistan. 4 WJP Rule of Law Index: Uzbekistan, 2022: https://worldjusticeproject.org/rule-of-law-index/global/2022/Uzbekistan/. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 8 National partners and LRP grantees shared the successes of LRP’s contribution to legislation protecting individuals with special needs. Respondents highlighted LRP’s contribution to the discussions on protection of rights for individuals and the resulting change in relevant legislation or adoption of relevant guidance. For example, a LRP grantee noted the success in incorporating a recommended measure to train deaf interpreters under the guidance of the Ministry of Labor and Poverty, which was subsequently included in the relevant law, mandating Pedagogical Institutions to provide training for qualified deaf interpreters, further protecting the rights of people with special needs. However, some NGO respondents indicated that individuals with hearing issues are still unable to participate in court hearings due to the absence of sign language interpreters in the judiciary. This problem may stem from a lack of implementation of the legislation. LEGISLATIVE DEVELOPMENT CHALLENGES A major concern is the lack of implementation of these laws and regulations. This was raised by NGO partners, LRP grantees, experts, the IP and USAID. The responses on the reasons for poor implementation reveals a common trend of heavy reliance on the mere passage of laws and insufficient attention and understanding of how to change behavior to comply with the law. Frequent administrative changes within government institutions pose challenges for stable and effective project implementation. As shared by IP staff and experts, the shifting leadership within national partners necessitates LRP to invest additional efforts in building relationships and conducting capacity-building activities for new leadership and staff. According to the respondents, since the onset of administrative reform in 2022, only 21 out of 60 state institutions and bodies remain unchanged. Due to administrative reforms, LRP had to organize new training activities on gender￾related topics for newly appointed leadership and staff, to ensure the continuity of implementing tasks. Furthermore, as reported by IP staff, some initially planned activities, such as the creation of an online platform on GBV, encountered challenges in finding a state institution willing to take the lead in its development after the reforms, resulting in delays and halting its progress. The shortage of state personnel, high staff turnover within state institutions and varying levels of professional capacity of government employees creates challenges for rule of law development. According to interviewed government representatives, NGO grantees and IP staff, these factors present barriers to the implementation of medium- and long-term strategic plans, and LRP grantees noted that the frequent rotation of state personnel at the district level diminishes the effectiveness of LRP-supported training activities. Upon the arrival of new employees, a comprehensive orientation process is required, along with building new relationships. The IP staff highlighted the challenge of conducting post-training assessment of participants from state agencies to gauge the use of training materials and knowledge, as participants either got reassigned to other institutions or were laid off. Additionally, fluctuating capacity within professional groups, such as the Chamber of Advocates, poses challenges in defining a baseline and meeting their training needs. LEGAL AID PROVISION The data suggests that LRP has made significant achievements in providing legal support to vulnerable groups through its NGO-grantees since 2020. One notable achievement, shared by a national partner, was the role of LRP in advancing the approval of free legal aid provision in social and administrative cases, expanding the scope beyond criminal cases. NGO partners described LRP as “Perfect laws that are not enforced have little benefit for the people.” (KII Respondent) GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 9 a pioneering initiative that offers comprehensive legal aid and services to people living with HIV/AIDS, tuberculosis, and other socially stigmatized diseases. As reported by the IP, the number of grantees has grown from 10 to 18, and as of June 30, 2023, a total of 423,667 individuals from low-income or marginalized communities received legal aid assistance from LRP grantees and NGO Madad. 5 Among them, 221,820 were females and 201,847 were males. The increased access to legal services and coverage of regions can be attributed to the growing demand from vulnerable groups, which is a result of awareness-raising activities and the diverse approaches6 employed by the grantees in providing legal services, shared by the IP. Lawyers during FGDs mentioned LRP conducted training sessions, indicating their importance in improving the quality of legal services. LRP’s grantees made a significant contribution to improving quality of life for beneficiaries of free legal aid. KIIs with IP, USAID, LRP grantees and some GoU suggest that this finding is further supported by group discussions with female and male recipients of legal aid services in Tashkent, Fergana, and Samarkand. The recipients expressed overall satisfaction with the process of receiving services from LRP grantees and praised the qualification of the lawyers.7 According to respondents, LRP's efforts have played an important role in ensuring that people receive their rightful property, alimony, and pensions, as well as assisting them in securing stay in government housing. For instance, a male beneficiary in Samarkand shared how the NGO lawyer helped him restore his rights to an increased pension, resulting in doubled pension payment. Female beneficiaries in Tashkent highlighted the importance of receiving free aid from qualified professionals, particularly for women who often lack the financial means to protect their rights. Perceptions of noticeable change in the behavior of legal aid beneficiaries and increased confidence among them were shared by IP and LRP grantees. Key informants noted that these behavioral changes extended to various areas, such as seeking legal assistance and asserting their rights. For instance, an NGO partner reported observing a significant increase in the number of citizens submitting appeals related to social issues. LRP grantees also emphasized the enhanced legal literacy of the targeted population, resulting in more detailed and focused appeals. These findings were supported by the results of FGDs. Divorced female beneficiaries shared their experiences, stating that with the assistance of free legal aid, they gained a better understanding of the necessary steps of obtaining alimony owed to them and successfully secured the support they were entitled to. LRP’s effective awareness-raising activities and outreach efforts are recognized as contributors to the increased legal literacy among disadvantaged and marginalized populations, fostering an enhanced legal culture and confidence in the legal system. This perspective was expressed by 13 out of 35 KII respondents. LRP grantees highlighted impressive outreach numbers resulting from their awareness-raising activities. For example, in Fergana, an LRP grantee conducted awareness-raising outreach in over 200 communities over a two-year period, with nearly 10,000 people participating in the events and meetings, including more than 7,500 women and nearly 2,500 men. Similarly, a LRP grantee in Samarkand reached out to over 9,766 disadvantaged groups of population residing in rural areas through awareness-raising events focused at promoting awareness about women’s rights and changes in legislation. They also provided legal aid to 7,024 5 This data was provided by the IP via email communication on July 25, 2023. 6 Including use of call centers, on-site services, and field portable services. 7 It should be noted that not all of the free legal aid recipients with whom the ET conducted group discussions were aware of LRP. However, they exhibited a high level of awareness regarding the names of NGOs, and specifically the names of lawyers who provided free legal assistance. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 10 individuals. LRP grantees in Tashkent reported that as a result of increased awareness, beneficiaries are able to submit more detailed and focused appeals to courts and feel more confident in navigating through the legal mechanisms. In Samarkand, female beneficiaries highlighted the importance of legal aid in supporting women who are victims of domestic violence, as discussing such issues openly in the community is culturally challenging. However, female respondents shared instances where they perceived the verdicts of judges to be influenced by gender biases, along with judges being late to court and frequent postponement of divorce court hearings. LRP’s collaboration with other NGOs, including Madat, in providing legal assistance has played a significant role in developing and enhancing the standards of legal aid service provision to the population. An NGO partner highlighted LRP’s valuable technical assistance and professional development support provided to their legal service providers in regional offices. The respondent specifically noted the positive collaboration experience with LRP in the development of legal aid provision methods and the identification of required courses to ensure the provision of qualified legal support to citizens. LRP grantees also emphasized the benefits of attending seminars and training sessions alongside other NGOs that provide legal consultations to the population. These opportunities have allowed them to build relationships, expand their knowledge and perspectives, and exchange experiences with their peers in the field. CHALLENGES IN THE PROVISION OF FREE LEGAL AID LRP grantees highlighted the significant challenge posed by the pending status of the law governing the provision of Free Legal Aid services, as it awaits official ratification. They expressed that the submission of proposals has not been successful in gaining acceptance, which hinders the creation of a unified system for managing, coordinating, and regulating legal aid. This, in turn, affects the determination of the categories of persons eligible to receive free legal aid. It also prevents the establishment of a standardized and systematic approach to the provision of legal services, including types, forms, and methods of delivery of legal aid by lawyers, NGOs, legal clinics, and other entities. Challenges in court representation have been identified. The implementing partner and LRP grantees noted that the signed memorandum of understanding with state ministries and khokimiyats (local governing bodies/administrations) has facilitated lawyers to represent beneficiaries at state agencies. Yet discussions with lawyers and some NGOs highlight obstacles in providing support in courts. These challenges include licensing limitations, which prevent lawyers from representing their clients in court hearings without necessary credentials. Additionally, state court fees pose a financial burden on those seeking legal representation. Another issue that requires attention is the need for regulation of pre-trial activities overseen by judges, whose role is to ensure due process is followed at this stage. However, interviews with the IP reveal that a few NGO grantees’ lawyers have provided court representation to 80 individuals out of 90 (as of June 2023), and their court fees were covered under the LRP grant. While LRP grantees acknowledge that legal services are being provided in nearly all regions of the country, some have highlighted limitations in accessing remote regions. The availability of call centers may vary due to differences in telecommunication infrastructure, and limited resources may restrict the ability to provide on-site visits or organize field trips to these regions. Free legal aid recipients in Samarkand also confirmed these findings and highlighted the challenges faced by individuals living in mountainous areas. The lack of means to travel to city centers poses a significant barrier for them in accessing the necessary legal assistance. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 11 EVALUATION QUESTION 1A: WHAT ARE THE STRENGTHS AND WEAKNESSES OF LRP’S ACTIVITIES/APPROACHES IN ACHIEVING ITS OBJECTIVES ON THE DEVELOPMENT OF THE RULE OF LAW AND THE PROVISION OF FREE LEGAL AID BY ACTIVITY-SUPPORTED NGOS/CSOS? STRENGTHS AND WEAKNESSES OF LRP APPROACHES The perspectives of key informants highlight several strengths of LRP approaches. Engaging experts, both national and international, are considered highly effective in improving the quality of legislation and developing relevant products. Creating discussion platforms fosters cooperation and joint actions between different stakeholders, while capacity-building activities enhance the knowledge and skills of government employees and legal professionals. LRP's geographic reach, research and training materials, engagement with local entities, provision of free legal aid to vulnerable groups, and flexibility and adaptability of the program are also recognized as significant strengths. While the evaluation did not find significant weaknesses in LRP approaches, it identified gaps and limitations that require attention. Capacity-building activities need to prioritize addressing language barriers and improving regional accessibility for better participants engagement, knowledge transfer, and inclusivity. Opinions differ on the effectiveness of LRP's study tours, with concerns raised by USAID and IP regarding limited follow-up and tangible results, while national partners emphasize the value of connections and exploration in legislative reform. Although specific actions or proposals were not cited by national partners, NGO respondents highlighted the importance of information exchange and advocacy efforts resulting from study tours. A summary of successful approaches and notable examples, as well as gaps and limitations highlighted by respondents can be found in Annex D. Evaluation Findings on LRP Approaches, Tables 8 and 9. CONCLUSIONS FOR EQ1 As the findings above indicate, the Legal Reform Program has supported the development of the rule of law and the provision of free legal aid by the project-supported NGOs to a large extent. Notable achievements have been observed in the development of legislation related to gender equality and amendments to criminal and administrative codes criminalizing GBV, economic rights, and entrepreneurship, bilateral investment treaty, and alternative dispute resolution. The inclusion of recommended measures, such as the provision of sign language interpreters in cases involving parties with hearing issues, and the training of deaf interpreters in relevant laws, further contributes to the protection and promotion of the rights of people with special needs in Uzbekistan. However, the implementation of these laws remains a challenge, highlighting the need for greater focus on changing behavior and enforcing the legislation. Despite its broad influence on the legal system, LRP faces challenges in aligning with the dynamic legislative priorities of the government, leading to difficulties in planning, prioritizing, follow-up, and effectively tracking the implementation of proposed changes across various areas of legislation. In terms of legal aid provision, LRP's NGO grantees have made significant strides in providing legal support to vulnerable groups. The expansion of free legal aid provision beyond criminal cases and into social and administrative cases has improved access to justice for marginalized communities. The positive contributions of LRP can be seen in the improvement of the quality of life for beneficiaries, including successful resolution of property disputes, support for victims of domestic violence to reclaim their belongings, and increased pension payments. LRP's efforts in raising legal awareness and improving legal literacy among disadvantaged populations have contributed to behavioral changes and GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 12 increasing confidence in the legal system. Collaboration with other NGOs has also enhanced the standards of legal aid service provision. However, challenges remain, including the lack of a law that hinders legal aid services, limited access to remote regions, the absence of sign language interpreters in courtrooms, and the need to address licensing and fee-related issues for legal practitioners. The evaluation identified a limitation in tracking legislative outcomes for the project's legislative and free legal aid activities, attributed to the project's outcome level indicator not effectively measuring desired changes. Overall, to ensure a comprehensive assessment of the project's impact on legal frameworks, it is necessary to revisit and refine the outcome level indicators to capture the desired changes in the legal landscape and document progress. This will provide a comprehensive record of the project's impact on the legislative environment and facilitate the evaluation of its effectiveness in driving legal reforms. The challenges faced by individuals in accessing their entitled benefits from the government, such as pensions and utility bill problems, are evident. However, the absence of systematic data collection on the specific barriers encountered by vulnerable groups, disaggregated by age, gender, disabilities, and other vulnerabilities, indicates a lack of monitoring requirements. Collecting and analyzing such data could uncover the root causes of these challenges and provide valuable insight to inform necessary bureaucratic and legislative changes. This, in turn, would help to improve access to entitlements in a more efficient and equitable manner, reducing the reliance on legal professionals for assistance. EVALUATION QUESTION 2: HOW EFFECTIVE HAS LRP SUPPORT BEEN TO THE GOU REQUESTS FOR AMENDMENTS TO THE CURRENT LEGISLATION AND/OR ADOPTION OF NEW LEGISLATION? LRP’s support is described as helpful in advancing the quality of legislative activity. The national partners interviewed shared several examples of state bodies’ participation in LRP events, seeking advice and requesting assistance from LRP in developing laws, regulations, roadmaps for implementing international obligations, and providing other support. Based on the responses of national partners, experts, and IP, the evaluation identified a few factors that support LRP’s valuable contributions to enhancing legislation. LRP is flexible to meet various government demands. The MoJ representative emphasized LRP’s responsiveness and adaptability to changing legislation development or amendment priority requests as a major factor of its success. LRP engages competent national and international legal experts to support the government in developing well-crafted laws. National partners underlined LRP’s capacity to organize public discussions on draft laws in line with the government’s requirements on the quality of legislative activity. For example, a respondent from the MoJ pointed to the expertise of LRP experts who deliberate via regional and national meetings and symposiums, and provide thorough analysis and recommendations. These experts also adapt the best international practices into national legislation, thereby offering expertise in drafting laws. The MoJ representative highlighted the substantial range of technical assistance received from LRP, including legal assessments of laws, formulation of regulatory statutes, and addressing anti-corruption related legislation. The assistance has been instrumental in developing several regulatory legal acts and advancing specific normative legal measures. LRP fosters new ideas and integrates them into requests. This factor was highlighted by national partners, IP, and NGO partners. In situations where mechanisms and tools are lacking, the project takes the initiative. For instance, LRP supported the development of a methodology and the GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 13 first-ever guidelines on conducting gender expertise in legislation. Additionally, according to an NGO partner, LRP provided support in developing a Public Engagement in Rulemaking law and devised a mechanism for its implementation. LRP continuously builds relationships, fosters confidence, and establishes close cooperation with national partners. This greatly influences LRP’s effectiveness in addressing government legislative requests, as shared by IP, NGO partners, and national partners. For instance, national partners frequently mention the consistent participation of LRP experts in the development of legislation, as well as their role in organizing and facilitating public discussions and seminars. This reportedly helps identify legislative issues that require follow-up or amendment. Ultimately the success of LRP’s efforts to address government requests for amendments to existing legislation and/or the adoption of new legislation is strongly influenced by the political will and priorities of the GoU. This finding was highlighted by 17 out of 35 KIIs. For instance, key informants described the MoJ request to support the development of the Code on NGOs in early 2020, and the MoJ expressed interest in independent analysis. However, when LRP proposed their approach, the MoJ indicated that their support in developing the code was no longer required. Ultimately the draft code was never finalized and adopted. Similarly, the law regulating free legal aid services discussed in EQ1 is still pending. On the other hand, the improvement and amendment process of legislation such as the Civil Code, Law on Ombudsperson, Law on Licensing, Permission and Notification Procedures, and Entrepreneurship Code, gender equality and GBV related legislation progressed relatively smoothly and successfully. Respondents attribute this fluctuation in supporting LRP’s efforts to the political will and priorities of the government. The GoU provided successful backing and support to LRP’s efforts in advancing legislation related to attracting foreign investment, impacting entrepreneurship, contributing to economic growth, and promoting gender-related legislation. Conversely, limited support for the NGO Code and the law regulating free legal aid is linked to low priority and interest on the part of the GoU. LRP’s support to government requests for amendments and adoption of legislation has been found to be broad rather than deep. The analysis of reviewed project documents and responses from IP staff, NGO partners and USAID indicates that unlike typical rule of law programming that focuses on specific areas, LRP was designed to be flexible to adapt to the changing political context at the time of the project design. This flexibility has helped the implementer navigate the rule of law reform process and be responsive to frequently changing legislative priorities and requests from the government. Consequently, the project has had to divert its attention to multiple diverse legislative areas, encompassing economic, environmental, public administration, technology, gender, and human rights, among others. Furthermore, respondents revealed that state employees still adhere to traditional practices of submitting to superiors, displaying low initiative, being circumspect about changes, resisting modernizing their operations, and not altering their way of interaction within the government hierarchy. This reportedly poses a significant challenge for the project in meeting the constantly shifting government legislative priorities and requests across numerous legal reform areas. For example, eight respondents highlighted that national partners lack the skills to effectively plan and anticipate changes in government priorities. One specific instance is the constitutional reform initiated by the President, which diverted the attention of all national partners towards drafting amendments to the constitution. As a result, this diversion affected the planned revisions of the Code on NGOs, which had already been long stalled, and other legislative acts. Priorities can shift based on changes in Presidential and Cabinet of Ministers’ decrees, as exemplified by the case of diplomatic law. Initially, diplomatic law was considered a priority, but it has now been deprioritized due to administrative reforms. This hampers GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 14 the project’s ability to effectively plan and follow up to ensure the acceptance of suggested recommendations and their implementation beyond mere existence on paper. CONCLUSIONS FOR EQ2 Overall, the LRP's support is described as helpful in addressing government requests as it enhances legislation and advances the quality of legislative activity. There are examples of state bodies seeking advice and assistance from LRP in developing laws, regulations, and roadmaps. The evaluation identified factors that support LRP's contributions, including alignment with the country's development strategy, engagement of competent experts, fostering new ideas, and building strong relationships with national partners. However, the ultimate effectiveness of LRP's efforts is strongly dependent on the political will and priorities of the government, as seen in the varying levels of support for different legislative requests. LRP's support, although broad, has had to navigate challenges arising from the old-fashioned operating methods, lack of initiative and anticipation skills among state employees within the government hierarchy. These challenges pose difficulties in meeting shifting legislative priorities and requests across multiple legislative reform areas. The project's flexibility and adaptability have been crucial. However, the changing decrees that deprioritize certain legislation over others create obstacles to effective planning, addressing requests, and gaining acceptance of recommended legislative changes. This highlights the dynamic nature of the legislative landscape, where government priorities and focus can change in response to various factors, including administrative reforms and policy shifts. Consequently, projects like LRP must be adaptable and responsive to these changes to effectively align their efforts with the evolving priorities of the government, which is exactly what the LRP team has accomplished so far. EVALUATION QUESTION 3: TO WHAT EXTENT DID THE GRANTS PROGRAM CONTRIBUTE TO THE DEVELOPMENT OF THE LEGAL-ENABLING ENVIRONMENT AND STRENGTHEN ORGANIZATIONAL CAPACITY FOR CSOS/NGOS AS PLANNED? The evaluation adopted the World Bank’s definition of a legal enabling environment, which encompasses a set of laws, policies, regulations, and norms that impact the capacity of NGOs to engage and contribute to development processes in a sustained and effective manner.8 The legal enabling environment plays a critical role in shaping the operational context for NGOs, providing the necessary framework for their operations. It influences their ability to advocate for specific causes, access resources, establish partnerships, and effectively carry out their development activities. LRP employs a few strategies to support the development of the legal enabling environment and organizational capacity that facilitates the effective engagement and contribution of NGOs to the development processes: ● First, it involves the distribution of grants to 20 local NGOs in 10 regions9 of Uzbekistan to provide free legal aid by a qualified lawyer, support for locally led initiatives, and contribute to the development of civil society. This aims to empower NGOs and strengthen their capacity to contribute to the development process. 8 http://web.worldbank.org/archive/website01029/WEB/IMAGES/_ENGL-60.PDF. 9 The 10 regions where NGO grantees operate include Tashkent city and its wider region, Namangan, Andijan, Fergana, Jizzakh, Samarkand, Kashkadarya, Bukhara, Khorezm, and the Republic of Karakalpakstan. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 15 ● Second, LRP focuses on building organizational and legal frameworks that regulate and enhance the capabilities of NGOs. This involves working with NGOs to improve their internal systems, governance structures, and compliance with relevant laws and regulations. ● Additionally, LRP promotes discussion and dialogue between state organizations, legislators, and CSOs. By facilitating these interactions, LRP aims to foster collaboration, understanding, and the development of policies and regulations that create a more supportive environment for NGOs. CONTRIBUTION TO THE DEVELOPMENT OF A LEGAL ENABLING ENVIRONMENT FOR NGOS LRP’s support in the development of a Presidential Decree on regulatory impact assessment is expected to address the existing disparity between the President’s encouragement of civil society to monitor state bodies and the absence of mechanisms to facilitate such monitoring. As shared by the IP and NGO partners, the efforts in developing this decree are viewed as a significant step forward in establishing the necessary mechanisms for civil society to exert influence on the government. Upon the decree’s anticipated adoption in the next few months, state organizations are projected to heighten their accountability towards civil society and be mandated to seek consultation with NGOs and other civil society groups. This regulatory impact assessment is reported to have far-reaching implications, as it will influence the entire spectrum of laws that shape public oversight of the government, thus elevating the standard of legislation. A review of the KII data indicates some minor changes in the interaction between state institutions and NGOs. LRP grantees highlight the benefits of participating in round tables and seminars with state employees, which have helped foster positive relationships and cooperation. For instance, a grantee in Samarkand shared that their lawyers started being consulted and invited by counterparts from state agencies on ways to assist citizens after attending an LRP-organized round table. The grantee believes that this cooperation can increase trust in state lawyers since NGO lawyers are perceived as more qualified in the eyes of the public. Similarly, an LRP grantee from Fergana shared a story of improved relations with a local government organization after participating in an LRP event that brought together state, community, and civil society organizations (CSOs). Grantees in Fergana and Samarkand also highlighted the significant improvement in their reputation, recognition, and collaboration with various organizations, ranging from health clinics to Khokimiats to the Prosecutor's Office. This positive development is evident through the invitations received to attend events relevant to the NGO's areas of focus. Moreover, employees from various state bodies actively seek consultations from the grantees, underscoring the trust placed in their expertise and insights. However, another grantee in Samarkand indicated that despite minor improvement in interaction between NGOs and the state bodies, the government agencies are still hesitant to interact with and engage in open dialogues with NGOs. The importance of engaging decision-makers and NGOs in round table discussions was highlighted by grantees in Tashkent as it provides an opportunity to clearly explain why certain regulations may have adverse effects on the development of democratic rule of law. The ongoing discussions with the MoJ and the Agency of Legislative Initiative regarding Decree #328 poses challenges for NGOs’ access to foreign funding was cited as an example. The evaluation has revealed some notable instances of LRP grantees contributing to the development of legislation amendments aimed at promoting civil society involvement in monitoring government activities, fostering social partnership, and protecting the rights of citizens with special needs. One such example is the Institute for Democracy and Human Rights, which has drafted commentary on the statute governing public oversight of the Law on Public Control. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 16 These amendments strive to enhance CSO participation in monitoring activities and citizen oversight of government bodies. The Civic Initiative Support Center conducted an analysis of the law on Social Partnership, focusing on understanding its perception among the public and NGOs. As reported, this analysis was utilized by the Oliy Majlis (parliament), the MoJ and the Center for Human Rights to formulate recommendations. NGO Mehr Ko’zda drafted comments and recommendations regarding regulations concerning the human rights and legal literacy of individuals with speech and hearing disabilities. As a result, considering the NGO’s recommendations, the Legislation and Parliamentary Research Institute, under the Oliy Majlis conducted revisions to the relevant regulations including the law on the rights of persons with special needs. Additionally, an LRP grantee in Samarkand mentioned their engagement in the development of legislation addressing GBV and other laws aimed at protecting vulnerable populations. However, discussions with IP, experts and USAID have highlighted persistent challenges, including the limited participation of NGOs in the process of amending laws, a lack of understanding of policy and law-making processes, and a restricted capability to effectively perform government oversight. These challenges remain despite LRP’s efforts to enhance the organizational capacity of NGOs and involve international experts in preparing appeals and drafting alternative reports for international human rights organizations. Another significant challenge, reflective of the broader NGO landscape in Uzbekistan, is the limited number of NGOs. This results in a relatively small pool of CSO available for engagement, leading to a situation where the “usual suspects” are frequently relied upon. These limitations can be attributed to the legacy of a restrictive environment created by the previous government, wherein NGOs were primarily engaged in social activities, and civic participation often followed a top-down mobilization approach. As discussed earlier, the current government’s restrictions also greatly contribute to these challenges. The lack of transparency and limited interest of the GoU in receiving LRP’s support in advancing the legal framework on NGOs, alongside the highly restrictive operating conditions, was highlighted as the main impediment to the development of an enabling environment, as expressed by NGO grantees and IP. As mentioned in EQ2, LRP and its grantees encountered challenges during the drafting process of the Code on NGOs, with the MoJ expressing a lack of interest in collaborating with LRP on any versions of the code, without providing clear reasons for their disinterest. Discussions with NGO grantees and experts reveal two contrasting trends that have emerged in relation to NGOs in Uzbekistan. On one hand, there is support from the President for strengthening and developing NGOs. On the other hand, there is a sense of caution and apprehension among the GoU towards NGOs, driven by concerns about potential protests and disorders. As a result, the state has tightened control over NGO activities, leading to a highly challenging process for registering new NGOs. Another aspect of this is that state organizations remain subject to national government orders and exhibit minimal independent initiatives, as per grantees and IP. Furthermore, there has been a notable reversal in the previously simplified access that NGOs had to foreign funding. This change in access is attributed to the provisions stated in Decree #328, which was approved by the Cabinet Ministers in June 2022. This decree establishes greater government control over foreign grants while limiting the right of NGOs to access and implement grants. The outcome remains uncertain of an appeal contributed to by some of the LRP grantees (and supported by an international expert hired by LRP), regarding these regulations. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 17 CONTRIBUTION TO STRENGTHENING ORGANIZATIONAL CAPACITY OF NGOS All the interviewed LRP grantees unanimously emphasized the significant improvement in their organizational and professional capacity, thanks to training sessions organized by LRP. According to respondents, the training topics were diverse and covered various aspects, such as drafting proposals aligned with organizational objectives, budget development, project management, reporting skills, and professional development in areas such as mediation, drafting laws, conducting legislation analysis, and provision of legal aid. One of the grantees highlighted that the project led to the establishment of an effective monitoring system, enabling the generation of weekly reports, and ensuring regular communication with the field office. This systematic approach has had a substantial impact, enhancing the capability of the NGO and significantly improving the overall monitoring and evaluation procedures. Another grantee in Tashkent mentioned that their knowledge and abilities in conducting public control were greatly strengthened as a result of LRP training. Similarly, in Samarkand, a grantee highlighted the practicality of the training methods, particularly through the use of case studies that provided insights into the procedural and practical aspects of issuing and applying protection orders for victims of domestic violence. During group discussions, lawyers among the grantees in Fergana and Tashkent expressed their appreciation for the interactive training methods employed, such as role-playing, case studies, experience sharing, group activities with subsequent presentations, Q&A sessions, as well as the provision of visual aids and handouts in both paper and electronic formats. The lawyers also highlighted the value of training sessions covering a wide range of topics that enhanced their knowledge in providing legal aid, and they especially appreciated specialized training sessions tailored to their specific needs, such as training on family disputes. Among the various sessions, the most recent professional development session focused on the financial sustainability of NGOs stood out for the grantees. They shared that this particular session equipped them with the necessary skills and expertise to address the challenges associated with financial sustainability. Moreover, they were encouraged to explore support from sources beyond donor funding, including engaging more effectively with government entities to ensure their rights and contributions are recognized and supported. LRP grantees have reported a noticeable enhancement in networking and collaboration opportunities with other NGOs as a direct outcome of their participation in LRP. This positive change can be attributed to LRP’s proactive efforts in creating a platform that fosters collaboration, knowledge exchange, and synergy within its grantees and the wider NGO community. A grantee based in Tashkent specifically emphasized the value of collaborating with 15 NGOs operating in different regions. This collaborative approach has enabled grantees to efficiently identify solutions to various legal issues faced by vulnerable groups. In cases where specialized expertise is required, the grantees refer individuals to relevant organizations for further assistance. Moreover, when dealing with legal proceedings, the grantees have the opportunity to consult with attorneys, ensuring effective legal representation. This approach ensures that LRP grantees are well-equipped to provide extensive legal aid and services to their target group, regardless of the specific challenges vulnerable groups may encounter. The issue of sustainability of legal aid and service provision emerged as a significant topic during interviews with LRP grantees and the IP. Several grantees mentioned that even after the grant period ended, they continued to provide pro-bono legal aid and services. For example, a grantee in Fergana highlighted their collaboration with a local network of NGOs and their memorandum of understandings with state bodies such as the Family and Women's Committee, mahalla (neighborhood) GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 18 committees, and health centers, which contributed to the ongoing provision of pro-bono legal services. Other grantees mentioned that their lawyers continued to offer free legal aid despite the lack of a salary. However, concerns were raised about the increasing difficulty of retaining lawyers without a sustainable source of income. Another group of grantees indicated that they would be able to extend their free legal aid services to more vulnerable groups if such projects continued. The IP recognizes the importance of sustainability strategies for NGOs, given the limited duration of LRP and the heavy reliance of NGOs in Uzbekistan on donor funding. The lack of financial support negatively impacts NGOs and the sustainability of the results they have achieved. Consequently, the IP is focusing on training NGOs to become more independent and self-sustainable. However, this is a challenging task, as the concept of carving out time for pro-bono work is not well-established among lawyers in Uzbekistan, and there is a limited culture of volunteering and community service. Additionally, the IP encourages NGOs to establish contacts with government organizations, recognizing their potential to provide financial support. However, bureaucratic obstacles often hinder NGOs from obtaining grants from the government, leading them to prioritize their activities over writing proposals and reports to the government. CONCLUSIONS FOR EQ3 The grants program made notable efforts towards the development of the legal enabling environment. However, the success of the suggested changes and improvements relies heavily on the political will of the GoU. The lack of transparency and limited interest of the government in engaging with LRP to advance the legal framework for CSOs/NGOs pose significant impediments to progress in this area. LRP grantees have played a role in shaping legislation to promote civil society involvement, protect rights, and foster social partnership. However, challenges such as limited NGO participation, a lack of policy process understanding, and restricted government oversight capacity persist due to the legacy of a restrictive environment and current government restrictions. The support in developing a Presidential Decree on regulatory impact assessment, if adopted and implemented, has the potential to empower civil society, increase state organizations' accountability, and mandate consultations with NGOs, leading to improved legislation and enhanced public oversight. While there have been positive changes in the interaction between state institutions and LRP grantees, some government agencies remain hesitant to engage in open dialogues with NGOs. Continued engagement of decision-makers is necessary to address regulatory challenges and ensure a supportive legal environment for NGOs in Uzbekistan. On the other hand, LRP’s grants program has significantly contributed to strengthening the organizational capacity of CSOs/NGOs. The project’s diverse training sessions have led to a notable improvement in the organizational and professional capacity of LRP grantees, enhancing their networking and collaboration opportunities to effectively address legal issues faced by vulnerable groups and provide comprehensive legal aid services with the support of relevant organizations and legal expertise. The focus on financial sustainability for free legal aid service provision has emerged as a significant concern, with ongoing provision of pro-bono legal aid services, but retaining lawyers without a sustainable source of income remains a challenge. The IP recognizes the importance of sustainability strategies for NGOs, including exploring alternatives to foreign donor funding such as financial support from government organizations; however, bureaucratic obstacles hinder NGOs’ access to state grants. Moreover, cultural factors such as the limited culture of volunteering and community service pose challenges to achieving long-term sustainability for NGOs in Uzbekistan. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 19 EVALUATION QUESTION 4: GOING FORWARD, GIVEN THE GOU’S AGENDA OF ONGOING LEGAL REFORMS IN UZBEKISTAN, IN WHICH AREAS SHOULD USAID￾FUNDED ACTIVITIES BE TARGETED TO ENABLE THE HOST COUNTRY TO BUILD A MORE SUSTAINABLE AND DEMOCRATIC RULE OF LAW SYSTEM/CULTURE? Interviews with key stakeholders identified seven areas that require targeted USAID-funded activities to support the host country in building a more sustainable and democratic rule of law. Capacity building and increasing legal literacy among government decision-makers, civil servants, and lawyers should be prioritized. This was emphasized by 23 out of 35 KIIs from various groups. However, stakeholders had slightly varied focuses and approaches to achieving this goal. NGO partners and experts unanimously believe that employees in government agencies and organizations have very limited awareness about new laws and legislative acts, hindering the development of a democratic rule of law. For example, an expert highlighted the need to retrain the Chamber of Advocates’ 4,000 lawyers, who still hold dated views on administrative law (likely due to its leadership’s cautiousness to changes) and are unaware of basic changes in the administrative procedures. It is therefore essential to train and provide opportunities for government employees to enhance their legal literacy and qualifications. Additionally, NGO partners stressed the need to prioritize training for leaders such as ministers, department heads, and decision-makers, in addition to regular employees. The current focus on training regular employees limits the impact of LRP, and strengthening the capacity of leaders with decision-making authority is necessary to create an environment conducive to fostering change. LRP grantees and some NGO partners highlighted the need to focus on the local level when providing training and support to the local government, local parliament, and mahallas. For instance, grantees from Samarkand and Fergana pointed out the necessity of conducting capacity-building sessions for state employees responsible for providing legal counsel and aid to the public, such as legal specialists, district court, mahalla committees, and khokimiyats. These institutions often have limited knowledge of updated laws and insufficient work experience, compromising their effectiveness. Therefore, there is a need to strengthen their capabilities and improve their performance through enhanced knowledge and oversight. National partners equally emphasized the need to increase the education of government employees, highlighting the preferences of internships and exchange programs in developed countries. For example, a representative of MoJ cited the contribution to bankruptcy law that was a result of collaborating with experts from Japan on another donor-funded project. Another respondent stressed the importance of internships for lawyers, mentioning the internship experience funded by Regional Dialogue and the OSCE in the US, which, reportedly, led interns from Chamber of Advocates to implement a project based on their learnings. Engaging government employees in extensive training sessions and seminars featuring reliable experts to incorporate international standards into anti￾corruption issues is considered beneficial for addressing corruption within governmental organizations and among officials, according to a legal professional from a national partner organization. Focus on conducting capacity-building events in various regions, including those that are less frequented such as areas bordering Afghanistan and Turkmenistan. There is a consensus among GoU, LRP grantees, NGO partners, the IP and training participants regarding to hold capacity-building events in these regions and avoid confining such events solely to the capital city. Respondents expressed that Tashkent already has abundant information and opportunities, while regions and rural areas have limited access to information. This creates an opportunity to educate and GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 20 enhance the legal knowledge of regional and local administrations, community structures, and the general population. LRP grantees and NGO partners particularly underscored the importance of focusing on the local level, including the training of local parliament members and engagement with mahalla (neighborhood) committees to increase their knowledge on legislative changes so that they can implement them in practice. Respondents mentioned that the current attention is primarily on the national parliamentary level, but it is crucial to put efforts into the development of critical thinking and increase participation and activism at the local level as well. They highlighted that mahallas play a significant and often monopolistic role in decision-making within communities, and working directly with them is essential for the overall well-being of society. Maintain a continuous line of communication and collaboration between NGOs and regional and local government bodies on the rule of law. Both the GoU and LRP grantees expressed the importance of maintaining an exchange between NGOs and regional and local government bodies. Respondents believe that such collaboration can have a positive impact on the overall development of the rule of law. At the regional and local levels, a grantee suggested the establishment of an interdepartmental committee consisting of NGOs, local government entities, schools, and mahalla committees. The purpose of this committee would be to facilitate official-level meetings and provide a platform for joint problem-solving and addressing common shortcomings. The creation of such a committee would enhance communication channels and enable NGOs to have a voice and be actively involved in decision-making processes. There is significant demand and importance for providing ongoing free legal aid to vulnerable groups and raising awareness about their rights and opportunities. This finding was consistently emphasized by the IP staff, LRP grantees, NGO partners, and experts. Respondents strongly believe that free legal aid plays a crucial role in building a culture of the rule of law. It not only increases the awareness of vulnerable groups about their rights, laws, and the legal system but also provides them with equal access and means to protect their human rights. Furthermore, according to some grantees, free legal aid has also contributed to increased confidence in the legal system. Through the assistance of pro-bono lawyers, beneficiaries were able to effectively protect their rights (as mentioned in EQ1). During FGDs, beneficiaries of free legal aid stressed the necessity of raising legal awareness and the provision of pro-bono legal services for vulnerable groups, especially in rural areas and remote villages where access to the internet is limited, and they have difficulties affording legal consultations. In Samarkand, male beneficiaries specifically highlighted the need for informational materials and legal services in languages understood in villages, such as Tajik. Respondents emphasized that relying solely on a single pro-bono lawyer provided by khokimiyats is insufficient to address the legal needs of such a large population. There is a need for a more extensive provision of free legal aid to effectively meet the demands of vulnerable groups. Female beneficiaries require psychological support in addition to pro-bono legal aid. This need was expressed in nearly all FGDs and was raised by grantees in Fergana, who emphasized the importance of psychological counseling, especially in cases of domestic violence, divorce, and child support. This sentiment was echoed by LRP grantees in Tashkent and Samarkand, who highlighted the need for both legal aid and mental health support for female beneficiaries. These women often experience depression, which can sometimes lead to suicidal tendencies. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 21 Continue strengthening the capacity of NGOs, particularly those working on sensitive and significant issues like human rights, women’s rights, children’s rights, and trade unions. The IP, NGO partners and grantees have expressed the need for this NGO capacity strengthening, and IP staff emphasized that the grantees have effectively demonstrated their capability to provide legal aid and contribute to legislative amendments. Furthermore, the grantees have successfully obtained MoUs from ministries, local government, and community organizations, allowing collaboration with state bodies. As a result, the government and state institutions have recognized these achievements and the importance of supporting the grantees. Additionally, lawyers have played a role in facilitating interactions between the grantee and state institutions through referral mechanisms, indicating a more streamlined and efficient process for providing legal aid. An important area of focus is the reinforcement of legislation and improvement of legal instruments, as well as the implementation of law. Experts and NGO partners recognized the essential need to address deficiencies in administrative and regulatory law and effective implementation of administrative procedures, given the significant interactions with the state that occur in the administrative realm. To tackle these issues, staff retraining, education, and active campaigns are necessary. Furthermore, it is crucial to update the outdated view that combines administrative and criminal actions under one specialty. Recognizing that administrative law requires specialized expertise is important. There should be a focus on advocacy and raising awareness among lawyers about the basics, such as the violation of principles that can lead to overruling decisions made by local authorities. Additionally, attention should be given to improving the accessibility of the administration within the well-developed legal system, ensuring that individuals can easily engage with administrative processes. EVALUATION QUESTION 4A: WHAT TYPE OF TECHNICAL PROGRAMMING, INNOVATIONS FOR DESIGN AND FUNDING (I.E., PILOTING, CO-CREATION, STAKEHOLDER INPUT, ETC.) OR PARTNERING ARRANGEMENTS SHOULD USAID PURSUE? A review of respondents’ responses yielded several types of programming suggestions for promoting democratic rule of law and fostering a culture of the rule of law: 1. Utilization of social media: Experts and NGO partners have emphasized the utilization of social media to enhance legal literacy and raise awareness about rights, principles, and the importance of the rule of law among the public. They believe that creating and disseminating social videos through social networks is an effective method, especially for educating children about human rights, respect for the rule of law, and independence. Cartoons or films can serve as powerful tools in achieving this goal. NGO partners emphasized the importance of social videos and their significant impact in addressing social issues, raising awareness, and promoting positive change within society. Additionally, grantees and beneficiaries of free legal aid have suggested utilizing social media as a platform to provide information and assistance to individuals seeking legal aid. The intention behind this suggestion is to leverage the popularity and reach of social networks and mass media to effectively promote and increase awareness about the availability of free legal aid services. By utilizing these channels, a wider audience can be reached, ensuring that individuals in need are aware of available support. Female beneficiaries proposed incorporating online help as part of free legal aid services, benefiting individuals who face challenges in traveling to receive assistance. They proposed developing a program or platform where users can initiate real-time discussions with legal specialists to address their legal issues remotely, ensuring timely assistance regardless of location. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 22 2. Utilizing Data Science and AI: The IP staff suggested using data science and artificial intelligence to analyze laws, gather relevant data, and provide recommendations to aid decision-making processes. This innovative approach would assist government officials in efficiently responding to the needs of the people. Furthermore, the suggestion includes utilizing technological tools to capture information, such as tracking laws people are searching for and collecting complaints against public servants. This enables the government to quickly gather and analyze information, becoming more responsive to the needs of the people. Crowdsourcing and seeking feedback on draft legislation or legislative acts are also proposed to identify and address specific needs and concerns effectively. 3. Creating a Centralized Online Portal: The IP suggested establishing a centralized online portal to host various educational and capacity-building materials and resources related to the LRP work and other projects supported by USAID. This portal would consolidate information and resources from various projects and initiatives, providing easy access to comprehensive and up-to-date materials related to the rule of law sector. The portal would serve as a repository for foundational materials, facilitating the dissemination and utilization of knowledge gained through training programs for national partners, civil servants, legislation employees, mahalla leadership, local government, members of the General Prosecutor's Office, and other interested entities. The IP believes that this educational portal would offer a valuable resource for ongoing learning and capacity building, benefiting stakeholders involved in the LRP and related initiatives. 4. Establishing an Integrated Database: NGO partners and the IP highlighted the importance of establishing a centralized database that consolidates information from various sources, including the government, NGO sectors, and foreign experts. Respondents emphasized the need for a comprehensive database to safely store and efficiently access valuable knowledge and statistical data scattered across different places and individuals. They expressed concerns about the current system's effectiveness in ensuring data safety, organization, sharing, and accessibility. The creation of such a database is seen as timely, particularly with ongoing legislative changes, enabling better utilization of collective knowledge. 5. Establishing a Knowledge-Sharing Platform: The interviewed NGO partners and experts have suggested the establishment of a knowledge-sharing platform, enabling scientists and professionals in the field to exchange expertise and information. The objective is to address the demand for competence and reliable contemporary information. According to the key informants, the MoJ faces challenges in accessing ideas or specific pieces of information. Therefore, a dedicated platform for discussions and knowledge sharing would simplify this process. Additionally, experts could share guidelines and tools to assist the Ministry in implementing these concepts. This initiative has the potential to foster productive collaboration with the ministry, facilitate information exchange, and serve as a valuable resource for the MoJ and other professionals. 6. Expediting Digitalization: NGO partners and grantees shared that increasing efforts to expedite the digitalization process would allow the legal system to fully leverage the benefits of digital technologies and increase its efficiency. Supporting the digitization of the legal system may involve addressing potential obstacles or challenges that are slowing down progress and exploring strategies to accelerate the digital transformation in the legal domain. CONCLUSIONS FOR EQ4 In conclusion, the respondents' suggestions for promoting democratic rule of law and fostering a culture of the rule of law encompass various innovative approaches. Utilizing social media emerges as a tool to enhance legal literacy, raise awareness, and offer online help through real-time discussions GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 23 with legal specialists. Incorporating data science and artificial intelligence could facilitate more efficient decision-making and responsiveness to people’s needs. The creation of centralized online portal and an integrated database aims to consolidate information, provide easy access to resources, and enhances knowledge-sharing among stakeholders. Additionally, establishing a knowledge-sharing platform and expediting digitalization are proposed to foster collaboration, facilitate information exchange, and improve the efficiency of the legal system’s transformation. Implementing these suggestions could significantly contribute to advancing the rule of law in Uzbekistan. 5. RECOMMENDATIONS Based on the findings and conclusions described in the previous section, the ET proposes the following recommendations to assist primary users in fostering learning and informing current and future USAID/Uzbekistan DRG programmatic strategies. RECOMMENDATIONS FOR USAID/UZBEKISTAN 1. USAID should limit the scope of areas of legislative development support, focusing on specific areas to ensure effective and targeted legislative assistance. One potential area of focus could be supporting initiatives that aim to improve legal instruments and enforce legislation, particularly in areas such as administrative and regulatory law. This can involve implementing retraining programs, education campaigns, and advocacy efforts to address deficiencies, enhance accessibility, and ensure compliance with international standards. 2. USAID should support activities that enhance legal literacy and qualifications among government employees, including decision-makers and leaders. This can be achieved through targeted training programs, workshops, and capacity-building initiatives that focus on updating knowledge and promoting an understanding of new laws and legislative acts, thus facilitating their effective implementation. 3. USAID should focus on local capacity building by providing training programs for local government officials, civil servants, mahalla committees, and community structures. By extending support and resources to these entities, the rule of law can be effectively promoted and implemented at the grassroots level. 4. USAID should support initiatives that strengthen communication and collaboration between NGOs, regional/local government bodies, and other stakeholders involved in the rule of law sector. This can be accomplished through the establishment of interdepartmental committees, regular meetings, and joint problem-solving initiatives. 5. USAID should continue to support the capacity building of NGOs, particularly those working on human rights, women's rights, children's rights, and trade unions. This includes providing resources, training opportunities, and financial support to enable NGOs to effectively contribute to legislative amendments, provide legal aid, and engage in collaborative efforts with government institutions. 6. USAID should prioritize the people-to-people approach and maintain funding for grants, supporting IP’s endeavors in strengthening long-term institutional capabilities of NGOs. This will empower local NGOs to establish partnerships with local communities and government organizations, fostering the culture of the rule of law and safeguarding the rights of vulnerable populations. 6. USAID should mandate implementing partners to collect outcome-level data to assess the effectiveness of USAID-funded activities in promoting a sustainable and democratic rule of law. This GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 24 data will provide valuable insights for programmatic adjustments and ensure efficient allocation of resources. 7. USAID should prioritize the people-to-people approach and allocate more funds toward grants, supporting IP’s efforts in building long-term institutional capabilities of NGOs. This will empower local NGOs and allow them to create partnerships with local community and local government organizations to advance the culture of the rule of law and protect the rights of vulnerable populations. 8. USAID should enhance coordination and information sharing among its rule of law, economic growth and local governance support programs, as well as other USG-funded initiatives. This will facilitate better communication, knowledge exchange, and the prevention of program duplication, double funding of beneficiaries, while also creating more opportunities for collaboration, joint actions and synergies. RECOMMENDATIONS TO THE IMPLEMENTING PARTNER OF LRP, TETRA TECH DPK 1. The IP should focus on strengthening the long-term institutional capabilities of grantees. Focus on building the capacity of local NGOs to ensure their sustainability beyond the project duration. Provide training, mentorship, and resources to enhance their organizational and technical skills, financial management, and strategic planning. 2. The IP should continue exploring alternatives to donor funding for NGOs. Support grantees in diversifying their funding sources by exploring opportunities for generating income, engaging in social enterprise initiatives, and establishing partnerships with government and local businesses. This will help reduce dependency on external funding and enhance the financial sustainability of NGOs, including free legal aid services. 3. The IP should consolidate efforts and develop a clear action plan for the post-LRP period. Identify areas with the greatest potential for sustainability and impact and develop a comprehensive action plan that outlines the steps, timelines, and responsibilities for continuing the progress achieved during the LRP. This plan should involve collaboration with the government, NGOs, USAID, and other potential funding opportunities to ensure continuity and sustained progress. 4. The IP should maintain a critical evaluation of government legislative requests to ensure their alignment with the LRP’s objectives of maximizing impact. This approach will enable the project to concentrate on specific areas for effective and targeted legislative assistance and adapt and respond to the dynamic changes in the government’s legislative priorities. 5. The IP should address language barriers and enhance accessibility to capacity-building activities for national partners, NGOs and grantees from different regions, aiming for better inclusivity, participants engagement and knowledge transfer. This can be achieved by gathering information on attendees’ language and accessibility preferences before events and organizing facilitators, materials, and logistics accordingly. 6. The IP should review and refine outcome level indicators to capture the desired changes and progress in the legislative landscape. Utilize tools such as Excel file to track and document the adoption of suggested regulations, as well as the outcomes of legislative activities. This approach will create a comprehensive record of the project’s impact and facilitate the evaluation of its effectiveness in driving legal reforms. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 25 7. The IP should systematically collect and analyze data on the barriers faced by beneficiaries of free legal aid in accessing entitled state benefits, disaggregated by relevant factors such as age, gender, region, disability, and other vulnerabilities. This data collection will uncover the root causes of challenges and provide insights to inform necessary bureaucratic and legislative changes. This will ultimately lead to improved access to citizens' entitlements in an efficient and equitable manner. Additionally, this data-driven approach can help reduce the reliance on legal professionals for free aid assistance. 8. The IP should facilitate dialogues and discussions that promote collaboration between government officials and civil society representatives. This platform will foster a shared understanding of the rule of law and its implications for democratic development in Uzbekistan. By bringing these stakeholders together, attitudes and perceptions about the rule of law culture can be effectively changed, especially in light of the country's adoption of a new constitution. This initiative will provide an opportunity for government officials to internalize the principles and values of the rule of law, potentially leading to behavioral and procedural changes, ultimately contributing to a stronger rule of law and democratic governance. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 26 ANNEX A. EVALUATION STATEMENT OF WORK SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK C.1 TITLE The Legal Reform Program (LRP) Evaluation C.2 PURPOSE The purpose of this evaluation is to test the primary objectives set forth in the LRP grant agreement. The evaluation will provide pertinent information, analysis, and judgments that assist USAID to learn what has been accomplished towards the intended results of the agreement and provide recommendations on how to achieve any results. Specifically, the evaluation will help USAID better understand results from its investments in legal reform and rule of law principles in Uzbekistan. C.3 BACKGROUND The USAID Legal Reform Program supports Uzbekistan’s efforts to ensure further development of rule of law principles, build the capacity of legal professionals and training institutions, enhance citizens’ access to justice, improve the legal operating environment for civil society, promote gender equality, and create a healthier investment climate. USAID partners with the Ministry of Justice, Chamber of Advocates, Academy of the GPO, Uzbekistan’s Higher Education Institutions, and key actors in civil society to meet these objectives. The GoU is in the process of implementing ambitious and comprehensive economic and social reforms aimed at boosting growth, modernizing its legal framework, promoting transparency and accountability, and improving services for citizens. The GoU has already made significant structural changes, such as reducing bureaucratic impediments to development and adopting policies to conform with international standards and best practices. To attract foreign direct investment and improve the country’s rankings in prominent international reports and indicators, the GoU has introduced measures to heighten investors’ confidence in the legal framework and key institutions that affect economic growth in Uzbekistan. C.4 PROJECT INTENT The LRP activity’s period of implementation is from October 1, 2018 to August 31, 2023, with a total estimated cost of $10,000,000. The USAID Legal Reform Program supports Uzbekistan’s efforts to ensure further development of rule of law principles, build the capacity of legal professionals and training institutions, enhance citizens’ access to justice, improve the legal operating environment for civil society, promote gender equality, and create a healthier investment climate. To achieve this goal, the project focuses on the following objectives: 1) Support for rule of law development in Uzbekistan; 2) Development of the legal profession; 3) Development of the legal-enabling environment for civil society and NGOs. C.5 EVALUATION QUESTIONS The evaluation will address the following key questions: GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 27 1. What gaps still exist in addressing the activity’s objectives? Where has the LRP activity succeeded and where has it failed to achieve the objectives and/or not achieved the intended results? Any unintended achievements, challenges or negative results? 2. To what extent does the Government of Uzbekistan support in practice its own request for LRP contributions for amending current legislation and/or adopting new legislation? To what extent has the GoU supported LRP contributions for amending current legislation and/or adopting new legislation? 3. To what extent did the grants program contribute to development of the legal enabling environment and strengthened organizational capacity for CSOs/NGOs as planned? 4. Going forward, given the GoU’s agenda of ongoing legal reforms in Uzbekistan, how and in which areas should USAID-funded activities be targeted to enable the host country to build a more sustainable and democratic rule of law system/culture? Include specific recommendations for technical programming, innovations for design and funding (i.e., piloting, co-creation, stakeholder input etc.) and partnering arrangements. C.6 DATA ANALYSIS METHODS This evaluation will be qualitative in nature with some secondary, descriptive quantitative data analysis. Prior to the start of data collection, as part of the evaluation work plan, the evaluation team will develop and present, for USAID’s review and approval: a data analysis plan; what procedures will be used to analyze qualitative data obtained from key informants and other sources; and how the evaluation will weigh and integrate qualitative data from various sources with quantitative data from project performance monitoring records, and public opinion polls implemented by local polling organizations to reach conclusions about the effectiveness and efficiency of LRP in Uzbekistan. The contractor will have flexibility in proposing the most appropriate methodology to accomplish the tasks outlined in the most efficient way. However, the contractor will address the following: Task 1: Desk review The contractor will conduct a desk review to gain a general overview of the rule of law situation in Uzbekistan and identify areas (as related to the LRP objectives and evaluation questions) that merit closer attention once the team begins its fieldwork. Reading materials will be available to the team upon award. The contractor will review a broad range of other relevant, non-project related documents, such as peer-review articles and other publications, and incorporate the review findings into the data collection instruments, the data collection design, and data analysis and interpretation. USAID will provide the evaluation team with a package of background materials, including: ● USAID/Uzbekistan Strategic Framework (2022-2025) ● LRP Contract Agreement, including any modifications that made changes to the Program Description ● LRP Work Plans, Years 1-4 ● LRP Annual Reports, Years 1-4 ● LRP Quarterly Reports, 2015-2018 ● LRP Initial Performance Monitoring and Evaluation Plan, 2018 ● LRP Performance Indicator table (Year 1-4) The final evaluation report shall triangulate information obtained from the pre-field work desk review and from the field to articulate LRP contributions to the objectives as well as any opportunities to strengthen achievements during the remaining time frame of the program and beyond. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 28 Task 2: Draft description of overall methodology and work plan In conjunction with USAID/Uzbekistan, the contractor will develop a plan for conducting the fieldwork and phone interviews using information from the desk review and this SOW. The work plan will clearly outline the methodological approach the team will use. The work plan shall be submitted for review to the USAID/Uzbekistan Contract Officer Representative (COR) for approval prior to arrival in the country. The work plan shall include a fieldwork and phone interview schedule, including a preliminary list of all proposed districts for travel and key organizations the evaluation team plans to meet. Task 3: Conduct a field data collection The in-country data collection and phone interviews include, but is not limited to, facilitated discussions through interviews with the following counterparts: LRP/TT DPK MoJ Partner GoU agencies Partner CSOs Other CSOs Donor counterparts Business counterparts Journalists Beneficiaries (as appropriate) The evaluation team will spend at least three weeks in the country with the evaluation team visiting the capital city of Tashkent and at least three additional regions in Uzbekistan. The identification of informants/respondents and scheduling data collection must be arranged as far in advance as possible to maximize efficient use of time while in the field. USAID/Uzbekistan will provide introductions and contact information to facilitate the data collection. SECTION D - EVALUATION PRODUCTS I. DELIVERABLES Work Plan The team will prepare a detailed work plan which will include the methodologies to be used in the evaluation, such as sampling method, sample size calculation, data collection methods and tools, data analysis plan and any known limitations and ways to reduce and/or mitigate them. The work plan will be submitted to the COR at USAID/Uzbekistan for approval no later than the sixth day of work. In-briefing During the first two days upon arrival, the evaluation team will meet with USAID/Uzbekistan for an in-briefing and planning session. Mission Debriefing and a final presentation Before departing, the evaluation team will debrief interested personnel from USAID/Uzbekistan on the team’s preliminary findings and recommendations at the end of the field assessment. Draft Report The team will provide USAID a draft write-up of the team’s preliminary findings and recommendations. USAID/Uzbekistan will review the draft report and submit comments to the assessment team no later GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 29 than five working days after receipt of the draft report. USAID/Uzbekistan stakeholders will have five additional business days to review and provide additional written comments to the evaluation team. Final Report The assessment team will finalize the report upon their return to their home country. The Final Report should be no more than 30 pages and will incorporate both comments from the debriefing and written comments and submitted to USAID/Uzbekistan within ten workdays after receipt of the written comments from USAID/Uzbekistan. The final report will contain the following: ● Executive Summary. An executive summary not to exceed three to four pages in length. The executive summary will present the major findings, conclusions, and recommendations. The executive summary should stand alone as an abbreviated version of the report. All content of the full report should be summarized. ● Evaluation Purpose and Questions. An introduction section for the overall evaluation purpose and questions will be clearly defined at the beginning of the report. It will describe in about one page or less why the evaluation is being conducted now, how the findings are expected to be used, what specific decisions will be informed by the evaluation, and who the main audiences are for the evaluation report. ● Project Background. A project background section will summarize the project being evaluated in one to three pages, including the original problem the project is designed to address, any changes that have occurred since the project has started, a description of the beneficiary populations, geographic area of the project, and the underlying development hypothesis, or causal logic, of the project. ● Methods and Limitations. This section will include a description of the methods and limitations explaining the evaluation methods and why they were chosen. This section will provide information on all aspects of the evaluation design and methods, including tradeoffs that led to the selection of specific data collection and analysis methods, a description of data availability and quality, and sampling strategies (purposive, random, etc.), including how interview subjects or site visits were selected. This section will describe any limitations in data collection and analysis, data quality, access to data sources, or any other factors that may result in bias. It would be useful to include a chart in the report that lists each evaluation question, the corresponding evaluation method to be used for data collection and analysis, data sources, sample sizes, and limitations. Note: the full description of evaluation methods should be expanded in the Annexes. ● Findings, Conclusions, and Recommendations. This section will include a detailed section of detailed findings, conclusions, and recommendations that should not exceed 20 pages. This section will make up the main body of the report, synthesizing what was learned during the evaluation, answering evaluation questions and presenting information in an easy￾to-understand and logical fashion. It must include recommendations with a detailed discussion of strategic opportunities for future USAID programming. ● Annexes. The following annexes (a.) through (e.) must be included, at a minimum: a. Statement of Work b. Full description of evaluation methods c. All evaluation tools (questionnaires, checklists, discussion guides, surveys, etc.) in English and all other translated languages. d. A list of sources of information (key informants, documents reviewed, other data sources) e. Disclosure of any conflict of interest by including a signed statement by evaluation team members that attests to a lack of conflict of interest or describes an existing conflict of interest relative to the project being evaluated. f. Only if applicable, include as an annex Statement(s) of Differences regarding any significant unresolved differences of opinion on the part of the funders, implementers, and/or members of the evaluation team. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 30 Information Dissemination The reports must be emailed to the COR and Alternate COR, with confirmation of receipt. If accompanying files are too large to email, then USAID will provide further instruction for receiving the files in electronic format. In addition to printed copies, all deliverables and supplementary data must be made available in electronic form using USAID Branding and Graphic Standards (available at http://pdf.usaid.gov/pdf_docs/PNADB334.pdf). In addition, the report must follow standard Feed the Future branding per USAID guidance on usage of FTF funds. Guidance requirements found at: http://feedthefuture.gov/lp/feed-future-branding. The final report must be 508 compliant, per Section 508 of the Rehabilitation Act of 1973 that requires federal agencies to provide software and website accessibility to people with disabilities. Once the data are cleared for use outside of USAID/CA, the contractor/recipient must submit to the Development Data Library, at www.usaid.gov/data, in a machine-readable, non-proprietary format, as required by USAID’s Open Data policy (see Automated Directives System 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. II. REPORTING QUALITY CRITERIA Per the USAID Evaluation Policy and USAID Automated Directives System 203, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. ● The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why. ● Evaluation reports shall address all evaluation questions included in the SOW. ● The evaluation report should include the SOW as an annex. All modifications to the SOW— whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline — must be agreed upon in writing by the Agreement Officer’s Representative/COR. ● The evaluation methodology must be explained in detail. All tools used in conducting the evaluation — such as questionnaires, checklists, and discussion guides—will be included in an annex in the final report. ● Evaluation findings will assess outcomes and impact on males and females. ● Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings must be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. ● Conclusions should be clearly distinguished from findings. ● Recommendations must be supported by a specific set of findings and conclusions. ● Sources of information must be properly identified and listed in an annex. ● Charts and graphs should include clear labels and captions. ● Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. SECTION E - EVALUATION TEAM COMPOSITION I. Team members and key personnel GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 31 The evaluation team will consist of the following staff at a minimum, with key personnel identified: 1. Team Leader/Evaluation Specialist (key personnel) 2. Local Technical Specialist (key personnel) 3. Translator 4. Logistics Assistant II. Desired qualifications The Team Leader/Evaluation Specialist will serve as chief technical advisor and will be responsible for management of the entire evaluation and technical management of the evaluation team. The Team Leader will have the following minimum qualification: ● Extensive professional experience in designing/conducting evaluations. Such evaluation experience in Uzbekistan and/or Central Asia is preferred but not required. ● Proven skills and experience in developing methodological tools and instruments for evaluation such as interview protocols or questionnaires, data processing sheets, etc. ● A graduate degree in a related field of study. ● Proven leadership in the management of similar-sized international donor-funded activities with skills in strategic planning, management, supervision and budgeting. ● Evidence of strong English communication skills, both interpersonal and written. ● Proficiency in Russian is desirable. Local Technical Specialist ● A degree in political science, law or other related field. ● At least five years of experience of working in the rule of law/justice reform sector and/or civil society sector. ● Strong analytical, communication and writing skills. ● Ability to work independently and as a team member. ● Advanced English communication skills, both orally and in writing. ● Proficiency in Russian is desirable. In addition to technical experts, the team is authorized to hire two local individuals with experience working for international donor programs. Translator This individual will be responsible for providing translation to the assessment team members. Fluency in English, Russian, and Uzbek is required. The individual must be familiar with the related terminology. Logistics Assistant This individual will be responsible for providing logistical support to the assessment team members including, but not limited to, scheduling appointments and arranging transportation. Fluency in English, Russian, and Uzbek is required. SECTION F - EVALUATION MANAGEMENT I. LOGISTICS The contractor is responsible for providing all logistical support. Office space will not be provided by USAID as it is the responsibility of the contractor to make the logistical arrangements necessary to fulfill the statement of work. Translation services and transportation to/from/within Uzbekistan are the responsibility of the contractor. USAID will support the evaluation team with Diplomatic Notes, GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 32 as necessary, to ensure the team will have access to government institutions and potential opportunities to interview local officials. II. DELIVERABLE SCHEDULE Table 4: Deliverable Schedule Activity Deliverables Schedule Delivery date* Desk Review After the contract award, USAID/Uzbekistan will provide project related documentation. The contractor shall conduct the desk review and incorporate findings into the proposed methodology and work plan. 7 working days Draft description of overall methodology and workplan The evaluation work plan shall be submitted for review to the COR at USAID/Uzbekistan for approval prior to the team arrival in the country. USAID/Uzbekistan will provide feedback within one week after receiving the work plan. 5 working days In briefing During the first two days upon arrival to the region, the evaluation team shall meet with USAID/Uzbekistan for an in-briefing and planning session. 2 days Conduct a Field Data Collection The evaluation team will conduct a field data collection in Uzbekistan. 15 working days Debriefing Before departing the country, the assessment team will debrief interested personnel from USAID/Uzbekistan on the preliminary findings and recommendations. 2 days Draft report The evaluation team will provide a draft write-up of the team’s preliminary findings and recommendations. USAID/Uzbekistan will review the draft report and submit comments to the assessment team not later than five working days after receipt of the draft report. 10 working days Final report (Approved English version) The contractor shall submit the final report with incorporated comments within five days after receipt of the comments from USAID/Uzbekistan. 5 working days GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 33 III. PAYMENT SCHEDULE A six-day workweek is authorized with no premium pay while in the country, a five-day work week is authorized while working in the US. No premium or overtime pay is authorized under this contract. The following table gives estimates of Level of Effort (LOE) to assist offerors in developing their cost proposals. Offerors will propose costs for each of the deliverables listed in the Deliverable Schedule (and not based on the tasks set forth in the table below). Table 5: Deliverable Table Task/Deliverable Estimated Duration/LOE Team Leader Technical Specialist Admin Assistant Translator Review background documents & submission of draft evaluation work plan 7 days 5 days 5 days USAID reviews the draft Work Plan and provides comments (5 days) - - - Team revises the work plan based on comments or questions and submits the final to USAID 5 days m 4 days - Travel to Uzbekistan 2 days - - Team Planning Meeting and meeting with USAID/Uzbekistan in Tashkent 1 days 1 days 1 days Information and data collection in Uzbekistan. Includes interviews with key informants (stakeholders and USAID/Uzbekistan staff) and site visits 15 days 15 days 15 days 15 days Travel to Tashkent and outbrief with USAID/Uzbekistan 2 days 2 days 1 days Depart Uzbekistan/Travel to US 2 days - - Draft report and send to USAID/Uzbekistan 10 days 10 days - USAID/Uzbekistan and an IP (TT DPK) provide comments on the draft report (7 days) - - - Team revises draft report and submits final to USAID/Uzbekistan (out of country) 5 days 5 days - USAID completes final review (5 days) - - - Total Estimated LOE 49 days 42 days 22 days 15 days GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 34 ANNEX B. EVALUATION DESIGN MATRIX Table 6: Evaluation Design Matrix Type of Question Methodological Approach to Data Analysis Data Sources Data Collection Methods Evaluation Question 1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Descriptive question Content and Comparative Analysis to identify and describe objective 1 and objective 3 (focus on free legal aid) successes and challenges, factors that support or limit LRP contribution, as well as response categories, patterns, and emergent themes from collected data. Primary ● MoJ/International Cooperation Department ● Legislation and Parliamentary Research Institute under the Oliy Majlis ● Representatives of the Legislative Chamber of the Oliy Majlis ● Senators, members of Gender Equality Committee under the Senate ● National experts on gender equality ● DSC under the Senate ● Antimonopoly Committee ● Intellectual Property Right Agency ● Scientific Research Institute “Mahalla and Family " under the MSMF ● National Center for Human Rights of the Republic of Uzbekistan (gender￾related activities) ● State Committee for Family and Women of the Republic of Uzbekistan ● Chamber of Advocates ● GPO Academy ● NGO Institute for Democracy and Human Rights ● NGO Civic Initiative Support Center ● NGO Madad ● NGO Legal Issues Research Institute ● Nationwide movement “Yuksalish” (think tank) ● Free Legal Aid Grantees operating in Tashkent, Fergana, and Samarkand ● Recipients of free legal aid services in Tashkent, Fergana, and Samarkand regions ● LRP-engaged national and international experts who provide technical assistance in developing regulations and recommendations for legislative amendments, adoptions, and improvements of legislation Secondary ● IP staff, including gender team ● Desk review ● KIIs ● FGDs GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 35 Type of Question Methodological Approach to Data Analysis Data Sources Data Collection Methods ● USAID/Uzbekistan Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Exploratory question Content and contribution analysis to identify and describe strengths and weaknesses of LRP activities/approaches and implications for achieving its objectives on the development of rule of law and provision of free legal aid by NGOs supported by the project. Gap analysis to examine which approaches/activities of LRP, if any, fell short of anticipated performance, and the likely factors behind these gaps. Primary ● IP relevant staff ● USAID/Uzbekistan ● MoJ/International Cooperation Department ● Legislation and Parliamentary Research Institute under the Oliy Majlis ● Representatives of the Legislative Chamber of the Oliy Majlis ● Senators, members of Gender Equality Committee under the Senate ● National experts on gender equality ● DSC under the Senate ● Intellectual Property Right Agency ● Antimonopoly Committee ● Scientific Research Institute "Mahalla and Family" under the MSMF ● National Center for Human Rights of the Republic of Uzbekistan ● State Committee for Family and Women of the Republic of Uzbekistan ● Chamber of Advocates ● GPO Academy ● NGO Institute for Democracy and Human Rights ● NGO Civic Initiative Support Center ● NGO Madad ● NGO Legal Issues Research Institute ● Nationwide movement “Yuksalish” (think tank) ● Free Legal Aid Grantees operating in Tashkent, Fergana, and Samarkand ● LRP-engaged national and international experts who provide technical assistance in developing regulations and recommendations for legislative amendments, adoptions, and improvements of legislation ● Desk review ● KIIs Evaluation Question 2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or adoption of new legislation? Descriptive/Normative question Content and comparative analysis to examine the effectiveness of LRP contributions for Primary ● IP relevant staff ● USAID/Uzbekistan ● Desk review ● KIIs GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 36 Type of Question Methodological Approach to Data Analysis Data Sources Data Collection Methods amending current legislation and/or adopting new legislation, as well as to explore the weak areas that need more support from the GoU for LRP in moving forward legislative reform. ● MoJ/International Cooperation Department ● Legislation and Parliamentary Research Institute under the Oliy Majlis ● Representatives of the Legislative Chamber of the Oliy Majlis ● Senators, members of Gender Equality Committee under the Senate ● National experts on gender equality ● Intellectual Property Right Agency ● Chamber of Advocates ● Ombudsperson’s Office ● Business Ombudsperson ● LRP-engaged national experts and international experts who provide technical assistance in developing regulations and recommendations for legislative amendments, adoptions, and improvements of legislation Secondary ● NGO Institute for Democracy and Human Rights ● NGO Civic Initiative Support Center ● NGO Legal Issues Research Institute ● NGO Madad ● Nationwide movement “Yuksalish” (think tank) Evaluation Question 3: To what extent did the grants program contribute to development of the legal-enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? Descriptive/Normative question Content and Comparative analysis to identify objective 3 grants program contributions to the legal-enabling environment, and CSOs/NGOs’ organizational capacity-strengthening activities’ success and challenges, as well as response categories, patterns, and emergent themes from collected data. Gap analysis to examine which aspects of grants program contributions to the development of legal-enabling environment, as well as NGOs capacity-building activities, if any, fell short of anticipated performance, and the likely factors behind these gaps. Primary ● LRP grantees operating activities that intend to contribute to development of legal environment in Tashkent, Fergana and Samarkand regions, including NGO Mehrjon (Fergana) ● LRP grantees participants of LRP’s organizational capacity-building activities ● NGO Institute for Democracy and Human Rights ● IP relevant staff Secondary ● USAID/Uzbekistan ● NGO Civic Initiative Support Center ● NGO Legal Issues Research Institute ● NGO Madad ● National gender, legal and civil society experts ● Desk review ● KIIs GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 37 Type of Question Methodological Approach to Data Analysis Data Sources Data Collection Methods Evaluation question 4: Going forward, given the GoU’s agenda of ongoing legal reforms in Uzbekistan, in which areas should USAID-funded activities be targeted to enable the host country to build a more sustainable and democratic rule of law system/culture? Sub-question 4.1: What type of technical programming, innovations for design and funding (i.e., piloting, co-creation, stakeholder input, etc.) or partnering arrangements should USAID pursue? Evaluation question 4 and its sub-question is a recommendation question, which will be formulated as a result of the analysis of all collected data. No additional data collection or analysis will be required. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 38 ANNEX C. PROGRAM BACKGROUND A snapshot of LRP’s main activities is shown in Table 7 below, followed by LRP’s theory of change, and planned achievements for project years 1-3. Table 7: Activities and Approaches by LRP Objective LRP Objective Approaches/Activities 1. Supporting Rule of Law Development in Uzbekistan ● Develops and/or modifies legislation at the national level to bring country’s legal framework in line with international standards and best practices. o Reviews laws, develops and submits recommendations to law-making bodies o Provides technical expertise to various working groups in drafting codes and decrees o Holds conferences, public hearings and discussions ● Administers international study tours, enhances capacity of legal professionals in rule of law related matters, and supports the development of resource (legal database, research assistance) ● 2. Development of the Legal Profession ● Develops capacity of legal educators and students of law schools ● Supports the development of systems for training institutions ● Strengthens capacity of state staff and legal professionals ● Supports creation of an Online Legal Clinic ● Develops resources to support business and legal educational needs ● Improves technological capacity of training institutions, state offices and NGOs 3. Development of Legal￾Enabling Environment for CSOs/NGOs ● Supports locally led efforts through distribution of grants to NGOs ● Builds organizational capacity of grantees ● Promotes discussions and dialogues between state organizations, legislators, and CSOs/NGOs ● Administers study tours for state officials and CSOs 4. Promoting Gender Equality (Cross-cutting) ● Led the development of the Gender Equality Law ● Conducts analyses of gender legislation and research ● Builds gender mainstreaming capacity and enhances skills of legislators, government employees, legal professions, NGOs, and journalists ● Develops and disseminates gender-related informational and educational resources ● Conducts awareness-raising campaigns and events LRP’s theory of change is as follows: IF LRP provides technical assistance and capacity development to draft, amend, and implement laws in harmony with international standards and sound policies, and simultaneously supports professional capacity building and a resource base for legal research, training, law practice, promotion of citizens’ interests, and development of a culture of lawfulness; and IF local actors make good use of that technical assistance and capacity development, THEN the legal framework, rule of law, and civil society will be strengthened. In year one, LRP’s programming was expected to strengthen legal framework, rule of law, and civil society in Uzbekistan in the following ways: GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 39 ● Establish and renew cooperative working relationships with GoU officials, civil society, United States (US) and international donors working on Rule of Law and Justice Sector reform. ● Approved roadmap that identifies required activities in years two and three that will lead to improved implementation of adopted reforms. ● Increase in the number of key stakeholders promoting judicial reform efforts in Uzbekistan. ● Draft legislation includes language that demonstrates an increased understanding of international standards, their bearing on Uzbek law and practice, as well as rating and ranking methodology from organizations that produce global indicators. In years two and three, LRP’s program was expected to achieve the following results: ● Increased awareness of new laws and reforms by the public, lawyers, judges, law students, professors, government officials and others. ● Improved research capacity, identification, installation and use of appropriate e-learning, e￾governance, online libraries, subscriptions, memberships to professional affinity groups, and other remote research and professional development resources. ● Increased quality and improved content of teaching methods and curriculum for basic and continuous education of judges, prosecutors and defense attorneys. ● Agency strategies, to include goals and benchmarks, established to measure progress towards achievement of improvements to rankings in global indicators. ● Improved technical knowledge and skill of Activity participants. The evaluation found LRP had several areas of strength in achieving rule of law and free legal aid objectives. These strengths, along with examples highlighted by KII and FGD respondents, are described in more detail in Annex D below. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 40 ANNEX D. EVALUATION FINDINGS ON LRP APPROACHES The evaluation found LRP had several areas of strength in achieving rule of law and free legal aid objectives. These strengths, along with examples highlighted by KII and FGD respondents, are described in more detail in the table below. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 41 Table 8: LRP Strengths and Examples Strength Context Successful Examples noted by KII/FGD Respondents Engaging Experts 26 out of 35 KII respondents considered engaging qualified national and international experts, legal profession practitioners, local champions and leaders the most useful and effective approach. It significantly influences the quality of legislation, training courses, and other LRP products; and assists in promoting changes in gender legislation through using social media to mobilize public support. NGO partners and grantees emphasized the role of international experts in assisting civil society groups in drafting appeals to higher-level authorities. ● LRP expert feedback was included in the final Entrepreneurship Code. ● LRP-supported 2022 research on the effectiveness of protecting women’s rights, preventing violence, and improving judicial practices in cases of GBV ● Involvement of Irina Matvienko, a prominent women’s human rights activist, and a member of the GoU Working Group. Support from the Vice President of the International Center for Not-for-Profit Law for Eurasian Countries helped fill skill gaps in drafting reports and appeals. Creating Discussion Patterns USAID representatives, NGO partners, and national partners consider this a helpful approach in increasing cooperation between public and private sectors and obtaining input on various rule of law topics. Discussion platforms in the form of roundtables, conferences, forums, and capacity-building events help civil servants and NGOs realize the necessity of joint actions to advance the development of rule of law and the policy-making process. ● The consolidation of multiple parties involved in the public participation process has led to the establishment of mechanisms such as regulatory impact assessment, anti￾corruption measures, and gender legal proficiency. Capacity-Building Activities National partners, NGO partners, LRP grantees, and training participants recognize training activities as an effective approach to enhance capacity, knowledge, and skills of government employees and legal professionals. The Training of Trainers (ToT) method, in particular, was praised for effectiveness and sustainability, facilitating learning and understanding of new concepts and empowering participants to teach and share their knowledge to colleagues. Legal professionals emphasized that LRP training provided them with practical knowledge and international guidelines on the provision of legal aid. All participants in FGDs with training participants affirmed that they utilize the acquired skills and knowledge in their daily tasks. Additionally, interpersonal connections were formed among the training participants. ● Fergana participants considered trainings more convenient than individual learning, as it is faster and allows for simultaneous learning of theory and practice of skills. ● Tashkent participants emphasized how trainings in intellectual property and gender audits deepened their understanding of the subject. ● Participants from various state institutions found the gender equality training highly beneficial for their application in their daily work. ● Samarkland FGD participants noted increased knowledge in methodologies for providing legal aid services. Free legal aid for vulnerable groups Vulnerable groups often face higher rates of legal illiteracy and have limited access to legal services to protect their rights. Enhancing legal literacy among vulnerable groups will empower them resolve their social and economic issues, ● The provision of free legal aid to socially and economically vulnerable and marginalized groups of the population is LRP's strongest and most significant asset, as noted by NGO partners, NGO grantees, and the IP. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 42 Strength Context Successful Examples noted by KII/FGD Respondents as well as protect their rights, and fostering legal culture among the general public will contribute to a more just and equitable society. Geographic Reach LRP’s geographic reach was described as a strength by 14 of 35 respondents across various stakeholder groups. Respondents highlighted the efforts of LRP in increasing legal literacy and providing free legal aid to disadvantaged groups in nearly all regions through its grant component. Access to legal services has significantly increased in regions mainly due to the organized field/portable services by lawyers. The capacity to offer legal consultations in regions was considered highly noteworthy and effective. ● In Fergana and Tashkent FGDs, female recipients of free legal aid emphasized the convenient location of legal services helped them save on expenses. ● LRP has provided equipment and technical support to human rights and business Ombudsmen, GPO Academy, and the Chamber of Advocates to ensure that they can provide services to remote areas. Research and training materials LRP’s research publications, training manuals, and book for people with special needs were described as instantly helpful and highly useful by the national partners, training beneficiaries, and NGO grantees. ● The handbook on parliamentary diplomacy and LRP materials on forms of violence against women were particularly successful. ● The Braille language book was considered a crucial resource that addresses visual impairment needs and provided information to navigate the legal system. Engaging and cooperation with local entities NGO grantees mentioned that engaging and cooperating with Mahalla Committees, and associations of people with special needs has been an effective approach in providing free legal aid to vulnerable groups. On-site consultations expand the reach and credibility of the grantees in target communities, fostering community confidence in the provided assistance. LRP supported legal aid centers in some areas further assist the population. ● Nearly all FGD participants indicated that they became aware of pro-bono legal aid services through their involvement with mahalla committees, Khokimiyats, and/or judicial bodies. Flexibility and adaptability LRP's strength lies in its flexibility and ability to provide the necessary support to the GoU, in line with the program's mandate. The program's long-term duration of 5 years, adequate funding, and ability to identify and address specific areas not covered by others were seen as advantageous, considering the political and legal environment in Uzbekistan. LRP's adaptability to the needs of national partners and beneficiaries allowed for meaningful and impactful work to be carried out. ● GoU respondents shared appreciation for LRP team's capacity to customize support based on the unique characteristics and needs of their national partners, transparency, and coordinated teamwork. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 43 Gaps and Limitations of LRP Approaches The evaluation did not find any significant weaknesses in the project’s approaches. However, certain gaps and limitations were identified, which are discussed in the table below. Table 9: LRP Limitations and Examples Gaps/ Limitations Context and Examples from KII/FGD Respondents Capacity￾building activities ● Some training events were conducted in Russian, which posed difficulties for some participants in understanding the concepts and content. Participants expressed a desire for training and materials to be offered in Uzbek, as not all are fluent in Russian. ● Most trainings took place in Tashkent, which affected participants ability to fully engage on the last day of the training. Study Tours ● USAID and IP believe that study tours have not been very successful in terms of follow￾up and have limited results in terms of changes in government bodies or legislation. USAID expressed reluctance to continue funding such study tours; however, with the project extension, there is a plan to shift the approach to focus on bringing representatives to countries with similar rule of law systems and a comparable political environment. ● Including high-ranking officials is an unspoken agreement. Excluding certain individuals from study tours may have an impact on those who would benefit and champion the ideas presented during the tours. ● National partners consider study tours highly beneficial for establishing connections with professionals, and providing an opportunity to analyze and understand diverse aspects that contribute to the formulation of new legislative proposals. However, national partners were unable to provide examples of actions as a result of study tours. NGO respondents highlighted a study tour to Georgia, which resulted in participants advocating for an NGO registration process as seen in Georgia. Tracking Legislative Outcomes and Data on Beneficiary Challenges ● While the project utilizes the publicly accessible legislative database to monitor the adoption of proposed legislative recommendations and amendments, there is no consistent and systematic method in place to tracking legislative outcomes in a document format.10 This could be attributed to the fact that the project’s outcome level indicator 1 may not effectively measure the desired changes in the legal landscape and instead aligns more with an output-level indicator.11 ● While satisfaction surveys are conducted, there is a lack of systematic data collection on the nature of complaints and specific barriers faced by vulnerable groups when applying for their rightful state benefits, disaggregated by age, gender, including individuals with disabilities, HIV, and other vulnerabilities. This is likely due to the absence of a monitoring requirement to disaggregate the data based on these factors. ● Collecting and analyzing such data could uncover the root cause of the challenges faced by citizens in accessing their state benefits and inform bureaucratic and legislative changes for efficient and equitable access to entitlements, leading to systematic and lasting improvements. Coordination ● Implementers described coordination as happening at the individual project level but emphasized the need for USAID or the Embassy to initiate broader coordination among 10 Interviews with project staff revealed that LRP-supported proposals were sometimes only partially incorporated into other regulations. For example, in the field of competition and entrepreneurship, LRP recommendations may be combined with other provisions during the adoption process. While there have been cases where LRP's legislative recommendations were fully adopted, as discussed earlier in the areas of gender, mediation, BIT, entrepreneurship, and civil law, there have also been instances where regulations were adopted with modifications and changes to their names. 11 LRP Narrative Monitoring, Evaluation and Learning Report, Attachment D, Annual Report October 1, 2021 – September 30, 2022, FY 2022. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 44 Gaps/ Limitations Context and Examples from KII/FGD Respondents all USG-funded sector-related programs. This would prevent duplication of project objectives and potential double funding of beneficiaries, which is currently considered problematic. ● One implementer indicated that while there is some inter-project coordination, mostly in the gender equality component, there are still opportunities for further coordination in this area. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 45 ANNEX E. EVALUATION METHODS AND LIMITATIONS Further elaboration on the evaluation methods and limitations mentioned in the report can be found below: CO-CREATION MEETINGS Data collected during the co-creation meetings informed the evaluation codebook that the ET used to ensure that each word in each EQ has a single, distinct, and meaningful definition. Additionally, the meetings helped to identify the guiding principles for selecting focal LRP Objectives and legislation focus, so that the ET provided a deeper and more rigorous evaluation than would otherwise be feasible. Finally, it added nuance in identifying potential data sources and discussed methodologies and outreach approaches that may be used to engage key stakeholders in a sensitive way. LRP OBJECTIVE PRIORITIZATION Within each LRP objective, the ET used the following criteria to identify the focal objectives for this evaluation: 1. Priority learning objectives: The first step was to prioritize LRP components that are pertinent to the Mission’s priority learning objectives, particularly as they relate to informing current and future programmatic strategy. The Mission expressed the importance of learning the successes and challenges of a) supporting rule of law development (Objective 1) as it will help the Mission to engage the GoU more effectively in legislation reforms; and b) the development of a legal environment for CSOs/NGOs (Objective 3), as the Mission considers it potentially impactful, particularly as a component of the provision of Free Legal Aid by NGOs supported by LRP. 2. Scope: The ET reviewed the scope and intensity of the activities/components and considered which ones would lend themselves to meaningful evaluation. For example, Objective 2 consisted of large, one-off activities;12 because of the expansive scope of these activities, these types of interventions were a lower priority to include in the evaluation scope. 3. Relevance of the evaluation questions: Lastly, the activities of LRP’s Objective 2 are not relevant in addressing the evaluation questions; therefore, Objective 2 is a lower priority in this evaluation scope. Based on the above discussed Mission priorities, scope, and relevance for addressing the EQs, the ET focused on evaluating LRP’s Objective 1 and Objective 3. LRP LEGISLATIVE FOCUS SELECTION The LRP’s legislative development efforts are large in scope. Given the limited time and resources, the evaluation could not sufficiently cover all 25 legislative activities, which involve diverse partners and approaches. As a result of the co-creation meetings, combined with the preliminary findings from the desk review, the ET focused on the following priority legislation topics identified based on the level of perceived success and learning priorities: entrepreneurship (large success), intellectual property rights (moderate success), and legal framework for NGOs (limited success). Another priority focus highlighted by the Mission is gender legislation, as it resulted in some changes in the practical application of two gender equality laws and has become a mandatory regulation in laws and 12 Example Objective 2 activities include providing equipment for distance learning, conferencing and office equipment, capacity development training for various training institutions and government offices, and development of web portals, textbooks, and manuals for legal educators and students. GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 46 procedures, both on local and national levels. Therefore, within LRP’s Objective 1, legislative activities were focused on gender, intellectual property rights, entrepreneurship, and NGO legal frameworks. SITE SELECTION The LRP project has activities across nearly the whole country. However, available time and resources were insufficient to facilitate data collection at all implementation sites. In consultation with USAID/Uzbekistan and the IP, combined with the ET’s review of key project documents, the ET focused on the Tashkent, Fergana, and Samarkand regions for primary data collection. The ET purposely selected the proposed sites based on the following criteria: ● Large representation of LRP partners and level of its main activities ● Level of LRP’s main activities implementation ● Greater demand for legal aid services, as these regions have high-density populations ● Diversity of legal aid issues faced by populations as these regions are geographically situated in international cross-border areas ● “Unrestricted” regions for LRP grantees to operate in, unlike recently enacted restrictions in the Karakalpakstan Republic KEY INFORMANT INTERVIEWS The KIIs provided the core sources of qualitative data collection and facilitated a deeper understanding of processes, issues, and contexts that impact the performance and results of LRP initiatives to support government reforms. KIIs focused on the successes and challenges of the project’s interventions in supporting the development of rule of law and provision of free legal aid by NGOs supported by LRP. Similarly, KIIs identified factors that support and limit LRP contribution to achieving its Objectives 1 and 3. To address EQ2, KIIs examined the effectiveness of LRP contributions in amending current legislation and/or adopting new legislation, as well as exploring the weak areas that need more support from the GoU to LRP’s effort in assisting legislative reform. Likewise, in addressing the EQ3, KIIs helped to assess the successes and challenges of factors that support or limit the grants component and its contribution to the creation of a legal-enabling environment for NGO development and improved interaction between the state and civil society. KIIs explored the perceptions of key stakeholders about the LRP benefits to the development of the rule of law, the adequacy of the GoU’s support to the LRP legislative assistance efforts, the quality and utilization of LRP’s support by the GoU, and the effectiveness of the grants component in strengthening organizational capacity of NGOs. FOCUS GROUP DISCUSSION Discussions with free legal aid recipients focused on their feedback in relation to received services and experiences, such as relevance, accessibility, and quality of provided services, as well as the impact of services in meeting their legal needs. In each focus region, NBT conducted at least one FGD with female legal aid recipients to ensure the comfortable and active engagement of each participant in the discussion. This helped to identify the differentiated needs and interests of vulnerable groups in obtaining legal aid. Discussions with LRP training participants were held around the effectiveness of training courses in increasing the capacity, strengths, weaknesses, and gaps of approaches/methods, value, and the extent of the practical application of obtained knowledge and skills in their professional activities. The ET consulted with LRP staff about lists of current and former free legal aid recipients and GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 47 capacity-building attendees for the sampling strategy. Figure 1: Map of FGD Locations SOURCE: GOOGLE MAPS PHASE III ANALYSIS AND REPORTING The ET performed the following analysis as described below: ● Rolling analysis. The ET populated a Findings, Conclusions, and Recommendations (FCR) Matrix to aggregate findings during data collection. The FCR matrix ensures that the ET is collecting data that systematically and thoroughly answer each evaluation question; verifies that preliminary analysis accounts for gender and social dimensions; identifies gaps where additional clarification or analysis may be necessary; and ensures that each recommendation is supported by evidence. ● Content and comparative analysis. From detailed notes taken during data collection, the ET utilized a structured coding approach to conduct content and thematic analysis using Excel software in a tally sheet structure for identification of emergent themes and contextual factors upon which to draw findings and conclusions and to develop evidence￾based recommendations. ● Gap analysis. The ET identified which aspects of LRP activities, if any, fell short of anticipated performance, and the likely reasons/factors behind these gaps. ● Data triangulation. The KII protocol and FGD guides included similar questions to allow for greater triangulation across sources – this helped to identify points of convergence and divergence and reasons behind these differences to come up with findings. The ET also triangulated desk review sources with data collected during the KIIs and FGDs to assess trends, consistency, affinity, and contradiction between data sources, aiming to better answer the EQs, generate more reliable findings, and reach well-founded conclusions. LIMITATIONS The evaluation’s most important limitations and the ET’s mitigation measures are presented below: GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 48 a. Data availability and data quality. There are two main challenges associated with remote and in-person data collection, namely: i. Availability/willingness of state officials and NGOs to participate. Given the time restraints of state officials and the potential for NGOs to be unwilling to participate due to their mistrust of respondent anonymity, the ET experienced challenges with participation. ii. Remote data collection was complicated by time differences, connectivity, and reception issues. Furthermore, the inability of the ET to conduct all interviews in person impeded the ability to build rapport and trust before the interview. To mitigate both of these challenges, the ET followed careful data collection preparations including formal government protocols and provided clear communication to respondents on the evaluation well in advance of fieldwork starting. The ET built in flexibility to the data collection schedule and utilized multiple methods (calling, utilizing the IP to support communications) to ensure the participation of key informants. a. Selection bias. A core aspect of an evaluation’s validity is the extent to which it can generalize findings beyond the data collection sample. The representativeness of the sample is constrained in three important ways. First, there are logistical and budget constraints. The ET did not collect in-person primary data in all regions of LRP activity implementation. As specific operational contexts may vary across regions, the locations selected for primary data collection do not necessarily reflect the entirety of LRP’s programming experience (i.e., there may be areas included in primary data collection where implementation was either more or less successful than others). Second, there was a fair degree of non-response. These challenges were mitigated through a combination of purposive sampling to ensure representation from key cross-sections of respondent populations, flexibility in scheduling interviews, and the identification of replacements in the sample selection. b. Recall bias. The evaluation covers a study period of over four years (LRP was awarded in October 2018), with many activities (current and completed) that address three objectives, implemented within a complex operating environment. As with any study, the further back the ET asked respondents to recall, the more likely and significant was the possibility of errors in memory. This could manifest in respondents saying they do not remember or respondents answering questions erroneously. Similarly, for partners that implemented multiple types of support over the years, recall for more recent activities is expected to be more accurate than for older support. These challenges were mitigated by triangulating findings against secondary sources (e.g., program documentation) as well as from other interviews. Furthermore, when interviewers were asking questions from a relatively long time ago, they used historical anchors to help respondents remember (i.e., important developments at the time that can help job respondent memory). c. Response bias. Every evaluation must contend with certain respondents not being fully forthright in their answers or misreporting their responses to questions, resulting in findings being based on the subjective perceptions of key informants. The challenge of respondents misrepresenting or omitting information is particularly acute in studies that deal with sensitive topics and/or with sensitive populations. Finally, there is often a perverse incentive on the part of beneficiaries to present a positive reflection of programming in the hope of continued support (whether through current or future programs). These challenges were mitigated through the use of local researchers familiar with the context and content; detailed consent forms that explained the role of the independent evaluator and the anonymity of respondents vis-à-vis evaluation findings; and triangulation of data (as noted above). d. Social desirability bias. This refers to the tendency of respondents to overreport more desirable attributes. For example, with the knowledge that evaluation results can inform the future funding or direction of an activity or of the Mission portfolio, respondents may have GS-10F-0033M /7200AA18M00016, Tasking N078 USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 49 provided evaluators with positive feedback because they want activities to continue. Likewise, activity staff may have overemphasized the activity’s successes in order to reflect well on their own performance. The team mitigated this type of bias by emphasizing to participants the desire to both hear about what has worked well in addition to what the challenges/weaknesses have been. The ET was clear in the informed consent that there are no direct benefits to respondents from participating in the evaluation and that there will be no retaliation against them for their responses. e. Halo effect: As mentioned above, there is a tendency among respondents to under-report socially undesirable answers and adjust their answers to approximate what they perceive as the social norm – this is called halo bias/effect. In the case of this evaluation, halo effect may have manifested if respondents (possibly free legal aid recipients and/or capacity-building activity participants) were not willing to reveal information that might reflect negatively on the implementer and/or government. However, in the context of the evaluation design outlined above, this limitation did not have significant impact as the ET obtained information from a variety of stakeholders and ensured anonymity in reporting their responses. USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 50 ANNEX F. DATA COLLECTION INSTRUMENTS CONSENT FORM FOR KEY INFORMANT INTERVIEWS Introduction: Thank you for taking the time to speak with us today. We are researchers from Social Impact (SI), an independent research firm or NBT Consulting, an independent data collection firm. We have been contracted by USAID to conduct a performance evaluation of the Legal Reform Program (LRP). Our task is to assess the performance of the LRP activities and the effectiveness of the mechanism and approaches to improve legal reform and rule of law development through information collected. You have been selected to participate in this research because of your knowledge of USAID LRP activities in Uzbekistan. Today’s interview is expected to last approximately one hour, and we will discuss USAID LRP activities that you are familiar with and how they contributed to legal reform and rule of law development in Uzbekistan. Risks and benefits: We do not anticipate that you will incur any risk or direct benefit from participating in this interview. Confidentiality: Your responses in this interview will be kept in confidence by SI and the researchers. If you give your consent to record, the interview will be recorded only for note-taking purposes to check for accuracy of responses and will be promptly deleted once notes have been finalized. Only SI staff and the research team will have access to the notes that are taken. Nothing that you say will be attributed to you unless you specifically authorize us to do so. However, for purposes of documenting evidence, statements by interviewees may be attributed to a category of interviewee, such as USAID staff, foreign IP, civil society or external expert. Voluntary participation: Your participation is voluntary. If you do not want to participate or to answer specific questions you do not have to. Should you choose to participate, please know that you may change your mind at any point during our discussion. There will be no consequences to your decision not to participate. Report: We expect to speak with approximately 40-60 people as part of this research. The information that you and others provide will be used to write a report. This report will be shared with USAID and other stakeholders for comment and eventually be made public. If you have any concerns, you may contact the Social Impact Institutional Review Board at irb@socialimpact.com with questions about the study or results. You can also contact the study team members Madina Khalmirzaeva (madina.khalmirzaeva@nbt.uz), Catherine Caligan (ccaligan@socialimpact.com), or Seara Grundhoefer (sgrundhoefer@socialimpact.com). Do you have any questions for us before we get started? [Researcher to answer any questions] Are you willing to participate in this interview? Yes__ (start interview) No__(do not start until provided) Are you willing to be audio-recorded in this interview? Yes__(start recorded interview) No__ USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 51 USAID/UZBEKISTAN INTERVIEW PROTOCOL Date Interviewer Notetaker Respondent Name Respondent Position, Organization Respondent Gender 1. Female____ 2. Male ____ READ CONSENT STATEMENT AND OBTAIN CONSENT FROM ALL RESPONDENTS. IF CONSENT TO PARTICIPATE IS NOT GIVEN, STOP THE INTERVIEW AND DO NOT PROCEED. Introduction 1. To begin, can you tell us a bit about your involvement with LRP? [Probe: role, duration, intensity, level of involvement] 2. Were you involved in the design of LRP? If so, does the project as implemented today differ in any significant way? If so, how, and why? 3. Has LRP activity been equally successful in achieving its stated objectives, or have results under a particular objective been stronger compared to others? Why? [If needed, list LRP objectives: 1) support rule of law development, 2) development of the legal profession, 3) development of legal environment for CSOs/NGOs, and 4) cross-cutting gender] EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? 4. To your knowledge, how does LRP define rule of law development? If unaware, how would you define rule of law development? [NOTE TO INTERVIEWER: keep handy the definition] 5. What changes have you seen in the ways legislation and other aspects of rule of law have developed since LRP began in 2018? Do you feel LRP has contributed to these changes? If so, how? 6. In your opinion, what have the LRP’s successes been in supporting rule of law development so far? What factors do you think contributed to these successes? 7. What have LRP’s biggest challenges been in supporting rule of law development? Do you know how those challenges were resolved? 8. What is your reflection on the provided free legal aid by LRP-supported grantees? a. Do you think the provision of free legal aid to the eligible population contributes to building sustainable and democratic rule of law system/culture? If so, why? If not, why not? Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 52 Rule of law development 9. In your opinion, what specific approaches that are employed by LRP bring leverage to achieving its rule of law development objective? Why? [List approaches to remind range of activities: hires experts to review, analyze, develop/modify legislation, develop recommendations to the law-making bodies, public hearings, enhance capacity of legal specialists in legislative drafting, develops databases and website to aid citizens to prepare correct legal docs] a. What would you say are the strongest points/assets of LRP approaches/activities? Can you please provide an example? 10. If any, what are the weaknesses/limitations of LRP’s approaches/activities for achieving its rule of law development objective? a. To your knowledge, what strategies does the project use to mitigate these weaknesses/limitations? Can you provide any examples? 11. Are there any other approaches/activities that LRP does not employ/use but that you believe would be more effective in achieving its rule of law development objective? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? 12. What would you do differently to improve LRP’s support to the GoU in building a more sustainable and democratic rule of law system and culture in the country? a. Are there any areas that you would rather target for LRP’s focus/activities? If yes, what areas? Why? b. What current LRP focus areas and approaches would you keep the same? Why? Provision of free legal aid 13. In your opinion, what are the strongest points/assets of LRP’s approaches in the provision of legal aid to vulnerable groups through the project-supported NGOs? Why? Can you provide an example? 14. Are there any limitations/weaknesses in LRP’s approaches in the provision of legal aid to vulnerable groups by the supported NGOs? If yes, what are they? a. If aware, what strategies does LRP and/or its NGO grantees use to mitigate these weaknesses/limitations? Can you provide any examples? 15. Are there any other approaches/activities that LRP or its NGO grantees do not employ/use, but that you believe would be more effective in the provision of legal aid to vulnerable groups? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 53 EQ2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or adoption of new legislation? 16. If any, what changes have you seen in the ways GoU requests support from LRP for moving forward its legislative reform since the project began in 2018? a. In your opinion, what are the GoU’s main drivers/motivations in engaging/cooperating with LRP in moving forward legislative reform? b. Are there any obstacles that minimize the GoU’s interest in collaborating with LRP? If yes, what are they? c. What factors do you think influence adequate GOU support to LRP’s efforts in fulfilling its legislation-related requests? d. Do you think the level of GoU’s support to LRP’s legislative assistance efforts depends on the type or topic of legislation? If so, why? 17. How do you feel that LRP has contributed to GoU requests to support amendments to the current legislation and/or the adoption of new legislation? a. In your opinion, what were the LRP’s biggest successes in providing legislative support to GoU? Can you please provide an example? b. If aware, what is your reflection on the quality of provided legislative support to GoU? 18. Are there any specific challenges that the project faced/faces in delivering legislative support to GoU? If so, what are these challenges? a. If aware, what strategies has the project employed to mitigate those challenges? EQ3: To what extent did the grants program contribute to the development of the legal￾enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? 19. What changes have you seen in the ways NGOs interact with state institutions and participate in legislative processes since LRP began in 2018? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? 20. In your opinion, what have the LRP’s grants program successes been in the development of the legal-enabling environment and in strengthening the organizational capacity of NGOs? What factors do you think contributed to these successes? 21. What have the LRP’s grants program challenges been in the development of the legal-enabling environment and in strengthening the organizational capacity of NGOs? Do you know how those challenges were resolved? Conclusion USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 54 22. Is there any innovative rule of law programming that USAID does not involve in Uzbekistan, but that you believe would be valuable to consider for helping the host country to build a more sustainable and democratic rule of law? 23. Knowing what you know today, is there anything you would have done differently in working on this project? 24. Is there anything else you would like to add or recommend that we include in our findings? Thank you for your time! USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 55 IMPLEMENTING PARTNER: TT DPK INTERVIEW PROTOCOL Interviewer Notetaker Respondent Name Respondent Position Respondent Gender 1. Female____ 2. Male ____ READ CONSENT STATEMENT AND OBTAIN CONSENT FROM ALL RESPONDENTS. IF CONSENT TO PARTICIPATE IS NOT GIVEN, STOP THE INTERVIEW AND DO NOT PROCEED. Introduction 1. To begin, can you tell us a bit about your role and level of involvement in LRP? 2. Were you involved in the design of LRP? If so, does the project as implemented today differ in any significant way? If so, how, and why? 3. Has LRP activity been equally successful in achieving its stated objectives, or have results under a particular objective been stronger compared to others? Why? EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 4. To your knowledge, how does LRP define rule of law development? If unaware, how would you define rule of law development? [NOTE TO INTERVIEWER: keep handy the definition] 5. What changes have you seen in legislation and other aspects of rule of law development since LRP began in 2018? Do you feel that LRP has contributed to these changes? If so, how? 6. In your opinion, what have the LRP’s biggest successes been in supporting rule of law development so far? What factors do you think contributed to these successes? 7. What have LRP’s biggest challenges been in supporting rule of law development? Do you know how those challenges were resolved? Legal aid provision 8. What changes have you seen in the ways legal aid is provided to the population since LRP began in 2018? Do you feel that LRP has contributed to these changes? If so, how? 9. In your opinion, what have the LRP’s biggest successes been in the provision of free legal aid to the eligible population by NGO grantees so far? Can you provide an example? a. What factors do you think contributed to these successes? 10. What are the major challenges in the provisions of free legal aid to the eligible population? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 56 a. What has been done by LRP and/or its grantees take to mitigate these challenges? Can you provide an example? Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 11. In your opinion, which specific LRP’s approaches/activities contribute the most to achieving its rule of law development objectives? Why? a. What would you say are the strongest points/assets of these approaches? Can you please provide an example? 12. Are there any weaknesses or limitations in the currently employed approaches by LRP in achieving its rule of law development objective? If yes, what are they? a. What has been done to minimize these weaknesses/limitations? 13. What would you change/adjust about the currently employed approaches in the achieving rule of law development objective? Why? a. Are there any external and/or internal barriers to adjusting the currently employed approaches? 14. Are there any other approaches/activities that the project does not employ/use currently, but that you believe would be more effective in achieving its rule of law development objective? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? Legal aid provision 15. Do you think the provision of free legal aid to the eligible population contributes to building a sustainable and democratic rule of law system/culture? If so, why do you think so? If not, why not? 16. In your opinion, what are the strongest points/assets of LRP’s approaches in the provision of legal aid to vulnerable groups through NGO grantees? a. What are the advantages of grant distributions in the provision of legal aid to vulnerable groups? b. What are the most important skills and expertise that grantee legal specialists should have for the effective provision of legal aid and identifying eligible clients for the service? c. Have you/LRP staff conducted any training to improve/strengthen these skills or expertise? If yes, what training activities were most effective? What makes you think so? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 57 17. Are there any limitations/disadvantages to LRP’s approach in the provision of legal aid to vulnerable groups via NGO grantees? If yes, what are they? a. If aware, what strategies do LRP and/or its NGO grantees use to mitigate these weaknesses/limitations? Can you provide any examples? 18. Which NGO grantees would you say are most successful in the provision of legal aid to the eligible population? Why? a. What makes these NGO grantees more successful compared to other grantees? Which specific approach/aspect of their work do you think contributes to this success? 19. Are there any other approaches/activities that LRP or its NGO grantees do not employ/use, but that you believe would be more effective in the provision of legal aid to vulnerable groups? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? EQ2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or adoption of new legislation? 20. How successful do you think LRP has been in providing support to GoU’s requests for legislative assistance? What makes you think so? 21. What were the LRP’s biggest successes in providing legislative support to GoU? Can you please provide an example? a. What factors do you think contributed to these successes? 22. Are there any specific challenges that LRP faced/faces in delivering legislative support to GoU? If so, what are these challenges? a. What strategies has the project employed to mitigate those challenges? 23. Has LRP established any processes/procedures to ensure that analytical, legislative, educational, informational, and other products developed by the project are of good quality? If so, what are they? 24. How does LRP ensure that submitted recommendations for improving the current or adopting new legislation are accepted/adopted by law-making bodies and/or relevant government institutions? 25. If any, what changes have you seen in the ways GoU requests support from LRP for moving forward its legislative reform since the project began in 2018? a. In your opinion, what are the GoU’s main drivers/motivations in engaging/cooperating with LRP in moving forward legislative reform? b. Are there any obstacles that minimize the GoU’s interest in collaborating with LRP? If yes, what are they? Can you provide an example? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 58 c. What factors do you think influence adequate GOU support to LRP’s efforts in fulfilling its legislation-related requests? d. Do you think the level of GOU’s support to LRP’s legislative assistance efforts depends on the type or topic of legislation? If so, why? 26. Are there any other approaches/activities that LRP does not employ/use but that you believe would be more effective in strengthening cooperation with the government? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? 27. What would you do differently to improve LRP’s support to the GoU in building a more sustainable and democratic rule of law system and culture in the country? a. Are there any areas that you would rather target for LRP’s focus/activities? If yes, what areas? Why? b. What current LRP focus areas and approaches would you keep the same? Why? EQ3: To what extent did the grants program contribute to the development of the legal￾enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? Grants program contribution to the development of legal-enabling environment 28. Have you noticed any changes in the ways NGOs interact with state institutions and participate in legislative processes since the start of the LRP in 2018? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 29. Has the LRP’s grants program improved the existing or established new mechanisms of interaction between the state and NGOs? If yes, could you please specify these mechanisms? a. Have NGOs been using these mechanisms? Can you provide any examples? 30. Which specific aspects of LRP’s grants program have been most effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? Why? 31. Which specific aspects of LRP’s grants program have been least effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? Why? 32. Are there any other approaches/activities that LRP does not employ/use in its grants program, but that you believe would be more effective for the development of the legal-enabling environment for CSO and NGOs? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 59 a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? Grants program contribution to strengthening organizational capacity 33. Have you noticed any changes in the capacity of NGOs to manage, monitor, report, and provide services and sustain activities since LRP began in 2019? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 34. What are the strongest points/assets of LRP’s grants program approaches in strengthening the organizational capacity of NGOs and improving NGO sustainability prospects? Why? a. Has LRP under its grants program conducted any training for NGOs to strengthen their organization management skills and capacity to sustain? If yes, what training activities were most effective? What makes you think so? 35. Are there any weaknesses/limitations in LRP’s grants program approaches in strengthening the organizational capacity of NGOs? If yes, what are they? a. What strategies does the LRP grants program use to minimize these weaknesses/limitations? Can you provide an example? 36. Are there any other approaches/activities that LRP’s grants program does not employ/use, but that you believe would be more effective in strengthening the organizational capacity of NGOs? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? Conclusion 37. What would you do differently to improve LRP’s support to the GoU in building a more sustainable and democratic rule of law system and culture in the country? a. Are there any areas you would rather target for LRP’s focus/activities? If yes, in what areas? Why? b. What current LRP focus areas and approaches would you keep the same? Why? 38. Is there any innovative rule of law programming that USAID does not include in Uzbekistan, but that you believe would be valuable to consider for helping the host country to build a more sustainable and democratic rule of law? 39. Knowing what you know today, is there anything you would have done differently in working on this project? Are there any lessons you have learned during implementation of LRP? 40. Is there anything else you would like to add or recommend that we include in our findings? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 60 Thank you for your time and cooperation! USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 61 GOVERNMENT OF UZBEKISTAN INTERVIEW PROTOCOL Date Interviewer Notetaker Respondent Position Organization Respondent Gender 1. Female____ 2. Male ____ READ CONSENT STATEMENT AND OBTAIN CONSENT FROM ALL RESPONDENTS. IF CONSENT TO PARTICIPATE IS NOT GIVEN, STOP THE INTERVIEW AND DO NOT PROCEED. Introduction 1. Please tell us a little bit about your organization’s experience receiving assistance from the USAID￾supported Legal Reform Program (LRP). a. How did your relationship with LRP begin? b. What specific support did you/your organization receive from LRP? EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 2. What major challenges did Uzbekistan face in making progress in ongoing legal reforms to build a democratic rule of law system and culture five years ago? What is different about the challenges it faces today, if any? 3. In your opinion, has the LRP support you described contributed to addressing these challenges? If yes, in what way? If not, why not? 4. Is there any support that could help you address these challenges better? If so, what? 5. Since your organization began receiving support from LRP, how have legislation development practices and processes, including public participation in legislative hearings of changes to civil and administrative legislation, changed? a. To what degree would you attribute these changes to LRP support to your organization? Provision of free legal aid USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 62 6. Since LRP began in 2018, have you noticed any changes in the access of women, men, youth, minorities, and vulnerable groups to legal aid? Do you feel that LRP has contributed to these changes? If so, how? Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 7. Do you think the LRP activities in which you participated or were involved helped your organization contribute to the development of rule of law and/or to making the legislative process more transparent and accountable in alignment with international standards and best practices? a. If yes, could you please elaborate on how? b. Which specific LRP activities/approaches do you think were most successful? Why? c. Which specific LRP activities/approaches do you think were least successful? Why? 8. Which specific LRP activities that you participated in or were involved in do you consider most beneficial for accomplishing your work? Why? 9. If any, what are the weak points/limitations of LRP approaches/activities in supporting your organization and contributing more effectively to achieve its objective on the development of rule of law? 10. Are there any other approaches/activities that LRP does not employ/use but that you believe would be more effective in achieving its rule of law development objective? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? Provision of free legal aid 11. In your opinion, what are the strongest points/assets of LRP’s approaches in the provision of legal aid to vulnerable groups? Why? Can you provide an example? 12. Are there any limitations/weaknesses in LRP’s approaches in the provision of legal aid to vulnerable groups? If yes, what are they? 13. Are there any other approaches/activities that LRP or its NGO grantees do not employ/use, but that you believe would be more effective in the provision of legal aid to vulnerable groups? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 63 EQ2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or the adoption of new legislation? 14. In your opinion, how well does LRP support or complement the GoU’s legislation reform efforts? 15. What were the LRP’s biggest successes in providing legislative support to GoU? Can you please provide an example? a. What factors do you think contributed to these successes? 16. What is your reflection on the quality of provided legislative support to GoU by LRP? 17. In your opinion, has the GoU’s support of LRP’s effort in fulfilling legislation-related requests been sufficient? What factors do you think influence adequate GOU support to LRP? 18. How do you think LRP could have done better in helping rule of law development and making the legislative process more transparent and accountable? Conclusion 19. Knowing what you know today, is there anything you would have done differently in working on this project? Are there any lessons learned during your participation in LRP? 20. Is there anything else you would like to add or recommend that we include in our findings? Thank you for your time and cooperation! USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 64 NGO PARTNERS AND TECHNICAL EXPERTS INTERVIEW PROTOCOL Date Interviewer Notetaker Organization Respondent Position Respondent Gender 1. Female____ 2. Male ____ READ CONSENT STATEMENT AND OBTAIN CONSENT FROM ALL RESPONDENTS. IF CONSENT TO PARTICIPATE IS NOT GIVEN, STOP THE INTERVIEW AND DO NOT PROCEED. Introduction 1. To begin, can you tell us about your involvement with the USAID-supported Legal Reform Program (LRP)? [Probe: role, duration, intensity, level of involvement] a. On what specific issues have you/your organization partnered with LRP or what specific support have you/your organization provided to or received from LRP? EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 2. What changes have you seen in the ways legislation and other aspects of rule of law have developed since LRP began in 2018? Do you feel LRP has contributed to these changes? If so, how? 3. In your opinion, what have the LRP’s successes been in supporting rule of law development so far? What factors do you think contributed to these successes? 4. If aware, what have LRP’s biggest challenges been in supporting rule of law development? Do you know how those challenges were resolved? Legal aid provision 5. Since LRP began in 2018, have you noticed any changes in the access of women, men, youth, minorities, and vulnerable groups to legal aid? Do you feel LRP has contributed to these changes? If so, how? 6. In your opinion, what have the LRP’s biggest successes been in the provision of free legal aid to the eligible population? Can you provide an example? a. What factors do you think contributed to these successes? 7. What have the LRP’s major challenges been in the provision of free legal aid to the eligible population? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 65 Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 8. In your opinion, which specific LRP approaches/activities contribute the most to achieving its rule of law development objectives? Why? [List approaches to remind range of activities: hires experts to review, analyze, develop/modify legislation, develop recommendations to the law-making bodies, public hearings, enhance capacity of legal specialists in legislative drafting, develops databases and website to aid citizens to prepare correct legal docs] a. What would you say are the strongest points/assets of these approaches? Can you please provide an example? 9. Are there any weaknesses or limitations in the currently employed approaches by LRP in achieving its rule of law development objective? If yes, what are they? 10. What would you change/adjust about the currently employed approaches by LRP in achieving the rule of law development objective? Why? Are there any external and/or internal barriers to adjusting the currently employed approaches? What are they? 11. Are there any other approaches/activities that the project does not employ/use currently, but that you believe would be more effective in achieving its rule of law development objective? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? Legal aid provision 12. Do you think the provision of free legal aid to the eligible population contributes to building a sustainable and democratic rule of law system/culture? If so, why do you think so? If not, why not? 13. In your opinion, what are the strongest points/assets of LRP’s approaches in the provision of legal aid to vulnerable groups? Why? 14. Are there any limitations/disadvantages to LRP’s approach in the provision of legal aid to vulnerable groups? If yes, what are they? a. If aware, what strategies does LRP use to mitigate these weaknesses/limitations? Can you provide any examples? 15. Are there any other approaches/activities that LRP does not employ/use, but that you believe would be more effective in the provision of legal aid to vulnerable groups? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 66 EQ2: How effective has LRP support been to GoU requests for amendments to the current legislation and/or adoption of new legislation? 16. How successful do you think LRP has been in providing support to GoU’s requests for legislative assistance? What makes you think so? 17. What were the LRP’s biggest successes in providing legislative support to GoU? Can you please provide an example? a. What factors do you think contributed to these successes? 18. If aware, are there any specific challenges that LRP faced/faces in delivering legislative support to GoU? If so, what are these challenges? Why do you think LRP encounter these challenges? a. If aware, what strategies has LRP employed to mitigate those challenges? 19. Are you aware of any processes/procedures established by LRP to ensure that analytical, legislative, educational, informational, and other products developed by the project are of good quality? If so, what are they? 20. Are you aware of any process that LRP uses to ensure that submitted recommendations for improving the existing or adopting new legislation are accepted/adopted by law-making bodies and/or relevant government institutions? If so, can you please elaborate on how LRP uses this process? 21. If any, what changes have you seen in the ways GoU requests support from LRP for moving forward its legislative reform since the project began in 2018? a. In your opinion, what are the GoU’s main drivers/motivations in engaging/cooperating with LRP in moving forward legislative reform? b. Are there any obstacles that minimize the GoU’s interest in collaborating with LRP? If yes, what are they? Can you provide an example? c. What factors do you think influence sufficient GoU support for LRP’s efforts in fulfilling its legislation-related requests? d. Do you think the level of GoU’s support to LRP’s legislative assistance efforts depends on the type or topic of legislation? If so, why? 22. Are there any other approaches/activities that LRP does not employ/use but that you believe would be more effective in strengthening cooperation with the government? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? b. What could be the weaknesses? EQ3: To what extent did the grants program contribute to the development of the legal￾enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? Grants program contribution to the development of legal-enabling environment USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 67 23. What changes have you noticed in the ways your organization/NGOs interact with state institutions and participate in legislative processes since the start of the LRP in 2018? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 24. In your opinion, has the LRP’s grants program improved the existing or established new mechanisms of interaction between the state and NGOs? If yes, could you please specify these mechanisms? a. Have your organization/NGOs been using these mechanisms to interact with state institutions? If yes, can you provide any examples? If not, why not? 25. Which specific aspects of LRP’s grants program have been most effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? What makes you think so? 26. Which specific aspects of LRP’s grants program have been least effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? Why? 27. Are there any other approaches/activities that LRP does not employ/use in its grants program, but that you believe would be more effective for the development of the legal-enabling environment for CSO and interaction between the state and NGOs? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? Grants program contribution to strengthening organizational capacity of NGOs 28. Have you noticed any changes in the capacity of your organization/NGOs to manage, monitor, report, or provide legal services and sustain activities since the start of LRP’s grants program in 2019? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 29. In your opinion, what are the strongest points/assets of LRP’s grants program approaches in strengthening the organizational capacity of NGOs and improving NGO sustainability prospects? Why? 30. Have you participated in any training sessions supported or conducted by LRP’s grants program? a. If not, INTERVIEWER GO TO QUESTION 31 b. If yes, which training activities did you attend? [EXAMPLES: organizational capacity-building, provision of legal aid to vulnerable groups, participation in legislative discussions, other] c. Do you consider LRP’s grants program training activities to be effective? Why? What makes them effective? d. Have you used what you’ve learned in your daily work? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 68 e. If yes, which specific knowledge or skill that you obtained at the LRP’s training do you consider most beneficial and useful for your work? Which is least beneficial and useful for your work? Why? 31. Are there any weaknesses/limitations in LRP’s grants program approaches in strengthening the organizational capacity of NGOs? If yes, what are they? 32. Are there any other approaches/activities that LRP’s grants program does not employ/use, but that you believe would be more effective in strengthening the organizational capacity of NGOs? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? Conclusion 33. What would you do differently to improve support for the GoU in building a more sustainable and democratic rule of law system and culture in the country? a. Are there any areas you would rather target for LRP’s focus/activities? If yes, what areas? Why? b. What current LRP focus areas and approaches would you keep the same? Why? 34. Is there any innovative rule of law programming that USAID does not include in Uzbekistan, but that you believe would be valuable to consider for helping the country to build a more sustainable and democratic rule of law? 35. Knowing what you know today, is there anything you would have done differently in cooperating with LRP? Are there any lessons that you have learned during your participation in LRP? 36. Is there anything else you would like to add or recommend that we include in our findings? Thank you for your time and input! USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 69 LRP GRANTEES INTERVIEW PROTOCOL Date Interviewer Notetaker Organization Respondent Position Respondent Gender 1. Female____ 2. Male ____ Region READ CONSENT STATEMENT AND OBTAIN CONSENT FROM ALL RESPONDENTS. IF CONSENT TO PARTICIPATE IS NOT GIVEN, STOP THE INTERVIEW AND DO NOT PROCEED. Introduction 1. To begin, can you tell us about your involvement with the USAID-supported Legal Reform Program (LRP)? [Probe: role, duration, intensity, level of involvement] 2. We understand that your organization is an awardee of LRP’s grants program. If this correct, what project activities has your organization implemented under this grant and when? [EXAMPLES: Grants were awarded to provide free legal aid to vulnerable groups, develop amendments to the existing legislation, exercise citizen oversight of government institutions, build capacity of journalists and media, provide legal assistance to journalists, and implement various awareness-raising activities] EQ1: To what extent has LRP supported the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 3. What changes have you seen in the ways legislation and other aspects of rule of law have developed since LRP began in 2018? Do you feel LRP has contributed to these changes? If so, how? 4. In your opinion, what have the LRP’s successes been in supporting rule of law development so far? What factors do you think contributed to these successes? 5. If aware, what have LRP’s biggest challenges been in supporting rule of law development? Do you know how those challenges were resolved? Legal aid provision USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 70 6. What changes have you noticed in the access of women, men, youth, minorities, and vulnerable groups to legal aid since the LRP began its grants program in 2019? Do you feel LRP has contributed to these changes? If so, how? 7. What has your organizations’ biggest successes been in the provision of LRP-support free legal aid to the eligible population? Can you provide an example? a. What factors do you think contributed to these successes? 8. What have your organization’s major challenges been in the provision of LRP-support free legal aid to the eligible population? a. What has been done by your organization and/or LRP to mitigate these challenges? Can you provide an example? 9. Since LRP awarded the grant to your NGO, have you noticed any changes in the ways and capabilities of your organization to provide legal aid to the eligible population? a. If yes, what changes have you noticed? Can you provide any examples? b. Do you think the LRP’s grants program has contributed to these changes? If so, how? Sub-question 1.1: What are the strengths and weaknesses of LRP’s activities/approaches in achieving its objectives on the development of rule of law and the provision of free legal aid by activity-supported NGOs/CSOs? Rule of law development 10. In your opinion, which specific LRP approaches/activities contribute the most to achieving its rule of law development objectives? Why? [List approaches to remind range of activities: hires experts to review, analyze, develop/modify legislation, develop recommendations to the law-making bodies, public hearings, enhance capacity of legal specialists in legislative drafting, develops databases and website to aid citizens to prepare correct legal docs] a. What would you say are the strongest points/assets of these approaches? Can you please provide an example? 11. Are there any weaknesses or limitations in the currently employed approaches by LRP in achieving its rule of law development objective? If yes, what are they? 12. What would you change/adjust about the currently employed approaches by LRP in achieving the rule of law development objective? Why? a. Are there any external and/or internal barriers to adjusting the currently employed approaches? What are they? 13. Are there any other approaches/activities that the project does not employ/use currently, but that you believe would be more effective in achieving its rule of law development objective? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 71 b. What could be the weaknesses? Legal aid provision 14. Do you think the provision of free legal aid to the eligible population contributes to building a sustainable and democratic rule of law system/culture? If so, why do you think so? If not, why not? 15. In your opinion, what are the strongest points/assets of LRP’s approaches in the provision of legal aid to vulnerable groups through NGO grantees? 16. Are there any limitations/disadvantages to LRP’s approach in the provision of legal aid to vulnerable groups via NGO grantees? If yes, what are they? Can you provide an example? 17. How would you know that your organization is successful in the provision of legal aid to the eligible population? a. What makes your organization successful? Which specific approach/aspect of your work do you think contributes to this success? 18. Do your organization/you apply any gender-sensitive approaches in the provision of free legal aid services to the eligible population? If yes, how? Has this been an LRP grant agreement requirement? Can you provide examples of gender-sensitive approaches your organization apply? 19. Are there any other approaches/activities that your organization does not employ/use or is not requested by LRP, but that you believe would be more effective in the provision of legal aid to vulnerable groups? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? EQ3: To what extent did the grants program contribute to the development of the legal￾enabling environment and strengthen organizational capacity for CSOs/NGOs as planned? Grants program contribution to the development of legal-enabling environment 20. What changes have you noticed in the ways your organization interacts with state institutions and participates in legislative processes since the start of the LRP in 2018? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 21. In your opinion, has the LRP’s grants program improved the existing or established new mechanisms of interaction between the state and NGOs? If yes, could you please specify these mechanisms? a. Has your organization been using these mechanisms to interact with state institutions? If yes, can you provide any examples? If not, why not? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 72 22. Which specific aspects of LRP’s grants program have been most effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? What makes you think so? 23. Which specific aspects of LRP’s grants program have been least effective in the development of the legal-enabling environment for civil society and in improving mechanisms of interaction between the state and NGOs? Why? 24. Are there any other approaches/activities that LRP does not employ/use in its grants program, but that you believe would be more effective for the development of the legal-enabling environment for CSO and interaction between the state and NGOs? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? b. What could be the weaknesses? Grants program contribution to strengthening organizational capacity of NGOs 25. Have you noticed any changes in the capacity of your organization/NGOs to manage, monitor, report, or provide legal services and sustain activities since the start of LRP’s grants program in 2019? a. Do you think the LRP’s grants program has contributed to these changes? If so, how? Can you provide an example? 26. In your opinion, what are the strongest points/assets of LRP’s grants program approaches in strengthening the organizational capacity of NGOs and improving NGO sustainability prospects? Why? 27. Have you participated in any training sessions supported or conducted by LRP’s grants program? a. If not, INTERVIEWER GO TO QUESTION 31 b. If yes, which training activities did you attend? [EXAMPLES: organizational capacity-building, provision of legal aid to vulnerable groups, participation in legislative discussions, other] c. Do you consider LRP’s grants program training activities to be effective? Why? What makes them effective? d. Have you used what you’ve learned in your daily work? e. If yes, which specific knowledge or skill that you obtained at the LRP’s training do you consider most beneficial and useful for your work? Which is least beneficial and useful for your work? Why? 28. Are there any weaknesses/limitations in LRP’s grants program approaches in strengthening the organizational capacity of NGOs? If yes, what are they? 29. Are there any other approaches/activities that LRP’s grants program does not employ/use, but that you believe would be more effective in strengthening the organizational capacity of NGOs? a. If yes, what are the strengths of this alternative approach/activity compared to the ones currently employed by LRP and its NGO grantees? USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 73 b. What could be the weaknesses? Conclusion 30. What would you do differently to improve support for the GoU in building a more sustainable and democratic rule of law system and culture in the country? a. Are there any areas you would rather target for LRP’s focus/activities? If yes, what areas? Why? b. What current LRP focus areas and approaches would you keep the same? Why? 31. Knowing what you know today, is there anything you would have done differently in implementing LRP’s grant? Are there any lessons that you have learned during the implementation of LRP’s grant? 32. Is there anything else you would like to add or recommend that we include in our findings? Thank you for your time and cooperation! USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 74 FREE LEGAL AID RECIPIENTS FOCUS GROUP DISCUSSION PROTOCOL Table 10: Free Legal Aid Pre-FGD Information Date: Months____ Day ___ Year____ Start time: Facilitator: End time: Notetaker: Consent granted to record: Yes/No *Note: If even one participant does not give consent to being recorded, the entire FGD will not be recorded. Region: Recording filename: Table 11: Free Legal Aid FGD Demographics and Service Received No. Participant Number Gender Age Issue of a legal aid service received [Note Facilitator should say a response is not required to this question if they prefer not to disclose this in front of the group] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 75 Comments on any aspect of the FGD: Introduction and Informed Consent Introduction: Hello, our names are ______, and we are researchers from the independent research firm NBT Consulting. We have been contracted by an independent consulting firm, Social Impact, which is commissioned by USAID to conduct a performance evaluation of the Legal Reform Program (LRP). Our task is to assess the performance of the LRP activities and the effectiveness of its mechanisms and approaches to improve legal reform and rule of law development through the information collected. You have been selected to participate in our evaluation because of your experience in receiving legal aid from LRP grantees: ● In Fergana: NGO Mehrjon ● In Samarkand: NGO Rahmdillik and NGO Yuksak Salohiyat ● In Tashkent: NGO Ishonch va Hayot, NGO Mehr Ko'zda, and NGO Association of Women in the Agriculture Sector of Uzbekistan, NGO Center for the Development of Modern Journalism. Purpose of discussion: This discussion group received similar legal aid assistance. The aim of today’s discussion is to learn about your experiences and hear your feedback on receiving free legal aid services as well as the impact of services in meeting your legal needs. Our role here is to ask questions and listen to your opinions and experiences. Please note that there are no “right” or “wrong” answers in this discussion. We would like everyone to share their experiences, and we ask that you allow people with different experiences than yours to express themselves freely. Risks and benefits: We do not anticipate that you will incur any risk or direct benefit from participating in this interview. However, it is possible that a sensitive topic such as your own need for legal assistance may arise, but we try to mitigate this risk by asking more general questions on the effectiveness of the services you received and by maintaining confidentiality. The local data collection firm will also mitigate this risk by including participants that received similar legal aid services, The local data collection firm will stop any discussions related to sensitive topics and re-direct the conversation back to the effectiveness of legal aid services. Structure: This focus group discussion will require 90 minutes of your time. As a part of this discussion, we will ask you questions and take notes. Voluntary participation: Your participation is completely voluntary, and you can choose to not answer any question or stop participating at any time. Deciding not to participate will not affect your access to and ability to obtain legal and other aid services now or in the future. There will be no consequences to your decision not to participate and [legal aid service provider] will not know whether or not you participated. You are not obligated to answer any question that you are not comfortable with. However, your feedback will be very useful and help to improve the current and design of future USAID rule of law programs in Uzbekistan. USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 76 Confidentiality: Your response will be kept confidential and anonymous. The information you provide will be used in summary form only and will not identify you as a participant in this discussion. Only SI staff and the research team will have access to the notes that are taken. We ask that everyone here respect each person’s privacy and confidentiality and not repeat what is said during this discussion. Nonetheless, there is a risk that other discussion participants will repeat what is shared. Remember that you are free to refuse to answer any question. Your voluntary feedback on the provided legal aid services will help to improve the program’s performance and help identify lessons from which future similar programs can learn. The information that you and others provide will be used to write a report. While all identifying information will be kept confidential, no identifying information will be included in the report. Results will only be presented in the aggregate and not connected to any individuals. The final report and high-level results will be shared with USAID and be made available on USAID's public database. Contact Information: If you have any concerns, you may contact the Social Impact Institutional Review Board at irb@socialimpact.com with questions about the study or its results, or you can contact Madina Khalmirzaeva, madina.khalmirzaeva@nbt.uz. Thank you very much for your participation. Do you have any questions for us before we get started? [Researcher to answer any questions] Consent and Recording: Are you willing to participate in this interview? Yes__ (start interview) No__ (do not start until provided) We would like to record this discussion with your permission. This recording will enable us to make sure we faithfully capture your thoughts and ideas during this discussion, and it will be deleted following the completion of the study. Do you agree to have this discussion recorded? Yes ___ (record the discussion)No___ (do not record the discussion) [GET VERBAL CONSENT FROM EACH PARTICIPANT OF THE FOCUS GROUP ONCE RECORDING STARTS. IF EVEN ONE PARTICIPANT DOES NOT GIVE CONSENT TO BEING RECORDED, THE ENTIRE FGD WILL NOT BE RECORDED] USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 77 Table 12: Free Legal Aid FGD Questions and Probes QUESTIONS FACILITATOR INSTRUCTIONS & PROBES 1. First, please share if you are comfortable: ● Where did you hear about the free legal aid service? ● When did you receive a free legal aid service? ● Who/which organization provided free legal aid? Before the discussion, be sure to review which NGO grantees were/are active in this community. If the respondent is unfamiliar with the name of the organization that provided the service, ask if they know the name of the provider. 2. Do you think free legal aid is a needed and relevant service in your community/region? ● If yes, why? ● If not, why not? 3. Have you experienced any challenges in accessing this legal aid service? In case challenges have been encountered, please ask to elaborate on what kind and how these challenges were solved. 4. In your opinion, what groups of people have benefited the most from free legal aid in this community/region? Do not read out these options, use the list below as a guideline for you. Do not probe, ask for spontaneous responses: ● labor migrants ● people living with HIV/AIDS ● victims of human trafficking ● victims of domestic and gender-based violence ● low-income families ● people with disabilities ● unemployed ● populations living in rural areas 5. Reflecting on the legal aid services you have received, have they met your needs? Probe: ● If yes, elaborate on how. ● If not, elaborate on why. Note: Facilitator to reinforce that responses should be focused on the legal services received, such as: ● financial assistance for people with disabilities (subsidies, benefits, allowances); ● employment subsidies (benefits, allowances); ● help with housing disputes; ● legal representation for divorce; financial assistance for low-income families (subsidies, benefits, allowances); ● alimony; ● employment; ● housing disputes; ● labor code violations Facilitator to reiterate that responses are optional so that participants who feel comfortable sharing more are encouraged to do so, while others don't feel pressured. Also being very explicit for participants to USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 78 know they don’t need to reflect or share personally, and that they can speak more generally. 6. Have you been satisfied with the quality of the provided legal service? ● If yes, please elaborate on why. ● If not, please elaborate on why. ● What needs to be done to improve the quality of service? 7. Is there anything else that you would like to share with me regarding your experience in receiving free legal aid services? Thank you again for agreeing to participate in this group discussion! [FACILITATOR: STOP RECORDER NOW, IF RECORDED] [FACILITATOR: WRITE THE END TIME OF FGD ON COVER PAGE] USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 79 TRAINING PARTICIPANTS FOCUS GROUP DISCUSSION PROTOCOL Table 13: Training Participant Pre-FGD Information Date: Months____ Day ___ Year____ Start time: Facilitator: End time: Notetaker: Consent granted to record: Yes/No *Note: If even one participant does not give consent to being recorded, the entire FGD will not be recorded. Region: Recording filename: Table 14: Training Participant FGD Demographics and Training Received No. Participant # Gender Organization Training topic 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Comments on any aspect of the FGD: Informed Consent Introduction: Hello, our names are ______, and we are researchers from the independent research firm NBT Consulting. We have been contracted by an independent consulting firm, Social Impact, USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 80 which is commissioned by USAID to conduct a performance evaluation of the Legal Reform Program (LRP). Our task is to assess the performance of the LRP activities and the effectiveness of its mechanisms and approaches to improve legal reform and rule of law development through the information collected. You have been selected to participate in our evaluation because of your experience in participating in capacity-building events supported by LRP. Purpose of discussion: The aim of today’s discussion is to learn about your experiences and hear your feedback on the effectiveness of training courses as well as the extent of practical application of obtained knowledge and skills in your professional activities. Our role here is to ask questions and listen to your opinions and experiences. Please note that there are no “right” or “wrong” answers in this discussion. We would like everyone to share their experiences, and we ask that you allow people with different experiences than yours to express themselves freely. Risks and benefits: We do not anticipate that you will incur any risk or direct benefit from participating in this interview. Structure: This focus group discussion will require 90 minutes of your time. As a part of this discussion, we will ask you questions and take notes. Voluntary participation: Your participation is completely voluntary, and you can choose to not answer any question or stop participating at any time. There will be no consequences to your decision not to participate. Participation or non-participation in the discussion will not affect your ability to receive training now or in the future. Only the evaluation team will know whether they participated or not. You are not obligated to answer any question that you are not comfortable with. However, your feedback will be very useful and help to improve the current and design of future USAID rule of law programs in Uzbekistan. Confidentiality: Your response will be kept confidential and anonymous. The information you provide will be used in summary form only and will not identify you as a participant in this interview/discussion. Only SI staff and the research team will have access to the notes that are taken. We ask that everyone here respect each person’s privacy and confidentiality and not repeat what is said during this discussion. The information that you and others provide will be used to write a report. This report will be shared with USAID and other stakeholders for comment and eventually be made public. Contact Information: If you have any concerns, you may contact the Social Impact Institutional Review Board at irb@socialimpact.com with questions about the study or its results, or you can contact Madina Khalmirzaeva, madina.khalmirzaeva@nbt.uz. Thank you very much for your participation. Do you have any questions for us before we get started? [Researcher to answer any questions] Consent and Recording: Are you willing to participate in this interview? Yes__ (start interview) No__ (do not start until provided) USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 81 We would like to record this discussion with your permission. This recording will enable us to make sure we faithfully capture your thoughts and ideas during this discussion, and it will be deleted following the completion of the study. Do you agree to have this discussion recorded? Yes ___ (record the discussion)No___ (do not record the discussion) [GET VERBAL CONSENT FROM EACH PARTICIPANT OF THE FOCUS GROUP ONCE RECORDING STARTS, IF EVEN ONE PARTICIPANT DOES NOT GIVE CONSENT TO BEING RECORDED, THE ENTIRE FGD WILL NOT BE RECORDED] Table 15: Training Participant FGD Questions and Probes QUESTIONS FACILITATOR INSTRUCTIONS & PROBES 1. First, let’s go around and introduce ourselves. Please share: ● What is your profession? ● How many times have you attended LRP￾supported training events? ● What was/were the topic(s) of the training sessions you attended? Do not read out these options, use the list below as a guideline for you. Do not probe, ask for spontaneous responses: ● organizational capacity-building ● provision of legal aid ● participation in legislative discussions ● drafting legislation ● gender legislation ● mainstreaming gender into work of legal professionals ● trademark registration ● licensing and franchising ● enforcement of intellectual property rights ● access to justice for citizens 2. Reflecting on your participation at training events, did you find the training(s) useful to your professional needs? If yes, how useful? If not, why not? Ask for any suggestions for improvement 3. Have you used or are still using what you’ve learned in your professional work? If yes, please ask them to elaborate on what specific knowledge, information, and/or skills obtained in training they use. How often do they use these specific skills? Ask for examples. If not, why not? Ask for any suggestions for improvement 4. What are the strongest points/methods used by facilitators in LRP-supported training events? Please ask them to elaborate on why. What makes these training events successful? 5. Were there any weaknesses/limitations in training methods/approaches? If yes, please ask them to specify what these weaknesses and limitations are? Ask for any suggestions for improvement 6. Have you experienced any challenges at the training? Example: understanding training material might be a challenge. USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 82 In case challenges have been encountered, please ask them to elaborate on what kind and how these challenges were solved 7. Overall, have you been satisfied with the quality of the provided training? If yes, why? If not, please ask them to elaborate on why. What needs to be done to improve the quality of service? 8. Is there anything else that you would like to share with me regarding your experience at the LRP-supported training? Thank you again for agreeing to participate in this focus group! [FACILITATOR: STOP RECORDER NOW.] [FACILITATOR: WRITE THE END TIME OF FGD ON COVER PAGE.] USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 83 ANNEX G. SOURCES OF INFORMATION Table 16: Sources of Information Organization Stakeholder Type USAID/Uzbekistan Donor East-West Management Institute Other USAID implementing partner Regional Dialogue USG implementing partner Tetra Tech DPK LRP implementing partners’ core team Ministry of Justice of Uzbekistan National partner/GoU Legislation and Parliamentary Research Institute (LPRI) under the Oliy Majlis National partner/GoU National Center for Human Rights of the Republic of Uzbekistan National partner/GoU Research Institute "Family and Women" of the Family and Women's Committee under the Ministry of Employment and Poverty Reduction National partner/GoU Academy of the General Prosecutor's Office of Uzbekistan National partner/GoU Chamber of Advocates National partner/GoU Business Ombudsperson National partner/GoU Legal Training Center Department of International Legal Disciplines and Human Rights National partner/GoU Institute for Democracy and Human Rights NGO partner/LRP grantee Civic Initiative Support Center NGO partner Legal Issues Research Center NGO partner NGO "Madad" NGO partner Nationwide movement "Yuksalish" NGO partner Technical experts National and international experts Mehrjon LRP grantee USAID.GOV LER II N078 UZBEKISTAN LRP EVALUATION REPORT | 84 Organization Stakeholder Type Association of "Women of the Agricultural Center" of Uzbekistan LRP grantee Yuksak Salohiyat LRP grantee Mehr Ko'zda LRP grantee Rahmdillik LRP grantee Ishonch va Hayot LRP grantee End users Recipients of free legal aid End users Training participants U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT 1300 Pennsylvania Avenue, NW Washington, DC 20523