Mid-Term Performance Evaluation of USAID/Georgia Judicial Independence and Legal Empowerment Project (JILEP) [September 2013] This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez, England & Associates. MID-TERM PERFORMANCE EVALUATION OF USAID/GEORGIA JUDICIAL INDEPENDENCE AND LEGAL EMPOWERMENT PROJECT (JILEP) Final Report 20 September 2013 Prepared under RFTOP No. SOL-114-13-000005 Submitted to: USAID/Caucasus Submitted by: Mary Noel Pepys (Team Leader) Keith Rosten Commercial Law Specialist Ritu Nayyar Stone (Evaluation Specialist, NORC) Vano Tavadze (Lawyer, PMCG) Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government CONTENTS EXECUTIVE SUMMARY .................................................................................................... .i Evaluation Purpose and Evaluation Questions ................................................................................... i Project Background .................................................................................................................................. i Evaluation Design, Methods and Limitations ...................................................................................... i Findings ....................................................................................................................................................... ii Conclusions .............................................................................................................................................. iii Recommendations .................................................................................................................................. iii 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS ............................... 1 1.1 Evaluation Purpose ................................................................................................................... 1 1.2 Evaluation Questions ............................................................................................................... 1 2.0 PROJECT BACKGROUND ..................................................................................... 2 3.0 EVALUATION METHODS & LIMITATIONS ...................................................... 4 3.1 Evaluation Methodology .......................................................................................................... 4 3.1.1 Qualitative Research and Analysis ...................................................................... 4 3.1.2 Quantitative Research and Analysis ................................................................... 4 3.2 Evaluation Limitations .............................................................................................................. 5 4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS .................................... 6 4.1 Question 1: Supply and Demand for an Independent Judiciary ..................................... 6 4.1.1 Findings ...................................................................................................................... 6 4.1.2 Conclusions ............................................................................................................ 10 4.1.3 Recommendations ................................................................................................ 12 4.2 Question 2: Results Despite a Lack of Political Will in the Judiciary for Reform .............................................................................................................. 13 4.2.1 Findings .................................................................................................................... 13 4.2.2 Conclusions ............................................................................................................ 17 4.2.3 Recommendations ................................................................................................ 20 4. 3 Question 3: Building the Civil Society Coalition ............................................................ 22 4.3.1 Findings .................................................................................................................... 22 4.3.2 Conclusions ............................................................................................................ 28 4.3.3 Recommendations ................................................................................................ 29 4.4 Question 4: Cost-Benefit of Support to Legal Education ............................................ 30 4.4.1 Findings .................................................................................................................... 30 4.4.2 Conclusions ............................................................................................................ 34 4.4.3 Recommendations ................................................................................................ 39 4.5 Question 5: Effectiveness of Support to Commercial Law ......................................... 39 4.5.1 Findings .................................................................................................................... 40 4.5.2 Conclusions ............................................................................................................ 45 4.5.3 Recommendations ................................................................................................ 47 ANNEXES Annex 1: Statement of Work Annex 2: Monitoring and Evaluation matrix Annex 3: List of Meetings Annex 4: List of Materials Consulted Annex 5: Interviewee Questionnaires FIGURES Figure 1: On a 5-Point Scale, To What Extent do You Agree or Disagree with the Opinion That, Overall, Judges are Independent in Georgia? (%) Figure 2: Advantages and Disadvantages of the JILEP Project Design Figure 3: On a 5-Point Scale, How Would You Assess the Urgency of Reform of the HCOJ in Terms of the Ultimate Goal of Creating a Fair and Impartial Judiciary in Georgia? (%) Figure 4: Challenges Faced by Working Groups Figure 5: CITJ Success in Achieving its Various Missions Figure 6: Challenges of Being a CITJ Member Figure 7: CITJ Members’ Views on the Process of Obtaining and Implementing EPF Grants Figure 8: To What Extent Are You Satisfied with the Level of Expertise in Georgian Lawyers and Judges in Commercial Law? (%) Figure 9: To What Extent are Opportunities for Continuing Lifelong Legal Education Accessible to You? (%) Figure 10: To What Extent is Commercial Law Training Accessible to Legal Professionals? Figure 11: How Confident Are You That the Outcome of the Cases in Commercial Disputes Are, on the Whole, Fair and Impartial? (%) Figure 12: How Confident Are You That the Outcome of the Cases in Commercial Disputes Are, on the Whole, Fair and Impartial? (%) (by case category) ACRONYMS ADR Alternative Dispute Resolution AmCham American Chamber of Commerce BSU Batumi State University CASI Computer Assisted Self-Interviews CATI Computer Assisted Telephone Interviews CAWG Court Administration Working Group CITJ Coalition for an Independent and Transparent Judiciary CLE Continuing Legal Education CLWG Criminal Law Working Group CRRC Caucasus Research and Resource Center CSO Civil Society Organization DOJ Department of Justice EPF Eurasia Partnership Foundation EWMI East-West Management Institute FGD Focus Group Discussion FreeUniFree University GAU Georgian-American University GBA Georgian Bar Association GEOLLA Georgian Law Libraries Association GIZ German International Cooperation GYLA Georgian Young Lawyers Association HCOJ High Council of Justice HSOJ High School of Justice IACC Inter-Agency Coordination Council ICC International Chamber of Commerce IRMS IT, Research & Metadata Solutions JAG Judges Association of Georgia JILEP Judicial Independence and Legal Empowerment Project KII Key Informant Interview KSU Kutaisi State University LAS Legal Aid Service LAWG Legal Aid Working Group LEWG Legal Education Working Group ME&A Mendez, England & Associates MOJ Ministry of Justice NCADR National Center for Alternative Dispute Resolution NCCL National Center for Commercial Law NGO Non-Governmental Organization NLC National Learning Center OSF Open Society Foundation PIL-net Public Interest Law Network STCL South Texas College of Law TI Transparency International TSU Tbilisi State University UNDP United Nations Development Programme UNCITRAL United Nations Commission on International Trade Law USAID United States Agency for International Development WUSL Washburn University School of Law MID-TERM PERFORMANCE EVALUATION OF JILEP i | Page EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS This is a report on the mid-term performance evaluation of the Judicial Independence and Legal Empowerment Project (JILEP) funded by the United States Agency for International Development (USAID) Mission in Georgia. JILEP is being implemented during the period September 24, 2010 – September 24, 2014, by the East-West Management Institute (EWMI). The evaluation of JILEP was conducted during the period June – September 2013, by a team assembled by Mendez, England & Associates (ME&A) with headquarters in Bethesda, Maryland. The team consisted of three specialists – two international and one local – with experience in rule of law reform, judicial independence and operation of court systems, and one Evaluation Expert that was provided by NORC. The main objectives of the evaluation were to determine the effectiveness of JILEP and to provide recommendations on corrective actions and new directions for the remaining years of project implementation and beyond. The main thrust of the evaluation included analyzing JILEP activities from the project’s start in September 2010 through the initiation of this evaluation in June 2013 to review progress toward achieving key expected results, as well as identify accomplishments, delays, challenges, and their impact on the project. An integral part of the evaluation mission was to answer a set of pre-determined questions, which focused primarily on judicial reform, building a civil society coalition, legal education, and commercial law. These questions are depicted in the body of this report, followed by sections related to findings, conclusions and recommendations. They concern:  The interplay between direct support for the judicial system and support for civil society organizations (CSOs) intent on advocating fundamental changes within the judicial system;  The project’s effect on judicial reform despite support from the judicial leadership;  The project’s approach to building a coalition of like-minded non-governmental organizations (NGOs) to support judicial reform;  The extent to which the project’s approach to legal education was effective; and  The extent to which the project’s approach to commercial law was effective. The results of the evaluation will be used by USAID/Georgia to improve ongoing and future interventions in the area of rule of law by focusing on the activities most meaningful and critical for developing more independent and efficient judiciary in Georgia, as well as strong civil society organizations that work in rule of law. PROJECT BACKGROUND JILEP was designed to support and strengthen the judiciary as an independent yet equal branch of government, and to improve Georgia’s commercial law system. The project is the most recent in a line of USAID projects focused on rule of law reform since the late 1990’s. JILEP is structured around four components: 1) strengthen judicial independence, accountability, and professionalism; 2) strengthen the institutional capacity of legal professional associations, legal rights NGOs, and the state legal aid system; 3) improve the quality of legal education; and 4) develop commercial law and improve commercial law-related practice. As JILEP was created to bring about a deep-rooted transformation of a flawed and complex system with various tightly intertwined elements, it was built upon the core principle of synergy. The methodology employed to achieve project objectives involved increasing the capacity of existing institutions and establishing new “USAID-legacy” institutions. EVALUATION DESIGN, METHODS AND LIMITATIONS The Evaluation Team collected quantitative and qualitative data from a broad range of stakeholders and beneficiaries to ensure independence of the evaluation process, as well as accuracy and completeness of the subsequent conclusions, recommendations, and lessons learned. Techniques that balance each other were MID-TERM PERFORMANCE EVALUATION OF JILEP ii | Page utilized: quantitative vs. qualitative data; individual vs. group responses; semi-structured interviews vs. analysis of existing surveys; and data sets. The following main sources of evidence were used:  Critical desktop review of materials related to JILEP, such as quarterly reports, annual work plans, monitoring and evaluation (M&E) plans, Caucasus Research and Resource Center (CRRC) baseline surveys, court monitoring reports, and reports from grantees, foreign organizations and donors.  Data sources from performance reports, evaluations of implementing partners, comprehensive baseline assessment conducted by CRRC with JILEP’s sponsorship, case data.  Project outputs against objectives and performance indicators.  Field visits to Kutaisi and Batumi.  Focus groups discussions (FGDs) with members of the Coalition for an Independent and Transparent Judiciary (CITJ), attorneys, law students, law professors, and judges.  Interviews including open-ended and semi-structured with USAID and JILEP implementers, project beneficiaries and stakeholders, and key informants from the government, judiciary, NGOs, JILEP staff and others.  Surveys conducted with a random sample of attorneys and the entire CITJ membership.  Direct observation of a JILEP program for law librarians and attendance at a conference in Georgia on the “Empowerment of the Judiciary.” The Evaluation Team encountered some limitations inherent to the design of this evaluation and during its fieldwork in Georgia. Some of the more relevant limitations are listed below:  Timing of the evaluation. The evaluation fieldwork took place at the height of the summer vacation season when some respondents were not available to speak with the Evaluation Team.  Biases in data collection methodologies. To identify key stakeholders and project beneficiaries, the Evaluation Team relied on assistance from USAID and JILEP staff.  Possible attenuated link between project activities and results. There are many factors or variables that may contribute to the results described in this report.  Availability of data. The Evaluation Team used data developed by/through JILEP as baseline data.  Short project timeframe. This kind of project may take many years to produce concrete results.  Time allotted for evaluation. The Evaluation Team had just over two weeks for its fieldwork to assimilate data from key informant interviews (KIIs), FGDs, and project personnel meetings. JILEP is a $19.3 project with numerous activities and sub-activities to look at in a short period of time. The above limitations, however, did not prevent the Evaluation Team from gathering the information and data needed to produce findings, conclusions and recommendations. FINDINGS JILEP was designed to support an independent judiciary by using direct assistance to core legal institutions and providing support to civil society. The project used critical tools to enhance cooperation and develop a robust and healthy dialogue between the judiciary and civil society. Project design included the flexibility to change course based on the political will for reform by reallocating efforts from one component to another. JILEP managed the coordination between various project components supporting civil society and judicial institutions. There was extensive interaction between civil society and core judicial institutions because, as the judicial sector’s appetite for reform was limited at JILEP’s outset, it shifted more activities to civil society. JILEP managed numerous activities with the judiciary, particularly with its counterparts at the High Council of Justice (HCOJ) and the High School of Justice (HSOJ), which were geared towards contributing to judicial independence, transparency, and professionalism. Activities commenced in September 2010, at a time when political will for judiciary independence, transparency, and professionalism was weak. Therefore, not all activities have been completed. JILEP has designed and implemented many diverse activities with universities, national learning centers, professional groups, law professors, students, and practitioners that provide critical support and significantly MID-TERM PERFORMANCE EVALUATION OF JILEP iii | Page contributed to the development and improvement of legal education in Georgia. The project arranged partnerships with U.S. institutions through which much of the support is channeled, provided support for three live client clinics, devoted considerable attention to teaching methodology to law school faculty, and supported lawyering skills development such as client counseling and trial advocacy. JILEP supported numerous law school competitions of trial advocacy, moot court, mediation, client counseling, and commercial arbitration. It offered scholarships to West Georgian students who agreed to work in underserved areas and laid the groundwork to establish an association of law schools. JILEP improved the commercial law environment through a series of activities related to improving commercial law implementation, including training judges in related subjects through the HCOJ, and training lawyers in commercial law through the Georgian Bar Association (GBA), organizations receiving advocacy grants through the Eurasia Partnership Foundation (EPF), and directly through JILEP. The project supported two major initiatives to prepare commentaries: a completed commentary on tax law, and an ongoing effort on the civil code. Despite these good efforts, the perception of the fairness of commercial adjudication has significantly dropped in the past year. Alternative dispute resolution (ADR) has gained acceptance, if not use, for commercial disputes. Mediators trained with JILEP funding are poised to commence mediations in the coming months in family law, inheritance, and neighborhood disputes. CONCLUSIONS JILEP’s design allowed it to allocate resources based on the political will or willingness of its target partners. Working with both judicial sector organizations and civil society allowed the project to play a broader role in judicial reform as it was able to moderate exchanges, facilitate cooperation, and seek opportunities for judicial system change that civil society may have been unable to determine without project intervention. In controversial areas, JILEP may have gained more traction in advancing its views on judicial reform when it was able to tie those views to those of civil society. JILEP’s activities effectively contributed to judicial independence, transparency and professionalism. Political will was not strong through the project’s midterm, so critical judicial reforms emanated from JILEP or CITJ, and more results could have been achieved had it been stronger. However, as a result of the changing political environment, new opportunities exist for JILEP to work directly with the judiciary to achieve its project goals. The HCOJ has prepared its own agenda on future judiciary priorities, which may serve as a starting point for continued cooperation with the judiciary. The effectiveness of CITJ’s activities demonstrates that there are more advantages than disadvantages in working through it. CITJ maintains respect and credibility with the judiciary, the Ministry of Justice (MOJ), and Parliament. Its successes and strengths are the result of a few active members, as well as its collaboration with JILEP. There is a marked difference of opinions regarding the quality of coordination and organizational development assistance provided by EPF. CITJ is capable of pursuing its stated goals with current support but will be better-positioned to survive after JILEP’s completion if foreign funding and programmatic support are continued. JILEP has been effective in designing and implementing activities that contribute to improved legal education in Georgia and its enhancement of clinical education to hone students’ skills is a major breakthrough in Georgia’s legal education system. Effective teaching methodology is essential to engage students but its impact is hard to gauge, especially in the short-term. Not enough time has lapsed to assess the influence of capacity building interventions on actual teaching and learning processes. JILEP has taken an effective approach to establish a resource group of law professors through an extensive training program, but sustainability and accessibility of the resource group is difficult to gauge. JILEP has been active in supporting the initiatives to develop teaching materials on commercial law-related areas, even though the texts have not been completed. There is a lack of teaching materials in all major areas of laws and JILEP’s efforts will narrow but not close this gap. The national learning centers at the universities provide an important input in the implementation of respective national policy initiatives and act as service providers for various stakeholders, including the GBA. While most of the universities have been successful in partnering with JILEP, Free University has been JILEP’s best partner, benefitting in the most efficient and effective manner from various opportunities. MID-TERM PERFORMANCE EVALUATION OF JILEP iv | Page JILEP’s activities have had a modest impact on the HSOJ’s ability to provide commercial law training to judges. The project, however, has significantly improved the capacity of law schools to provide effective training in commercial law, and strengthened continuing legal education for lawyers in commercial law, even though from disparate sources. JILEP’s support for a commentary on tax law and the forthcoming commentary on the civil code is a significant step in implementing commercial law. Due to JILEP’s various activities, ADR has gained wider acceptance in both mediation and arbitration and commercial adjudication is perceived as less fair by attorneys than it was a year ago. RECOMMENDATIONS Support for civil society and core legal institutions in one project is an effective lever for reform. This approach worked well for JILEP and can work well in other countries. Separating support to civil society from assistance to the judiciary or other the government agencies would be less desirable. The challenge that remains is an aggressive exit strategy to enshrine accomplishments and assure that JILEP’s approach and other achievements are sustainable after the project’s end. JILEP should continue to work directly with the HCOJ, the HSOJ and the MOJ in the same technical areas it has pursued since inception, with a greater focus on those that have not been completed. It should continue to support CITJ. JILEP should expand its work focusing on the empowerment of judges by assisting the Conference of Judges and the newly-created Unity of Judges. In order to achieve sustainable results from its activities, JILEP should encourage collaboration and moderation among justice institutions, MOJ, Parliament, Procuracy, GBA, and law faculties. It should also develop a strategic plan for the justice system as the imbalance of power among the three branches of government could impede JILEP from reaching its project goals resulting in short-lived successes. JILEP should continue to work through CITJ as the two-prong approach in pursuing judicial reforms has been successful. The EPF should be more active in the internal management, communication and coordination activities of CITJ to achieve greater results and sustainability. JILEP should develop a sustainability strategy for the legal education component to ensure that the initiatives it started continue beyond project completion. Because there is little quantitative data to confirm JILEP’s important accomplishments in legal education, the project should undertake performance assessments and follow-up studies to gauge the extent of education intervention impacts. JILEP laid a solid foundation of cooperation among Georgian law schools and should now facilitate partner university discussions on the establishment of an association of law schools. Practical skills trainings have produced good results and JILEP should design and provide further practical skills development activities. JILEP can focus its efforts on commercial law with the HSOJ, which has been generally receptive to commercial law training. JILEP should look to fill gaps in judicial training on domestic commercial law and should produce handbooks or bench books on handling specific types of cases. JILEP should develop a blueprint for introducing an institution to handle arbitration for commercial disputes. The project may have some opportunities in the process of drafting commercial legislation with MOJ. There are good opportunities that JILEP has created for providing more frequent commercial law training for lawyers on a wide breadth of commercial law topics. MID-TERM PERFORMANCE EVALUATION OF JILEP 1 | Page 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE This is a report on the mid-term performance evaluation of the Judicial Independence and Legal Empowerment Project (JILEP) funded by the United States Agency for International Development (USAID) Mission in Georgia. The project is being implemented by the East-West Management Institute (EWMI) between September 24, 2010 and September 24, 2014. The evaluation covered the timeframe from JILEP’s start through July 2013 and was conducted by a team of four key experts: Ms. Mary Noel Pepys (Team Leader and Rule of Law Specialist), Mr. Keith Rosten (Commercial Law Expert), Dr. Ritu Nayyar-Stone (Evaluation Expert), and Vano Tavadze (Local Expert). The main goal of the performance evaluation was to determine the effectiveness of JILEP and to provide recommendations on corrective actions and new directions for the project’s remaining years of implementation and beyond. The Evaluation Team was also tasked with reviewing actual progress toward achieving key expected results and identifying accomplishments, delays, challenges, and their impact on the project from its start in September 2010 through the initiation of this evaluation in June 2013. Specifically, the Evaluation Team was asked to assess the effectiveness of: 1) including the supply and demand for an independent judiciary in the same award; 2) JILEP’s activities in contributing to judicial independence in Georgia; 3) JILEP’s work through the Eurasia Partnership Foundation (EPF) to provide assistance to the Coalition for Independent and Transparent Judiciary (CITJ); 4) JILEP’s contribution to improving legal education in Georgia; and 5) the project’s efforts to improve commercial law related practice. The results of the evaluation will be used by USAID/Georgia for improving ongoing interventions in the area of rule of law by focusing on the activities that are most meaningful and critical for developing a more independent and efficient judiciary in Georgia, as well as strong civil society organizations (CSOs) that work in the rule of law arena. The results will also contribute to the design of USAID’s next generation of projects supporting the rule of law and will be used for reporting purposes to Washington-based stakeholders. The intended audience of the evaluation includes USAID and, in particular, its Democracy and Governance (DG) office. The results may be shared with local stakeholders, such as the High Council of Justice (HCOJ), High School of Justice (HSOJ), the Supreme Court of Georgia, other donors working in the rule of law area, and interested CSOs. 1.2 EVALUATION QUESTIONS The Evaluation Team was asked to answer a number of specific evaluation questions, outlined below: 1. Supply and Demand for an Independent Judiciary. JILEP was designed with components both for strengthening the judiciary (Component I) and generating increased civil society advocacy for judicial independence (Component II). The theory was that these components would be utilized proportionally to the political will for reform, i.e. when there has been will within the judiciary, JILEP has worked directly with the court administration to foster improvements. When political will has been scarce, JILEP has facilitated relevant reforms through civil society. - How effective has it been to include both the supply and demand for an independent judiciary in the same award? - What has been the nature and level of interaction and relationship between these components, including among project beneficiaries and from a project management perspective? - Is this approach recommended in the future? 2. Results Despite a Lack of Political Will in the Judiciary for Reform. - Recognizing that political will for judicial independence has been weak, which (presumably less politically sensitive) technical areas and/or activities has JILEP managed to work in and/or complete with the judiciary itself? - How effective have those activities been in contributing to judicial independence? MID-TERM PERFORMANCE EVALUATION OF JILEP 2 | Page - Considering the changed political environment, what new openings and opportunities exist for working directly with the judiciary to achieve program goals, both through the end of JILEP and beyond? 3. Building the Civil Society Coalition. - Taking into consideration the Coalition’s accomplishments to date, what has been the JILEP and EPF approach to building the Coalition? - What have been the advantages and disadvantages of working through a Coalition for Independent and Transparent Judiciary? - How effective has it been to work through EPF to provide coordination and organizational development assistance to the Coalition for Independent and Transparent Judiciary? - What additional support is required in order to ensure the Coalition continues to pursue its stated goals? 4. Cost-Benefit of Support to Legal Education. - Taking into consideration the cost and scale of JILEP support to legal education, how effective have the activities been or are they likely to be in contributing to improved legal education in Georgia (i.e. partnerships between Georgian and U.S. University law schools)? - Which Georgian universities have been the best partners and why? - Moving forward, what approaches would best accomplish the overall objective of improving the quality of legal education in Georgia? 5. Effectiveness of Support to Commercial Law. Sustainable economic growth in Georgia depends on an effective legal framework and judicial environment related to commercial law. - To what extent have JILEP activities contributed to improvements to commercial law related practice, including the transparency, efficiency, and effectiveness of commercial adjudication, alternative dispute resolution (ADR) and enforcement of the legal framework? - Moving forward, what approach can best contribute to improvements in the legal framework, judicial environment and commercial law practice for sustainable economic growth? 2.0 PROJECT BACKGROUND Since the Rose Revolution of 2004, Georgia’s justice system underwent significant changes pertaining to the restructuring of the court system, improving court facilities, modernizing court administration, cleansing the judiciary to reduce corruption among judges, increasing judges’ salaries and benefits, and the government’s assuming financial and administrative responsibility to provide initial training for prospective judges and continue training sitting judges through the creation of the HSOJ. However, serious deficiencies remained in establishing an independent and accountable judiciary. An imbalance of power existed between the executive and judicial branches of government, with the President of Georgia wielding significant influence over the judiciary directly through his power to appoint the Chief Justice of the Supreme Court, and indirectly through his control of the Prosecutor’s Office. The Chief Justice controlled the HCOJ, which operated with a lack of transparency regarding the appointment, discipline, transfer and dismissal of judges, and the HSOJ, which trained judicial “listeners” (those who are undergoing training to become a judge) at a moderate pace and was not eager to collaborate with foreign donors. Furthermore, the self￾regulating Conference of Judges was inactive and the Judges Association of Georgia (JAG), essential to promoting the independence of judges, was moribund. While unstructured in the past, Georgia’s legal profession is today organized under the aegis of the Georgian Bar Association (GBA), an independent professional association whose goals are to protect the rights of attorneys, raise the professionalism of GBA members, establish ethical standards, and facilitate the administration of justice and the rule of law. GBA provides continuing legal education (CLE) courses, although most attorneys remain unfamiliar with commercial law issues, such as consumer protection, copyright, competition, and bankruptcy. Legal education has undergone several stages from the existence of few qualified law schools, to a proliferation of schools which graduated ill-prepared students and, due to an accreditation MID-TERM PERFORMANCE EVALUATION OF JILEP 3 | Page process, to the current situation of fewer yet better schools. Problems with the quality of legal education, however, still exist. The USAID Mission in Georgia has supported strengthening the independent rule of law sector in the country for many years, with interventions dating to the late 1990’s. Previous efforts to reform the judiciary have had limited success, although several waves of reforms created fundamental court changes. Under the 1997-1999 judicial reform package, the Law on Courts and the Law on the Supreme Court extracted essential functions that define the court system’s autonomy from the Ministry of Justice (MOJ) and dispersed them among new organizations. The capacity of these new institutions failed to keep up with the changes, necessitating a new round of judicial reform in 2005-2007. The President of Georgia acted as Chair of the HCOJ before its reform in 2007, at which time the Chair of the Supreme Court became the Chair of the Council. Unfortunately, the new reforms did little to support the independence of Georgia’s judiciary. According to some reports, judges were asked to resign and threatened with disciplinary action if they failed to comply. New judges were drawn primarily from prosecutorial ranks, resulting in courts heavily favoring the prosecution in criminal cases and state agencies in administrative cases. A 2006 Freedom House report stated that the judiciary "rarely makes decisions that run counter to the will of the executive." The report further explained that only 37 of almost 17,000 individuals taken to court in 2005 were acquitted. As early as March, 2007, the Venice Commission plenary session criticized the system of disciplining judges in Georgia as a threat to judicial independence. It was under this context that JILEP was designed to support and strengthen Georgia’s judiciary as an independent yet equal branch of government, and to improve the country’s commercial law system. JILEP is a four-year, $19.3 million project that works to improve the process by which judges are appointed, trained, and disciplined to increase their independence and professionalism. Further, JILEP is intended to: 1) strengthen the state free legal aid system and the legal non-governmental organization (NGO) community and increase public understanding of the role of the judiciary; 2) support reform in legal education both for law students and for practicing attorneys; and 3) streamline procedures for handling commercial cases and enforcing commercial judgments. JILEP is structured around four components:  Component 1: Strengthen judicial independence, accountability, and professionalism. Under this component, JILEP supports the development of the HCOJ; works with the HCOJ to develop written procedures and performance evaluation tools for assessing judicial performance and guiding advancement and disciplinary decisions; helps the HCOJ improve its transparency and public outreach; increases access to information in the courts; and assists in strengthening the long-term sustainability and training capacity of the HSOJ.  Component 2: Strengthen the institutional capacity of legal professional associations, legal rights NGOs, and the state legal aid system. Under this component, JILEP works in conjunction with its sub-awardee, the EPF, to implement a grants program targeting court performance monitoring and public education, and providing free legal assistance to the indigent population of Georgia; supports the work of CITJ, which unites approximately 30 legal rights NGOs, professional associations, business associations and media outlets in a platform that coordinates efforts to promote reforms in the justice system; and helps strengthen the institutional capacity of the state legal aid agency, Legal Aid Service (LAS), through targeted assistance to its Monitoring Board. In addition, through targeted grants supporting the work of two major legal rights NGOs in Georgia, the Georgian Young Lawyers Association (GYLA) and Transparency International (TI), JILEP helps increase access to justice.  Component 3: Improve the quality of legal education. Under this component, JILEP provides intensive support to select law schools, introducing more practical skills training, making more specialized trainings available, and establishing a collaborative relationship between Georgian and U.S. law schools. JILEP also works with the GBA to increase its capacity to provide CLE programs and set ethical, legal practice standards for its member lawyers.  Component 4: Developing commercial law and improve commercial law related practice. Under this component, JILEP assists in the development of training modules on commercial law for use in MID-TERM PERFORMANCE EVALUATION OF JILEP 4 | Page law schools, the HSOJ, and Bar; awards grants to monitor commercial law developments and their disposition in the courts and to educate the public on commercial law issues and property rights; and works closely with the HCOJ to develop mediation as an alternative to courts. 3.0 EVALUATION METHODS & LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation methodology was carefully designed to respond to the five evaluation questions, and to determine whether JILEP achieved its initial objectives and whether results achieved are sustainable. The Evaluation Team collected quantitative and qualitative data from a broad range of stakeholders and beneficiaries to ensure independence of the evaluation process, as well as accuracy and completeness of the subsequent conclusions, recommendations, and lessons learned. Techniques that balance each other were utilized: quantitative vs. qualitative data; individual vs. group responses; semi-structured interviews vs. analysis of existing surveys; and data sets. 3.1.1 Qualitative Research and Analysis The qualitative evaluation began with a critical desk-top review of materials related to JILEP, including project quarterly reports and annual work plans, performance management plans (PMP), project design, and communications among partners. The Evaluation Team reviewed a wide range of project-related documentation from several sources, such as the Report and Recommendations on Legal Education in Georgia; sources from international organizations such as the International Bar Association’s Human Rights Institute; documents from CITJ and its various members on judicial reform, court monitoring, and preventive measures; and news articles in the lay press for background information (for a list of materials reviewed, see Annex 4). The Evaluation Team conducted interviews with USAID and JILEP staff; key informant interviews with partners, beneficiaries, and stakeholders such as TI Georgia, EPF, GYLA, the HCOJ, the HSOJ, the JAG, the Georgia Bar Association (GBA), the National Center for Alternative Dispute Resolution (NCADR), the National Center for Commercial Law (NCCL), the MOJ, the Prime Minister’s Office, Parliament, Free University, Tbilisi State University (TSU), and others. The Evaluation Team retained the services of IT, Research & Metadata Solutions (IRMS) to assist in implementing surveys and FGDs. To obtain qualitative data and provide context and background on information gained through individual interviews, the Evaluation Team conducted several focus group discussions (FGDs), which were held separately with law students, law professors, members of CITJ, attorneys, and judges. FGDs covered topics including legal education, operations and influence of CITJ, continuing legal education, and commercial law, among others. The FGDs with law students and law professors were conducted in Georgian by the Georgian-speaking member of the Evaluation Team. Two separate FGDs with members of the CITJ and attorneys were conducted by a trained facilitator in Georgian by a local research company while members of the Evaluation Team monitored the discussion in another room. The FGD with judges was conducted by members of the Evaluation Team through a translator. Local representatives from each of the target groups composed the FGDs. Participants in the judge FGD were all members of a new judges’ association, the Unity of Judges Association. The goal was to facilitate an open discussion in a non￾threatening environment to probe the issues in this Evaluation. Participants provided verbal consent at the start of each FGD. Discussions were then recorded, transcribed, and coded into NVIVO, a qualitative data analysis software. Questions asked during FGDs are provided in Annex 5. The Evaluation Team traveled to Kutaisi and Batumi to conduct fieldwork and attended a training outside of Tbilisi for law school librarians, as well as an all-day conference with international experts and judges from throughout Georgia on “The Empowerment of the Judiciary: The Way Forward,” held on July 13, 2013. 3.1.2 Quantitative Research and Analysis The Evaluation Team conducted two mini-surveys: one of the Coalition members, and the second of GBA MID-TERM PERFORMANCE EVALUATION OF JILEP 5 | Page members that had participated in continuing legal education. The methodology for these two surveys was:  Coalition Survey: IRMS created an online survey using Survey Monkey and sent the link to one contact person for each of the 32 coalition members via email. Each contact was given two follow-up phone calls (if needed), urging them to complete the survey. EPF also contacted Coalition members via email to urge survey completion. There was a response rate of 59% for this survey, with 19 of 32 members submitting a response.  GBA Survey: The Evaluation Team received two comprehensive reports from the Caucasus Research Resource Centers (CRRC), including “Attitudes Towards the Judicial System in Georgia,” and “Attitudes to the Judiciary in Georgia: Assessment of Legal Professionals and Business Leaders” (Attorney Baseline Survey).1 JILEP provided the raw data from CRRC’s Attorney Baseline Survey, from which the Evaluation Team developed a survey for attorneys to gauge progress. GBA provided the Evaluation Team with a full contact list of members who attended its training courses or those from one of its accredited providers to meet the mandatory continuing legal education requirements for GBA members. IRMS cleaned and prepared data on the sampling frame for further use, merging participants of trainings in both 2012 and 2013. As a result, 3,566 GBA members were identified as having attended at least one course offered by GBA. IRMS randomly sampled 150 members from this list and collected the data using two methods: 1) computer assisted telephone interviews (CATI); and 2) computer assisted self-interviews (CASI). CATI was used since some contact information included only telephone numbers and no email addresses, and obtaining email addresses was not efficient. A special data-entry system was pre-programmed and included rules and controls in line with the questionnaire’s logical flow. The data, using both CASI and CATI, was exported and processed using SPSS software, and checked for invalid and missing data. Open￾ended questions were recorded and standardized. 87 people completed the survey for a response rate of 58%. The results from this survey are referred to in this report as the GBA Survey. IRMS also surveyed the entire membership of the Coalition and the results from this survey are referred to in this report as the Coalition Survey. The Evaluation Team developed written surveys in English for attorneys and members of the CITJ. Many questions for attorneys tracked the language from the Attorney Baseline Survey, which was commissioned by JILEP in early 2012 for legal professionals and business leaders. After USAID approved the survey instruments, they were translated into Georgian. The Georgian-speaking Evaluation Team member ensured accuracy of the translations. 3.2 EVALUATION LIMITATIONS The Evaluation Team encountered few limitations during its fieldwork in Georgia. Some of the more relevant limitations are listed below: 1. Timing of the evaluation. Due to the fact that the evaluation took place in the summer, when Georgians, USAID staff members, and JILEP staff were on vacation, there was a shorter period in￾country than optimal to conduct the evaluation. The Evaluation Team mitigated this limitation by conducting extensive key informant interviews (KIIs), FGDs, and two surveys so that the required evaluation methodology could be completed in a shorter time frame. 2. Biases in data collection methodologies. To identify key stakeholders and project beneficiaries, the Evaluation Team relied on assistance from USAID and JILEP staff. Although there was the risk of selection bias, during meetings, the Evaluation Team did not discover any omissions of individuals with whom they had wanted to meet on the list of proposed meetings provided by USAID and JILEP. JILEP provided invaluable assistance in meeting with the Evaluation Team on numerous occasions, enhancing their understanding of the issues, and in helping to set up meetings. 3. Possible attenuated link between project activities and results. There are many variables that may have contributed to some of the results described in this report. Trying to ascribe a result solely to JILEP’s activities or to draw a causal link between JILEP’s activities and a particular result is difficult 1 This survey included the responses from both legal professionals (39) and business leaders (44). The Evaluation Team was supplied with questions and raw data and used only the responses of the legal professionals on which basis to make a comparison with the responses of a random sample of attorneys (87), discussed infra., as the GBA Survey. MID-TERM PERFORMANCE EVALUATION OF JILEP 6 | Page since the Evaluation Team cannot determine if some of these changes and results would have naturally evolved in the absence of JILEP (this is not an impact evaluation). 4. Availability of data. The Evaluation Team relied on baseline data developed through JILEP’s research firm, CRCC, which at the time of data collection was part of EPF. JILEP provided both summary and raw data and detailed the research methodology to allow the Evaluation Team to review and analyze the data. 5. Short duration of the project. JILEP only had two full years, 2011 and 2012, to show results before this evaluation was conducted in mid-2013. A project of this type, however, may take years to show results. The Evaluation Team analyzed data collected only a little more than a year apart to gauge what progress had been made. This is a very short period. 6. Time allotted for evaluation. The Evaluation Team was in-country for just over two weeks. The project’s activities and sub-activities are complex, with numerous counterparts in the government, institutions of higher learning, and NGOs. The scope of the evaluation narrowed some of the questions and mitigated some of this limitation. Narrowing the scope also meant that some of JILEP’s major activities were not included in the inquiry of the evaluation. While important, the above limitations did not prevent the Evaluation Team from gathering information and data to draw conclusions and make recommendations for similar, future USAID-funded projects. 4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS 4.1 QUESTION 1: SUPPLY AND DEMAND FOR AN INDEPENDENT JUDICIARY How effective has it been to include both the supply and demand for an independent judiciary in the same award? What has been the nature and level of interaction and relationship between these components, including among project beneficiaries and from a project management perspective? Is this approach recommended in the future? 4.1.1 Findings JILEP was designed: 1) to support an independent judiciary by both using direct assistance to core legal institutions and providing support to civil society; and 2) with flexibility to respond proportionately to the political will for reform. The project implemented activities to enhance cooperation between the judiciary and civil society and manage their supporting components. There was extensive2 interaction between civil society and core judicial institutions. As the judicial sector’s appetite for reform was limited at the outset of the project, JILEP shifted more activities to civil society. The overarching goal of JILEP was “to support and strengthen the judiciary as an independent yet equal branch of government…” JILEP combined two approaches to achieve this goal: 1) direct support of and engagement with state institutions; and 2) assistance to CSOs. Its design incorporated support to the supply-side (judicial institutions), and active assistance to the demand-side (non-governmental organizations (NGOs)). Component 1 was intended to strengthen judicial independence by directing interventions towards core judicial institutions, primarily HCOJ and HSOJ. The major issues in Component 1 included the appointment, promotion and discipline of judges, and decisional independence. Although support to NGOs was generally part of Component 2, Component 1 also specifically included a demand element that included civil society oversight of appointment and disciplinary proceedings. 2 The HCOJ was, according to a KII with a former executive of a large NGO, “absolutely closed.” Starting with a meeting in July 2011 on “Working Toward Court Transparency” between NGOs and members of the judiciary, interaction between NGOs and the HCOJ became more routine. In Ocotber 2011 alone, the HCOJ conducted three meetings with NGO representatives, international donors, and legal professionals to exchange viewpoints on the proposed draft amendments to the laws and regulations related to judicial selection, transfer, discipline of judges as well as the composition of the HCOJ. Quarterly Report 2011, Q4, p.6. MID-TERM PERFORMANCE EVALUATION OF JILEP 7 | Page Component 2 was directed at supporting legal professional associations and legal rights NGOs to stimulate demand for judicial reform. The Cooperative Agreement required the project to “monitor enforcement of laws and judicial performance and to build the capacity of the organizations.” This component was designed specifically to establish a new coalition of NGOs to create a demand for judicial reform. Major emphasis on supply and judicial independence The fundamental point of JILEP was to assess what plagued the judiciary, develop tools to address the identified issues, and stimulate a coalition of interests to accomplish the reforms necessary to strengthen the judicial system. When JILEP commenced its operations, it met with its counterparts in HCOJ and HSOJ who, although one would expect them to show a strong interest to work with the new project, were not eager partners. Indeed, according to one respondent, the Chief Justice submitted a letter to USAID expressing reservations about JILEP even before the project commenced. The Evaluation Team was unable to verify this communication3 but, based on KIIs, found that JILEP encountered challenges4 in creating a common agenda and determining priorities of cooperation with both HSOJ and HCOJ. KIIs disclosed several reasons for HSOJ’s and HCOJ’s ambivalence. First, the political atmosphere had shifted since the project’s design. Second, there may have been some donor fatigue as Georgia was the target of international assistance from numerous countries. Major donors had been beating a path to the Supreme Court and HCOJ so the core institutions could pick and choose with whom they wanted to work and in which areas. Third, there was limited absorptive capacity as the target organizations lacked personnel and other resources to devote to JILEP’s proposed activities. In any event, JILEP learned early on that there were very narrow areas on which its primary judicial counterparts, HCOJ and HSOJ, were willing to engage. Stimulating demand through JILEP The major mechanism through which JILEP was designed to stimulate demand was an NGO coalition, CITJ, organized by sub-awardee EPF, and inaugurated in April 2011. CITJ is comprised of about 30 NGOs with the goal “to strengthen the capacity of legal professional associations, legal rights NGOs, business associations, and the media in monitoring relevant judicial practices and advocating for an independent judiciary.” CITJ has been a major advocate5 of judicial independence through policy statements, studies, conferences, public forums, public information, and other means described in more detail in Section 4.3. JILEP’s design also envisioned stimulating demand through a series of grants provided either directly from or through EPF. In the first round, JILEP and EPF provided legal advocacy grants directly to two and eight organizations, respectively. In another round, EPF made an additional four awards. The activities under some grants thrust these organizations into the middle of judicial reform while other grants stimulated demand in different ways. For example, GYLA and TI monitored the activities of HCOJ, including publication of HCOJ’s decisions and meeting agendas. Under their grant activities, GYLA and TI also implemented a court monitoring program for criminal and administrative cases in select courts in various cities in Georgia. Under other legal advocacy grants, NGOs monitored the activities of LAC, conducted a campaign for greater court transparency, and undertook activities to stimulate demand within Georgia’s judicial system. JILEP also included a series of grants for free legal assistance. In the initial round, JILEP provided grants directly to TI and GYLA and, through EPF, to five other organizations to offer legal assistance by representing clients in human rights, property rights, labor rights, and violations in tax administration cases. TI, for example, provided legal counsel for those who thought they did not receive fair and reasonable compensation for expropriated land. GYLA provided legal assistance to a citizen who was assaulted by an officer of the Ministry of Interior Affairs. Some of these cases were litigated all the way to the Constitutional Court of Georgia and the 3 USAID is determining whether a copy of this letter can be found. 4 These challenges are discussed in more detail in Section 4.2.1 of this report and include, by way of example, the HCOJ’s failure to: implement the program to introduce a computer-assisted program for the judicial application and qualification examination; introduce changes in the standards for promoting judges; and adopt the recommendations of the JILEP expert on developing a communications strategy. As diplomatically related in an early quarterly report, the “HCOJ might not completely embrace JILEP’s message that increasing transparency should be a central goal of HCOJ’s reform efforts.” Quarterly Report, 2011, Q4, p.5. 5 Before formation of the Coalition and USG support through JILEP, the NGO community was “not taken seriously,” according to a KII with a government official previously in civil society. MID-TERM PERFORMANCE EVALUATION OF JILEP 8 | Page European Court of Human Rights and one case, filed in the European Court of Human Rights, challenged Georgian courts’ ordering of preventive measures without articulating a factual basis for their decisions. Another way in which the design of JILEP contributed to stimulating demand was through the support of two national centers of learning at two Georgian law schools. JILEP helped organize NCADR and NCCL. Assistance to these centers was directed towards the professional development of law professors, as well as creation of new courses and teaching materials. However, as they developed, NCADR and NCCL also stimulated demand for their services through their legal aid clinics and by submitting friend of the court (amicus curiae) briefs to the Constitutional Court of Georgia. JILEP was not involved in the grantees’ daily operations. Therefore, the project’s connection with activities was sufficiently attenuated so that interviewed government respondents did not raise the issue of U.S. Government (USG) involvement with, or interference in, supporting complaints against Georgian judicial or other governmental bodies. In no KIIs was JILEP criticized for providing funding for these grants or for playing both sides of the game. Cooperation between the judicial system and civil society contributed to judicial reform JILEP’s orchestration of cooperation between the judicial system and civil society, as well as its support for the HCOJ and CITJ, played a pivotal role in Georgia’s judicial reform. JILEP expertly used several tools to stimulate demand for this reform and to influence HCOJ not only to accept but also to champion some of the reforms. Through direct assistance to HCOJ, and indirect influence over CITJ, JILEP highlighted the need for the reform of judicial selection, evaluation, and disciplinary processes. Significantly, HCOJ drafted amendments to the Law on Courts and its internal rules. For a working meeting in December 2011, JILEP brought together representatives from HCOJ, NGOs, and the international community. JILEP properly indicated that the legislative proposals were “an important step in the direction of establishing a lasting framework for judicial independence and system transparency” even though “they did not go far enough.”6 Cooperation between HCOJ and civil society culminated in Parliament’s adoption7 of amendments to the Law on the Courts, the Law on the Disciplinary Responsibility and Disciplinary Proceedings of Judges, and the Law on Transfer of Judges in March 2012 – months before parliamentary elections in October 2012 brought a new government to power. NGOs did not have easy access to the judiciary, and particularly to HCOJ, because their meetings were not open before JILEP. One of the purposes of providing legal advocacy grants to GYLA and TI was to monitor HCOJ policies for bonuses to judges, financial declarations, transfers, and disciplinary proceedings. Not coincidentally, around the same time that these organizations started monitoring HCOJ, HCOJ held its first open meeting on April 3, 2012. JILEP provided direct assistance to HCOJ to launch a new website providing information about its structure and activities, and a calendar and agenda for upcoming meetings. According to one former HCOJ member, the “most important contribution to opening the system [up] was getting civil society involved in the system.” This monitoring eased pressure on individual judges, who, according to many judicial KIIs, were subject to capricious decisions of the HCOJ. These decisions related particularly to disciplining judges and transferring judges to new courts. JILEP provided grants to TI and GYLA to monitor criminal and administrative case proceedings in select courts. Court monitoring was new to Georgia, and without JILEP’s support and expertise, it would have not occurred.8 JILEP developed court monitoring methodology and trained NGOs in accordance with international standards. HCOJ was brought into the discussion and, despite roadblocks that some of the courts tried to place in front of NGOs, the initiative went forward. NGOs produced reports in June 2012 and April 2013 based on their objective monitoring of numerous cases. The reports were not critical of individual judges but 6 Quarterly Report 2011, Q4, p.3-4. 7 As discussed above in footnote 2, after the formation of CITJ, HCOJ held several meetings with civil society, entertained suggestions and recommendations, and, as it prepared its draft amendments, incorporated some of these suggestions into its proposed draft. As related during a KII with a judge, the “most important contribution to opening [up] the system was getting civil society inolved in the system.” 8 According to JILEP, the experience with court monitoring was “minimal” and the capacity “limited.” Quarterly Report 2011, Q1, p. 12. In the Evaluation Team’s KII with one of the organizations involved in court monitoring, this view was confirmed. MID-TERM PERFORMANCE EVALUATION OF JILEP 9 | Page exposed a system that was unfairly tilted in favor of the government in administrative and criminal cases. According to a KII with a government official, before this comprehensive monitoring initiative, the major actors in judicial reform, including the MOJ, Parliament, and the HCOJ, could dismiss anecdotal evidence; however, with the court monitoring reports showing the seemingly automatic granting of prosecution motions and the extremely low acquittal rates, the findings could not be readily dismissed.9 CITJ and GYLA published a comprehensive report that included a systematic analysis of the core problems facing the judiciary, which generated international attention that resulted in renewing momentum for continued judicial reform. Thereafter, CITJ and GYLA publicized the report’s highlights on transparency of disciplinary proceedings, promotion processes, and court proceedings in a forum in July 2012. The forum brought together both the NGO community and the Chair of the Supreme Court. The good will that JILEP had fostered between civil society and the judicial system came under severe strain in light of the parliamentary elections. After the elections, the new government was more receptive to changes, not all of which were endorsed by the HCOJ. Nevertheless, according to several KIIs with representatives both in and out of government, the Minister of Justice began in earnest to prepare the legislative framework for fundamental changes, including the composition of the HCOJ. According to the Deputy Minister of Justice, the MoJ’s proposed changes were based on CITJ’s July 2012 Report on the Judicial System in Georgia. JILEP itself admitted that it was important to “maintain an objective approach and a long-term view of judicial reform” and understood that it needed to take “special care…to make the court leadership understand that JILEP greatly values their cooperation and wishes to continue to work together no matter what the eventual composition of the HCOJ.”10 JILEP’s role as honest broker between competing visions of the judiciary was put to the test and JILEP had to maneuver through difficult political waters. According to the Chair of the Supreme Court, JILEP did not fill this role well as both JILEP and CITJ failed to value their continuing cooperation with HCOJ. As JILEP reported, the Chair “noted that the HCOJ had been working with CITJ for more than a year and managed to agree on many important reforms,” but that after the elections, “the new government and MOJ took the lead and submitted all the amendments to Parliament that the HCOJ intended to submit but included two sections with which the judiciary disagreed.”11 During the Evaluation Team’s KII with the Chair, he reiterated his view that JILEP and CITJ sent their ideas directly to the new MOJ and did not work collaboratively with the HCOJ.12 According to the Chair, even though some of the amendments were directly contradictory to the independence of the judiciary, no one from CITJ opposed the amendments. JILEP disagrees with the Chair’s assertion and maintains that it went to great lengths to keep the door open with the HCOJ and took an even-handed approach during this politically volatile period. In any event, the project and CITJ submitted their recommendations not directly to the MOJ, but to the Judiciary Reform Working Group created under the Criminal Justice System Reform Inter-Agency Council that is led by the MOJ. The Judiciary Reform Working Group is a collegial body comprised of representatives of civil society, international donor organizations, GBA, MOJ, and importantly, the judiciary. In light of the Chair’s attitude, after parliamentary elections, cooperation with the HCOJ cooled and it became more closed, returning to a “less-than-transparent mode of decision making.”13 After a cooling off period, 9 According to the GYLA Monitoring Report, judges did not automatically grant the prosecution’s motion on pre-trial detention. In cases that it monitored, GYLA noted that the judges ordered bail over the proseuction’s objection for 13 defendants, representing 27% of the cases it monitored. GYLA Court Monitoring Report, July-December 2012, p. 9. GYLA points out that these hearings occurred after the October 2012 parliamentary elections and ten of the defendants were former government officials. Consequently, it is not possible to solely credit the court monitoring program for the judges’ new-found independence from prosecutors, although the court monitoring program documented the progress and may have contributed to this outcome. Nevertheless, according to several KIIs with both Coalition members and judges, the court monitoring had a major impact. One judge suggested that the results of court monitoring significantly shaped the recent packet of amendments on the judiciary. 10 Quarterly Report 2012, Q4. 11 Quarterly Report 2012, Q4, p. 16. 12 An opposition member of parliament offered a similar view during a KII, maintaining that since the elections, cooperation with the parliamentary minority did not continue. 13 Quarterly Report 2013, Q1. MID-TERM PERFORMANCE EVALUATION OF JILEP 10 | Page HCOJ began again providing notice of its meetings, in line with previous commitments. Nevertheless, tensions between civil society and the judicial system, and MOJ and HCOJ may continue through the project’s end. The Minister of Justice, who had urged amendments requiring a new selection of judge representatives on the HCOJ, was hopeful that a new composition of the HCOJ would favor her views on judicial reform. With the election in June 2013, the allies of the Chair were easily elected by the Conference of Judges and the Chair’s influence will likely remain intact. Disappointed, the Minister of Justice conveyed her views that the selection of the new judges was “strange” and that she “…cannot but note that these figure[s] were elected without any problems in the first round.”14 After these major changes that ensued following parliamentary elections and the new composition of the HCOJ, JILEP has largely kept intact its successful working relationship with the HCOJ, MOJ and CITJ. GBA Survey of Attorneys shows that attorneys still consider the judiciary not independent According to the GBA Survey, over 40% of respondents “completely disagree” that judges are independent (see Figure 1 below). In the comprehensive Public Opinion Survey15 that JILEP commissioned, 23% of respondents who had a court experience during the last two years or knew a close friend or relative who had, believed judges to be independent (4 or 5 on a 5 point scale). Using the same question on the GBA Survey, only 9% of respondents agreed. Although, as discussed below, some judges certainly believe that they have become more independent, based on the GBA Survey, there remains considerable doubt among attorneys about the level of independence of judges in Georgia. Figure1: On a 5-Point Scale, to What Extent do you Agree or Disagree with the Opinion that Overall, Judges are Independent In Georgia? (%) 4.1.2 Conclusions Still early to assess the long-term results of the cooperation. It is too early to assess how these structural changes may affect the judiciary and the perception of judicial independence. Although judges, during KIIs, unequivocally and unanimously discussed that HCOJ and Chairs of the Courts have interfered less in their work, this critical finding was not translated into results of the GBA Survey. The perception of independence is perhaps a lagging indicator. Design allowed JILEP to reallocate resources. JILEP’s flexible design allowed the project to shift resources and devote more time, energy, and money into other components.16 If one component lacked the requisite political will to forge change or absorptive capacity, then resources could be reallocated. For example, when work with HSOJ was not gaining much support, JILEP spent more time and resources on other components, such as support for civil society. Design allowed JILEP to play a broader role. If the project worked with just one side, it would tilt the balance in favor of one or the other. If there was no support within state institutions for change, then civil society could petition for all of the changes it wanted, but without the capacity and desire to change from within state organizations, there would have been no reforms. 14 Justice Minister Unhappy Over HCoJ Election Results, Civil Georgia, Tbilisi, 10 June 2013. 15 The Public Opinion Survey implemented in May-June 2011 included 4,318 respondents throughout Georgia. 16 The Evaluation Team was not asked to and did not conduct an analysis of project expenditures in comparison to project budget. The conclusion is based on discussions with JILEP personnel who, as initiatives bogged them down with HSOJ and HCOJ, mentioned that project resources of time and energy were reallocated to other activities in which there was a desire on the part of the counterparts to cooperate. 40.2 12.6 37.9 5.7 3.4 0 10 20 30 40 50 Completely disagree Somewhat disagree Neutral Somewhat agree Completely agree MID-TERM PERFORMANCE EVALUATION OF JILEP 11 | Page Not every state institution views the role of civil society as constructive and necessary. One would not expect that civil society would be as suspicious of the project as it provides support to state institutions. It is not likely that NGOs would refuse assistance and support because the project is directly assisting state organizations. However, state organizations could have closed the doors if the project was seen to unfairly favor CSOs. In short, as one government respondent said: “The government likes what the [Coalition] is doing. They expose problems in a gentle way…The [Coalition] was the correct strategy. It worked.” JILEP and CITJ, although influenced by each other, maintained their separate and distinct identities. For example, when CITJ planned and organized seven public forums, it and its leadership prepared the agenda. As JILEP related to the Evaluation Team, “we aren’t removed from the process but we are observers.” Similarly, in the FGD with members of CITJ, one participant remarked that JILEP “supports us” but “does not influence our decision.” According to this participant, JILEP shares its opinion and experience, but never influences the decision and does not apply pressure. JILEP was generally allowed to increase its influence in the reform process as it was not perceived automatically to endorse the views of all CITJ members. JILEP’s position may have been reduced since the highly charged debate relating to the package of amendments in the wake of the parliamentary elections in October 2012, but generally JILEP’s role is still secure both within the NGO community and the judicial system. JILEP moderated the exchanges between civil society and government institutions. As civil society and state institutions engaged one another there was a natural friction between these two actors in developing a mutually agreeable plan for reform. By working with both sides, JILEP moderated the more strident views of the various actors to reach compromise or consensus.17 As the project gained allies within the civil society sector and less traction within judicial organizations, it could tilt its assistance and energy to that side. This approach worked especially well before the parliamentary elections in October 2012 and it has been severely tested since then. On the one hand, JILEP contends that this approach shielded the Supreme Court from the harshest criticism from the new government and civil society. The Chief Justice, for his part, was critical of both JILEP and CITJ after the Parliamentary elections and suggested that JILEP sent its proposals directly to MOJ, without first airing them with the HCOJ.18 He contends that the new packet of legislative amendments is contradictory to the independence of judges. His primary objection is that JILEP and CITJ worked with the HCOJ before the parliamentary elections, and altered its approach afterwards. Without accepting either version, the Evaluation Team concluded that JILEP has been an expert buffer between the various actors. Having the purse strings attached to both civil society and the court system, and having the backing of the USG allowed JILEP to play this role. Moreover, JILEP has ably navigated its role in perilous political waters. The design allowed JILEP to probe the opportunities for reform in the judicial system. By working with both state institutions and civil society, the project could condition assistance to state institutions on requirements that they become more transparent and open, as well as identify areas of opportunity for civil society to be more effective in the reform process. JILEP expertly orchestrated the cooperation of both judicial institutions and civil society to advance the cause of judicial reform. JILEP had to confine its direct efforts with the HCOJ and HSOJ to those activities that the institutions chose. If the HSOJ did not want JILEP to work with it on curriculum development, then JILEP would not do so. To JILEP’s credit, the project continued to press and look for areas of opportunity so that it could remain engaged. 17 According to one KII who was not part of the COTJ, JILEP was able to push some things forward and “put the brakes on some crazy ideas.” A member of parliament phrased it slightly differently, maintaining that NGOs are “less arrogant” than before JILEP’s commencement. Similarly, in those cases in which JILEP agreed with the positions of CITJ, it could leverage its views by tying JILEP's position to that of civil society. 18 As discussed above, JILEP submitted recommendations not to the MOJ but to a working group comprised of a diversity of interests, including members of the judiciary. MID-TERM PERFORMANCE EVALUATION OF JILEP 12 | Page 4.1.3 Recommendations Support for civil society and core legal institutions in one project is an effective lever for reform. Based on the conclusions set forth above, there were many advantages and few disadvantages in implementing a design in which the same project provided support to the supply side, the core judicial institutions, and the demand side, civil society and those who use the judicial system. Table 2 below, shows some of these advantages and disadvantages. Figure 2: Advantages and Disadvantages of the JILEP Project Design Advantages Disadvantages Project may reallocate resources from one target of assistance to another Project may be perceived to be unduly influenced by one or the other targets of assistance Project may leverage its influence to play a broader role Project may neglect one or the other targets of assistance Project may moderate the exchanges between civil society and government institutions Conserve resources and no duplication of efforts, time and human resources No extra layer of coordination between projects Project can gather further intelligence on how to provide effective assistance to both civil society and judicial institutions Project may gain more traction in advancing a reform agenda with judicial organizations by tying its views to those of civil society Working with both groups nurtures cooperation and understanding This approach may not work in every situation, for example in countries in which judiciary corruption runs so deep that any cooperation with it could undermine the integrity of the implementing organization. In general, in most circumstances in which both civil society and core legal institutions are targeted for assistance, activities should be provided under one project. An alternative approach is to split the assistance in separate projects; however, this approach would be unwieldy, costly, and unproductive because it would require the creation of parallel projects with parallel and potentially overlapping activities. Not only would there be a risk of duplication of efforts and lack of coordination, but there would be the additional expense of multiple offices and staff. Another approach would be to eliminate assistance to either the judiciary or to civil society. The result would be to either assist only the courts, neglecting civil society and thus providing no assistance for checks on the courts, or to assist only civil society, thereby attempting to reform the courts without providing any tools to respond to the demand for reform. Either approach would be unbalanced and would likely not provide significant sustainable reform. Based on JILEP’s experience, a design whereby assistance is provided in the same mechanism to both the judicial system and civil society is the preferred approach and a similar one can be implemented in other countries. An aggressive exit strategy should be implemented. The Cooperative Agreement, dated September 24, 2010, and JILEP envisaged a sound exit strategy that focuses on “ensuring the sustainability of JILEP’s expected achievements” structured around “strengthened governmental and non-governmental institutions” and “establishing new USAID legacy institutions.”19 The Evaluation Team endorses this view and recommends that every remaining activity be viewed through the prism of sustainability. The project has invested in establishing CITJ and building the capacity of various NGOs within CITJ and the judicial sector organizations. The success of the cooperation between civil society and the judiciary will continue after the end of JILEP if there is continuing internal support for these organizations from their members, the government, and third parties. We strongly recommend that sustainability should guide the allocation of scarce resources to the end 19 Cooperative Agreement, Program Description, p. 27. MID-TERM PERFORMANCE EVALUATION OF JILEP 13 | Page of the project. This principle should guide the assistance to both civil society and the court system. Court monitoring is a good example of how JILEP supported the introduction of a new methodology for judicial reform and then worked to standardize and institutionalize it. Local organizations now have the expertise and resources to continue court monitoring after the project’s end. 4.2 QUESTION 2: RESULTS DESPITE A LACK OF POLITICAL WILL IN THE JUDICIARY FOR REFORM Recognizing the political will for judicial independence has been weak, which (presumably less politically sensitive) technical areas and/or activities has JILEP managed to work in and/or complete with the judiciary itself? How effective have those activities been in contributing to judicial independence? Considering the changed political environment, what new openings and opportunities exist for working directly with the judiciary to achieve program goals, both through the end of JILEP and beyond? 4.2.1 Findings JILEP managed numerous activities with the judiciary geared towards contributing to judicial independence, transparency, and professionalism. Activities commenced in September 2010 at a time when political will for judicial independence, transparency, and professionalism was weak. Therefore, not all were completed. JILEP’s activities closely followed the technical areas listed in the Cooperative Agreement between USAID and EWMI, and are examined below for completion. Strengthen appointment, reappointment, and promotion processes for judges The HCOJ enjoyed extensive discretionary powers over the appointment, reappointment, promotion, and transfer of judges, which led to concerns over the objectivity of these fundamental judicial processes and the qualifications of judges. In order for Georgia’ system to meet international standards of judicial independence, and at the request of the HCOJ, JILEP employed the services of Mr. Timothy Baland, a retired U.S. judge, to work directly with the HCOJ to develop a merit-based and transparent judicial selection process. Judge Baland conducted a workshop with the HCOJ, which included the Chief Justice of the Supreme Court and Secretary of the HCOJ, and provided written recommendations for the HCOJ to improve the candidate application and interview process. As a result of his advice, JILEP helped the HCOJ create a detailed judicial application form that was partially used for the November 2012 processing of judicial applications. Following the November 2012 application process, JILEP offered support to the HCOJ to improve the judicial examination system by creating an aptitude section of the exam. Also at the request of the HCOJ, JILEP worked with the HCOJ and ONYX, a Georgian software company, to develop a software system to provide a platform for the application, administration, and grading of judicial qualification examinations, and support the judicial candidate interview and evaluation process. The goal of this effort was, by April 2012, to develop software that provided anonymity and random selection in the grading and interview processes to reduce arbitrary and politically-motivated judicial selection. Although an aptitude test is fundamental to the judicial selection process and the HCOJ requested that JILEP assist in developing it, the work was stalled and is thus incomplete because the National Examination Center has continually postponed its contract with JILEP. Additionally, in November 2012, the HCOJ ignored the software system’s benefits by failing to publish critical information related to judicial vacancies and the application process such as the number of court vacancies, applications submitted, short-listed candidates, and applicants selected. In a recent application process, the HCOJ used the software platform’s online application portal only to receive applications from candidates. At the request of the Criminal Justice Reform Inter-Agency Coordination Council (IACC), JILEP participated in roundtable discussions on HCOJ’s internal judicial selection rules. JILEP sponsored working meetings with the HCOJ, CITJ, and others to provide recommendations to HCOJ’s judicial selection rules. Ultimately, the HCOJ drafted amendments to their rules, which included JILEP’s recommendations pertaining to: 1) evaluation criteria for listener judges and former judge candidates; 2) documents required for the application process; and 3) in-person interviews with short-listed judicial candidates. Regarding the promotion of judges, JILEP sponsored two judicial conferences with the Supreme Court Chief Justice and staff, HCOJ Secretary and staff, and chief trial court judges. The purpose was to discuss the efficacy of the new court performance measurement system developed by the HCOJ and its new performance MID-TERM PERFORMANCE EVALUATION OF JILEP 14 | Page evaluation rules, which appeared to favor a top-down administrative approach to controlling judicial behavior and discourage judicial independence. Recognizing the need to develop objective criteria to measure judicial performance and a systematic mechanism to evaluate their work as grounds for promotion, JILEP employed the services of Mr. Richard Mohr to assess HCOJ’s new rules. Mr. Mohr’s assessment highlighted several provisions of the rules, which, if not revised, could dampen judicial independence. The HCOJ has yet to act on Mr. Mohr’s assessment. Improve judicial disciplinary procedures According to the disciplinary provisions of the Law on Disciplinary Responsibility and Disciplinary Proceedings of Judges of the Common Courts of Georgia (Law on Disciplinary Responsibility), judges were dismissed by the HCOJ on the basis of a “gross violation” of applying the law. Rather than appeal a lower court decision, claimants would file with the HCOJ a complaint about a judicial decision with which they disagreed. Claimants remained ignorant of the outcome of their complaints because disciplinary proceedings and decisions were confidential. The appellate process dealing with the way in which incorrect interpretations of the law were to be reviewed and corrected was ignored. Disciplinary actions were also undertaken for insignificant violations of the labor code as a means of punishing noncompliant judges. JILEP utilized the services of Ms. Victoria Henley, Director and Chief Counsel of the California Commission on Judicial Performance, to provide advice and guidance to the HCOJ to ensure its disciplinary procedures were consistent with international standards. Ms. Henley provided recommendations on the legislative amendments drafted by the HCOJ, particularly with regards to the need for transparency of disciplinary procedure outcomes. In light of the legislative amendments, and at the request of the HCOJ, Ms. Henley conducted a workshop for HCOJ staff and members on assessing the grounds for disciplinary action and improving internal processes for investigating complaints. As a result of JILEP’s expert advice, the Law on Disciplinary Responsibility was amended in March 2012 to eliminate the abovementioned grounds for judicial discipline, require that disciplinary decisions, with names redacted, be posted on the official HCOJ website, and require that the claimant be notified of the decision. However, in recent disciplinary proceedings, the HCOJ reprimanded judges on the “improper fulfillment of duties,” a criteria which JILEP had recommended be eliminated for being too broad for discipline and an infringement on judicial independence. The Law on Disciplinary Responsibility was further amended in May 2013 to ensure independence of the Disciplinary Collegium from the HCOJ. Develop practical guidelines for the HCOJ, judges and legislative advocacy While JILEP’s reports do not specifically address this technical area, the project did engage in activities relating to the punitive transfer of judges and HCOJ voting procedures, which required the acquiescence of all bodies represented on the HCOJ rather than a simple majority vote that, in effect, gave the executive and legislative branches control over HCOJ decisions. JILEP assisted the HCOJ in drafting amendments to the Law on the Transfer of Judges, which Parliament passed in March 2012, to reduce the length of a judge’s transfer to one year and to require the judge’s consent to the transfer. This revision was necessary to deal with improper influence on a judge’s decision, particularly court￾based meetings between a chair of the court and an individual judge to discuss the judge’s decision-making process for a specific decision. According to one key informant, the rate of transfers has been reduced by approximately 50% since the amendment’s passage. As noted by all eight judges of the Unity of Georgian Judges with whom the Evaluation Team met, such meetings could result in the lengthy transfer of a judge to another court as punishment if he/she did not accommodate the Chair’s advice. During meetings with MOJ and IACC, JILEP and CITJ presented policy recommendations related to the composition of and appointment to the HCOJ. Following the October 2012 elections, the new Minister of Justice drafted an amended Law on the Common Courts, passed by the Parliament in May 2013, which incorporated many of the policy recommendations of JILEP and CITJ, including the transfer of power from the President of Georgia to the Conference of Judges to nominate and appoint judge members to the HCOJ. Improve transparency and public outreach for the HCOJ A major concern with the HCOJ was its tendency to conduct operations and decision-making processes in secret. The public, including judges, were informed of essential matters affecting the judiciary, such as appointment and discipline, after the HCOJ’s deliberations. MID-TERM PERFORMANCE EVALUATION OF JILEP 15 | Page As a means of increasing the transparency of the HCOJ, JILEP utilized the expert services of Dr. Mark West to advise the HCOJ on its outreach and communications strategy. Dr. West conducted a workshop with the HCOJ in which recommendations, such as making HCOJ meetings regular and public, and enhancing the HCOJ’s new website to include judicial disciplinary issues, were favorably received. Following Dr. West’s recommendations, the HCOJ hosted numerous meetings with NGO representatives, journalists, and donors to discuss judicial reforms and draft amendments to laws. With JILEP’s support, the HCOJ created a website (http://hcoj.gov.ge/) assisted by Digital Design Studio to inform the public about its composition, mission and activities, including HCOJ meeting dates and agendas. As a result of JILEP’s assistance and Dr. West’s recommendations, the HCOJ began in 2012 to schedule regular quarterly meetings open to the public, publicize agendas in advance of the meetings and minutes following meetings, and include in the public record documents and decisions related to the meetings. Following the October elections, the HCOJ closed its meetings to the public and refused to publicize agendas or meeting minutes. During these meetings, crucial decisions were made regarding judicial appointments, transfers, and bonuses. With the new composition of the HCOJ, meetings again were opened to the public. Dr. West also assisted the HCOJ in drafting its first “Public Trust and Confidence Plan” to assist courts in educating the public about the justice system, which the HCOJ adopted and JILEP helped to implement, including the creation of a public service announcement. As part of its work with the HCOJ to improve transparency and public outreach, the Supreme Court reached out to JILEP to assist in its development of a public communication and outreach strategy for the entire court system. During his fourth visit to Georgia, Dr. West drafted a Communications Strategy for the Supreme Court of Georgia and Common Courts, which he later revised following suggestions by the Supreme Court. After Dr. West submitted the revised Communications Strategy, the Supreme Court did not have further meetings with him nor did it demonstrate any interest in implementing the Communications Strategy. Civil society oversight of appointment and disciplinary procedures Given the secrecy with which the HCOJ conducted its deliberations, JILEP provided a targeted grant to GYLA and TI to monitor the HCOJ and publish its findings. With JILEP’s grant funding, GYLA and TI notified the HCOJ of their intention to monitor HCOJ’s meetings to ascertain the transparency of its decision-making processes dealing with selection, appointment, transfer, evaluation and promotion of judges, and judicial discipline and remunerative policies. In response, HCOJ welcomed the initiative and pledged to cooperate. While the HCOJ initially cooperated with GYLA and TI and not only welcomed them to its meetings but also responded to various requests for information, it gradually became closed and less willing to engage, either by not responding to information requests or by providing incomplete information to GYLA and TI. After a ten￾month effort to monitor the HCOJ, GYLA and TI presented their written findings at a conference in which many shortcomings of the HCOJ, particularly its lack of transparency, were highlighted. While HCOJ members disagreed with some of the findings, there was an open exchange of views resulting in the Deputy MOJ offering to initiate legislative amendments to resolve some of the shortcomings. While the HCOJ was closed for a period of time, which was highlighted as a shortcoming, the meetings have again become open. Strengthen the High School of Justice Having been created by law in 2005, the HSOJ, as a government entity with its own line item in the courts’ budget, provided sufficient training courses to judicial applicant “listeners” and CLE courses to sitting judges with support from donors. However, it did not have training standards focused on curriculum design, program evaluation, and interactive teaching methodologies, nor did it have a system of measuring the impact of its training and identifying future training programs. Essential topics, such as judicial ethics, judicial skills, and gender-related issues, were not mainstreamed into its courses. JILEP began its activities with the HSOJ at its request by providing two tax code trainings for sitting judges. After much resistance to JILEP’s offers of assistance, particularly with developing a long-term planning process, designing a CLE curriculum for sitting judges, and assessing its new judge training program, the HSOJ ultimately agreed to host JILEP’s expert, Professor Delaine Swenson, to conduct a teaching methodology course for 12 Georgian judges who teach at the HSOJ. Following the election, the HSOJ also agreed to let JILEP develop and deliver CLE courses on various topics. JILEP’s experts, Professor Jim Alfini and Seana Willing, conducted two training courses on judicial ethics for sitting judges, and just recently completed four additional trainings on MID-TERM PERFORMANCE EVALUATION OF JILEP 16 | Page judicial ethics which received significant praise from the judges who attended, according to the Chief Justice. JILEP trained judges from the Tbilisi and Kutaisi Appellate courts on international trade law, and, in collaboration with the HSOJ and the United Nations Commission on International Trade Law (UNCITRAL), trained judges on international and domestic commercial arbitration. Lastly, JILEP conducted a course on legal reasoning and statutory interpretation for appellate and trial court judges. According to a senior justice official, “JILEP’s approach to training is excellent by partnering American judges with Georgian judges.” While JILEP made several attempts to assist the HSOJ to undergo a fundamental evaluation of its training programs, JILEP was repeatedly rebuffed and fell considerably short of completing its activities under this technical area prior to the October 2012 election. Strengthen the Judges Association of Georgia (JAG) Founded in 1999, JAG, a voluntary association, was created to: improve the judicial profession; protect the independence of judges; enhance the prestige of judges; protect judicial rights and professional, material, and social interests; increase public confidence in the judiciary; and improve the court system. While a percentage of judicial salaries are allocated to JAG, and while almost all judges are members, most are not active members. As previously stated, JAG has become a moribund organization and the leadership seemed to prefer that the organization remain passive. In an attempt to twin JAG with a successful judges association, JILEP sent five JAG members to Poland to meet with representatives of Justitia, the Polish Judges Association, to learn about their organizational development and activities, including engaging in legislative advocacy, developing a funding base, and improving public reputation. JILEP was not able to interest JAG in hosting Justitia leadership in a reciprocal visit to Georgia as JAG did not believe there were benefits from associating with judges who operated in a justice system managed by MOJ. Thus, all potential activities with JAG were put on hold, except for a JILEP and German International Cooperation (GIZ)-supported international roundtable with JAG on mediation and ADR. The small grants program to JAG to support activities that promote public education, and transparency and fairness in judicial appointment, advancement and disciplinary procedures was not activated, nor did JILEP develop and implement an Institutional Development Index with JAG to guide and measure its overall development. As a result of JAG’s lack of interest, JILEP was unable to complete its activities under this technical area. While several of JILEP’s activities were incomplete, according to a KI JILEP did not spend money for the sake of spending money. It spent money to shift things. JILEP looked at the situation realistically and knew when to put the breaks on some activities. JILEP was not alone. The United Nations Development Programme (UNDP) suspended its work with the Supreme Court when it did not find a cooperative partner. Increase access to information in the courts As with the HCOJ meetings, court hearings were closed to the public and court documents unavailable. There was scant freedom of information compliance and standards for attaining information from the courts were non-existent. JILEP provided targeted grants to GYLA and TI to monitor the courts with the aim of increasing access to court information. After conducting several meetings with GYLA and TI, JILEP realized that although GYLA and TI had extensive experience in legal aid and legal advocacy, they were inexperienced in court monitoring. Thus, JILEP engaged the services of Mr. Neil Weinstein to assist GYLA and TI in developing a court monitoring plan, which included selecting and training court monitors. GYLA began to monitor criminal cases at the Tbilisi City Court, while TI began monitoring administrative cases at the Tbilisi and Batumi City Courts and later extended its efforts to first instance courts in Gori, Telavi and Khelvachauri. Mr. Weinstein returned twice to Georgia to assist GYLA and TI in analyzing the data from their court monitoring, revising their methodology, and presenting their court monitoring reports in a user-friendly format. To further improve GYLA’s and TI’s court monitoring methodology, JILEP organized a workshop with U.S. Department of Justice (DOJ) court monitoring expert, Ms. Ani Morgoshia, whose advice resulted in GYLA and MID-TERM PERFORMANCE EVALUATION OF JILEP 17 | Page TI modifying their methodologies. With JILEP support and CRRC training, GYLA and TI began using an electronic database program for inputting data and Excel for analyzing it. Following several months of court monitoring, GYLA and TI presented their findings at a public event in Tbilisi attended by representatives of the HCOJ, Supreme Court, MOJ, GBA, and the media, and at subsequent public events. The findings revealed a preference for prosecution and plea agreements, for the government in administrative cases, an overuse of pre-trial detention, a low acquittal rate, a lack of respect for the defendant, judicial passivity, and accessibility of public hearings. 4.2.2 Conclusions JILEP’s activities effectively contributed to judicial independence, transparency and professionalism, but more results could have been achieved had political will been stronger. Without a doubt, the combination of JILEP’s activities, particularly its support of and collaboration with CITJ, contributed to a structural change in enhancing judicial independence, transparency, and professionalism as a result of several legislative amendments to the Organic Law on Common Courts, the Law on Disciplinary Responsibility and Disciplinary Proceedings of Judges of Common Courts, the Law on the Assignment of Cases and Transfer of Judges to Courts, and the Law on the High School of Justice. One fundamental, albeit theoretical, outcome of the legislative changes is that the judiciary is no longer an institution of subordination. In practice, however, there is still work to be done to ensure that judges are no longer subordinated to the chair of the courts, and that the chair of the courts is no longer subordinated to the HCOJ. According to several CITJ members, judicial and other governmental officials, and international donors, JILEP and CITJ were influential with regards to the first set of legislative amendments in March 2012, and according to the Chief Justice, were responsible for at least 90% of the substance of the second set of legislative amendments in May 2013. Even the new Minister of Justice publically proclaimed that the origin of many of the 2013 amendments were from JILEP and CITJ. According to a former Chair of the CITJ Steering Committee, JILEP was a strategic partner of CITJ. JILEP opened the previously closed doors of justice institutions to CITJ, particularly the HCOJ, which were “unimaginable to ever be opened to civil society organizations.” Several KIIs indicated that JILEP’s direct work with justice institutions and the advice and guidance its experts provided in various technical areas complemented CITJ’s work to such a degree that legislative successes are attributable equally to JILEP and CITJ. However, one international donor stated that “you cannot underestimate the influence of JILEP.” Despite several setbacks, delays, resistances, and refusals, JILEP was able to achieve results with its individual activities that, collectively, also led to a more independent, transparent and professional judiciary. While the steps were small, due to a lack of political will to carry through JILEP’s activities to their completion, they laid the foundation and were essential for achieving an independent and accountable third branch of government. The results, however, could have been considerably greater had political will been stronger. JILEP’s activities to strengthen the judicial appointment, reappointment, and promotion processes, particularly with the revisions to HCOJ’s internal rules – Evaluation of Efficiency of Judges’ Performance – were an initial step in developing a legal and administrative framework for judicial independence and professionalism and thus deserve credit, even though some activities were thwarted or ignored by the HCOJ when political will was weak. The HCOJ continues to have extensive discretionary power over judicial appointment and promotion as flexible criteria for judgeships and considerable arbitrary decision-making processes remain within the HCOJ. While the amendments to the Organic Law on Common Courts – changing the fundamental role of the HCOJ to safeguarding the independence of the judiciary and the individual judge, eliminating the veto power of the President and Parliament over judicial candidates and, instead, requiring that judicial candidates be appointed by a majority vote of the HCOJ members – are laudable, they did not address specific criteria for judicial qualifications and selection that would result in objective, fair and transparent judicial appointment procedures. JILEP-supported software can be a useful tool for the application, administration, and grading of judicial qualification examinations. While the HCOJ used the new online portal to process judicial applications, there is no indication that HCOJ systematically used it or utilized its full benefits, resulting in a less transparent process than intended. MID-TERM PERFORMANCE EVALUATION OF JILEP 18 | Page JILEP’s work with improving judicial disciplinary procedures nominally helped enhance judicial independence and transparency by limiting the control of the HCOJ over the decision-making process. The disciplinary process is no longer shrouded in secrecy, thus providing the public with a modicum of transparency in the outcome of disciplinary proceedings. Although the amendments to the Law on Disciplinary Responsibility can be applauded, and while disciplinary decisions with names redacted are publicized, the substance of the disciplinary process remains flawed. Parties to a case must utilize the appellate process to seek redress for an incorrect interpretation and application of the law, yet they continue to file complaints with the HCOJ for judicial decisions with which they disagree. JILEP’s recommended amendments to the Law on the Transfer of Judges limiting the term of transfer and requiring the consent of the judge, gave judges more confidence in rendering decisions based on the facts and applicable law rather than on the request of their superior, according to several judges who are members of the Unity of Georgian Judges, described on page 16, as such “independence” would have been punished by lengthy transfers to other courts. While the terms of judicial transfer have been revised, this is the first step towards improving judicial transfer proceedings. The May 2013 amendments to the Law on the Common Courts, which JILEP and the Coalition, among others, recommended, had a significant impact on the independence of the judiciary as the amendments changed not only the composition of the HCOJ, thereby giving more power to the judicial branch, but also the election of members to the HCOJ, which empowered the Conference of Judges with authority to participate in the justice system of Georgia and to engage in judicial reforms. The results of the Conference of Judges’ election of members to the HCOJ, while democratic, were disappointing to many informants as they believe the new members appear to represent the old ideas of the former administration. Additionally, they believe the new members owe their allegiance to, and need the protection of, the Chief Justice. As a result of JILEP’s work with the HCOJ, through the services of Dr. Mark West, the transparency of the HCOJ’s deliberations were improved, and its outreach and relations with the public and media through its meetings and website were extended. Although JILEP’s work with the HCOJ was intensive, particularly because the HCOJ showed signs of genuine interest in improving its transparency and outreach to the public, and while JILEP’s activities produced outputs such as the Outreach and Communications Strategy, the HCOJ website, and the HCOJ’s public meetings, when the political will of the HCOJ altered after the election, it was inconsistent in ensuring full transparency of its operation causing JILEP’s effectiveness to wane. According to the GBA Survey of members, lawyers in Georgia still see a scope and urgency in undertaking further reform in both the HCOJ and HSOJ. As seen in Figure 3, below, 77% of respondents said that the reform of the HCOJ in terms of the ultimate goal of creating a fair and impartial judiciary in Georgia is “very urgent.” Only 3.3% felt it is “not urgent at all” or “not urgent.” For the HSOJ the corresponding responses were 67% and 5.7%. Figure 3: On a Five-Point Scale, How Would you Assess the Urgency of Reform of the HCOJ in Terms of the Ultimate Goal of Creating a Fair and Impartial Judiciary in Georgia? (%) JILEP’s inability, despite its willingness, to work closely with the HSOJ resulted in minimal impact on the professionalism of judges; however, JILEP’s recent judicial ethics courses and other substantive courses this 0 20 40 60 80 Not urgent at all Not urgent Neutral Urgent Very urgent 2.3 1.1 9.2 10.3 77 3.4 2.3 5.7 21.8 66.7 HCOJ HSOJ MID-TERM PERFORMANCE EVALUATION OF JILEP 19 | Page year portend a brighter future with the HSOJ and considerably greater results if this trend of cooperation continues. While JILEP had a nominal working relationship with JAG and little results, an unexpected outcome of its activities is that two of the five JAG members who visited Justitia in Poland are founders of The Unity of Georgian Judges, a new judges association comprising approximately 50 judges who now feel empowered to speak freely about the problems within the justice system and to fully engage in recommending and implementing judicial reforms. One major reason for the creation of the Unity of Georgian Judges was that they were disappointed with the failure of JAG to generate any meaningful initiatives that support ordinary members. Following the release of GYLA and TI’s findings from their court monitoring projects - which the former chair of GYLA stated “would never have been successful without Neil’s well-established methodology,” - that revealed serious deficiencies in the courts, with JILEP’s support, the Law on the Common Courts was amended in May 2013 to enhance the transparency of court proceedings by obliging the courts to allow audio/video recording and broadcasting of trials. With less dramatic results, but nevertheless important, TI observed that judges monitored by TI began to alter their courtroom conduct. Also, as a result of the court monitoring, both GYLA and TI found a slightly positive trend towards court objectivity and professionalism. While the Chief Justice praised JILEP’s judicial ethics courses and its recommendations of certain legislative amendments, he did not constructively comment on any other activity that JILEP pursued with the Supreme Court or the HCOJ. As a result of the change in the political environment, new openings and opportunities exist for JILEP to work directly with the judiciary to achieve its project goals through the end of JILEP and beyond. Although JILEP was unable to complete all of its activities with the HCOJ, HSOJ and JAG, it engaged in a successful balancing act in supporting the justice institutions by working directly with them, while also challenging them by supporting CITJ’s activities. When political will was weak, JILEP patiently, yet persistently, proffered support even when it was repeatedly discouraged from continuing its activities. JILEP now has an advantage in accomplishing the goals of its activities, and pursuing new openings and opportunities, as it has built a solid foundation of professional relationships based on trust and respect with the judicial leadership. The various activities it initiated within the justice institutions were at the request of the institutions and although some results were modest, given the wavering political will, the activities with the institutions represented appropriate contributions to judicial independence, and could have resulted in greater impact had political will been available. It should be mentioned that the window of opportunity in working with the institutions is not certain. Although legislative amendments have allowed for some disbursement of power, there still exists a concentration of judges on the HCOJ who represent the attitude of the previous government toward the role of the courts and individual judges. While the power of the Chief Justice has been diminished by recent changes to the Law on the Common Courts, he still retains considerable influence over the judiciary. There is hope, indeed, an expectation, that the change in government and the MOJ’s engagement in seeking judicial reforms, may cause the Chief Justice to become more proactive in pursuing improvements of the justice system, either out of a genuine interest, or as a result of the change in government and competitive nature of the Supreme Court’s relationship with the MOJ. The Future Priorities of the Judiciary, July 10, 2013, issued by the Chief Justice on behalf of the HCOJ, lists three priority areas with numerous activities that resemble many of the activities JILEP and CITJ have pursued during the past two years: 1) Strengthening Judicial Independence; 2) Improving the Quality of Judicial Decisions; and 3) Improving Judiciary’s Administrative Systems. JILEP is well-placed to respond to the exuberance among individual judges to influence the direction of the judiciary, as well as their own professionalism. With the increased authority of the Conference Judges, the creation of The Unity of Georgian Judges, and the potential revitalization of JAG, JILEP has new openings and opportunities to empower judges as individuals and, collectively, in the Conference of Judges to enhance their self-governance. MID-TERM PERFORMANCE EVALUATION OF JILEP 20 | Page 4.2.3 Recommendations JILEP should continue to work directly with the HCOJ, the HSOJ and the MOJ in the same technical areas as it has pursued since inception with a greater focus on those activities that were not completed while also continuing its support of the Coalition. Given the change in political environment, the solid foundation that JILEP developed with the judicial leadership and the initiation of numerous activities within the eight technical areas, many of which follow the same direction as those in the HCOJ’s Future Priorities of the Judiciary, JILEP’s continued assistance with these activities has the potential to produce greater results than originally anticipated. While all technical areas should continue to be pursued, there are specific recommendations that emerged from the evaluation which deserve attention. With regards to strengthening appointment, reappointment, and promotion processes for judges, the HCOJ’s internal rules and the Law on Organic Courts need further revisions, including more specificity to ensure that international standards of judicial independence are incorporated into the selection and appointment processes. Having begun a software system that provides a platform for judicial qualification examinations, the HCOJ needs to return to its full usage, not sidestep any of its transparent processes, and support the development of the aptitude test. If there is political will for a revision of the HCOJ’s new rules, Evaluation of Efficiency of Judges’ Performance, which was absent during JILEP’s first two years, the evaluation criteria needs to be revised to limit the use of the rate of appeals as it can be a disingenuous measure of judges’ work since corrupt appellate judges may reverse a properly-decided lower court decision for their own interests. Also, the involvement of the individual judges in the evaluation process must be increased. With regards to improving judicial disciplinary procedures, even with the recent legislative amendments allowing the Conference of Judges to nominate and elect members to the new Disciplinary Committee, deficiencies remain that need to be addressed in order to ensure judges are disciplined by an objective body of individuals for appropriate causes and with appropriate punitive measures. With regards to developing practical guidelines for the HCOJ, judges, and legislative advocacy, there is a need to ensure that the HCOJ provides evidence-based justification for all judicial transfers. Without transparency of the decision-making process, including justification by the HCOJ that the recipient court is in need of additional judicial personnel, a judge could still be punished even though the transfer is now for only one year and even though he or she “consented” to the transfer. JILEP’s efforts, in collaboration with CITJ to improve transparency and public outreach for the HCOJ, need to be redoubled, particularly since the HCOJ ignored its outreach and communications strategy, and closed its meetings after the election. According to a new non-judge member of the HCOJ, JILEP and CITJ should continue pursuing the publication of HCOJ agendas and minutes and the broadcasting of its meetings as it’s hard under these conditions for the HCOJ to hide behind its decisions. While the HCOJ membership prompts concern as four of the new members are on the Supreme Court and presumably beholden to the Chief Justice, there are new opportunities as a result of its Future Priorities of the Judiciary of July 2013. Without a doubt, strengthening the HSOJ is one technical area where JILEP has already had and could continue to have substantial new openings and opportunities. With a new head of the HSOJ nominated and elected by the Conference of Judges, a new Minister of Justice, and with the Chief Justice’s favorable opinion of JILEP’s judicial ethics course, JILEP could not only complete the judicial ethics courses by training 120 judges, it could also conduct numerous other courses, and support the HSOJ in conducting a comprehensive evaluation of its curriculum and teaching methodology. There may be new openings and opportunities to revitalize the JAG, particularly since there is now “competition” with the Unity of Georgian Judges, discussed below. While it is unclear whether JAG will acquiesce to assistance from JILEP, it is essential for JILEP to continue offering assistance and not abandon it in favor of another judges’ association. JILEP should expand its work focusing on the empowerment of judges by assisting the Conference of Judges and the newly-created Unity of Georgian Judges. During JILEP’s first two years MID-TERM PERFORMANCE EVALUATION OF JILEP 21 | Page of activity, the Conference of Judges was not in a good position to direct the judiciary as it possessed few powers and did not show any interest in working with JILEP. JILEP was not invited to its annual meetings on July 30, 2011 and October 27, 2012, as they were only for judges and confidential. However, following the election of October 2012 and the amendments to the Law on Common Courts, which gave the Conference of Judges authority in electing members to the HCOJ and determining leadership of the HSOJ, JILEP, along with other foreign donors, was invited to the Conference of Judges’ annual meeting held on June 9-10 and June 16- 17, 2013. The meetings were covered by most major new channels and live-streamed over the internet. Judges were free to comment publically and voted privately and anonymously. The changes in the authority of the Conference of Judges and the transparency of its most recent annual meeting provides JILEP with an opportunity to extend its reach to the Conference of Judges and assist in strengthening it. The Unity of Georgian Judges is an unexpected outcome of the elections and deserves assistance from JILEP, along with JILEP’s continued assistance to the JAG, if interested. In a FGD with eight members of the Unity of Georgian Judges, which has a membership of over 50 judges, all expressed a unified vision of the Unity of Georgian Judges as strengthening the role of an individual judge. With less intrusion by the chairs of the courts in the decision-making processes of individual judges, one key informant stated that judges are now “breathing easier,” and are willing to take public stances of their views as witnessed in the recent international judicial conference sponsored by the European Union and other international donors. They no longer fear, or to the degree they feared before the election, that the HCOJ would apply undue pressure, which, in the opinion of the judges, was a result of the HCOJ protecting the executive branch rather than the judicial branch. It must be pointed out that most judges who are members of the Unity of Georgian Judges are civil judges as many believe that criminal and administrative judges still operate under a psychological fear of retribution. The judges in the FGD stated that the Unity of Georgian Judges must improve the judiciary from within to enhance its independence and transparency and are eager to take advantage of the changed political environment to proactively pursue self-governance. According to an active CITJ member, judges are lost today. They have never learned how to be independent. As there are still legislative gaps in securing the power of the judge, the judges will seek to eliminate those gaps so that a government with a weak political will for judicial independence cannot take advantage of those gaps as was done with the former government. With this as a backdrop, JILEP has numerous new openings and opportunities to respond to requests proffered by the Unity of Georgian Judges to strengthen its development, and to work collaboratively with other international donors who will, undoubtedly, also assist the Unity of Georgian Judges. In order to achieve sustainable results from its activities, JILEP should encourage collaboration and moderation among the justice institutions, the MOJ, Parliament, Procuracy, GBA and law faculties to develop a Strategic Plan for the Justice System of Georgia, which would be geared towards ensuring balance of power among the three branches of government. Without such a Strategic Plan, the imbalance of power, particularly between the MOJ and the HCOJ, could impede JILEP from reaching its project goals resulting in short-lived successes. In order for sustainable, systematic changes to result in the judicial branch becoming an independent and accountable third branch of government, the legislative framework and judicial infrastructure must solidly favor decision-making authority within the justice system with appropriate checks and balances. Without a solid framework supporting judicial independence, the judicial branch of government will be subjected to the political whims of those in power, as has been demonstrated. Although it is admirable, and perhaps the only feasible avenue available, to pursue individual activities within several technical areas as required by the Cooperative Agreement between USAID and EWMI, it is advisable for JILE -, along with the other donors which, to date, have developed into a collegial and cooperative group of international assistance providers - to pursue the creation of a Strategic Plan for the Justice Sector of Georgia that has the participation of the Supreme Court, HCOJ, HSOJ, MOJ, Parliament, Procuracy, JAG, the Unity of Judges, GBA, law faculties and relevant NGOs. While several key informants suggested that the HCOJ develop a strategic plan for the justice system, this is a MID-TERM PERFORMANCE EVALUATION OF JILEP 22 | Page short-sighted solution as it will perpetuate HCOJ’s political competition with the MOJ and Parliament. Until all three branches of government collectively determine the proper role of the judiciary in Georgia, judges will continue to be the pawns of those in power. While the current political environment will clearly not support such an inclusive involvement by all relevant stakeholders listed above, particularly with the political tensions between the Supreme Court, HCOJ, and MOJ, it would, nevertheless, be advisable for JILEP to work with the other donors to encourage the development of a Strategic Plan for the Justice System of Georgia. Such a plan would encompass broader areas of justice system development than those areas covered in the Criminal Justice System Reform Strategy of 2008, such as the values associated with a democratic justice system, the relationships between judges, advocates and prosecutors, the role and responsibilities of bar associations, and the education of law students. The strategic planning process would help guide essential decisions and actions that shape and describe what the critical functions of the justice system are and should become. It would help identify fundamental values of the Georgian justice system and create a vision to guide and coordinate justice system reforms so that all reforms move toward a common goal. The strategic planning process would also provide all relevant stakeholders with an opportunity to collectively discuss numerous issues central to the creation of an independent judiciary by focusing on the basic purposes of the justice system, defining the fundamental issues, identifying strategies to address those issues and setting forth an action plan that would include benchmarks, milestones to allow measurement of progress, a delineation of resource needs and a time-line for completion. By addressing justice reform objectives and goals in a holistic manner, the appropriate checks and balances will, presumably, be inserted into the strategic plan enabling the judicial branch to ultimately withstand external political pressure. With a strategic plan, the HCOJ will be able to direct and coordinate execution of judicial reform activities initiated not only by Georgian individuals and entities, but also by international donors who pursue their own goals on their own time line, and in an ad hoc manner. 4. 3 QUESTION 3: BUILDING THE CIVIL SOCIETY COALITION Taking into consideration the Coalition’s accomplishments to date, what has been the JILEP and EPF approach to building the Coalition? What have been the advantages and disadvantages of working through a Coalition for Independent and Transparent Judiciary? How effective has it been to work through EPF to provide coordination and organizational development assistance to the Coalition for Independent and Transparent Judiciary? What additional support is required in order to ensure the Coalition continues to pursue its stated goals? 4.3.1 Findings The Coalition for an Independent and Transparent Judiciary participated in many activities to promote civil society engagement and government accountability for an independent and transparent judiciary Forums Since its creation, CITJ has conducted seven forums with a broad range of stakeholders, including senior officials within the justice sector and government, who engaged in active discussions and were willing to receive as well as provide information on the relevant topic of the forum. The seven forums focused on: 1) transparency of the courts and public access to the courts; 2) judicial appointment, promotion, transfer, and disciplinary processes; 3) controversial provisions within the Tax Code and the Law on Enforcement Proceedings, which were later repealed by the Parliament and ADR; 4) administrative detention and imprisonment provisions of the Code of Administrative Offenses and their conformity with human rights standards; 5) judicial administration, judge empowerment, and self-government of judges; 6) proposed legislative amendments to laws concerning judicial independence; and 7) establishment of a temporary Commission of Miscarriages of Justice. Working groups CITJ created five working groups among its members to operate within the framework of the Coalition’s mission. According to JILEP’s quarterly reports, the most active working group is the Court Administration Working Group (CAWG), which has been engaged in analyzing issues related to the transparency of the courts and judicial appointment, promotion, transfer, and disciplinary processes. Based on the analysis, CITJ MID-TERM PERFORMANCE EVALUATION OF JILEP 23 | Page prepared a report entitled The Judicial System in Georgia Report 2012, which was presented during the fifth forum. Based on its analysis, CAWG made recommendations, through CITJ, on legislative amendments to increase the judiciary’s transparency and to encourage a fairer judicial selection process to the judicial leadership and other governmental representatives. As a result of the forums, particularly the fifth, the publication of The Judicial System in Georgia Report, and CITJ’s large-scale advocacy campaign, Parliament adopted several legislative amendments in March 2012 that were proffered by CITJ and supported by JILEP, specifically relating to the voting procedure of the HCOJ, the transfer of judges, the publication of disciplinary decisions, and the criteria for selecting judges. After the October 2012 election, the CAWG continued its work on draft legislative amendments with MOJ, to which the sixth forum was dedicated. As with the June 2012 legislative amendments, several of the May 2013 legislative amendments could also be attributed to the recommendations of CITJ and JILEP, particularly those relating to the structure and composition of the HCOJ, the authority of the Conference of Judges to elect members of the HCOJ and the head of the HSOJ, and the membership of the Disciplinary Committee. The Criminal Law Working Group (CLWG), also quite active, analyzed administrative detention and imprisonment issues and made recommendations for legislative amendments to the Code of Administrative Offenses, to which the fourth forum was dedicated. The CLWG, after extensive research, published a study on the Application of Preventative Measure in Criminal Proceedings which included recommendations for reform. Both sets of recommendations are to be the subject of active advocacy campaigns by CITJ. Lastly, the CLWG conducted an analysis of plea bargaining by reviewing pertinent laws, studying cases in which a plea was entered, and conducting in-depth interviews. The Legal Aid Working Group (LAWG), after distributing a questionnaire to legal aid organizations around the country, created a map of NGOs providing such services; identified law firms that provide pro bono legal services; and drafted recommendations regarding amendments to the Law on Legal Aid. The Legal Education Working Group (LEWG) studied various requirements for entry into the legal profession and reviewed CLE courses for attorneys. The LEWG is currently researching the quality of legal education in Georgia. The Commercial Law Working Group, the least active working group, focused on a draft law relating to property rights and on the CITJ’s sponsorship of the forum “Controversial Provisions within the Tax Code Related to Enforcement Proceedings.” The survey with CITJ members included questions covering the effectiveness of the above groups. The results of the survey indicated that the CLWG was considered the most effective, with 58% of respondents stating that it is “very effective,” followed by the CAWG at 47% and the LAWG at 32%. When asked if effectiveness of the working group is determined by its membership or by the policy issues it covers, 90% responded “membership” while 58% responded “policy issues it covers.” Figure 4, below, shows respondent views on the main challenges facing the working groups. “Lack of commitment on the part of the working group members” was considered the main challenge by 68% of respondents, followed by “passivity of the working group members.” Figure 4: Challenges Faced by Working Groups Challenges faced by the working groups Count % Lack of funding 6 32 Lack of expertise 6 32 Lack of commitment on the part of the working group members 13 68 Passivity of the working group members 11 58 Capacity of the working group members 4 21 Source: Survey of Coalition Members by the Evaluation Team MID-TERM PERFORMANCE EVALUATION OF JILEP 24 | Page A majority of respondents to the CITJ Survey, 53%, said that their own association has been very active in attending the Steering Committee meetings, regular CITJ meetings, working group meetings and forums, and attending at least one meeting monthly. Advocacy CITJ members regularly met with the judicial leadership and other governmental officials, including parliamentarians, as well as foreign donors to present their recommendations on various topics and exchange views. CITJ issued several public statements on current issues relating to judicial independence and transparency. The Coalition also produced, as part of its advocacy campaign, four video clips: 1) CITJ’s presentation; 2) administrative imprisonment; 3) audio recording in courts; and 4) legal aid map. When asked “does the Coalition fulfill its mission by successfully facilitating legislative and regulatory advocacy for an independent and transparent judiciary?” 47% replied “yes” and 47% replied “somewhat.” Figure 5 shows Coalition Survey responses to how successfully CITJ is achieving its mission. 53% of respondents believe that CITJ is successful in promoting collaboration with major justice system stakeholders. Figure 5: CITJ Success in Achieving its Various Missions Question: Does the Coalition fulfill its Mission by Successfully: Yes No Somewhat Count % Count % Count % Promoting government accountability for an independent and transparent judiciary? 9 47 1 5 9 47 United civil society efforts in judicial system reform monitoring? 8 42 1 5 10 53 Facilitating legislating and regulatory advocacy for an independent and transparent judiciary? 9 47 1 5 9 47 Promoting collaboration with major stakeholders in the justice system, such as the High Council of Justice, the Supreme Court, the High School of Justice and the Ministry of Justice 10 53 1 5 8 42 Source: Survey of Coalition Members by the Evaluation Team JILEP’s and Eurasia Partnership Foundation’s mutual approach to building the Coalition was to encourage collaboration in seeking an independent and transparent judiciary CITJ was created to provide an umbrella for CSOs that would pursue, while avoiding duplication in their programming, an independent and efficient judiciary and would collaborate in advocating for improvements in the judiciary and monitoring judicial reforms. JILEP and EPF’s approach in building CITJ was inclusive and broad-based. Membership was available to NGOs, business associations, professional associations and media organizations whose mission was similar to CITJ’s goal of promoting an independent and transparent judiciary and who demonstrated a willingness to work collaboratively with other CITJ members. To ensure that prospective CITJ members met the requirements, an application form was required to be completed and, in some cases, meetings with prospective members, particularly business associations, were held to encourage them to join. When asked why their association joined CITJ, the majority of responses, both at 68%, were because it “helps us advocate for national policies on reforming the judiciary” and “helps us expand our knowledge of the issues that impact judicial reforms.” JILEP and EPF’s approach in building CITJ is to ensure: 1. CITJ leadership is flexible, allowing for a variety of members to be on the Steering Committee as well as to serve as its Chair and Deputy Chair. This is accomplished by the members of the Steering Committee being elected for two years while the Chair and Deputy Chair have one-year terms. 2. All members participate in the activities of CITJ and are visible to the judicial leadership, other governmental officials, and the donor community. 3. CITJ is not dominated by a few active members. 4. Consensus about the subject of CITJ statements and the topic of forums is reached before decisions are made. 5. CITJ statements are developed in a timely manner and are well-drafted. MID-TERM PERFORMANCE EVALUATION OF JILEP 25 | Page 6. CITJ activities and outputs are internally-driven and not influenced by its two creators – JILEP and EPF; 7. The activities of CITJ are not Tbilisi-centered but include regional organizations and issues. 8. Strict adherence by the members to the governing documents of CITJ, particularly the Bylaws and the Memorandum of Understanding. As a result of JILEP and EPJ’s approach, CITJ consists of approximately 30 members, many of whom are active in CITJ, with a Steering Committee of changing membership, and a revolving Chair and Deputy Chair. CITJ has engaged in numerous activities, is visible to the judicial leadership, governmental officials and donors, and, as demonstrated in Section 4.2.1, had an impact on judicial reforms. Nevertheless, there are advantages and challenges to working through CITJ in pursing an independent and transparent judiciary, as discussed below. Working through the Coalition to pursue an independent and transparent judiciary has advantages and challenges In KIIs with representatives of EPF and individual Coalition members, in the FGD with six representatives of the Steering Committee of the Coalition, and from the results of the Coalition Survey, the following advantages and challenges of working through the Coalition have emerged. Advantages The most oft-cited advantages of working through CITJ were the financial support USAID provided to CITJ and the programmatic support JILEP provided. Many CITJ members believe it obtained a seat at the judicial reform table because of the foreign support it received. CITJ members attribute to JILEP its approach in addressing judicial leadership and other governmental officials with its concerns and recommendations. Rather than take a confrontational stance, CITJ, upon JILEP’s guidance, acted respectfully and professionally in dealing with stakeholders, and, as a result, its voice was heard. Even members of the judiciary whom the Evaluation Team interviewed commented on CITJ’s professionalism in presenting its views at forums and in statements. According to several CITJ members, JILEP taught CITJ how to be professional, how to negotiate with the government, and how to present its opinions to justice officials without being confrontational. In the words of one justice official “JILEP taught the Coalition members how to expose problems in a gentle way.” According to NGOs well-known for advocating for judicial independence and transparency, there is a distinct advantage of presenting recommendations for judicial reform from a cohesive, yet disparate group of associations that have developed a consensus on issues. While their presence is acknowledged within the legal and judicial community, these NGOs recognize the strength of a Coalition statement rather than an individual NGO statement. A senior governmental official commented that “it was more efficient to work through the Coalition rather than 30 individual organizations.” The results of the FDG confirm this Finding and highlight CITJ’s major achievements in seeking judicial reform:  “The fact that 30 organizations have gathered together to work out one vision and are using this one vision to conduct advocacy.”  “The coalition organized various forums where different interested parties participated, prepared recommendations, and these were considered in the Organic Law on Courts, and most of the recommendations were taken into account.”  “The coalition has become the main player in judiciary reform in a very short time; this was demonstrated in the last reform of the judiciary where coalition participants participated in all meetings and media events and played a main role in the process.” According to the survey conducted by the Evaluation Team, 42% of respondents said that membership in CITJ has met their NGO’s expectations for joining; and 53% said that participation in CITJ has helped increase their capacity to fulfill their mission. Challenges KIIs, FGDs and Surveys conducted by the Evaluation Team demonstrated that challenges still exist. Respondents stated that the main challenges of working through CITJ are that “only a few members of the Coalition are active and do most of the work” followed by “members have different levels of institutional or MID-TERM PERFORMANCE EVALUATION OF JILEP 26 | Page organizational capabilities that impede decision-making.” Figure 6 shows the challenges highlighted by members, followed by further findings and analysis. Figure 6: Challenges of being a CITJ Member Challenges Count % Only a few members of CITJ are active and do most of the work 11 58 The decision-making process of CITJ is too cumbersome and takes too long 3 16 The decision-making process of CITJ does not give sufficient time to non￾members of the Steering Committee to participate in a meaningful way 6 32 The focus of CITJ is only on national issues, and needs to take into consideration regional issues 3 16 The activities of CITJ are too time consuming 4 21 Members have different level of institutional or organizational capabilities that impede decision-making 8 42 Source: Coalition Members Survey done by the Evaluation Team One of the major challenges, as seen in Figure 6, is that despite JILEP’s and EPF’s approach to ensure an equal playing field among the members, the largest NGOs well-known for their involvement in judicial independence and transparency issues have emerged as the most active members of CITJ and are the recognizable leaders and spokespersons for CITJ, thereby diminishing the contributions of other lesser-known and less active members. Involving CITJ members who are located outside Tbilisi in the activities of CITJ is challenging. Coalition members in the regions have difficulty attending meetings in Tbilisi, and for this reason alone, have shied away from seeking Steering Committee membership. Even attending CITJ or working group meetings electronically is challenging as the internet connection is often faulty. In the FGD with CITJ Steering Committee members, discussants mentioned that other organizations that are not members of the Steering Committee, as well as regional organizations, need to have a more active role in CITJ.  “I wish that organizations that are not members of the steering committee had been more engaged; I wish media organizations were more active.”  “The steering committee is built from the most active organizations, those who are actively engaged in the working process of the coalition, but I’d like to see other organizations more active as well.”  “Regional organizations are minimally engaged, because they are geographically far, and the main working organizations are located in Tbilisi.” Additionally, CITJ members who serve as the Chair or Deputy Chair of the Steering Committee have lapsed into speaking on their own behalf even though a consensus on a position has been reached by CITJ thus giving the individual CITJ member as Chair or Deputy Chair more recognition than it deserves. To overcome minority participation in CITJ activities, EPF maintains a constant outreach to CITJ members to ensure that all members are well-informed of CITJ, Steering Group and working group meetings, meetings with stakeholders, fora, and EPF-awarded grants so that those who are interested in particular issues can actively participate. In the Coalition Survey, 63% of respondents agreed that “the Coalition provides timely updates on the important developments within the Coalition, on important decisions of the Steering Committee and Working Groups, and on activities of the Coalition.” Another major challenge is developing a consensus among the members on issuing public statements within a timely manner and with well-drafted documents. Concerns were expressed by various members of CITJ as to the disparate opinions among NGOs involved in human rights issues and business associations regarding judicial reform issues on which CITJ will issue a public statement.  “The main challenge was prompt reactions to existing problems……sometimes it is difficult to achieve consensus in a short time, so this is one very important challenge.” MID-TERM PERFORMANCE EVALUATION OF JILEP 27 | Page  “It is quite difficult to organize working groups; some organizations are participants of more than one working group.”  “Working group is a very necessary and positive tool….but often this intensive work is not possible because everyone should find time.”  “Some members are passive and do not participate in the process and activities and then do not express their positions regarding some issues, this is also a problem.” Some CITJ members, particularly the American Chamber of Commerce (AmCham), were critical that procedures for members to consider supporting a CITJ position and approving a CITJ statement were not always followed. In some cases, the statements were issued too quickly without consideration and review by the full membership of CITJ, whereas other CITJ members criticized the process as being too slow. Ensuring that CITJ takes a proactive approach rather than simply respond to JILEP’s or EPF’s direction has been challenging and has become a balancing act for EPF. To mitigate reliance on EPF, CITJ was encouraged last year to create working groups to initiate activities related to the topic of the working group. Some working groups, as demonstrated in the first finding, were instrumental in propelling the work of CITJ. Others, however, have been slow and inactive. To encourage more activity, EPF recently offered grant support to the working groups to develop a collaborative project among its members. According to the Survey, only one working group has utilized these funds, while representatives of EPF indicated that no working group has submitted a project for these funds. Of the 19 coalition members that responded to the survey, 7 said that they were not aware of the availability of funding by EPF for working groups. Another challenge arises when CITJ members draft statements or forum agendas are not professionally presented in terms of form and substance. EPF is often engaged in severe editing of the initial statements and revising forum agenda to ensure the topics are well-formulated and the invited speakers represent various points of view from the three branches of government. Working through the EPF in providing coordination and organizational development assistance to the Coalition is challenging According to KIIs, challenges are centered on the EPF being too strict with issuing grants to CITJ members and too passive in encouraging active involvement by all members in CITJ’s activities. EPF acknowledges its strict procedures in issuing grants and believes they are necessary to ensure responsible expenditure of funds. EPF also acknowledges its passivity in pursuing judicial reforms as the Secretariat of CITJ does not have expertise in legal advocacy or legal aid issues; its skills center on coalition-building. There is concern that EPF’s standards for issuing grants to CITJ members are so strict that some members who deserve the opportunity to grow as advocates or legal aid providers are denied. While there is the need for high standards to ensure the grant funds are used appropriately and efficiently for the purpose intended in the grant application, and that such funds are not ill-managed, there is also the need to develop the capacity of smaller or nascent NGOs who are well-intentioned, but have yet to develop the professional skills to meet EPF’s qualifications and strict standards. By only favoring NGOs, which have demonstrated significant capability to engage in EPF-funded projects, EFP may cause a continuation of their high profile as CITJ member and in the NGO community. By not taking any risks in issuing grants, coupled with its passive approach to developing member capacity, EPF has yet to expend an appropriate amount of funds allocated to grant-making. The Coalition Survey indicates that the respondents are satisfied with EPF’s leadership of CITJ. The Coalition Survey asked several questions about grants offered by the EPF and the process for obtaining the grant. Eleven of 19 respondents (58%) said that they have received a grant from the EPF. Only 42% said the process for obtaining and implementing grants is fine, while others highlighted several problems with this process as seen in Figure 7, below. Figure 7: CITJ Members’ Views on the Process of Obtaining and Implementing EPF Grants How do you View the Process for Obtaining and Implementing EPF Grants? Count % The process is fine 8 42 The process is too strict 3 16 The process is too time-consuming 3 16 MID-TERM PERFORMANCE EVALUATION OF JILEP 28 | Page How do you View the Process for Obtaining and Implementing EPF Grants? Count % The process is too bureaucratic 5 26 Source: Survey of Coalition Members by the Evaluation Team A fundamental challenge is EPF’s “hands-off” approach to coordinating and providing developmental assistance to CITJ. EPF’s philosophy is to break the umbilical cord with CITJ members so that they take responsibility for their own actions and activities. EPF is of the opinion that, given Georgia’s socialist history, too many individuals and organizations continue to assume another entity, be it the government or a superior organization, that will take the responsibility for the well-being of the individual or organization. Thus, EPF is purposely taking a more passive approach in working with CITJ and individual members so that they can learn to grow on their own. A member of the Coalition, whose sentiments were shared by other CITJ members, stated that the EPF “does not inspire Coalition members to fully engage in Coalition activities, nor is proactive,” citing the need for EPF to ensure that when a working group meeting is scheduled, members will attend as it is a waste of time to call a meeting with only one member in attendance. Additionally, there is little information-sharing among working groups. “ “The CLWG is discussing important topics, yet other working groups have no idea what’s going on” lamented one member who stated that information-sharing among working groups is crucial as they are initiating the policy work of CITJ. EPF was compared to the Open Society Foundation (OSF) which, according to an active CITJ, is “very, very active creating incredible results” with the coalition it manages. EPF countered the criticism with the observation that it would prefer to take a “back seat” so that CITJ is known by the success of its members rather than its manager. EPF also countered that OSF has more freedom to make decisions whereas EPF often needs JILEP’s and USAID’s approval which can cause considerable delays. Respondents to the Coalition Survey were asked three questions regarding the coordination and developmental assistance provided by EPF. When asked “how effective has the EPF been in managing and organizing the activities of the Coalition?” 68% stated “very effective” and 32% “somewhat effective.” To the question “has the EPF been sufficiently active in encouraging the Coalition to pursue judicial reforms and enhance judicial independence?” 58% responded “very active” and 42% “somewhat active.” However, when asked if another Georgian NGO could accomplish the role of the secretariat, 84% said “yes.” 4.3.2 Conclusions The effectiveness of the Coalition’s activities demonstrate there are more advantages than disadvantages working through the Coalition; the Coalition has the respect and credibility with the judiciary, the Ministry of Justice and Parliament; the successes and strengths of the Coalition are the result of a few active Coalition members, as well as its collaboration with JILEP. Based on the KIIs with CITJ members, the FGD with CITJ Steering Committee, and the Coalition Survey, the advantages of being a member of CITJ were substantial, particularly since CITJ was able to achieve that which individual members could not. The legislative amendments to various laws, which were recommended by CITJ carried significant weight whereas one association pursuing amendments may not have been heard by the judicial leadership, MOJ, or the Parliament. The disadvantages, or rather challenges, represent EPF’s view of developing CITJ. EPF faced numerous challenges as listed above to create a cohesive group of disparate associations who have a similar goal of judicial independence, transparency and professionalism, which EPF has addressed and attempted to overcome. In all of the KIIs with JILEP, judges, representatives of the MOJ and members of Parliament, CITJ was viewed as an effective organization as it engaged in a professional manner in pursuing judicial reforms with the very institutions it was attempting to reform, particularly the HCOJ. According to the former Secretary of the HCOJ and other judges, CITJ acted respectfully in presenting its position, which was often critical of the operations of the HCOJ. CITJ members were open to hearing other points of view in their discussions with the HCOJ and were willing to compromise in order to achieve certain results. In KIIs with members of CITJ, JILEP was viewed as the silent teacher and essential partner of CITJ guiding it in its presentations on controversial topics and publically supporting its positions. MID-TERM PERFORMANCE EVALUATION OF JILEP 29 | Page According to KIIs with CITJ members as well as the FGD with CITJ Steering Committee, JILEP played a central role in CITJ achieving its goals.  “It would be difficult for the Coalition to reach any goal without JILEP.”  “It was using JILEPs technical platform that we could provide assistance in all fields, and that is why without JILEP it would be very difficult.”  “Yes, [JILEP] helps with everything that would help us….financial and technical support.” There is a marked difference of opinion regarding the quality of the coordination and organizational development assistance provided by the EPF. The results of the Coalition Survey show a marked difference between the perceptions CITJ members have of EPF’s assistance to CITJ. Perhaps the diverse perceptions are a result of different expectations. Except for the most active members, many of whom participated in KIIs and in the FGD, most CITJ members, on the other hand, are satisfied with EPF’s leadership role; however, they may not be highly engaged associations and may have lower expectations for EPF involvement. Therefore, any provision of assistance by EPF to them as individual members or to CITJ, whether it is coordination or organizational development, is acceptable. The Coalition is capable of pursuing its stated goals with the current support, but will be better￾positioned to survive after JILEP’s completion if foreign funding and programmatic support is continued. CITJ’s success is largely attributable to: 1) USAID funding as USAID is respected in Georgia, and, as a USAID-funded project, CITJ often received positive responses according to several key informants representing EPF and members of the CITJ and 2) JILEP’s professional advice and guidance. In many instances, JILEP’s support was as a silent partner, which allowed CITJ to gain credibility on its own merits. CITJ needed JILEP in the beginning to open government doors, and to broker the communication between the judiciary and civil society. Members of Parliament stated that “JILEP helped explain each side to the other.” Today, CITJ’s reputation, according to several key informants, is on solid ground; thus some believe that the need for programmatic support, after completion of JILEP, may not be necessary. However, when asked “does the Coalition need a foreign sponsor to obtain respect among government officials?” 90% of the Coalition Survey respondents said “yes” and only 10% said “no.” At the rate the funds for CITJ are being expended by EPF, CITJ could not absorb additional financial support today, although after JILEP’s completion, according to KIIs and the FGD with CITJ members, external funding is essential as CITJ has yet to develop a plan of financial self-sustainability. The respondents had mixed feelings about the ability of CITJ to continue its activities after JILEP funding ends. Some believed that a solution would be found for continuation, while others found that CITJ would continue to function and undertake activities but were unclear as to how effective it would be.  “Of course there will be a solution, but I wonder how effective this work will be.”  “I believe there will be a problem, because JILEP’s financial support defines the work of the coalition. The same is with the working groups.”  “I think that the Coalition’s activities will be continued but this will be difficult, and difficult to judge how effective it will be.” One of the advantages of having a disparate group of associations with similar goals to pursue judicial independence, transparency and professionalism is not just its strength in numbers, but also its strength in diversity. Statements made by CITJ were more credible, according to some judicial and other governmental officials, as they represented a consensus of views from NGOs, business associations, professional associations and media organizations. If CITJ loses its diversity and becomes only an umbrella organization of advocacy and legal aid NGOs, its influence of governmental officials could possibly wane. All CITJ members responding to the Survey seem committed to staying with CITJ and continuing being a member of CITJ after JILEP has ended. 4.3.3 Recommendations JILEP should continue to work through the Coalition as the two-prong approach in pursuing judicial reforms has been successful. JILEP achieved considerable results due to its programmatic and financial support of CITJ and of individual CITJ members, while also engaging directly with the justice MID-TERM PERFORMANCE EVALUATION OF JILEP 30 | Page institutions. The Evaluation Team did not receive any suggestions or improvements that needed to be made to the two-prong approach. Thus, continuing its collaborative work with CITJ using the same approach as it has over the past two years is recommended. The EPF should be more engaged in the internal management, communication and coordination activities of the Coalition to achieve greater results and sustainability. Given the recommendations that emerged from the KIIs with JILEP, some CITJ members, the FGD with the Steering Committee and the Survey, EPF would be well-advised to become more active in the internal management, communication, and coordination activities of CITJ along the following lines:  Use an online forum, including a closed Facebook group for discussions.  Conduct field trips for CITJ members to see first-hand the activities of individual members.  Proactively seek greater involvement of CITJ members in all activities of CITJ.  Take a “hands-on” approach, particularly since CITJ is a nascent entity and its members are of varying stages of development.  Broaden the membership of CITJ to include other types of organizations.  Ensure a greater participation of CITJ members at meetings.  Foster more public debates and engage a broader range of stakeholders in the discussions of judicial reform.  Qualify more CITJ members for grants.  Ensure members of CITJ approve CITJ statements before publication.  Provide more opportunities for CITJ members to deliver their collective opinion on legal development issues.  Provide more opportunities for CITJ members to engage in meaningful dialogue with the courts, the government and other relevant stakeholders.  Provide more support to working groups to create studies and publications on topics of critical importance in legal and judicial reform.  Conduct press conferences and other public outreach events to inform the stakeholders and the public.  Update and maintain CITJ website.  Partner with similar organizations from other countries. Although the CITJ has effectively achieved some results, given the variety of suggestions above, the EPF should seriously consider developing a Strategic Plan of CITJ to help guide CITJ members to reach a common goal that may be broader than the original goals of CITJ. By engaging all members of CITJ in the strategic planning process, the members will have an opportunity to discuss among themselves the direction they would now like CITJ to take as well as the activities they want CITJ to pursue, both of which will assist EPF in managing, coordinating and providing organizational assistance to CITJ. In fact, 89% of the survey respondents said “the Coalition should develop a strategic plan for its future activities.” 4.4 QUESTION 4: COST-BENEFIT OF SUPPORT TO LEGAL EDUCATION Taking into consideration the cost and scale of JILEP support to legal education, how effective have the activities been or are they likely to be in contributing to improved legal education in Georgia (i.e. partnerships between Georgian and U.S. University law schools)? Which Georgian universities have been the best partners and why? Moving forward, what approaches would best accomplish the overall objective of improving the quality of legal education in Georgia? 4.4.1 Findings JILEP has designed and implemented a large number of diverse activities with universities, national learning centers, professional groups, law professors, students and practitioners to contribute to significant improvement of legal education in Georgia JILEP’s legal education component provides a large number of interventions designed to enhance the quality of legal education in Georgian law schools. While most of JILEP’s legal education interventions are to be viewed as long-term investments that will most likely have a tangible impact on the quality of legal teaching and MID-TERM PERFORMANCE EVALUATION OF JILEP 31 | Page learning in the near future, the project’s immediate results are indicative of major improvements in the legal education system and demonstrate the enhanced performance of partner law schools and increased motivation of universities, law professors, students, and practitioners to engage in project activities or benefit from opportunities not available prior to JILEP. Importantly, the project’s capacity building interventions have a dual focus. On one hand, JILEP’s legal education component seeks to strengthen the institutional or organizational capacities of the law schools to offer quality legal education to students; on the other, many of its activities are also designed to enhance the individual teaching skills of law professors and to deepen the knowledge of students and practitioners in substantive areas of law. JILEP’s legal education component consists of nine major activities that are implemented in cooperation with Georgian universities, national learning centers, and CSOs. Washburn University School of Law (WUSL), South Texas College of Law (STCL), and the Public Interest Law Network (PIL-net) provide important inputs in designing and implementing the capacity building measures for the Georgian law schools. Law Clinics The 2011 baseline Assessment Report of the Legal Education in Georgia identifies some gaps in the clinical learning experience offered by the Georgian schools of law. Namely, many law school-based clinical students received limited, if any, feedback on their performance, reducing the clinic’s educational value. Importantly, a lack of capacity in the teachers and practitioners who supervise the students was also found. To address the identified gaps, JILEP, in cooperation with PIL-net, provides technical and grant support to three live-client clinics: Batumi State University (BSU), Kutaisi State University (KSU), and the Georgian￾American University (GAU). While JILEP does not offer stand-alone clinical grants to TSU and Free University (FreeUni), it does provide some general assistance through its university partner grants awarded to NCADR and NCCL, housed in TSU and FreeUni, respectively. Recently, other law schools demonstrated a strong interest in receiving JILEP’s assistance to improve the performance of their law clinics. The number of beneficiary clinics is expected to increase in September 2013. JILEP’s assistance to Georgia’s clinical education has three main areas of focus:  Improvement of clinical management and operations  Enhancement of supervisory skills - over both students and client matters  Improvement of teaching skills Throughout project implementation, JILEP has provided a number of workshops for the existing live client clinics to improve the teaching skills of law clinic professors and administrators. With input from PIL-net clinical experts, a series of workshops have been delivered to improve the fact finding, problem solving, and office practice skills of clinical students and staff. JILEP also organized its first annual Law Clinic Summer School focused on law clinic administration and live-client file management procedures. Through a JILEP grant, GAU, with support from a PIL-net clinical expert, created a Legal Clinic Handbook in Georgian that includes information on various models of clinical education, specific details of clinical programs in the Georgian legal environment, and a compilation of forms, procedures, and protocols used by live client clinics. The Handbook, distributed to other Georgian legal clinics, is intended to be a useful instrument for those working in the legal education field, university administrators, professors, and student supervisors. In 2013, JILEP delivered follow-up workshops to six different Georgian law school legal clinics to determine how well they are applying the Handbook. A report was developed detailing the findings of the assessment with the overall conclusion that in five of the six universities, clinics were fully operational, following established and professionally acceptable procedures, attracting (in varying degrees) clientele, and addressing, to a similar if not identical extent, the educational value of exposing students to hands-on work. As noted in the report, the incremental nature of assistance given to the clinics seemed to be paying dividends and the strategy of joint and individual training worked well. Another major finding is that the creation of a manual is a major step to formalizing and standardizing processes and, in consequence, securing quality. It is noteworthy that the organizational arrangements and operational procedures of clinics differ from school to school. Therefore, the issues encountered by the clinics vary. In September 2013, a summer school will be organized with input from a Pil-NET clinical education expert to assess the quality of clinical education in JILEP’s partner universities and facilitate an exchange of experiences and best practices among the clinic’s administrators and instructors. As MID-TERM PERFORMANCE EVALUATION OF JILEP 32 | Page planned, an assessment tool will also be developed to measure the application of procedures and forms recommended by the Handbook. Finally, in 2013, JILEP supported a group of Georgian law school clinicians to conduct a study visit to Poland to expose Georgian University law clinic directors to successful Polish legal clinics and give them the opportunity to develop institutional relationships with those clinics. Promoting interactive teaching methodologies in Georgian law schools JILEP, through its partner western universities – WUSL and STCL – designed and delivered a large number of training workshops to law professors and teachers from various Georgian universities to introduce more interactive teaching methodologies in law schools. Part of the workshops and masters classes discussed student assessment theory and included a peer-to-peer information exchange and participatory exercises involving Georgian law professors. JILEP engaged lead experts in the field of teaching methodology to introduce modern concepts of teaching theory, cognitive learning theory, and best teaching methodologies. The workshops covered a wider range of topics such as choosing and stating course goals and learning objectives, evaluating the learners, using of variety of teaching and learning methods to enhance students’ learning, identifying material to support course goals, assessments, teaching/learning methods, syllabus drafting, and evaluating and revising the course design. In 2012, JILEP, together with the NCCL, launched a long-term teacher development program to prepare a team of Georgian instructors to be able to train other Georgian professors in modern teaching methodology and teaching techniques. JILEP conducted a series of masters training programs in teaching methodology for the resource group that consists of law professors from eight Georgian law schools. The final training workshop was delivered in May 2013. Part of the sessions at the Advanced Teaching Method Conference, organized by JILEP in cooperation with WUSL and the Institute of Law Teaching and Learning on May 10-11, 2013, were led by Georgian-trained law professors. Until that point, the resource group had not actively engaged in delivering teaching methodology workshops to Georgian law professors. Further, since 2012, JILEP has implemented a teaching methodology and instructional development program at TSU that seeks to assist TSU professors in adopting more modern, effective, and interactive teaching methods. During the final project stage, JILEP will evaluate the use of the new teaching methods in TSU classes. Practical skills development Georgian universities historically placed insufficient emphasis on legal skills development because the traditional law school teaching culture undermined the importance of students mastering practical skills to prepare them for entry into the profession. There has been an overall trend in western countries to teach students lawyering skills and practical knowledge, as opposed to only legal theory, to better prepare them to become successful lawyers. JILEP undertook efforts to expand the use of skills-oriented instruction in Georgian law schools and, throughout its implementation, delivered a large number of workshops with a view to improving students’ practical skills, as well as those of law professors and practicing lawyers. Since 2012, JILEP has implemented legal ethics and a trial advocacy program to improve students’ practical skills necessary to function in an adversarial court environment. Nine law schools committed to making these courses part of their curriculum during the academic year. JILEP trained the law professors in important advocacy skills, such as adversarial trial models, witness examination, building case theory, exhibit usage, and closing arguments. Further, a legal ethics workshop was delivered to Georgian law professors with input from STCL law professors and GBA experts on key issues of professional ethics such as client relationships, conflict of interest, and the principle of confidentiality. International experts assisted Georgian law schools to draft the legal ethics course syllabi. Client counseling is yet another focus of JILEP’s practical skills development efforts. JILEP sponsored the Client Counseling Competition, National Criminal Trial Advocacy Competitions for law students, and a summer school on Lawyers’ Practical Skills. Practical skills development efforts were also directed towards law professors (mediation skills training), practicing lawyers, and other legal professionals (professional ethics and legal writing training workshops). JILEP has not yet undertaken a follow-up study to assess the extent to which the skills were retained by trainees. However, by the time this report was developed, CRRC and JILEP were designing a study to measure MID-TERM PERFORMANCE EVALUATION OF JILEP 33 | Page the effects of JILEP’s capacity building and learning interventions on various target groups including students, law professors, and practitioners. As anticipated, the study’s results will be available by the end of 2013. Teaching materials and textbooks JILEP has contributed to and supported the development and publication of teaching materials and textbooks, mostly in the area of commercial law. National learning centers have been actively using JILEP’s assistance in developing the teaching materials for various learning courses offered to students or practicing lawyers. A textbook on the law of obligations has been developed by NCCL. Further, JILEP co-sponsors scholar groups to develop commentaries on tax legislation and civil code. Nonetheless, the lack of teaching materials in Georgian remains a major issue in the country’s legal education system. National learning centers JILEP supported the establishment of national learning centers (NLCs) – NCADR (TSU) and NCCL (FreeUni). Through technical assistance afforded by STCL and WUSL, both NLCs have gained visibility among students and professionals around the country, proven by the gradually increasing number of participants at their events organized and the increased number of student and practitioner applications to participate in training workshops, conferences, and competitions (specific quantitative data is given on below). Both NCADR and NCCL function pursuant the eventful operations plans offering diverse services to targeted audiences including students, practitioners, academicians, and the wider public. While housed in two Georgian universities, the centers are equally accessible to other university students, professors, and practitioners. They focus on improving legal education through curriculum and teaching materials development, and provide courses to build practical skills and analytical thinking of law students. Both NLCs also provide clinical legal education through respective accredited learning courses. Activities implemented by the NLCs through JILEP’s support include winter and summer school programs, student competitions including moot court competitions, law journals, a Georgian business law review, a public lecture series on commercial law and mediation, national symposiums, CLE conferences, a commercial law library, case books, and pilot project implementation. STCL provided TSU with intensive assistance to establish and make operational the NCADR. Further, STCL engaged its professors to deliver a variety of training courses to TSU and other law school students, including topics such as legal writing and reasoning, legal ethics, and mediation skills. STCL organized a trip for NCADR administrators to visit the United States in order to help them develop activity plans, observe STCL’s operation and management learning centers, and gain insight into U.S. ADR in the justice system. Likewise, the NCCL, housed in FreeUni, has received considerable assistance from WUSL in designing and delivering a number of training courses to students and practitioners on the topics of commercial law and of other substantive areas of law. Competitions JILEP supported a number of competitions for law students on a wide range of topics. Most were organized by NCADR and NCCL. International experts and law professors engaged by the partner western universities provided input in designing competitions for law students as well as practitioners. JILEP has supported and sponsored the following nationwide competitions:  National Criminal Trial Advocacy Competitions  Willem C. Vis International Commercial Arbitration Moot Competition Tbilisi Pre-Moot  Writing Competitions in Business Law for Georgian Business Law Review  National Student Competitions in Mediation  Writing Competition for NCADR Journal  Commercial Law Moot Court competition  Regional moot court competitions organized by the Batumi State University  Client Counseling Competition  First Annual Georgian Arbitration Competition As part of its effort to expand experiential learning opportunities for law students, JILEP sponsored Georgian students’ participation in international law student competitions – International Chamber of Commerce (ICC) MID-TERM PERFORMANCE EVALUATION OF JILEP 34 | Page Commercial Mediation Competition held in Paris, France, and International Client Consultation Competition held in Glasgow, Scotland. The Evaluation Team found an increasing interest among students from various law schools to participate in national competitions and the possibility of participating in international contests or moot court competitions is a major motivating factor. West Georgia Scholarships Through the West Georgia Masters of Law Scholarship Program, JILEP provides students from Georgia’s western regions the opportunity to pursue master’s level legal study at BSU and KSU. The program is managed by the Foundation for the Support of Legal Education, a local NGO founded by the GYLA. While all students from the western regions can apply for a scholarship, preference is given to students from the high mountain regions and nearby villages of Kutaisi and Batumi, as well as to students representing socially vulnerable segments of the population. There are a total of 12 supported students – eight in Kutaisi and four in Batumi. The main rationale of this intervention is to secure access to education and retain bright professional cadres in peripheral regions. Student beneficiaries of the W-Georgia Scholarships Program commit to participate in a number of activities aimed at enhancing their practical skills and providing a valuable contribution to NGOs providing free legal aid services. Namely, program fellows provide legal consultations, prepare legal documents, attend and monitor trials, prepare legislative proposals and participate in debate clubs. Importantly, they also conduct training workshops and public lectures for MA and high school students on a variety of legal topics. Currently, seven out of 12 students are employed in local CSOs, law firms, and public sector organizations (Parliament, police). Association of Law Schools Although JILEP has failed to facilitate the establishment of a national association of law schools, its efforts helped establish the Georgian Law Libraries Association (GEOLLA) to facilitate access to legal textbooks and materials for students, professors, and professionals. TSU’s school of law with the largest library is least interested in networking with other universities and, therefore, is not a part of GEOLLA. JILEP has provided capacity building measures to members of GEOLLA and students interviewed for this evaluation mentioned considerable improvement in the performance of libraries. 4.4.2 Conclusions JILEP has been effective in designing and implementing activities ultimately contributing to the improved legal education in Georgia. By the scale of activities, legal education is JILEP’s largest component. As noted above, most of the project’s legal education activities are long-term investments that should impact the quality of teaching and learning in a few years. Most of JILEP’s interventions, including those designed to improve teaching methods and introduce practical skills development elements in the learning process, should better prepare students for their entry into the dynamic and competitive environment of the legal profession. While JILEP has accomplished a great deal during its implementation, the basic knowledge conveyed to Georgian counterparts must still be well integrated in law schools’ learning processes and properly adjusted to Georgian realities. Change in the teaching and organizational cultures at universities takes time since significant barriers, including those related to individual perceptions and behaviors, must be overcome. Thus in the GBA Survey, 60% of respondents expressed that legal education in law schools (law faculties) needs “great improvement” and 35% stated that it needs “some improvement.” Nonetheless, in discussions with the Evaluation Team, students and professors observed that the orientation of law schools on practical skills development, introduction of interactive teaching methodologies, and creation of supplementary learning opportunities through NLCs have increased the motivation of both students and teachers to engage in the educational process. JILEP has been a major motivating factor among various stakeholders to enhance the institutional and professional capacities for improved legal education. Project beneficiaries and stakeholders observed improvements in three major areas of legal education: 1. Modern teaching methodology. According to the 2011 Baseline Assessment Report of legal education in selected Georgian law schools, the typical teaching methodology employed by law professors and lecturers was the “large lecture format” in which students had little interaction with MID-TERM PERFORMANCE EVALUATION OF JILEP 35 | Page professors in the classroom. Heavy emphasis placed upon rote memorization, with a single, summative assessment at semester’s end by either written or oral examination. While there is no indication that all Georgian law professors at JILEP’s partner universities have changed their lecturing methods, according to students, some classes are being delivered in a “more interactive format that makes the learning process more interesting for the students as well as professors.” Following the training workshops, some of the trained law professors adapted their curricula and testing procedures. However, according to informants and members of the certified trainer resource group, by the time of this evaluation, most of the academic staff and participants of teaching methodology workshops were still in the process of developing syllabi to align their courses with contemporary teaching methods. 2. Practical skills development and clinical education. According to Delaine R. Swenson, JILEP’s lead expert on teaching methodology and a member of the baseline assessment team, Georgian universities have made considerable progress in integrating the practical skills development elements in the learning process. While Georgia’s higher legal education still encounters major issues, as far as practical skills development is concerned, JILEP’s partner Georgian universities may be ahead of universities in some Eastern European Countries, including those in Poland. KIIs and law professor and student FGD participants emphasized the orientation of Georgian universities on legal skills development over the past two years. That said, students still have an expectation that classes and overall teaching and learning processes should become even more interactive and participatory to help them master practical skills prior to entering the profession. Further, as noted by a law professor during a FGD, clinical education in Georgian universities was nearly dead prior JILEP’s assistance to improve the management, performance, and operational procedures of clinics, as well as skills and capacities of clinic administrators and instructors. 3. Supplementary educational opportunities and access to learning resources. According to students, professors, and university/clinic administrators, learning opportunities provided by NLCs, access to online educational resources, improved operation of law libraries, and institutionalization of nationwide student competitions has significantly influenced the quality of legal education in Georgia. Nonetheless, there is a need to create an efficient strategy for beneficiaries and stakeholders to sustain and expand the project’s positive results beyond JILEP. Enhancement of clinical education for mastering students’ practical legal skills is a major breakthrough in Georgia’s legal education system. JILEP provided or supported activities to improve student’s practical skills through legal ethics, trial advocacy, and client counseling workshops. Clinical education, however, is the main avenue for students to master practical skills. According to the managers of JILEP’s partner law clinics, the revival of clinical education in Georgia is attributed to JILEP’s rigorous efforts. As noted in the 2011 Baseline Assessment Report of the Legal Education in Georgia which was conducted by JILEP in selected Georgian law schools, although few universities had clinical education in place prior to the project, their efficiency was not high due to the lack of teaching materials and capacity gaps of clinic instructors. JILEP-supported clinics are not evenly successful since some serve more individuals than others in settlements of approximately the same size. As clarified through interviews and discussions with clinic administrators and JILEP staff, the Evaluation Team found that there are a few factors influencing a clinic’s success, including motivation and qualification of clinic administrators and instructors, public and community awareness of the free legal advice offered by clinics, and availability of alternative free legal consulting and aid services offered by NGOs in the areas where clinics are located. Accredited and mandatory clinical courses do provide a rare possibility for building learners’ practical skills. The technical assistance provided through PIL-net to legal clinics helped to build the management capacities of clinic administrators and improve the teaching skills of law professors supervising students undergoing the clinical courses. All of the interviewed clinic administrators confirm that the Legal Clinic Handbook provides useful guidelines to managers and teachers in their efforts to design and deliver clinical courses. It should be noted that some JILEP-supported law clinics do not only provide live client counseling activities but also allow clinic students to provide public lectures and training workshops to fellow students and community members. Some have even developed partnerships with municipalities that refer individuals to the clinics for free legal counseling. MID-TERM PERFORMANCE EVALUATION OF JILEP 36 | Page Further, the municipality of Kutaisi provides funding to the legal clinic of the Akaki Tsereteli State University. The success of the clinical education in five Georgian universities has motivated other Georgian law schools to apply for JILEP’s technical assistance to improve the performance of their clinics. For the next academic year, commencing in September 2012, three other Georgian law schools will benefit from JILEP’s assistance. It must be noted that law schools have different structures and procedures of clinical education. In some schools, students have the option of replacing the mandatory clinical course with an externship program made available by agreements between law schools and different state and private institutions. Law schools and clinic administrators, however, cannot supervise the performance of students undergoing the externship programs. Notably, the recent trend in Georgia’s legal education system is for universities to generally become more oriented on practical skills development and invite practitioners to deliver lectures. While students take special interest in practitioner-led classes, engaging them, including lawyers and judges, entails the risk of disrupting the normal educational process whenever the practitioners designated or summoned to court hearings. Currently, not enough time has lapsed to assess the influence of capacity building interventions on the actual teaching and learning process. JILEP, together with WUSL and STCL, delivered a large number of training workshops to law professors from various Georgian universities to introduce more interactive teaching methodologies in law schools. JILEP designed a teaching methodology program specifically for TSU to help its professors adopt more modern, effective, and interactive teaching methods, and improve the overall teaching culture at TSU. According to law professors from various Georgian universities, the training workshops delivered by teaching methodology experts helped them to align their teaching techniques with the best U.S. teaching and learning practices. As noted by a law professor and members of the teaching methodology working group, the workshops delivered by JILEP‘s international experts were both interesting and useful for Georgian law professors who were introduced to the insights of modern law teaching methods. The Georgian professors appreciated the training course topics including development of course syllabi, evaluation of course design, evaluation of students, etc. As a part of the training workshops, law professors were assisted in developing or redesigning their course syllabi. Some of the interviewed law teachers were concerned that the syllabi limited their flexibility, to a certain extent, to make adjustments to their courses. Notably, participants of the students FDG also observed the change in lecturing method since some classes are being delivered in a more interactive, engaging manner with an emphasis on practical skills development. Nonetheless, according to FGD participants, students still want and expect a more engaging learning process not limited to only the communication of theoretical knowledge. Almost all law professors who participated in FGDs or were interviewed by the Evaluation Team stated that they effectively use the knowledge communicated by the international experts and professors during the teaching methodology workshops. However, it is yet too early to assess the impact of JILEP’s capacity building interventions on the actual teaching and learning process since most of the trained law professors are still in the process of syllabus development or redesign. According to teachers and school administrators, not all teaching methodologies presented at the workshops can be effectively applied in Georgian law schools due to the local teaching culture and school infrastructure. Further, additional guidance and materials (teaching method textbooks) are being requested by law professors and teachers who went through the training courses. It is noteworthy that JILEP has been less successful in engaging the older generation of law professors in teaching methodology interventions. Language is one of the major barriers preventing older professors from participating in the interactive teaching methodology workshops, which are conducted in English. However, stakeholders noted that universities are undergoing a somewhat painful process of changing generations of law professors – graduates of western universities and young scholars come teach alongside the older generation of professors who are less enthusiastic about interactive teaching methodologies and are mostly focused on strictly theoretical knowledge. Younger teachers have the willingness and motivation to absorb western teaching methods. MID-TERM PERFORMANCE EVALUATION OF JILEP 37 | Page Interviewed teachers observed that many Georgian law professors who had been less enthusiastic about JILEP’s offer to help enhance their teaching skills were actively engaged in capacity building initiatives after the initial rounds of training workshops. To retain teaching methodology know-how in-country, JILEP has taken an effective approach of establishing a resource group of law professors through an extensive ToT program. Sustainability and accessibility of the resource group is yet to be resolved. Through its teacher development program, JILEP prepared a team of Georgian instructors from eight Georgian law schools to train other Georgian professors in modern teaching methodology and teaching techniques. This is a smart approach to retain know-how in the country and create a resource group of professionals readily available to conduct interactive teaching methodology workshops for professors at interested law schools. This operational strategy successfully used the experience of the international teaching methodology experts engaged by EWMI. The experts completed similar capacity building projects in a number of countries, some of which were financed by USAID (Poland, Cambodia). Further, JILEP’s approach to building a resource group of academic trainers is in line with EU recommendations emphasized in the 2013 Report on Improving the Quality of Teaching and Learning in Europe’s Higher Education Institutions, developed by the High Level Group on the Modernization of Higher Education. While the Report recommends that all staff teaching in higher education institutions should receive certified pedagogical training by 2020 and should be required to seek continuous professional education, these recommendations, as well as others, should be accompanied by guidance, counseling, and coaching methods, the professionalization and development of teachers, trainers and staff, and mobility in and exchanges of academic staff for teaching assignments. While the final training workshop for the teaching method train-the-trainers program was delivered in May 2013 and the resource group has not been actively used by law schools as of now, it is not yet clear what strategy will be deployed to make the group visible and accessible for the universities. There are a couple of options currently considered among group members: 1. Establish a law teaching methodology association as an independent NGO to offer trainings and coaching services to interested schools or individuals. 2. Form a teaching methodology center in one of the lead Georgian universities (FreeUni, TSU) that will be accessible for all other universities and schools. Both options have pros and cons. For instance, establishing an association of trainers ensures a high degree of institutional independence of the resource group. On the other hand, however, managing an association’s work will require administrative work from professors, many of whom are employed in other organizations or are practicing law, who have limited time resources. Alternatively, although forming a teaching methodology center in one of the selected universities may relieve the resource group members from administrative or managerial burdens, it may affect the motivation of other universities from applying for coaching from a competitor university where the Teaching Methodology Center will be housed. As international best practices demonstrate, the similar resource groups providing capacity building to teachers or instructors are most effectively used when managed and coordinated by the associations of schools. JILEP has been active in supporting the initiatives of development of teaching materials on commercial law-related areas. While there is a lack of teaching materials in all major areas of laws, it is least likely that JILEP’s efforts alone will be sufficient to address this gap. JILEP supported its partner universities and learning centers to develop teaching materials and textbooks, mostly in the area of commercial law. According to university professors and students, there is a huge lack of quality in Georgian teaching materials for almost every substantial law subject. It is noteworthy that the publication of law textbooks is not commercially viable in Georgia for cultural and financial reasons. Although there is a considerable demand on law textbooks among law students and practitioners, the market is too small to generate profit for authors and publishers. Further, students prefer to photocopy textbooks at lesser costs. Most textbooks published during the past few years have been sponsored by international donor organizations. Law professors, for their part, struggle to find appropriate teaching materials for their courses. While some teaching materials are available in English, not every Georgian student is fluent enough to understand the contents of English law books, especially outside of Tbilisi and its suburbs where knowledge of English is lower. MID-TERM PERFORMANCE EVALUATION OF JILEP 38 | Page It seems that JILEP’s scope and resources will not be sufficient to address this major gap. So far, the project’s strategy has been the promotion of online resources through NLCs and, most importantly, establishment of GEOLLA though the information services development program. JILEP intends to purchase textbooks for all law school library members of GEOLLA. The Memorandum of Understanding signed among the six member law schools provides for securing access to the services of all libraries for the students of all member schools. The NLCs – NCCL and NCADR provide an important input in the implementation of respective national policy initiatives and act as service providers for various stakeholders, including GBA. The NLCs housed in two lead Georgian universities – TSU and FreeUni – provide diverse services to students, practitioners, and the wider public through their educational programs. According to students from various Georgian law schools, these centers provide a rare opportunity for learners to access supplementary legal education courses. The increased number of student and practitioner applications to attend the NCL’s fee￾based courses demonstrates their success. In an interview with the NCCL administrators, the Evaluation Team learned that initially, the center experienced some difficulties in attracting participants to their courses. However, after initial events, the center rapidly gained popularity among students, academicians, and practitioners. As the data provided by NCCL suggests, the number of participants in CLE courses, student competitions, winter and summer school, open public lectures, and other NCCL events increased over the past year. Throughout 2012, 705 people (366 students and 339 practitioners) participated in 14 events organized by the center, while in the first seven months of 2013, 773 people (497 students and 276 practitioners) participated in 11 NCCL organized events, with more yet to be organized before year end. For some events (e.g. summer and winter school), participants are in the pre-selection process due to the high number of applications. As noted by student FDG participants, free learning opportunities in Georgia are still rare, so motivated students seek out courses offered by NLCs (NCCL, NCADR) and NGOs. According to managers of the learning centers, nearly all services are accessible for other university students, professors or practitioners. Clinical education courses and online law libraries are the only exceptions. NCADR, a national training center, provides input in the implementation of national policy reforms and is involved in the HCOJ-initiated court-annexed Mediation Pilot Project. NCCL received an organizational accreditation with the GBA and provided CLE courses for bar members. While most of the universities have been successful in partnering with JILEP, FreeUni has been JILEP’s best partner and the one that has benefitted the most from various opportunities. Nearly all of the partner universities have been efficient in implementing their programs supported by JILEP. The clinical education program, implemented in five Georgian law schools, produced good results in terms of building the capacities of legal clinics to develop students’ practical skills (allegedly, BSU is the only underperforming legal clinic). Likewise, NCADR and NCCL established in two law schools, demonstrate an example of good practice of NLCs. That approach can be replicated in other universities or schools. Based on feedback from project stakeholders and JILEP staff, the Evaluation Team found that FreeUni benefitted the most from JILEP’s individual and institutional capacity building interventions. This was based on the following factors:  FreeUni, although a young school, has a modern school management system in place that is based on western models.  FreeUni managed to engage motivated staff and professors who actively participated in and provided valuable contributions for JILEP’s legal education interventions.  Compared to TSU, FreeUni is still at its early stage and does not have as many international contacts. Therefore, FreeUni is more motivated to absorb the developmental opportunities available through partnering with well-established western universities.  FreeUni has the education quality monitoring systems in place that distinguishes it from other law schools.  FreeUni contemplates other initiatives with WUSL beyond JILEP. MID-TERM PERFORMANCE EVALUATION OF JILEP 39 | Page 4.4.3 Recommendations Develop sustainability strategy for legal education component. JILEP should continue to support institutions and professionals to improve the legal education system and facilitate the efforts of beneficiaries to produce clear strategies and operations plans through which local partners will be able to sustain activities and results accomplished to date. Currently, it appears that the sustainability of JILEP’s legal education interventions can be maintained. The West Georgia Scholarship Program is the only intervention that, from this standpoint, necessarily requires raising external funds by GYLA’s Foundation for Legal Education. According to university administrators interviewed by the Evaluation Team, upon completion of JILEP, they should be ready to continue implementing interventions now supported by JILEP by optimizing activity costs (e.g. by organizing or holding workshops and summer/winter schools in Tbilisi instead of remote areas that will require less administrative costs), and by properly advertising capacity building and learning opportunities offered by NLCs and universities to the legal community. Moreover, some of JILEP’s legal education activities, designed to improve the management skills and capacities of local beneficiaries, will not require continuous external funding but proper knowledge management systems among local partners enabling them to duly document and retain the knowledge communicated by JILEP. Nonetheless, it is recommended to identify necessary financial, human, or technical resources that are required from JILEP’s partner organizations to uphold the institutions created through JILEP’s efforts. JILEP should facilitate the efforts of the certified teaching methodology trainers to design an operational strategy of making this resource group visible and widely available for all law schools across the country. However, a wider spectrum of stakeholders will need to be involved in the discussion of ideas that will lead to a more institutionalized and sustainable format. Undertake performance assessments and follow up studies to learn about and analyze the impact of legal education interventions. JILEP should regularly conduct performance assessments of its local partner institutions and, where relevant, design or provide additional assistance measures to partner institutions. JILEP should undertake follow-up studies during the next academic year to identify to what extent teaching methodology trainings have been useful for actually introducing new interactive teaching techniques. Facilitate partner university discussions on the establishment of the association of law schools. JILEP should encourage law schools to reconsider the benefits of forming an association. While founding an association depends on the will of potential members, JILEP can pursue these members to support such an association by bringing as an example the project’s achievements to-date, including developing common legal education standards, advocating for common interests, sharing information and experiences, accommodating certified trainers resource groups, developing textbooks and teaching materials, continuing positive cooperation beyond JILEP, etc. Design and provide further practical skills development activities. Lack of legal research skills among students is being identified as one the major challenges for Georgia’s legal education system. While JILEP provided a series of legal writing training workshops, most project beneficiaries requested more comprehensive legal research training workshops, a part of which will include legal writing skills development. Communicate information and materials of the ongoing capacity building interventions across all stakeholders. While engaging the older generation of professors in teaching methodology training workshops is a difficult task for JILEP and the resource group of certified trainers, it is recommended to keep the professors informed on modern teaching techniques and provide related materials. 4.5 QUESTION 5: EFFECTIVENESS OF SUPPORT TO COMMERCIAL LAW Georgia depends on an effective legal framework and judicial environment related to commercial law. To what extent have JILEP activities contributed to improvements to commercial law related practice, including the transparency, efficiency, and effectiveness of commercial adjudication, alternative dispute resolution and enforcement of the legal framework? Moving forward, what approach can best contribute to improvements in the legal framework, judicial environment and commercial law practice for sustainable economic growth? MID-TERM PERFORMANCE EVALUATION OF JILEP 40 | Page 4.5.1 Findings JILEP contributed to the commercial law environment with a series of activities related to improving its implementation. Perception of the fairness of commercial adjudication has dropped significantly during JILEP whereas ADR has gained acceptance, if not use, for commercial disputes. There are two aspects to commercial law. First, the commercial law framework includes subjects such as contracts, tax, secured transactions, intellectual property, bankruptcy, and other substantive areas of the law. Most of these areas have been developed in Georgia, although further elaboration and amendments may still be required. Second, implementation of commercial law includes attorneys’ representation of their clients and judges’ application of the law to disputes. There are certain corollary institutions that should be supported, for example entry of secured rights into a registry. Law students must be trained in commercial law. Bankruptcy trustees must be trained and organized in company reorganization. In general, economic growth programs are assistance programs that seek to establish the framework of commercial law, devoting their efforts to writing discrete pieces of legislation, implementing regulations to establish this framework, and setting up pledge registries. Rule of law programs devote their efforts, in contrast, to supporting the capacity of major actors who must understand and apply the law and are more concerned about the process and procedures for effective, efficient,and transparent mechanisms to resolve disputes. Rule of law programs are tasked with providing commercial law training. They work with law schools to train students, judicial training centers to train disposed judges, and bar associations to train practicing lawyers. They are disposed to provide assistance to developing transparent, efficient, and effective processes for resolving disputes. Transparent regards whether commercial adjudication is open, fair and impartial; efficient, whether resorting to a body, judicial or not, allows parties to resolve commercial disputes at a reasonable cost in relation to the amount in controversy; and effective, whether resorting to a body allows parties to resolve disputes with reasonable predictability based on the law. ADR is not a substantive field of commercial law. It is, as it name implies, an alternative mechanism by which parties may resolve their disputes, commercial or otherwise. Some of these mechanisms are included in court processes and others are used as a way to stay out of court. In some cases, arbitral tribunals may be the only fair way to resolve a dispute as the courts are seen as so corrupt that neither party trusts it will receive a fair and impartial hearing. In other cases, the courts are overloaded and alternative disputes mechanisms are attractive almost as a case management tool. Finally, in other cases, judges may simply lack the expertise to handle commercial cases leading parties to alternative mechanisms to resolve their disputes. Lawyers in Georgia are quite familiar with mediation and arbitration as an element of ADR. When asked, 41.3% of respondents in the GBA Survey said that they were “familiar” or “completely familiar” with mediation; and 63.2% said they were “familiar” or “completely familiar” with arbitration. The mandate of JILEP was limited to “strengthening commercial law practice.”20 One of the underlying purposes of JILEP was to implement the commercial law framework by building the capacity of core legal institutions, including law schools, the bar and judiciary, to review, apply and interpret commercial laws. As JILEP matured, it used the commercial law component as a cross-cutting measure to increase the capacity of its counterparts and some of the activities described in this section that have also been addressed in previous sections. GBA Survey shows continuing need for support to lawyers and judges For this evaluation, a GBA Survey was given to attorneys to probe how critical the need is for commercial law training for judges and attorneys. No clear trend emerged from respondents regarding their perception of the level of expertise of lawyers and judges, and both groups were judged similarly. While only 26% of respondents were not satisfied (1 or 2 on 5 point scale) with the level of commercial law expertise of lawyers and only 23% were not satisfied with the level of expertise of judges, there was only tepid confidence in the abilities of both attorneys and judges in the area of commercial law (see Figure 8, next page). 20 Cooperative Agreement, Program Description, p. 25. MID-TERM PERFORMANCE EVALUATION OF JILEP 41 | Page Consequently, it was not surprising that the Evaluation Team found that commercial law training remains absolutely necessary. The Supreme Court provided the Evaluation Team with recent statistics on the kinds of commercial cases which trial and appellate courts are reviewing and rendering decisions. Although the number of cases that relate to obligations’ law, including rent, transportation, and pledge agreements, jumped last year by 36% to almost 9000, based on statistics for the first six months of this year, the number might decline by as much as 70%. Trial-level cases related to entrepreneurship declined by 28% last year and only 49 were reviewed, including two bankruptcy cases. Disputes related to administrative contracts declined by 78% last year to 164, and the number of tax related disputes fell by 30% to 132. Figure 8: To What Extent Are You Satisfied with the Level of Expertize in Georgian Lawyers and Judges in Commercial Law? (%) Source: GBA Survey At the appellate level, the number of commercial cases dropped last year by 11% to 1,330, with over a third related to sales or loans. At the administrative level, law cases declined by 75% to 126, and tax and customs cases, by 67% to 80. At the cassation level, cases related to obligations, about a third of which dealt with sales or loans, fell by 8% to 574, administrative law cases by 70%, and tax and customs cases, by 67% to 87 cases.21 Training of judges in commercial law JILEP worked through HSOJ to conduct judicial training on commercial law topics. HSOJ is a success story of international assistance. It originated as an NGO in 2006, largely supported by donor assistance, and evolved into a government institution capable of making its own decisions about the kind of support that it wants. As described previously, HSOJ made clear that it was not interested in working on institutional development with JILEP, but was interested in course development on select commercial law topics. HSOJ and JILEP worked together to introduce a tax course for judges who hear tax cases. JILEP offered three trainings, two days each, on the new tax code in February – April 2011. At the end of the series, about 65 judges had received training. According to JILEP, the course evaluation by the judges who attended was positive. There was a break of almost two years on commercial law. As related by JILEP, “finding avenues to expand cooperation with the HSOJ continues to be a challenge”22 and HSOJ was “reticent to expand its cooperation beyond occasional, narrowly targeted courses.”23 The next training on commercial law was in April 2013 for 14 judges on international trade law and a course to 13 judges in May 2013 on international and domestic commercial arbitration. 21 These upublished data were provided on August 2, 2013 to the Evaluation Team by the Supreme Court Department of Court Statistics and Analysis 22 Quarterly Report 2011, Q2. 23 Quarterly Report 2012, Q1. 4.6 21.8 54 10.3 10.3 9.2 12.6 49.4 18.4 9.2 0 10 20 30 40 50 60 Not satisfied at all Not satisfied Neutral Satisfied Completely satisfied In Georgian Lawyers In Georgian Judges MID-TERM PERFORMANCE EVALUATION OF JILEP 42 | Page HSOJ provides about 60 training programs throughout the year, most of which are two days. HSOJ has limited personnel and must allocate its resources to accommodate donors who are vying to insert topics into the curriculum. For example, GIZ has scheduled 10 trainings in the upcoming year. Commercial law is a priority because, according to the acting director, judges generally have little experience in commercial law. GBA requires continuing legal education for lawyers GBA boasts a membership of about 3,500. Working with an EU program on “Support to the Reform of the Criminal Justice System in Georgia,” JILEP assisted the GBA to introduce mandatory continuing legal education (CLE) requirements that started with a minimum of six hours in 2012, 10 hours in 2013, and 12 hours in 2014 and beyond. A critical component of CLE is that GBA provides an accreditation to providers and there is open competition among accredited providers to provide this training. JILEP has worked to expand the capacity of GBA to provide CLE to lawyers. As one respondent candidly assessed GBA, “We weren’t a good partner two years ago because we didn’t have the capacity.” The Evaluation Study showed progress in CLE in general. The Attorney Baseline Survey conducted by CRRC in early 2012 found that 54% of respondents thought that “continuing lifelong legal education” was very accessible (5 on a 5 point scale). This result improved to 66% in the Evaluation study. When it comes to commercial law training, the respondents were somewhat more equivocal in the Evaluation Study (see Figure 9 below). Figure 9: To What Extent are Opportunities for Continuing Lifelong Legal Education Accessible to You? (%) Answer Baseline (June 2012)* GBA survey (July/August 2013) Not accessible at all 2.6 3.4 Not accessible 15.4 3.4 Neutral 10.3 10.3 Accessible 17.9 17.2 Very accessible 53.8 65.5 * Response by 39 legal professionals, CRRC data Sources: 2012 Attorney Baseline Survey, n = 39; 2013 GBA Survey, n= 87 Those who attended CLE thought the trainings were useful to their practice of law (72%) and that the CLE was effective (70%). According to the studies, the percentage of respondents who think that CLE needs “great improvement” declined from 56% to 29%. (In the original study, there may have been some ambiguity about the term “legal education” – but for evaluation purposes, it was assumed that the question related to CLE rather than law school education). In the evaluation study, by comparison, 60% of the respondents thought that law school education needs “great improvement.” Similarly, the results showed that respondents to the GBA Survey are generally satisfied with CLE offerings. On a five point scale, 86% (4 or 5 on a 5 point scale) of respondents agreed that the current content of CLE training offers detailed, high quality yet diverse courses, which facilitate and improve attorney performance. Referring to a CLE course, one attorney participating in the attorney focus group exclaimed, before the training “we were blind.” The GBA Survey showed that CLE, through its various providers, has made significant progress, despite less than stellar assessment of GBA as an organization. GBA must still gain the confidence of its members. Almost 53% of respondents think GBA reform is “very necessary” (5 on a 5 point scale) in terms of creating a competent, ethical bar in Georgia. These data also reflect that, since the Attorney Baseline Survey, GBA has not made significant improvement as 64% of respondents saw GBA reform as necessary (4 or 5 on a 5 point scale) in the Attorney Baseline Survey and 66% in the GBA Survey. However, GBA fared better than HCOJ, which 77% of respondents believed “very urgent” reform. JILEP’s activities improved CLE in commercial law Before JILEP, attorneys who wanted or needed CLE, particularly in commercial law, had few opportunities. In the FGD of GBA members lawyers indicated that they have limited experience in commercial law and need further training. Some also said that they could use commercial law only if they were trained in it. MID-TERM PERFORMANCE EVALUATION OF JILEP 43 | Page Still others said that they had not requested this in the past since they were not aware of this branch of law. “We have no practice in commercial law and we need more training.” “If you trained us in [commercial law], we would use it more.” “How can I request something that I don’t know?” JILEP engaged in a series of activities to broaden the profile of commercial law and to enable providers to provide CLE on commercial law. At the outset of the project, JILEP tentatively selected commercial law subjects “develop as learning modules” for the CLE program, including tax, insolvency, drafting contracts, corporate law and governance, and corporate finance for lawyers. JILEP has implemented several approaches to provide trainings in commercial law: 1) GBA support; 2) advocacy grants; 3) support to the national centers of training; and 4) direct training through JILEP. GBA, with JILEP funding, provided 14 trainings on the tax code in 2012 and 3 as of this evaluation in 2013. Article 42 received 2 grants to implement trainings and provided 10 in the first year and will provide more this year. The NCCL was particularly active in providing CLE on commercial law, organizing and implementing three major conferences since December 2011 on commercial law with various break-out sessions on other related topics. Over 100 lawyers attended each conference and even more expressed interest in attending. NCCL also provided a three-day training on the law of obligations. NCADR provided trainings, with JILEP funding, on mediation. JILEP has directly provided training along with the International Council for Commercial Arbitration on the Convention on the Recognition and Enforcement of Foreign Arbitral Awards (New York Convention). Based on the results of the GBA Survey, commercial law training has not become significantly more accessible in the little over a year since the original Attorney Baseline Survey was conducted. 23% of respondents in both studies thought that commercial law training was “very accessible.” The baseline study showed that 33% of respondents thought that commercial law training was not accessible (1 or 2 on 5 point scale), whereas 31% shared this opinion in the GBA Survey (see figure 10 below). Figure 10: To What Extent is Commercial Law Training Accessible to Legal Professionals? Answers Baseline, June 2012* GBA Survey July/August 2013 Don’t Know 2.6 Not accessible at all 5.1 21.8 Not accessible 28.2 9.2 Neutral 20.5 25.3 Accessible 20.5 20.7 Very accessible 23.1 23.0 *Response by 39 legal professionals, CRRC data. Drafting of commercial legislation and commentaries At project outset, it appeared that JILEP may have planned to engage more with MOJ in preparing legislation. However, its efforts with MOJ to gain a seat at the table did not materialize. In its first quarterly report in 2010, JILEP indicated that MOJ planned a comprehensive reform of Georgian private law and JILEP “stands ready to assist,” but ultimately did not become engaged in the actual drafting of legislation until 2013. JILEP worked with MOJ to organize a “Consulting Council of Private Law Reform” to consider changes needed to the private legal framework in April 2013. JILEP currently sits on working groups on property law, company and insolvency law, civil procedure and ADR at the MOJ, although it is one of numerous actors, does not set the agenda and, according to two KIIs at MOJ, does not have critical input into draft legislation. JILEP has devoted considerable time and energy to support writing two significant commentaries. In cooperation with the USAID Economic Prosperity Initiative, JILEP supported a working group to prepare a comprehensive commentary in two volumes on tax legislation, which was completed in mid-2012. JILEP recognized that there was a significant need for a new commentary to the Civil Code as the last one was prepared 12 years ago. JILEP is now working with GIZ to organize and support a working group on preparing a new commentary on the Civil Code, which is scheduled to be completed in 2014. MID-TERM PERFORMANCE EVALUATION OF JILEP 44 | Page Commercial law integration at the law schools JILEP created a commercial law environment to establish and integrate topics and trainings in law schools. The project funded the creation of two textbooks on contract and property law which, although delayed, are scheduled to be completed in 2013. JILEP worked primarily with NCCL on commercial law and with NCADR on ADR, although in some areas both centers worked together. JILEP and NCCL worked to create mini-courses on commercial law topics, hold writing competitions on commercial law topics for students and professionals, establish the Georgian Business Law Review, and hold a moot court competition in commercial law. With JILEP funding, NCCL has held numerous public lectures on commercial law, developed a library on business and commercial law, is developing commentary on certain commercial law judicial decisions, and is creating video lectures on commercial law. Acceptance of ADR has improved JILEP set out to develop ADR mechanisms throughout Georgia. One of the two centers of learning that JILEP established, was devoted to ADR. Following an assessment, JILEP found that ADR has a bad reputation in Georgia because commercial institutions, primarily banks, reportedly use mandatory clauses to require arbitration in captive arbitral tribunals that are seen as corrupt and not independent. Accordingly, JILEP decided to focus ADR activities more on mediation than arbitration, although some activities were continued teach and train students on arbitration. JILEP worked to amend the Code of Civil Procedure and then trained 17 mediators through the UK Center for Effective Dispute Resolution. Training mediators was a significant step. The mediators’ activities are limited to certain areas such as neighborhood disputes and inheritance. JILEP recognized that the introduction of mediation was an evolutionary process, but as part of this evolution, the acceptance of ADR in general is a necessary prerequisite. JILEP trained judges on the Convention on the Recognition and Enforcement of Foreign Arbitral Awards (New York Convention) in June 2013. Since the Attorney Baseline Survey, ADR, both mediation and arbitration, has gained wider acceptance as shown in the GBA Survey. 5% of baseline survey respondents thought that there was a “great demand” for mediation whereas this percentage rose to 20% in the GBA Survey. Similarly, 10% of baseline respondents thought that there was a great demand for arbitration while 23% of the GBA Survey thought so. In commercial disputes, 40% of the respondents in the GBA Survey thought that it was very necessary to introduce mediation for commercial disputes, compared to 28% in the original Attorney Baseline Survey. The acceptance of arbitration was even more pronounced, as the percentage who thought it “very necessary” for commercial disputes grew from 46% to 63%. This preference for arbitration was reflected in the percentage of respondents who thought that mediation will definitely contribute to efficiency of reaching settlement in commercial disputes, which dropped from 49% to 46%, while arbitration grew from 38% to 49%. Perception of fairness in commercial law cases has significantly deteriorated Since 2005, the integrity of the judicial system has seriously been in question. One bright spot, however, was the resolution of civil cases. As related to the Evaluation Team by several respondents during interviews, in cases affecting the government, particularly criminal and administrative cases, there was serious question about judicial independence. In sharp contrast, civil commercial case disputes were resolved fairly, according to several KII with interviewees from the courts, private practice and civil society. Despite these subjective views expressed during interviews, results of the GBA Survey produced surprising results showing a significant deterioration in the perception of the fairness of commercial disputes in the courts. In the Attorney Baseline Survey, 44% thought the outcome of cases in commercial disputes was fair (4 or 5 on a 5-point scale). In the Evaluation study, only 16% would reach the same conclusion. Only 5% of respondents in the Attorney Baseline Study were “not confident at all” that these disputes would be resolved fairly, while 31% of respondents in the GBA survey were not confident at all that these disputes were fair and impartial (see Figure 11, below). MID-TERM PERFORMANCE EVALUATION OF JILEP 45 | Page Figure 11: How Confident Are you That the Outcome of Cases in Commercial Disputes are, on the Whole, Fair and Impartial? (%) Sources: 2012 Attorney Baseline Survey, n = 39; 2013 GBA Survey, n= 87 The Evaluation Team also looked at this result in comparison with how the respondents looked at criminal and administrative cases. Only 10% of the respondents thought that criminal cases were fair (4 or 5 on a 5 point scale), in comparison with 18% of the respondents for administrative cases—but only 16% of the respondents in commercial cases. Figure 12: How Confident Are You That the Outcome of Cases in Commercial Disputes are, on the Whole, Fair and Impartial? (%) (by case category) 4.5.2 Conclusions JILEP’s activities have had a modest impact on the HSOJ to provide training in commercial law. The HSOJ limited JILEP’s ability to provide institutional support. Nevertheless, JILEP was able to cooperate with HSOJ on providing training on tax law in the first half of 2011. According to JILEP (QR 2011-2), “[F]inding avenues to expand cooperation with the HSOJ continues to be a challenge.” JILEP intended to offer additional courses, for example in tax and economics, to HCOJ on commercial law topics, although these other courses were not given and there was almost a two-year hiatus until the next commercial law course with JILEP funding and support was given at the HSOJ. The two new courses scheduled for this year do not relate to the 0 5 10 15 20 25 30 35 40 Not confident at all Not confident Neutral Confident Very confident 31 18.4 34.5 10.3 5.7 37.9 18.4 33.3 5.7 4.6 21.8 21.8 37.9 11.5 6.9 Commercial Criminal Administrative 5.1 12.8 30.8 33.3 10.3 31.0 18.4 34.5 10.3 5.7 0 5 10 15 20 25 30 35 40 Not confident at all Not confident Neutral Confident Very confident 2012 2013 MID-TERM PERFORMANCE EVALUATION OF JILEP 46 | Page vast majority of commercial disputes that judges hear. The HSOJ appears to have an appetite for additional assistance in commercial law topics, despite its recalcitrance on assistance on institutional development. JILEP’s activities significantly improved the capacity of law schools to provide law students commercial law subjects. JILEP substantially improved the capacity of law schools to provide training to law students on commercial law and ADR. JILEP’s support for the NCCL at Free University and, to a lesser extent, for the NCADR at TSU, has significantly24 improved the capacity of these two universities to provide law students with essential tools in commercial law, although it is too early to assess the impact of these activities over the long-term and whether these two centers of training will become sustainable without JILEP’s assistance. JILEP significantly strengthened CLE in commercial law. The quality of continuing legal education has substantially improved during the project as JILEP has provided support to providers of continuing legal education. An overwhelming majority, 86%, of the respondents in the GBA Survey, agree that the current content of CLE training offers detailed, high quality yet diverse courses which facilitate and improve the performance of attorneys. Yet, the responses in the GBA Survey were substantially similar to the Attorney Baseline Survey when asked about accessibility of commercial law training. Commercial law training comes from disparate providers and training offerings are periodic rather than systematic. As reflected in the GBA Survey, the quality of course offerings is generally respected, but attorneys want to see more frequent opportunities for commercial law training in a breadth of commercial law subjects. JILEP’s commentaries are a significant step in implementation of commercial law. The comprehensive commentary on the tax code was a significant achievement and it is expected that the commentary will be used by practitioners, judges and students. Once completed, the civil law commentary will also be an important accomplishment of the project. The Program Description of JILEP in the Cooperative Agreement did not include drafting activities and JILEP’s drafting activities in commercial law have been modest. Not surprisingly, JILEP had “no major influence” on private law reform, as related during one KII. The MOJ has organized a Consultative Council on Private Law Reform, which includes working groups on, among other topics, entrepreneurship and bankruptcy, and obligations. JILEP has recently started sitting on these working groups and is one of their many members. JILEP could monitor the work on these working groups through one of its partner organizations, such as GBA or CITJ or one of the recipients of the advocacy grants. ADR has gained acceptance. JILEP’s assessment uncovered serious skepticism about the institution of arbitration in Georgia. JILEP devoted time and energy into the institution of court annexed mediation with a substantial effort working with the courts, with Parliament to amend the Code of Civil Procedure, and with training mediators who are poised to being their work later this year. The institution of mediation will change neighborhood disputes and family law matters. It is too early to determine whether parties will soon or ever agree to mediation in commercial matters. This institution took a long time to evolve in the United States and other countries and there is no reason to believe that Georgia will be any different. At the time of the assessment, arbitration was disfavored. JILEP’s activities in training judges on recognition or international arbitral awards, and the training of lawyers, in addition to the student training appear to have contributed to an increasing acceptance of arbitration. There is more willingness within the attorney ranks to revisit the institution of arbitration in Georgia. Commercial adjudication is now perceived as less fair than it was a year ago. There is no clear explanation why the perception of fairness in commercial adjudication deteriorated substantially. During several KIIs, respondents indicated that fair commercial adjudication should be a model for administrative and criminal cases. The GBA Survey suggested that adjudication in commercial cases should not be considered a model. 24 JILEP’s work in supporting these institutions is detailed above in Section 4.4.1, and in the commercial law area, includes numerous public lectures on alternative dispute mechanisms and commercial law, commercial law moot cour program, arbitration moot competition, the preparation of textbooks on commercial law subjects in progress, and other activities. For example, intensive short-term courses are offered not only to students and Free University and TSU, but to law students throughout Georgia. The summer school on commercial law in 2012 attracted 34 students from various universities. MID-TERM PERFORMANCE EVALUATION OF JILEP 47 | Page We recognize that this finding is not consistent with interviews that we undertook in connection with the evaluation fieldwork. We prepared a cross tabulation to find which cohorts were the most skeptical of commercial law adjudication. Gender did not play a role. Age played a significant role as did the primary field of practice of the respondents. 52% of those respondents 46 or over were “not confident at all” in the fairness or impartiality in the resolution of commercial law cases, versus 23% who were 45 or younger. Similarly, there was no difference between the responses of the men and women in the survey. We also note that the number of commercial cases substantially declined in several categories, but without further investigation we cannot say whether this is because there are fewer disputes as parties are resolving their differences, whether parties do not believe in the impartiality of judges in these cases, or whether there is a different reason. 4.5.3 Recommendations JILEP can focus its efforts on commercial law with the HSOJ. Further information is needed to determine whether finding on fairness of commercial adjudication is pervasive throughout the legal profession. Nevertheless, we recognize that the absorptive capacity of the HSOJ is limited. With GIZ’s substantial programming, there may be limited opportunities for further training on commercial law in 2013. Nevertheless, judges and the HSOJ still appear interested in further development of commercial law curriculum for those judges who hear commercial law cases. We would recommend developing bench books, short and concise step-by-step summaries of how to handle certain kinds of cases. These bench books can be devoted to the most frequent commercial law cases, for example in sales or rent or transportation, or whatever area on which JILEP and HSOJ can agree. These bench books can then later be used as written materials for later training through the HSOJ. Target opportunities in the process of drafting commercial legislation. JILEP has had influence on commercial law development through its grant and support of CITJ and Article 42 and a grant for the improvement of commercial law legislation. JILEP also attends several MOJ working group sessions on commercial law and other areas. JILEP is one of many actors, including international organizations, which endeavor to influence the substance of the drafts. There may be a point of diminishing returns on how much influence the project may have on these sessions and from a sustainability perspective, once JILEP is over, its influence will cease. Especially in the area of commercial law, the Georgians give substantially more weight to German expertise and with many actors JILEP’s voice may pass the point of diminishing returns. We recommend that JILEP still support these proceedings through an NGO by providing an advocacy grant. There are several able NGOs that may be able to work with MOJ, such as Article 42, GBA, CITJ, and others. One approach where JILEP may provide direct support is by assisting in the process of drafting commercial legislation. MOJ has the core strength in legal drafting with an expert staff of 14 lawyers, the majority of whom have five or more years work experience. They currently have an old drafting manual which they use. We understand that the EU will have a project next year on legal drafting, although the details of the project are not known. In the meantime, we recommend that JILEP try to engage the MOJ to further develop the capacity of its legal drafters—and use one of the pieces of commercial legislation as an example of legal drafting techniques. Accordingly, we recommend that JILEP shifts its emphasis from reviewing commercial legislation to the process of drafting commercial legislation. Continue to provide tools to integrate and build commercial law training. With JILEP’s support primarily to GBA and to the NCCL, quality commercial law trainings are becoming part of the landscape of the legal community in Georgia. The structure under which various providers accredited by GBA may offer trainings to fulfill mandatory CLE requirements is sound and sustainable. JILEP may want to accelerate these efforts through the end of the project to make sure that the quality and consistency of commercial law offerings is maintained after the termination of the project. JILEP has already supported CLE and local organizations such as NCCL and other accredited GBA providers have the capacity to maintain a consistent repertoire of commercial law training for example, the three day program on obligations last year. JILEP should encourage and support targeted short-term training on commercial law topics with greater frequency and geographic coverage in more subject areas that are relevant to practitioners (which JILEP has already identified). MID-TERM PERFORMANCE EVALUATION OF JILEP 48 | Page Develop blueprint to support arbitration in commercial cases. JILEP should continue to support the introduction of mediation although JILEP itself recognizes that mediation in commercial cases is a long-term goal rather than a short-term opportunity. JILEP has already felt the shifting winds regarding the acceptance of arbitration and has embarked on a series of activities that should be supported to build local capacity. JILEP recognized that after the Parliamentary Elections that the new government has been more inclined to explore the introduction of arbitration. JILEP worked to introduce the Arbitration Code of Conduct in April 2013 to overcome some of the previous objections. JILEP has worked to establish a new arbitration association has been established. JILEP is on a working group to establish state institution under the Chamber of Commerce and it recognizes that there are differing approaches on whether to license arbitrators. JILEP should continue to support the introduction of arbitration as an effective and efficient method to resolve commercial disputes. MID-TERM PERFORMANCE EVALUATION OF JILEP 49 | Page ANNEXES MID-TERM PERFORMANCE EVALUATION OF JILEP 50 | Page ANNEX 1: STATEMENT OF WORK MID-TERM PERFORMANCE EVALUATION OF JILEP 51 | Page SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK STATEMENT OF WORK I. SCOPE Nonpersonal services for a mid-term evaluation of the Judicial Independence and Legal Empowerment Project (JILEP) program. The contractor must:  review the results achieved by the project to date,  determine the extent to which the project’s activities have contributed to these results,  consider how the implementation approach could be adjusted to improve the probability of attaining strengthened judicial independence and accountability, and  provide recommendations on changes to the project in view of the current political situation in Georgia and the progress achieved by JILEP over the past two years. The contractor must provide the following deliverables within the terms defined by the contract: - Detailed evaluation design to be submitted with the proposal. - In-brief and out-brief with preliminary findings with the USAID management and staff upon arrival to and departure from Georgia. - Final evaluation report in accordance with the Reporting Guidelines. Project Number: Cooperative Agreement No. AID-114-A-10-00008 Period of Performance: 09/24/10 – 09/24/2014 Project Funding: $19,314,000 Implementing organization: East-West Management Institute (EWMI) Agreement Officer’s Representative (AOR): Rusudan Tabatadze II. Purpose of the Evaluation and Its Intended Use The purpose of this mid-term performance evaluation is to determine the effectiveness of the project1 and to provide recommendations on corrective actions and new directions for the remaining years of project implementation and beyond. The contractor must review actual progress toward achieving key expected results and identify accomplishments, delays, challenges, and their impact on the project. The timeframe to be covered by the evaluation is from the start of the project in September 2010 through the initiation of this evaluation. The results of the evaluation will be used by USAID/Georgia for improving ongoing interventions in the area of rule of law by focusing on the activities that are most meaningful and critical for developing more independent and efficient judiciary in Georgia as well as strong civil society organizations what work in the rule of law arena. They will also contribute to the design of USAID’s next generation of support to the Rule of Law. The audience of the evaluation will be USAID and, in particular, its Democracy and Governance (DG) office. The results of the study may be shared with local stakeholders (HCOJ, HSOJ, and the Supreme Court of Georgia), 1 For detailed information on USAID previous and current interventions in Rule of Law area and JILEP please see attachment 1. MID-TERM PERFORMANCE EVALUATION OF JILEP 52 | Page other donors working in this area, and interested NGOs. Finally, evaluation results will also be used for reporting purposes to Washington-based stakeholders. III. Evaluation Questions The evaluation report must provide evidence-based answers. The contractor must answer the following questions in the evaluation: 1. Supply and Demand for an Independent Judiciary. JILEP was designed with components both for strengthening the judiciary (component I) and generating increased civil society advocacy for judicial independence (component II). The theory was that these components would be utilized proportionally to the political will for reform, i.e. when there has been will within the judiciary, JILEP has worked directly with the court administration to foster improvements. When political will has been scarce, JILEP has facilitated relevant reforms through civil society. How effective has it been to include both the supply and demand for an independent judiciary in the same award? What has been the nature and level of interaction and relationship between these components, including among project beneficiaries and from a project management perspective? Is this approach recommended in the future? 2. Results Despite a Lack of Political Will in the Judiciary for Reform. Recognizing that political will for judicial independence has been weak, which (presumably less politically sensitive) technical areas and/or activities has JILEP managed to work in and/or complete with the judiciary itself? How effective have those activities been in contributing to judicial independence? Considering the changed political environment, what new openings and opportunities exist for working directly with the judiciary to achieve program goals, both through the end of JILEP and beyond? 3. Building the Civil Society Coalition. Taking into consideration the Coalition’s accomplishments to date, what has been the JILEP and Eurasia Partnership Foundation (EPF) approach to building the Coalition? What have been the advantages and disadvantages of working through a Coalition for Independent and Transparent Judiciary? How effective has it been to work through EPF to provide coordination and organizational development assistance to the Coalition for Independent and Transparent Judiciary? What additional support is required in order to ensure the Coalition continues to pursue its stated goals? 4. Cost-Benefit of Support to Legal Education. Taking into consideration the cost and scale of JILEP support to legal education, how effective have the activities been or are they likely to be in contributing to improved legal education in Georgia (i.e. partnerships between Georgian and US University law schools)? Which Georgian universities have been the best partners and why? Moving forward, what approaches would best accomplish the overall objective of improving the quality of legal education in Georgia? 5. Effectiveness of Support to Commercial Law. Sustainable economic growth in Georgia depends on an effective legal framework and judicial environment related to commercial law. To what extent have JILEP activities contributed to improvements to commercial law related practice, including the transparency, efficiency, and effectiveness of commercial adjudication, alternative dispute resolution and enforcement of the legal framework? Moving forward, what approach can best contribute to improvements in the legal framework, judicial environment and commercial law practice for sustainable economic growth? MID-TERM PERFORMANCE EVALUATION OF JILEP 53 | Page IV. Evaluation Methods The finalized evaluation design must be submitted to the TOCOR three workdays prior to the team’s arrival in-country. The evaluation design must outline in detail what methods the contractor will use to get answers for each evaluation question. The evaluation design must include a detailed evaluation matrix (including the key questions, methods and data sources used to address each question and the data analysis plan for each question), draft questionnaires and other data collection instruments or their main features, known limitations to the evaluation design, a work plan, and a dissemination plan. This information together with the Mission’s comments will be discussed in detail during the in-brief meeting with USAID. The work plan must include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team. V. Projects Documents for Review and Logistics The JILEP AOR, through the Mission’s Democracy and Governance (DG) office will put the contractor in contact with its implementing partner and may provide help with a small number of meetings (such as meeting with USG agencies). Relevant reports and other project documentation will be provided by the Mission to the contractor prior to travel to Georgia. These documents are:  Project Description as is stated in the award;  Implementing partners Quarterly Reports;  Initial list of in-country contacts;  PMP indicator tables;  M&E plans submitted and approved by USAID;  AOR monitoring reports;  Other deliverables (expert report, publications) produced by partner. Prior to arriving to Georgia, the contractor may decide to interview USAID/E&E and USAID/ DCHA officials in the United States on programing in rule of law generally, and Georgia specifically, in addition to any other Washington-based experts as appropriate. The contractor might decide to meet with the management of East West Management Institute and JILEP back stop in the US. The Mission will not be involved in arranging these meetings. While in Georgia, the contractor will conduct most of the meetings in Tbilisi. Some meetings will require traveling to regions outside Tbilisi to Batumi and Kutaisi where JILEP counterpart universities and some sub-grantees are located. The Mission’s DG Team will brief the evaluation team on its perception of the developments in the rule of law area. END OF SECTION C ANNEX 2: MONITORING AND EVALUATION MATRIX JILEP Monitoring and Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis Evaluation Question 1: Supply and Demand for an Independent Judiciary 1.1 1.2 1.3 How effective has it been to include both supply and demand for an independent judiciary in the same award? What has been the nature and level of interaction and relationship between Components I and II, including among JILEP beneficiaries and from a project management perspective? Is a supply and demand approach in the same award recommended in the future?  USAID RFP for JILEP  USAID Cooperative Agreement with EWMI  JILEP’S Work Plans and M&E Plans  JILEP’s Reports and Documents  GoG legislation and relevant documents of the MOJ, SC and the Legal Issues Committee of the Parliament  HCOJ and HSOJ documents and reports  Relevant reports of JILEP’s partners and beneficiaries, such as The Coalition, EPF, GYLA, TI and GBA.  Independent research on the supply and demand approach to enhancing judicial independence  Independent Research on the political will for an independent judiciary in Georgia  Document Review  KIIs with JILEP’s partners and beneficiaries, as well as USAID personnel, EWMI managers, JILEP staff; governmental officials, and international rule of law donors in Georgia  FGD with members of the Steering Committee of the Coalition  Survey of all members of the Coalition  Site Visits: Tbilisi, Batumi and Kutaisi  On-site observations of the activities of JILEP The ET defined the term “effective” which was approved by USAID and is provided in the Work Plan.  Analysis of targeted results, outputs, and outcomes of JILEP  Verification of JILEP reporting  Non-JILEP documents and independent research  Opinions and recommendations elicited during the KIIs and FGD  Quantitative analysis of the results of the Survey  Critical synthesis and triangulation analysis of opinion data JILEP Monitoring and Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis Evaluation Question 2: Results Despite a Lack of Political Will in the Judiciary for Reform 2.1 2.2 2.3 Recognizing the political will for judicial independence has not been strong, which technical areas and/or activities has JILEP managed to work in and/or complete with the judiciary itself? How effective have those activities been in contributing to judicial independence? Considering the changed political environment, what new openings and opportunities exist for working directly with the judiciary to achieve program goals, both through the end of JILEP and beyond?  USAID’s RFP  USAID Cooperative Agreement with EWMI  JILEP’S Work Plans and M&E Plans  JILEP’s Reports and Documents  GoG legislation and relevant documents of the MOJ, SC and the Legal Issues Committee of the Parliament  HCOJ and HSOJ documents and reports  Relevant reports of partners and beneficiaries, such as the NGO Coalition, EPF, LAS, GYLA, TI and GBA.  Document Review  KIIs as stated above  FGDs with members of the Steering Committee of the Coalition  Survey of all members of the Coalition  Site Visits: Tbilisi, Batumi and Kutaisi The ET defined the term “effective” approved by USAID and is provided in the Work Plan.  Analysis of targeted results, outputs, and outcomes of JILEP  Verification of JILEP reporting  Opinions and recommendations elicited during the KIIs and FGDs  Quantitative Analysis of the results of the Survey  On-site observations of the activities of JILEP and the Coalition  Critical synthesis and triangulation analysis of opinion data Evaluation Question 3: Building the Civil Society Coalition 3.1 3.2 Taking into consideration the successes achieved through the Coalition, what has been the JILEP and EPF approach to building the NGO Coalition? What have been the advantages and  USAID RFP for JILEP  USAID’s cooperative agreement with EWMI  JILEP’S Work Plans and M&E Plans  JILEP’s Reports and  Document Review  KIIs, as stated above, particularly key representatives of the members of the NGO Coalition  Analysis of targeted results, outputs, and outcomes of JILEP  Verification of JILEP reporting  Non-JILEP Documents  Opinions and recommendations elicited during the KIIs and FGDs JILEP Monitoring and Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis 3.3 3.4 disadvantages of working through the NGO Coalition? How effective has it been for JILEP to work through EPF to provide coordination and organizational development assistance to the NGO Coalition? What additional support is required in order to ensure the NGO Coalition continues to pursue its stated goals? Documents  GoG legislation and relevant documents of the Legal Issues Committee of the Parliament  HCOJ documents and reports  Relevant reports of the NGO Coalition and its 30+ NGO members, EPF, GYLA, TI and GBA.  FGDs with members of the Steering Committee of the Coalition  Survey of all members of the Coalition  Site Visits: Tbilisi, Batumi and Kutaisi The ET defined the term “effective” approved by USAID and is provided in the Work Plan.  Analysis of the results of the Survey  On-site observations of the activities of JILEP and the Coalition  Critical synthesis and triangulation analysis of opinion data Evaluation Question 4: Cost-Benefit of Support to Legal Education 4.1 4.2 4.3 Taking into consideration the cost and scale of JILEP support to legal education, how effective have the activities been or are they likely to be in contributing to improved legal education in Georgia (i.e. partnerships between Georgian and U.S. law schools)? Which Georgian universities have been the best partners and why? Moving forward, what approaches would best accomplish the overall objective of improving the quality of legal education in Georgia?  USAID RFP for JILEP  USAID’s cooperative agreement with EWMI  JILEP’S Work Plans and M&E Plans  JILEP’s Reports and Documents  GoG laws and relevant documents of the MOJ, and the MOE  HSOJ documents and reports  Relevant reports of the JILEP partner Georgian and U.S. Law Schools  Document Review  KIIs, as stated above, particularly key representatives of JILEP partner Georgian and U.S. Law Schools, Washburn University School of Law and South Texas University School of Law  FGDs with students and law professors  Site Visits: Tbilisi, Batumi and Kutaisi The ET defined the term  Analysis of actual versus planned progress toward achieving expected results  Verification of JILEP reporting  Non-JILEP Documents  Opinions and recommendations elicited during the KIIs and FGDs  Quantitative Analysis  Analysis of the results of the FGDs with law students and law professors  On-site observations of the activities of JILEP and relevant law schools  Critical synthesis and JILEP Monitoring and Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis  Relevant reports of the GBA. “effective” approved by USAID and is provided in the Work Plan. triangulation analysis of opinion data Evaluation Question 5: Effectiveness of Support to Commercial Law 5.1 5.2 To what extent have JILEP activities contributed to improvements to commercial law related practice, including the transparency, efficiency, and effectiveness of commercial adjudication, ADR and enforcement of the legal framework? Moving forward, what approach can best contribute to improvements in the legal framework, judicial environment and commercial law practice for sustainable economic growth?  USAID RFP for JILEP  USAID’s cooperative agreement with EWMI  JILEP’S Work Plans and M&E Plans  JILEP’s Reports and Documents  GoG legislation and relevant documents of the MOJ, and the Supreme Court  HCOJ and HSOJ documents and reports  Relevant reports of recipients of JILEP’s commercial law training such as the GBA and law schools.  Document Review  KIIs, as stated above, particularly MOJ representatives, HCOJ leadership, HSOJ trainers, GBA Board members, GBA Training Center representatives, selected judges, practicing attorneys and law students  FGD with members of the GBA  Survey of members of the GBA  Site Visits: Tbilisi, Batumi and Kutaisi The ET defined the terms “transparency,” “efficiency” and “effectiveness” approved by USAID and is provided in the Work Plan.  Analysis of targeted results, outputs, and outcomes of JILEP  Verification of JILEP reporting  Non-JILEP Documents  Opinions and recommendations elicited during the KIIs  Analysis of the results of the FGD with GBA members  Analysis of the results of the Survey with GBA members  On-site observations of the activities of JILEP  Critical synthesis and triangulation analysis of opinion data ANNEX 3: LIST OF MEETINGS 1. USAID 2. JILEP staff 3. Former members of the High Council Of Justice 25 4. Supreme Court of Georgia 5. Members of the Parliament 6. Current Judge-Members of the HCOJ26 7. (Effective starting from June 22) 8. Judicial Associations 9. High School of Justice 10. Current non-judge members of the HCOJ 11. (Effective starting from June 22) 12. Ministry of Justice of Georgia 13. Selected practicing Lawyers benefiting from JILEP Component 4 activities 14. Trainers of the High School of Justice (HSOJ) 15. JILEP Partner Law Schools 16. Eurasia Partnership Foundation (EPF) 17. JILEP Direct Partner NGOs 18. Georgian Young Lawyers’ Association (GYLA)27 19. Transparency International Georgia (TI)28* 20. JILEP Experts 21. National Center for the Development of Quality in Education (National Education Accreditation Center) 22. Georgian Foundation for the Support of Legal Education 23. Georgia Bar Association 25 Bios of the members can be found here http://hcoj.gov.ge/en/about/organizational-charter/sabchos￾shemadgenloba 26 Please see details of the Council member elections and on political context following the links below: http://civil.ge/eng/article.php?id=26162 http://civil.ge/eng/article.php?id=26180 27 GYLA website: http://gyla.ge/eng/news 28 TI website: http://transparency.ge/en *TI and GYLA are the Coalition Steering Committee members 24. Commercial Law Advisory Council 25. Universities 26. Association of Georgian Arbitrators 27. USAID project: 28. Economic Prosperity Initiative 29. US Embassy Department of Justice and INL Office 30. Rule of Law Donors 31. Prime Minister’s Office 32. Coalition for Independent and Transparent Judiciary29 29 Coalition website: http://www.coalition.org.ge/en/ ANNEX 4: LIST OF MATERIALS CONSULTED I. USAID documents Outline of the HCOJ legislative changes, November 2011 Georgia Country Context for Evaluation team, as of April 2013 Judicial Administration and management Reform Project Final Report, November 2011 Judicial Administration and management Reform Project Final Evaluation Report, January 2012 II. JILEP Documents Assessment of ADR in Georgia, Oct. 2011 CRRC Study Qualitative and Quantitative Research methodology Documents CRRC Study: Attitudes to the Judiciary in Georgia: Assessment of Legal Professionals and Business Leaders, June 2012 Victoria Henley Report on Judicial Discipline, May 2012 Gender Balance in Judiciary as of July 2012 JILEP Comments on Reform Strategy of Judiciary, February 2012 Judge Stuart’s Trip Report on Communications Training for Judges, November 2011 Consultant report on Judicial Evaluation, June 2012 Arbitration Moot Competition, Feb 2012 West Georgia Scholarship Amendment, August 2012 Consultant Report, Professor Delaine R Swenson, February 2012 Memo on changes to the law on Court Annexed Mediation, Nov 2011 Description of mediation Pilot, Oct 2012 CRRC Baseline Survey, January 2012 JILEP Comments on Judiciary Reform Strategy, Feb 2013 Memorandum of Understanding between JILEP and GIZ on Civil Code Commentary, March 2013 Civil Code Commentary Concept, March 2013 JILEP Arbitration Reform Plan, May 2013 Report of NCADR Director on visit to South Texas College of Law, Feb 2013 Comments on the draft Code of Administrative Violations Provided by The Judicial Independence and Legal Empowerment Project (JILEP) Description of Mediation Pilot Selection and appointment of judges in Georgia: Explanation of Current Process and Suggestions for Improvement, JILEP, May 2011 III. Documents prepared by JILEP Partners/Grantees A. Court Monitoring GYLA Court Monitoring Methodology TI Court Monitoring Methodology TI Administrative Cases Court Monitoring Report; May 2012 GYLA Criminal Cases Court Monitoring Report; April 2012 TI Administrative Cases Court Monitoring Report; March 2013 GYLA Criminal Cases Court Monitoring Report; March 2013 GYLA/TI Court Monitoring Reports: Key Findings, Feb 2013 B. Coalition for Independent and transparent Judiciary Coalition Report on Preventive Measures, Dec 2012 Coalition Statement on Legislative amendments on Jury Trial, Jan 2013 Coalition Report on Preventive Measures, Executive Summary GYLA/Coalition Report on Judiciary, June 2012 Coalition recommendations on Judicial Transparency, 2012 Coalition Recommendations on Judiciary Reform Strategy, Feb 2013 Statement of Coalition member NGOs on the Conference of Judges on June 9, 2013 Coalition Comments on the Code of Administrative Violations, March 2012 C. Other Documents produced by JILEP partner NGOs HCOJ Monitoring Report by GYLA and TI, Jan 2012 Center Point Group-Georgia’s Biggest Construction Scandal, Dec 2012 “Justice Minister Unhappy Over HCoJ Election Results,” Civil Georgia, Civil Georgia, June 10, 2013. Freedom House Report, Georgia, 2006. Venice Commission Opinion on the law on disciplinary responsibility and disciplinary prosecution of judges of common courts of Georgia adopted by the Venice Commission at its 70th Plenary Session (Venice, March 2007). ANNEX 5: INTERVIEWEE QUESTIONNAIRES ILLUSTRATIVE KII QUESTIONS for the HCOJ and HSOJ: Did HCOJ’s political will to enhance judicial performance, judicial advancement and judicial disciplinary measures alter throughout JILEP’s first two years? If so, what were the causes for the change in political will? What were the successes that the HCOJ achieved in enhancing judicial performance, judicial advancement and judicial disciplinary measures? What were the challenges that the HCOJ faced in enhancing judicial performance, judicial advancement and judicial disciplinary measures? Did HCOJ’s political will for improving transparency and public outreach, and increasing access to court information alter throughout JILEP’s first two years? If so, what were the causes for the change in political will? What were the successes that the HCOJ achieved for improving transparency and public outreach, and increasing access to court information? What were the challenges that the HCOJ faced for improving transparency and public outreach, and increasing access to court information? Did HSOJ’s political will for JILEP’s assistance to strengthen its training capacity and sustainability alter throughout JILEP’s first two years? If so, what were the causes for the change in political will? What were the successes that the HSOJ achieved in strengthening its training capacity and sustainability? What were the challenges that the HSOJ faced in strengthening its training capacity and sustainability? Was there a noticeable difference between the political will of the HCOJ and the HSOJ, given the leadership of each judicial institution? II. ILLUSTRATIVE FOCUS GROUP QUESTIONS for members of the Coalition for Independent and Transparent Judiciary 1) Why did your NGO join the Coalition? Please check the relevant reasons.  Helps us to reach the goals of its mission  Helps us to advocate for national policies on reforming the judiciary  Helps us to become more visible on a national level  Increases our contact with the leadership of the judiciary  Increases the level of awareness of our NGO among other like-minded NGOs  Helps us to expand our knowledge of the issues that impact judicial reform  Increases our chances of obtaining a grant from the Eurasia Partnership Foundation 2) Have you received a grant from the Eurasia Partnership Foundation?  Yes  No 3) If not, why not?  Have not applied  Applied, but was denied 4) How do you view the process for obtaining and implementing grants? Please check the relevant reasons.  There process is fine  The process is too strict  The process is too time-consuming  The process is too bureaucratic 5) How active is your NGO in attending meetings of the Coalition such as the Steering Committee meetings, Coalition meetings, Working Group meetings and Forums?  Very Active, attending at least one meeting every month  Active, attending at least six meetings annually  Not very active, attending less than six meetings annually 6) What are the challenges of being a Coalition member? Please check the relevant reasons.  Only a few members of the Coalition are active and do most of the work  The decision-making process of the Coalition is too cumbersome and takes too long  The decision-making process of the Coalition does not give sufficient time to non- members of the Steering Committee to participate in a meaningful manner  Less active NGOs in the Coalition are consuming too much time of the active members  The focus of the Coalition is only on national issues, and needs to take into consideration regional issues.  The activities of the Coalition are too time-consuming  Members have different level of institutional or organizational capabilities that impedes decision-making 7) Does membership in the Coalition meet the expectations of your NGO for joining the Coalition?  Yes  No  Somewhat 8) Does your participation in the Coalition help to increase your NGO’s capacity to fulfill its mission?  Yes  No  Somewhat 9) How effective has the Eurasia Partnership Foundation been in managing and organizing the activities of the Coalition?  Very effective  Somewhat effective  Not effective 10) Has the Eurasia Partnership Foundation been sufficiently active in encouraging the Coalition to pursue judicial reforms and enhance judicial independence?  Very active  Somewhat active  Not active 11) Could another Georgian NGO accomplish the role of the secretariat?  Yes  No 12) How effective are the Working Groups? Court Administration  Very effective  Somewhat effective  Not effective Criminal Law  Very effective  Somewhat effective  Not effective Legal Aid  Very effective  Somewhat effective  Not effective Legal Education  Very effective  Somewhat effective  Not effective Commercial Law  Very effective  Somewhat effective  Not effective 13) Is the effectiveness of the Working Groups determined by its membership or by the policy issues it covers? Please check the relevant reasons  Membership  Policy Issues 14) What are the challenges faced by the Working Groups? Please check the relevant reasons  Lack of funding  Lack of expertise  Lack of commitment on the part of Working Group members  Passivity of the Working Group members  Capacity of the Working Group members  List other reasons for some Working Groups to be active and others not active 15) Are you aware of available funding by the Eurasia Partnership Foundation for Working Groups initiatives?  Yes  No 16) If yes, has the Working Group in which you are a member utilized these funds?  Yes  No 17) If no, why not?  State reasons 18) Does the Coalition fulfill its mission by successfully promote government accountability for an independent and transparent judiciary?  Yes  No  Somewhat 19) Does the Coalition fulfill its mission by successfully unite civil society efforts in judicial system reform monitoring?  Yes  No  Somewhat 20) Does the Coalition fulfill its mission by successfully facilitate legislative and regulatory advocacy for an independent and transparent judiciary?  Yes  No  Somewhat 21) Does the Coalition fulfill its mission by successfully promote collaboration with major stakeholders in the justice system, such as the High Council of Justice, the Supreme Court, the High School of Justice and the Ministry of Justice?  Yes  No  Somewhat 22) Does the Coalition provide timely updates on the important developments within the Coalition, on important decisions of the Steering Committee and Working Groups, and on activities of the Coalition?  Yes  No  Somewhat 23) Are you well informed about the activities of JILEP?  Yes  No  Somewhat 24) What would you suggest to improve internal communication between the Coalition members?  List suggestions 25) Do you think the Coalition has chosen the right strategic activities to impact public policy on judicial independence and transparency?  Yes  No  Somewhat 26) Which of the following activities are the most effective for enhancing judicial independence and transparency? Please prioritize  Conducting Forums  Issuing Public Statements  Monitoring Judicial Institution activities  Having Meetings with Public Officials  Issuing Public Service Announcements  Participating in Talk Shows  Developing policy briefs or policy papers  Advocating regulatory amendments 27) What activities would make the Coalition become a stronger and more visible force in impacting judicial independence and transparency?  List suggested activities 28) Do you believe you had sufficient opportunities to actively engage in the activities of the Coalition?  Yes  No 29) Do you think the Coalition should develop a Strategic Plan for its future activities?  Yes  No 30) Which public policy issues affecting an independent and transparent judiciary should the Coalition pursue?  List suggested public policy issues 31) What additional support does the Coalition need to continue its stated goals?  List suggested support 32) Does the Coalition need a foreign sponsor to obtain respect among governmental officials?  Yes  No 33) Will you continue being a member of the Coalition after the JILEP program has ended?  Yes  No 34) What percentage of impact did the Coalition’s demand for judicial independence have on the revisions to laws affecting the judiciary?  None at all  Less than 25%  Between 26% - 50%  More than 50% 35) What is the type of your organization?  NGO supporting legal aid  NGO supporting legal advocacy  Business Association  Media Association 36) What percentage of the staff members of your NGO are women?  Less than 10%  Between 11%-25%  Between 26%-50%  Between 51% - 75%  Over 76% END OF SURVEY III. ILLUSTRATIVE FOCUS GROUP QUESTIONS for members of the GBA 1. How difficult is it to become a lawyer in Georgia? What do you think of the role and capacity of the Bar Qualification Commission? What would you like it to do differently? 2. Are familiar with GBA’s activities and programs? From what sources? What do you think are the major benefits of your membership in GBA? Do you think that GBA is responsive to (meet the needs of) its members? What are GBA’s strengths? What would you like GBA to do differently? 3. Lawyers just like everyone else sometimes get into trouble, and sometimes in their personal lives and sometimes in their professional lives. How prevalent do you think that ethical violations are among lawyers? Do you think that you have adequate information to guide your professional conduct? How do you think that the system of providing information on professional ethical conduct can be improved? How do you think that the system for disciplining attorneys can be improved? 4. What do you think about mandatory continuing legal education for practicing lawyers? Is the current training program deficient in any way? What do you like about the program? Are the offerings adequate for your needs? Are there sufficient offerings in areas like commercial law, administrative law, tax law, alternative dispute resolution? 5. In your practice, what areas do you think that you would most benefit in training offerings? If your practice includes tax and sales of goods, can you comment on the training offerings that are available through GBA or any other organization? 6. What additional support in commercial law do you need to serve your clients? Are there sufficient resource materials for lawyers? What areas would you like to have more resource materials like commentaries, handbooks or other materials? 7. For the training programs that you have attended, did you learn anything useful? What would like to see different? What did you learn about that you did not know before? What were you least comfortable about before the training? How has that changed since the training? 8. Have you used any form of alternative dispute resolution in your practice? Was it useful? Would you be inclined to use it again? [If the members of the focus group have used both mediation and arbitration, the facilitator should divide up the discussion and first discussion arbitration and then mediation.] What additional training or other support do you need regarding alternative dispute resolution? Mini-survey questions 1. On a five point scale, how would you assess how present GBA is in traditional media? One means not visible at all, and five means highly visible. 2. On a five point scale, how would you assess how present GBA is in new media? One means not visible at all, and five means highly visible. 3. On a five point scale, to what extent do you agree with the statement that the current content of continuous legal education for attorneys through GBA offers detailed, high quality yet diverse courses which facilitate and improve the performance of judges. One means completely disagree, and five means completely agree. 4. Does GBA need improvement? a. Needs great improvement b. Needs some improvement c. Needs no improvement 5. On a five point scale, to what extent is there the necessity of reform of the GBA in terms of the ultimate goal of creating a competent, ethical and professional bar in Georgia? One means not necessary at all, and five means very necessary. 6. On a five point scale, to what extent is there the accessibility of opportunities for continuing lifelong legal education. One means not accessible at all, and five means easily accessible. 7. Does continuing legal education for lawyers need improvement? a. Legal education needs great improvement. b. Legal education needs some improvement c. No opinion 8. Does legal education at law schools (law faculties) need improvement? a. Legal education needs great improvement. b. Legal education needs some improvement c. No opinion 9. On a five point scale, to what extent is there the necessity of reform of the legal education at law schools (law faculties) in terms of the ultimate goal of creating a competent, ethical and professional bar in Georgia? One means not necessary at all, and five means very necessary. 10. On a five point scale, to what extent are you familiar with mediation as an element of ADR? One means not familiar at all, and five means completely familiar. 11. On a five point scale, to what extent are you familiar with arbitration as an element of ADR? One means not familiar at all, and five means completely familiar. 12. On a five point scale, to what extent is there the presence of information about ADR in the traditional media? One means not visible at all, and five means highly visible. 13. On a five point scale, to what extent is there the presence of information about ADR in the new media? One means not visible at all, and five means highly visible. 14. On a five point scale, to what extent are you satisfied with the level of expertise of Georgian lawyers in commercial law? One means not satisfied at all, and five means very satisfied. 15. On a five point scale, to what extent is there the accessibility of commercial law training to legal professionals. One means not accessible at all and five means easily accessible. 16. On a five point scale, to what extent are you confident in the impartiality of court case outcomes in commercial disputes? One means not confident at all, and five means very confident. 17. To what extent is there a demand for mediation as a form of ADR? a. There is a great demand for mediation as a form of ADR b. There is some demand for mediation as a form of ADR c. There is little demand for mediation as a form of ADR d. Don’t know. 18. To what extent is there a demand for arbitration as a form of ADR? a. There is a great demand for arbitration as a form of ADR b. There is some demand for arbitration as a form of ADR c. There is little demand for arbitration as a form of ADR d. Don’t know. 19. On a five point scale, what is the necessity of introduction of mediation mechanisms for the resolution of commercial disputes? One means not necessary at all and five means very necessary. 20. On a five point scale, what is the necessity of improving arbitration services for commercial disputes? One means not necessary at all and five means very necessary. 21. On a five point scale, to what extent are you confident in the current level of legal expertise of arbitrators to resolve commercial disputes? One means not confident at all and five means very confident. 22. How cost effective is arbitration in commercial disputes as compared to the cost effectiveness of litigation? a. Definitely provides more cost-effective dispute resolution b. May or may not provide more cost effective dispute resolution c. Definitely does not provide more cost-effective dispute resolution d. Don’t know 23. How cost effective is mediation in commercial disputes as compared to the cost effectiveness of litigation? a. Definitely provides more cost-effective dispute resolution b. May or may not provide more cost effective dispute resolution c. Definitely does not provide more cost-effective dispute resolution d. Don’t know 24. What will be the result of the introduction of mediation mechanisms in Georgia on efficiency of resolving commercial disputes? a. Will definitely contribute to efficiency of reaching settlements b. May or may not contribute to efficiency of reaching settlements c. Nothing will change even if mediation is introduced d. Don’t know 25. What will be the result of the option of referral to arbitration in terms of the fairness of rulings in commercial disputes? a. Will definitely contribute to fairer rulings b. May or may not contribute to fairer rulings c. Nothing will change even if alternative arbitration is introduced