December 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc., under USAID Contract No. AID￾OAA-I-10-00003 and Task Order No. 169-TO-13-00005 MID-TERM EVALUATION of the SERBIA JUDICIAL REFORM AND GOVERNMENT ACCOUNTABILITY (JRGA) PROGRAM Final Evaluation Report MID-TERM EVALUATION of the JUDICIAL REFORM AND GOVERNMENT ACCOUNTABILITY (JRGA) PROGRAM Final Evaluation Report DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. The cover photograph depicts a USAID/JRGA sponsored Round Table Discussion among Independent Agencies and National Assembly, September 18, 2013. TABLE OF CONTENTS Table of Contents.......................................................................................................................................................... i Acronyms........................................................................................................................................................................ ii Executive Summary...................................................................................................................................................... iv Introduction ........................................................................................................................................................................ 1 Evaluation Purpose & Evaluation Questions........................................................................................................... 1 Project Background ...................................................................................................................................................... 1 Evaluation Methods & Limitations............................................................................................................................. 3 Findings, Conclusions & Recommendations................................................................................................................ 5 Annex I: Evaluation Statement of Work .................................................................................................................... 27 Annex II: Evaluation Methods and Limitations.......................................................................................................... 35 Annex III: Data Collection Instruments..................................................................................................................... 37 A. Examples of Semi-Structured Interview Questions....................................................................................... 37 B. Association of Misdemeanor Judges Survey..................................................................................................... 45 C. Questionnaire for Grantees, including Results............................................................................................... 54 D. PMP Analysis, Year 1 Summary.......................................................................................................................... 56 Annex IV: Sources of Information ............................................................................................................................... 57 A. Bibliography............................................................................................................................................................. 57 B. Evaluation Interviewees........................................................................................................................................ 61 C. Sites Visited............................................................................................................................................................. 66 Annex V: Evaluation Logic Flow................................................................................................................................... 67 i ACRONYMS AC Administrative Court ACA Anti-Corruption Agency AMJ Association of Misdemeanor Judges APJ Acting Presiding Judge APPO Association of Public Procurement Officers APPP Association of Public Prosecutors and Deputy Public Prosecutors of Serbia BEG MC Belgrade Misdemeanor Court BEP Business Enabling Project BIRODI Bureau for Social Research CAFV Counseling Against Family Violence CDS Center for Development of Serbia CIP Court Improvement Plan CIPIPDP Commissioner for Information of Public Importance and Personal Data Protection CPE Commissioner for Protection of Equality COR Contracting Officer’s Representative CRTA Centre for Research, Transparency and Accountability CSO Civil Society Organization DG Democracy and Governance DPI Development Professionals Inc. EPC European Project Center EU European Union GOS Government of Serbia HCC High Court Council HMC High Misdemeanor Court JA Judicial Academy JAS Judges Association of Serbia JRGA Judicial Reform and Government Accountability IA Independent Agencies IQC Indefinite Quantity Contract IRI International Republican Institute ISC Institute for Sustainable Communities IT Information Technology JA Judicial Academy JAS Judges Association of Serbia JRGA Judicial Reform and Government Accountability LoM Law on Misdemeanors MC Misdemeanor Court MCCMS Misdemeanor Court Case Management System M&E Monitoring and Evaluation MOJ Ministry of Justice MOJPA Ministry of Justice and Public Administration MSI Management Systems International NA National Assembly NCSC National Center for State Courts NDI National Democratic Institute NPC New Policy Center OMB Ombudsman ii OSCE Organization for Security and Cooperation in Europe PDC Partners for Democratic Change PJ Presiding Judge PMP Performance Management Plan POC Point of Contact PPO Public Procurement Office SAI State Audit Institution SCC Supreme Court of Cassation SI Social Impact, Inc. SPP Separation of Powers Project SSI Semi-Structured Interview SOW Statement of Work TS Transparency Serbia UNDP United Nations Development Programme USAID U.S. Agency for International Development WB World Bank YUCOM Lawyers’ Committee for Human Rights iii EXECUTIVE SUMMARY Evaluation Purpose The purpose of this mid-term performance evaluation is to conduct a full, evidence-based and independent review of the USAID/Serbia-funded Judicial Reform and Government Accountability (JRGA) project implemented by the National Center for State Courts (NCSC), and to produce a report that provides a qualitative and quantitative evaluation of project activities. The primary audience for this evaluation report includes the following: USAID/Serbia, the US Embassy Belgrade, USAID Washington D.C., NCSC, Management Systems International (MSI), JRGA’s Chief of Party and Judicial Reform and Government Accountability Component Leaders, and the broader democracy promotion community. Project Background JRGA is a five-year program consisting of two components: Component 1--Judicial Reform and Component 2--Government Accountability. The JRGA project was designed to improve Serbia’s rule of law, judicial efficiency, independence and integrity and the openness and accountability of government operations overall, both for the sake of the people of Serbia, and to help Serbia move closer to European Union (EU) accession and participation in other Euro-Atlantic institutions. The JRGA contractor’s charge was to ensure that progress in these areas is promoted to the general public. The principal goals of the Judicial Reform component are to build the capacity and performance of the new Misdemeanor Court (MC) system (previously part of the Executive Branch) and provide training and technical assistance to the new Administrative Court (AC) to develop public outreach capacity and automate the court’s decisions. The principal goal of the Government Accountability component is to strengthen the ability of Serbia’s independent agencies (IAs) and civil society to demand and ensure transparency and accountability from Government of Serbia (GoS) institutions. Serbia has five IAs: Anti-Corruption Agency (ACA), State Audit Institution (SAI), Ombudsman’s Office, Commissioner for Information of Public Importance and Personal Data Protection (CIPIPDP), and Commissioner for Protection of Equality (CPE). Evaluation Design, Methods and Limitations The evaluation utilized key informant interviews, mini-surveys/questionnaires and document review. The Evaluation Team spent three weeks collecting data in Serbia conducting interviews with all direct beneficiaries, indirect beneficiaries, selected grantees, and external interlocutors. To mitigate bias, as many interviewees were identified by JRGA, the team conducted in-depth interviews and used multiple data sources to triangulate on each evaluation issue. Summary of Findings, Conclusions and Recommendations EVALUATION QUESTION 1: To what extent is the program on-track (meeting estimated milestones) to accomplish the objectives and results expected under both Components? Q. 1 Judicial Reform Q. 1 Judicial Reform Findings: Under the Judicial Reform Component, JRGA is on track to accomplish each of the nine Tasks specified in the JRGA Project Scope of Work (SOW): 1. Make open hearings routine, dignified, accessible, safe, efficient, and fair; 2. Reduce backlogs and improve case processing times and procedures 3. Increase the percentage of assessed court costs and fines collected; iv 4. Increase the physical safety of victims of family and domestic cases 5. Improve the coordination of cases involving related issues and/or parties 6. Increase the competency of judges in all the MCs and High Misdemeanor Court) HMC; 7. Develop and disseminate practical and user-friendly information for court users on court operations; 8. Improve the reputation of MCs and HMC among court users and the public; and 9. Increase non-governmental support for judicial reform through grants. Evidence of achievement of these Tasks is reflected by documented progress to date in producing the following deliverables: • Drafting and implementing Amendments to the Law on Misdemeanors (LoM); • Assessing and prioritizing renovation of all MC facilities; • Compiling case law establishing the basis for standardization of judicial decisions in the AC and HMC; and • Development of the Misdemeanor Court Case Management System (MCCMS). Q. 1 Judicial Reform Conclusions: While JRGA is on track in achieving its principal objectives, progress in two areas lagged behind targets: • Development of case processing, backlog reduction and judicial performance standards; and • Renovation of highest priority facilities. Delays experienced in developing standards reflect, in part, the unwillingness of the Ministry of Justice and Public Administration (MOJPA) to transfer the responsibility to the High Court Council (HCC) and of HCC to accept responsibility for management of all judicial branch functions. Q. 1 Judicial Reform Recommendations: • Within the context of JRGA’s overall judicial reform objective to “implement activities and tasks that improve the efficiency and independence of Serbian courts” 1 USAID/Serbia and its partners/implementers should facilitate HCC’s assumption of ownership over the judicial system by encouraging HCC to lead implementation of the National Judicial Strategy and Action Plan. • The HCC with JRGA’s assistance should encourage the Ministry of Justice and Public Administration (MOJPA) and EU to fund the renovation and construction of highest priority MC facility relocation and renovation projects. • JRGA should facilitate the development and implementation of an HCC research and development organization/function; and • JRGA should disseminate the results of Civil Society Organization (CSO) grants as well as assist judicial branch agencies in implementing grantee recommendations. Q. 1 Government Accountability Q. 1 Government Accountability Findings: Under the Government Accountability component, JRGA is required to complete six tasks and achieve the following expected results: (1) The ACA effectively executes its responsibilities relating to conflicts of interest prevention, detection and sanctioning; (2) The ACA, its Board, and any successor organizations effectively regulate the financing of political parties, and election campaigns; 1 Evaluation Statement of Work, see below Annex I. v (3) The recommendations and decisions made by IAs are implemented and/or enforced by other government agencies; (4) Administrative agencies and relevant courts process and enforce laws and rules efficiently; (5) Individuals and organizations use information from or processes supervised by the IAs to monitor and improve political party financing, compliance with conflict of interest provisions, administrative practices and/or other government operations; and (6) Individuals and organizations work to improve governance and official integrity, reduce corruption, monitor political party financing (including the financing of local and national elections), and to promote compliance with EU norms. JRGA has utilized several short-term experts to conduct comprehensive needs assessments and prepare detailed recommendations for improvements in IA policies and procedures, conduct training for IA staff, and advise the Government of Serbia (GoS) and IAs on new legislation and anti-corruption policy. As a result: • The ACA implemented JRGA STTA’ recommendations to improve systems for conflict of interest and income and asset management and is starting to implement recommendations to improve its complaints system; • The ACA established a network of campaign monitors and completed its reporting obligations under the Law on Financing Political Activities in connection with the 2012 elections in Serbia; • IAs have increased capacity to prepare better Annual Reports and formulate actionable recommendations; • Ombudsman staff are better able to process and respond to complaints; • The Commissioner for Information of Public Importance and Personal Data Protection CIPIPDP has increased capacity to perform its duties under the Data Protection Law; • The CIPIPDP, ACA and MoJPA have produced drafts of a whistleblower law; • The CPE has a new website and outreach materials focused on vulnerable populations; and • The GoS prepared a new Anti-Corruption Strategy and Action Plan. JRGA has organized roundtables, workshops and joint trainings to improve cooperation within IAs and with Courts, government agencies and other bodies in order to increase implementation/enforcement of IA decisions and recommendations. These activities have been very successful and are highly praised by counterparts and external parties. Through its grants program, JRGA is supporting local CSOs that implement projects that complement JRGA’s activities, such as tracking implementation of IA recommendations, and raising awareness and engaging the public in connection with good governance and anti-corruption efforts. As noted above, two of JRGA’s six tasks involve targeted assistance to the ACA. Cooperation with the ACA was difficult during the first year of the Project due in part to poor management and unclear legal provisions. The Agency was also understaffed. As a result, the Agency was unable to absorb all of the provided assistance. However, the Director has been replaced, there have been changes on the Board and the Agency is now fully staffed. JRGA is now working closely with the ACA and recently organized a retreat for Senior Management and the Board. vi Q. 1 Government Accountability Conclusions: JRGA appears to be on track to accomplish Component 2 objectives. Although progress with the ACA was problematic during the first year of the Project, it is now on track. Q. 1 Government Accountability Recommendations: • JRGA should continue to build its relationship with the revitalized ACA and follow-up on issues raised at the management retreat and by ACA staff, including reorganization of internal procedures and advanced management training for key staff. JRGA should also pace activities to ensure that the ACA can absorb the assistance. • JRGA should continue its facilitation of inter-agency cooperation and assist IAs in publicizing improvements in enforcement and implementation of IA decisions and recommendations in order to improve public trust in the IAs. • Additional recommendations regarding accomplishment of Component 2 objectives are included in Evaluation Questions 2 and 5. EVALUATION QUESTION 2: What emerging impact has the project had on strengthening Misdemeanor Courts and Independent Agencies in Serbia? Judicial Reform Q. 2 Judicial Reform Findings: JRGA’s principal contributions to strengthening the MCs have been drafting and facilitating the implementation of the LoM; development of the MCCMS; strengthening the competencies of Misdemeanor Judges; and assessment and improvement of MC physical facilities. Q. 2 Judicial Reform Conclusions: Comprehensive support provided by JRGA at both the national/policy and local/operational levels to MCs and AC has significantly advanced prospects for sustainability of modern judicial system management in these courts. Q. 2 Judicial Reform Recommendation: JRGA should assist HCC and the Judicial Academy (JA) in developing a national court system research and development organization to assist in the development and implementation of court performance standards and ensure that the results of judicial system reforms initiated and implemented with JRGA assistance will be sustainable. Q 2 Government Accountability Q. 2 Government Accountability Findings: JRGA has provided significant assistance in three overarching aspects of government accountability: development of the new Anti-Corruption Strategy and Action Plan; facilitating communication and cooperation between IAs and courts, government agencies and Parliament and among IAs; and increasing the visibility of IAs. Q. 2 Government Accountability Conclusion: JRGA assistance was critical to developing the detailed Action Plan that increases the prospects for timely and transparent implementation. The roundtables and other activities that foster communication and cooperation between the IAs and MCs and between the IAs and National Assembly (NA) provide the foundation for ongoing cooperation and are also improving the credibility of the IAs. The increased visibility of IAs is raising the awareness of Serbian citizens of the role of the IAs in fighting corruption and promoting good governance. However, vii counterparts continue to rely on JRGA support and there is some risk of over dependence on JRGA. Accordingly, the sustainability of these initiatives is not yet certain. Q. 2 Government Accountability Recommendations: • JRGA should support the IAs in connection with implementation of the Anti-Corruption Action Plan. JRGA should also avoid actions that could foster the perception that JRGA is only aligned with the MoJPA and to the extent possible, highlight the work of the IAs. • JRGA should try to shift some responsibility for organization of roundtables to IAs and the NA, as appropriate. • JRGA should monitor the IAs public outreach and media activities to ensure that messages are on point and that the quality of outputs remains high. EVALUATION QUESTION 3: To what extent has the project design been appropriate in achieving intended objectives and results? Findings: The JRGA project design merged the Mission’s primary Democracy and Governance (DG) objectives and recognized the need for technical assistance and training to help establish new judicial branch and independent agencies whose creation required both development of capacity and establishment of protocols for inter-institutional coordination. In addition, the flexibility built into the project allowed JRGA to respond easily to structural changes in government and to address targets of opportunity within the scope of the Project’s overall goals. While several donors provide assistance to the IAs, JRGA is the only program that works on IA-MC cooperation. Conclusion: The project design allowed for needs assessment and consideration of previous or ongoing donor assistance. For example, after the 2012 national elections and change in government in Serbia, ACA leveraged its relationship with MoJPA established under Component 1 to assist that institution, which now has primary responsibility for implementation and monitoring of the Anti￾Corruption Strategy and Action Plan. Further, combining the two JRGA components into one program allowed for important synergies as the new MCs would have jurisdiction over adjudicating violations of some of the same laws and regulations that the IAs are responsible for detecting and preventing. As both the MCs and IAs were JRGA counterparts, JRGA could bring these two groups together. Recommendations: The principal project design recommendations during the second half of the JRGA project are to expand the scope of IA-MC cooperation and focus on transition and sustainability. EVALUATION QUESTION 4: Within the current contract scope, what opportunities can be identified to expand support for Serbian judicial reform efforts? Findings: JRGA has already identified and begun to support targets of opportunity within the project scope, including outreach to petitioners to facilitate implementation of the LoM amendments; capacity development of the HCC and the JA; and assessment of court facility renovation requirements. Conclusions: Interviews and documentary sources served to inform the Team’s professional judgment about additional opportunities for expanded support for Serbian judicial reform efforts, including development of a research and development function at the HCC to assure long-term sustainability of judicial system reform efforts, focusing initially on the development of judicial performance standards; viii expansion of outreach to petitioners to implement the LoM amendments; expanded implementation of facility assessment recommendations, particularly highest priority MCs in buildings shared with Basic Courts; and, in addition to continuing grants to CSOs to improve the administration of justice, increased efforts to promote improvements to the judiciary among the public and to conduct surveys regarding the public’s confidence in the courts. Recommendations: Possible opportunities for expansion of support for judicial reform include: • Expanding assistance to the MOJPA and the MCs to renovate those facilities identified as highest priority projects by the JRGA facilities inventory, particularly those in buildings where MCs share space with the Basic Courts. • By tracking revenues attributable to MCs, help MOJPA and/or other donors to justify increased allocation of funds for construction and renovation of all MCs; • Increased outreach to petitioners to facilitate implementation of the LoM Amendments; • Offer assistance to the chief justice and presiding judges (PJs) to develop and deliver annual state of the judiciary addresses to highlight court system improvements, especially as defined in the National Judicial Strategy and Action Plan; and • Initiate discussions among counterparts regarding the feasibility of developing a research and development function at the HCC to assure long-term sustainability of judicial system reform efforts, focusing initially on the development of judicial performance and case management standards, and ultimately on the development of the kinds of mechanisms and institutions deemed critical to the establishment and maintenance of judicial competence in other countries, such as the establishment of a transparent judicial discipline mechanism in which incompetent or corrupt judges could be publicly disciplined or removed. EVALUATION QUESTION 5: To what extent are Component 2 Government Accountability activities sufficiently focused and strategically targeted toward the broader program goal? Findings: JRGA has followed the tasks as outlined in the RFP and conducted activities as set forth in the approved Annual Workplans. JRGA experts have focused on critical needs and provided concrete recommendations for action. Significant assistance focused on preparing the new Anti-Corruption Strategy and Action Plan that will guide Serbia’s effort to achieve an open, transparent and accountable government over the next five years. Conclusions: Assistance to the IAs has focused on building institutional capacity and visibility and improving cooperation and coordination as originally planned. JRGA has leveraged its relationship with the MCs to increase cooperation between the IAs and MCs to improve the courts’ actions in misdemeanor cases initiated by IAs. Further, as the ability of IAs to fulfill their duties and promote increased government transparency and accountability depends on implementation of recommendations for legislative changes, JRGA has included the NA in program activities. To a limited extent, JRGA has provided assistance on topics not foreseen in the RFP, such as public procurement, although such activities are consistent with the broader program objectives. JRGA’s support to the ACA and MoJPA in preparation of the Anti-Corruption Strategy and Action Plan target the broader program goal. Recommendations: Given that the JRGA project is at midterm, it should focus on completing ongoing activities. Project staff should continuously follow up on IA implementation of recommendations provided by JRGA short-term experts (e.g. amendments to the Law on the Anti-Corruption Agency to clarify provisions related to ix conflicts of interest and responsibilities of ACA; finalizing whistleblower legislation, drafting a lobbying law; and supporting the GoS commitment to the Open Government Partnership). JRGA should limit the use of international experts and only do so to the extent that Project staff or local experts can provide follow up. JRGA should only initiate new activities to the extent that this will not undermine the Project’s ability to complete ongoing activities. In cases where JRGA has been requested to provide assistance based on the success of earlier activities under Component 1 (e.g. prepare integrity plan templates and facilitate training for local self-government or conduct IA facilities inventory), JRGA should encourage the relevant IAs to take the lead in implementing activities. The Mission should encourage more formal coordination among anti-corruption/good governance donors. This cooperation could be launched through a JRGA supported high level donor coordination meeting that includes Serbian counterparts to discuss assistance in implementing the Anti- Corruption Strategy and Action Plan. As the ACA is responsible for monitoring implementation and the MoJPA is tasked with coordinating GoS activities, these two bodies could co-chair the meeting. x INTRODUCTION EVALUATION PURPOSE & EVALUATION QUESTIONS Evaluation Purpose The objective of this mid-term performance evaluation is to conduct a full, evidence-based and independent review of USAID/Serbia’s JRGA Project, and to produce a report that provides a qualitative and quantitative evaluation of project activities. This will include the identification of assistance gaps and opportunities for potential new activities that could help the project accomplish its objectives and identify areas where progress is lagging or insufficient, and recommend course changes as appropriate as well as to show what has JRGA accomplished to date. The Evaluation Team consisted of a pair of regional rule of law and civil society experts. Francis Bremson and Marilyn Zelin, Team Leader and Evaluation Team Member, respectively, conducted this evaluation between September and December 2013. The audience for this evaluation will be USAID/Serbia and, more broadly, U.S. Embassy Belgrade, USAID Washington, NCSC, and the broader democracy promotion community. Evaluation Questions This mid-term performance evaluation will address, at a minimum, the following questions: 1. To what extent is the program on track (i.e., meeting established milestones) to accomplish the objectives and results expected under both components? 2. What emerging impact has the project had on strengthening Misdemeanor Courts and Independent Agencies in Serbia? 3. To what extent has the project design been appropriate in achieving intended objectives and results? 4. Within the current contract scope, what opportunities can be identified to expand support for Serbian judicial reform efforts? 5. To what extent are Component 2 Governmental Accountability activities sufficiently focused and strategically targeted toward the broader program goal? PROJECT BACKGROUND The JRGA project was designed to improve Serbia’s rule of law, judicial efficiency, independence and integrity and the openness and accountability of government operations overall, both for the sake of the people of Serbia, and to help Serbia move closer to EU accession and participation in other Euro￾Atlantic institutions, by focusing on the work of Serbia’s courts (particularly the MCs and the new AC), Serbia’s (originally four, now five) IAs and the work of other groups and organizations supporting these reforms. The JRGA contractor’s charge is to ensure that progress in these areas is promoted to the general public. JRGA contributes to Assistance Objective 2 “Democratic structures in Serbia strengthened” (in USAID/Serbia’s Amended Country Strategy 2011 - 2015), which has two key elements: improving government operations – “Intermediate Result 2.1 Government Operations Improved”; and increasing civil society engagement in public life – Intermediate Result 2.2 Civil Society Engagement in Public Life Increased” and Sub intermediate Result 2.1.1 “Efficient, Transparent and Accountable Provision of Government Services Enhanced”, Sub-intermediate Result 2.1.2 “ Checks and Balances Strengthened” and Sub-intermediate Result 2.2.2 “Government Accountability and Responsiveness to Citizens Improved”. 1 Component 1: Judicial Reform Serbia’s judicial system has experienced substantial reform over the past several years, initially as a result of initiatives included in the Judicial Reform Strategy (2006-2011) and legislation and rules of procedure adopted to implement the Strategy. Among other changes, the reform laws established the HCC to assume management of the judiciary and instituted a new court network which established some courts, consolidated others, and reduced the number of judges and court seats overall. The MC system was transferred from the executive to the judicial branch effective January 1, 2010 and its work expanded to include responsibility for adjudication of a number of laws not previously within its jurisdiction. Appeals from the 45 MCs are to the HMC. Another development during this period was the establishment of the AC, which is located in Belgrade and has three branch offices. Appeals are to the Supreme Court of Cassation (SCC). Serbia’s JA, which provides pre-service training as well as continuous training for judges, court staff and prosecutors, was also established in 2010. During the course of the JRGA project, the GoS adopted Amendments to the LoM (adopted in June 2013 to be effective March 1, 2014); adopted a five year extension of the National Judicial Strategy and Action Plan (2013-2018) in August 2013 that advocates transfer of authority over all judicial branch operations from the MoJPA to the judicial branch (HCC), and approved a new court network that became effective November 2013. Component 2: Government Accountability Serbia’s five IAs—the ACA, the SAI, the Ombudsman’s Office, the CIPIPDP and the CPE--were established to promote open, accountable, and efficient government; to fight corruption and poor administrative practices; and to generate public demand for and participation in good governance. Serbia adopted a five-year anti-corruption strategy in 2005, which noted that previous anti-corruption efforts were insufficient and that successfully combatting corruption was a prerequisite for EU accession and harmonization with EU standards. Under that Strategy, implementing laws for the Republic Ombudsman’s Office and the SAI, institutions that are provided for under the Constitution, were passed although the Directors of the SAI and Ombudsman were not appointed until 2007; the Law establishing the ACA was adopted in 2008 and the first director was appointed in 2009; and the CPE was established by law in 2009 and the Commissioner was appointed in 2010. The CIPIPDP was originally established by law as the Commissioner for Information of Public Importance in 2004 and its competence was expanded when the Law on Data Protection was adopted in 2009. Although legislation was passed and the IAs were established, the 2005 Strategy was widely held to be ineffective. At the time that the JRGA Program started, the 2005 Strategy had expired and there was no replacement. Moreover, the capacity and profile of the five IAs differed significantly. The ACA was faltering due to a combination of poor management, unclear legal provisions and insufficient resources. As a result, the capacity of the ACA was quite low, internal cooperation and communication was poor and contributed to inefficiencies. The Director was dismissed in November 2012 and a new director was appointed shortly thereafter. In contrast, the SAI, Ombudsman’s Office and CIPIPDP had been operating for several years and were considered strong, credible organizations. Clearly the most significant political development during the course of JRGA’s first two years of operation was the 2012 national elections (President, NA and Local Assemblies) and the change in Government. The new Government has a strong “zero tolerance” message regarding corruption. A new National Anti-Corruption Strategy and Action Plan were adopted in 2013. The new Anti￾2 Corruption Strategy establishes several priorities and emphasizes corruption prevention, implementation of IA recommendations and civil society participation in legislative drafting and oversight. It also provides that the MoJPA be responsible for coordinating activities within the government, that the Anti-Corruption Council (which advises the GoS) follows implementation, and that pursuant to its authority, the ACA will monitor implementation and report to the NA. Priorities under the new Strategy are, among others, financing political activities; public finance (tax and customs); public expenditures (procurement); public internal financial control (SAI); privatization; and the judiciary (financial independence, merit based career system, new criminal procedure code and cooperation with national and European institutions). Another significant development is that the European Council formally agreed to continue open accession negotiations with Serbia on 28 June 2013. The 2013 EU Progress Report states that Serbia will have to sustain the momentum of reforms over time in the key areas of the rule of law, particularly judicial reform and anti-corruption policy, independence of key institutions, media freedom, anti￾discrimination policy, the protection of minorities and the business environment. Continued progress in normalization of relations with Kosovo is also critical. Significant EU support for accession will continue to be provided through the Instrument for Pre-Accession Assistance. Regarding areas within the scope of the JRGA Project, the EU Progress Report notes 1) increased transparency of the NA and use of public hearings although urgent procedures with limited hearing continued to be used extensively2 ; 2) increased transparency in GoS legislative drafting 3; 3) a new public administration reform strategy (PAR); 4) steps to address logistical constraints of independent agencies; 5) the new 5 year Judicial Reform Strategy; 6) the new Anti-Corruption Strategy; and 7) the new Law on Public Procurement. EVALUATION METHODS & LIMITATIONS The Evaluation Team initially reviewed background documents provided by the Mission, including Laws4, PMPs5, Work Plans6, JRGA Monthly, Quarterly and Annual Reports7, Partner Court Selection Report8, the original JRGA RFP9, the JRGA Round 3 grantee RFA10, and the Round 3 Grants Summary11. These were supplemented during the course of the Team’s field visit with documents provided by JRGA, including the Year 3 1st Quarterly Report (September 2013), and the JRGA Year 3 Work Plan (June 2013), as well as documents collected from interviewees and other sources, such as IA Annual Reports and independent studies.12 The Evaluation Team reviewed all relevant project documentation, and then conducted semi-structured interviews with all direct and indirect beneficiaries and external observers to obtain information needed to answer the five evaluation questions posed by USAID. The Team organized data according to expected results under each JRGA component into findings, then drew conclusions based on those findings that provided direct or at least inferable answers to the five overarching evaluation questions; 2 A women’s parliamentary network was established in 2013. 3 EU Delegation has sent JRGA legislative drafting model to Brussels as an example of “best practice”. 4 The Law on Misdemeanors; and Draft Amendments to the Law on Misdemeanors, Law on Courts, Law on Judges, and Law on Public Prosecutor; and the Action Plan for Implementation of the National Judicial Strategy 5 Original PMP Plan (December 2011) and the Year I Results and Year 2 Targets Report (April 2013) 6 Life of Project Work Plan (August 2011), and Year 1 (June 2011) and Year 2 (June 2012) Work Plans 7 Monthly (November 2011 – August 2013), Quarterly (Year 1 Quarters 1, 2 & 4 and Year 2 Quarters 1-3) and Annual Report (Year 1), including Annexes. 8 September 2011 9 September 2010 10 September 2012 11 June 2013 12 See Annex IV-A for a complete list of documents reviewed. 3 and where logical recommendations flowed from that analysis, including whether to continue or to change course on JRGA activities, proposed activity-specific recommendations. For further detail regarding the Team’s methodology, see Annex II, Evaluation Methods and Limitations. Judicial Reform Component interviewees initially identified as direct beneficiaries included Partner Misdemeanor Courts, HMC, AC and JA; Indirect Beneficiaries included SCC, HCC, ROL grantees including the Judges Association of Serbia (JAS) and APPP, MOJPA and NA Secretariat; and relevant external organizations included the Separation of Powers Project (SPP), EU, United Nations Development Programme (UNDP), World Bank (WB), Organization for Security and cooperation in Europe (OSCE) and the Department of Justice (DOJ) and Political Officer at USAID Belgrade. (A complete list of interviewees appears in Annex IV-B). In order to efficiently conduct interviews for the Judicial Reform Component, the Evaluation Team attended the Misdemeanor Court Judges Association (AMJ) Meeting held 23-25 October in Kopaonik, as nearly every Misdemeanor Judge in Serbia as well as other key counterparts were scheduled to attend. The focus of the meeting, co-sponsored by AMJ and USAID Serbia/JRGA, was on key activities supported by JRGA, including implementation of the LoM Amendments, best practices of MCs and development of the MCCMS. The Evaluation Team Leader attended the AMJ Meeting and conducted meetings with APJ of the HMC and APJs of two Partner Courts. In addition, the Team conducted a voluntary survey of all Conference attendees (Annex III-B). The Team also conducted site visits to four Misdemeanor Partner Courts (Belgrade, Novi Sad, Prokupje and Zrenjanin) as well as the AC and the JA. Government Accountability Component interviewees initially identified included direct beneficiaries ACA, CPE, SAI, Ombudsman and the CIPIPDP, as well as indirect beneficiaries - the PPO, MOJPA and NA Secretariat and AC Grantees including Transparency Serbia, European Policy Center, Center for Research, Transparency and Accountability, Eutopia/Pistaljka and YUCOM; and external beneficiaries as the EU, UNDP, OSCE and DOJ and Political Officer at USAID. A survey of all JRGA grantees was also prepared and disseminated (see Annex III-C). The survey was designed to obtain information for purposes of assessing the grantees’ perspectives on the impact of their activities as well as their perspectives on JRGA’s overall results and impact. Semi-structured interview questions drafted in advance of fieldwork13 were revised on an ongoing basis to tailor the scope of the interviews to the responsibilities and activities of the particular agency as well as to reflect information learned during the course of the evaluation. In addition, potential interviewees that were identified during the evaluation were incorporated into the Team’s schedule to the extent possible. The Team endeavored in selecting interviewees and in conducting interviews to minimize the level of selection bias due to the fact that many individuals and agencies as well as grantees were identified by JRGA staff, including Partner Courts. In order to mitigate bias, the Team conducted in depth interviews to probe general responses more thoroughly and verify respondent familiarity with the material discussed. In-depth interviews with external sources, including other international organizations whose work complements JRGA activities (such as EU, UNDP, and OSCE) also served to mitigate bias. With regard to gender equity, more than half of the interviewees were female, including individuals in key positions at the MC, AC and IAs. Following the completion of interviews and review of as much documentation as was available, the Team prepared an annotated outline (preliminary analysis) of findings, conclusions and recommendations, 13 See Annex III-A. 4 which was submitted to USAID Serbia at the evaluation Out-Brief on 5 November 2013 for feedback. USAID provided comments to Social Impact on 13 November, SI submitted the first draft of the Final Report to USAID on 22 November 2013; USAID staff provided feedback on the Final Draft to the Team on December 4 and SI responded to those comments on December 10; that version was then shared with JRGA and the Mission’s and JRGA’s last set of comments was provided to SI on 16 December, after which SI submitted the Final report to USAID on 20 December. The report will be published following formal USAID approval of the final version. FINDINGS, CONCLUSIONS & RECOMMENDATIONS The following section presents the Evaluation Team’s Findings, Conclusions and Recommendations prepared in response to Evaluation Questions I-5. EVALUATION QUESTION 1: To what extent is the program on-track (meeting estimated milestones) to accomplish the objectives and results expected under both Components? Component I (Judicial Reform) Findings: Under the Judicial Reform Component, JRGA is on track to accomplish each of the following nine Tasks specified in the JRGA Project Scope of Work (SOW): 1. Make open hearings routine, dignified, accessible, safe, efficient, and fair; 2. Reduce backlogs and improve case processing times and procedures 3. Increase the percentage of assessed court costs and fines collected; 4. Increase the physical safety of victims of family and domestic cases 5. Improve the coordination of cases involving related issues and/or parties 6. Increase the competency of judges in all the MCs and HMC; 7. Develop and disseminate practical and user-friendly information for court users on court operations; 8. Improve the reputation of MCs and HMC among court users and the public; and 9. Increase non-governmental support for judicial reform through grants. Evidence of achievement of these Tasks is reflected by documented progress to date in producing the following deliverables: • Drafting and implementing amendments to the LoM; • Assessing and prioritizing renovation of all MC facilities; • Compiling case law establishing the basis for standardization of judicial decisions in the AC and HMC; and • Development of the MCCMS. The following is a detailed analysis of the extent to which JRGA is on track in achieving the above Tasks: Open Hearings 5 Findings: JRGA determined early along that strengthening the capacity of MCs to assure that open hearings are the rule rather than the exception required both availability of adequate facilities as well as judge and staff training on best practices in support of open hearings. Facilities: To determine the most cost-effective venues for renovation or construction, JRGA conducted the first-ever comprehensive assessment of all MC facilities and categorized all 45 MC sites as well as the four HMC sites and assigned renovation/construction priorities to all buildings, as follows: Urgent Relocation (A); Urgent Renovation (owned by the state) (B1); Urgent Renovation (not state-owned) (B2); Renovation in the Next Five Years (C); and Adequate (D); and shared this Inventory with USAID, the SCC, the HMC, the HCC and the MOJPA, to facilitate a determination of how best to improve MC facilities. Other than focusing on consolidating the Belgrade MC’s 15 buildings into a central facility, JRGA initially provided facilities design and renovation support to 8 MCs, all of which had been initially categorized as lower priority projects.14 This is believed to be attributable, at least in part, to the limited resources available to the project for renovation and construction purposes as well as to the unwillingness of Basic Courts to cooperate with Misdemeanor Courts in shared facilities. Best Practices: At the same time, JRGA also began working with Partner Courts to document best practices, including the development of public information mechanisms to increase public awareness; and with the JA to develop and begin conducting training programs on the need for and the means to conduct open hearings. Partner Courts report that the number of open hearings in MCs continues to increase15, primarily in upgraded facilities. Conclusion: Implementation of open hearings in all courts will continue to remain a function of the court system’s ability to obtain expanded facilities and related security services. Other than Belgrade MC, all renovation projects to date have been limited to providing upgrades to adequate facilities, such as installation of one-stop-shop counters, improvement of court archival facilities, etc. The prospects for improvement of facilities shared with Basic Courts remain dim as long as PJs of the highest jurisdiction courts in shared buildings have no incentives to cooperate with lower courts regarding allocation of resources.16 Recommendation: To the extent feasible within the scope of project resources, JRGA should encourage GoS and international donors to begin renovating/relocating the top priority (Categories A and B1) MCs as specified in the Facilities Inventory. By sharing data with the MOJPA (and the EU) on the significant amount of General Fund revenue attributable to MCs, the MOJPA (and the EU or other donors) may be more inclined to increase infrastructure investment to improve the MC facilities of highest priority on the Inventory. Case Management Findings: Nine Partner Courts (including 6 MCs, the HMC and the AC) adopted Case Backlog Action Plans through Year 2, and all reported a combined 17% reduction in backlog (where backlog is loosely defined as the total number of cases pending at the end of the reporting period) through Year 3 YTD. 14 Category B2: the HMC branch in Kragujevac; Category C: Arandjelovac, Kikinda, Ruma; and Category D: Valjevo, Zrenjanin, Zajecar. Loznica was unrated as it was to be relocated to a new judicial facility. 15 PJ respondents to the AMJ QX indicated by a 4.1 out of 5.0 score that the number of open hearings in their courts has increased. See Annex III-B. Partner Court responses to JRGA PMP surveys report increased numbers of open hearings as well. 16 By Court Rule, the PJ of the highest court in a building with shared facilities is solely authorized to make all decisions regarding allocation of resources within the facility. 6 Thus, at the end of 2012, while the 9 Partner Courts collectively reported a total of 373,791 cases pending17, those same courts also reported a reduction in backlog as a result of the courts’ combined clearance rate18 of 108% (450,000 cases filed and 486,000 cases disposed), reflecting disposition of 36,000 more cases than were filed during 2012. (The HMC reported a 98% “efficiency factor” for 2012, but as they had no initial start-up backlog, they had only 1100 matters pending at year’s end). JRGA has facilitated the establishment of a committee to develop case processing time standards, as well as more appropriate backlog definitions. It is anticipated that this process will proceed in concert with the development of the MCCMS, so that backlogs and time standards will be established as metrics to be generated by the automated system, rather than as official judicial performance standards. Conclusion: No official case processing time standards or backlog definitions currently exist in Serbian judicial law, procedure or practice. Operational changes in misdemeanor case processing authorized by the Amendments to the LoM should substantially reduce case processing time for the vast majority of MC cases (particularly traffic cases, which account for about 57% of all MC cases). Implementation of the Misdemeanor Order provision should reduce case processing time by reducing personal service time at the initiation of traffic cases19 from a minimum of 60 days to a maximum of 8 days. Further, implementation of the plea bargaining agreement will allow the petitioner to resolve the case in a more reasonable period of time, while the implementation of the electronic register function will substantially decrease enforcement time as interfaces between the register and other licensing agencies will eliminate the need for the court to actively search to find the judgment debtor. Recommendation: The Working Group responsible for developing time standards and backlog definitions should establish a case processing time baseline in Partner Courts, then compare current times to new times expected to result from reduction of summoning and enforcement case processing times anticipated to result from implementation of the Amendments to the LoM, at which point the HCC can consider adoption of case processing times reflecting the Partner Courts experiences under the new procedures. Establishment of case processing time standards and monitoring of courts compliance with those standards should enable the SCC, the HMC, and the MCs to better manage their caseloads, process cases in reasonable times and improve allocation of judicial system resources. Generating these reports as byproducts of the operating MCCMS system will increase the first instance court’s ability to control the pace of litigation and thereby strengthen judicial independence. Collections Findings: Collections for all MCs declined following transition of MCs from the executive to the judicial 17 All Courts in Serbia began self-reporting summary disposition data (i.e., # of cases pending, filed and disposed during the reporting period) to the HCC in 2012; 2/3 of the 45 Misdemeanor Courts reported dispositions exceeding filings during 2012, resulting in net reduction in MCs total backlog. Data provided by SPP. Most of the MCs backlogs arose during the first six months of 2010 when the MCs were transitioning from the executive to the judicial branch. As noted above, once the courts established procedures and began operating full-time, that backlog began to decline while the courts continued to process more cases than were filed, so that the original backlog has since been reduced by about 20%, as reported by JRGA in its Performance Monitoring Plan (PMP) results. 18 Also known as “efficiency factor”: Refers to the percentage of cases disposed divided by the number of cases filed in a given period. Any rating over 100% means the court is disposing of more cases than are being filed; thereby resulting in a decrease in the number of cases pending before the court. Backlog is defines as the raw number of cases that have been pending beyond the period of tolerable delay, where “tolerable delay” is, in turn, defined as the maximum amount of time within which the court is required to dispose of a particular type of case. Typically, courts (or legislatures) who establish case processing time goals give disposition time priority to criminal cases over civil cases, and within criminal cases, to cases in which the defendant is incarcerated prior to trial/disposition. 19 In fact, these citations will no longer even be counted as court cases. If the defendant opts to pay the discounted fine within 8 days, the transaction will only be reported to the issuing agency and the court will never need to open a case file. 7 branch on January 1, 2010, due to a combination of factors, including: inefficient MC service of process procedures, a change in the legal framework governing collection (transfer of jurisdiction for collecting costs and fines for misdemeanors from the Tax Administration to the Basic Courts), the introduction of the function of bailiffs that took place only in June 2012, low levels of voluntary payment and poor tracking mechanisms within the courts. At the same time, Four Year 1 Partner Courts reported collections within time standards in 23% of all cases through Year 2; while 3 more Year 2 Partner Courts reported 18% collections within time standards through Year 2, at or above PMP target levels. Conclusion: The most significant initiative under the LoM Amendments will be the implementation of the electronic register function that will automatically link the register to other licensing agencies, thereby compelling the payment of unpaid fines. A marked improvement in the collection rate is expected to occur once the Amendments are adopted and become operational in practice during Year 320. For example, the Misdemeanor Order provision under the new Law is likely to increase the rate of collections, if not the total amount, as defendants opt to close their cases by paying a reduced fine within 8 days of citation rather than contest the matter in court. Likewise, implementation of automated collection tracking through introduction of MCCMS, scheduled to begin implementation on or around the effective date of the Amendments on March 1, 2014, and centralization of enforcement functions in the Belgrade Misdemeanor Court are expected to dramatically increase the capacities of all 6 Year 1 & 2 Partner Courts to collect fees and fines on a timely basis during Year 4. At the same time, the amount of increase may be tempered by implementation of other provisions of the Amendments shifting the burden of proof to the petitioner as well as the expansion of the court’s authority to imprison or impose alternative sanctions. Recommendation: JRGA has already begun to reach out to agencies/institutions acting as authorized petitioners in the misdemeanor procedure to facilitate implementation of many provisions of the Amendments. This effort should be expanded to include a coordinated public information campaign to assure that licensing agencies and licensees are aware of the new linkages between unpaid traffic fines and agency privileges. Domestic Violence Findings: During Year 2, JRGA engaged STTA Susan Keilitz to assess the handling of family violence cases in Serbia’s MC system. Ms. Keilitz identified several issues that limited the ability of the MCs to deal with domestic violence, including overlapping jurisdiction between the MCs and Basic Courts and sub￾standard facilities of the MCs which limit the court’s ability to isolate victims of domestic violence. She recommended that the MCs: begin systematic identification of domestic violence cases within Public Order caseload and consider assigning all Public Order cases involving domestic violence to one or two judges who can specialize in those cases; designate an area of the court as a “safety zone”, assign court staff to monitor the safety zone and advise victims of the availability of safety zones in the court summons; collaborate with police and domestic violence safe houses to provide accompaniment of victims from safe houses to the court; allow the victim to leave the court before the offender/defendant and require the offender to remain in a place under the surveillance of court staff; and consider legislative changes to avoid duplicative filings in both the Basic and Misdemeanor Courts. Nine (9) Partner Courts (PCs) (20% of total MCs) through Year 2 and 3 new PCs in Year 3 have partially adopted procedures to provide greater safety to victims of domestic violence, although full 20 Although centralization of Belgrade Misdemeanor Court functions in Year 3 will understandably impact timeliness of collections in that court during construction and implementation of new procedures. 8 implementation has been limited by available space. Those procedures reflect the best practices of the pre-existing program in Zrenjanin, which has served as the model for the program reportedly being implemented by Valjevo and three other jurisdictions with implementation assistance being provided by JRGA grantee Association of Public Prosecutors and Deputy Public Prosecutors of Serbia (APPP). During Year 2, all MCs in Serbia were provided with a supply of brochures designed by JRGA covering the topic of domestic and family violence and treatment of such cases before MCs, as well as concrete information for victims on available assistance. Conclusion: While JRGA has encouraged adoption by other PCs of the domestic violence victim protection program ongoing in Zrenjanin, implementation in other courts has been limited to date to dissemination of information informing the public of the courts’ commitment to protection of domestic violence witnesses. While delay in implementation in other courts is generally attributed to lack of available space, it is not clear that lack of space is the most critical factor. Recommendation: JRGA should facilitate dialogue among MCs considering adopting domestic violence programs to determine the courts’ reasons for delaying implementation. Case Coordination Findings: STTA Susan Keilitz identified the lack of an effective means (such as automated case information exchange) of identifying and coordinating the processing of cases arising from the same incident or involving the same family, recommended that JRGA promote the development of coordinated community responses to domestic violence and implementation of multidisciplinary teams to improve case processing in the MCs, and contribute to greater court security; and seek to amend the LoM to eliminate the problem of multiple filings in Basic and MCs. Through Year 2, JRGA facilitated Round Table discussions (RTs) between the MCs and three IAs: three RTs with the CIPIPDP regarding Agency requests for misdemeanor case-related information and data privacy standards; four RTs with the SAI regarding identity of persons against whom misdemeanor charges should be filed; and two RTs with the ACA regarding the Agency’s planned filings of misdemeanor charges against political parties for failure to comply with the law on financing of political activities as well as differing amendments being proposed by both the ACA and the MCs to the LoM. JRGA subsequently assisted in drafting Amendments to the LoM to resolve overlapping jurisdiction between MCs and Basic Courts as regards domestic violence matters. Conclusion: MCs and IAs are committed to developing and implementing procedural solutions to matters of jurisdictional conflict. Twelve (12) PCs and the JA are preparing to implement provisions of the Amendments to the LoM in March 2014 that will substantially eliminate the problem of multiple filings in basic and misdemeanor courts and improve coordination between misdemeanor courts and independent agencies. Recommendation: JRGA should work with the JA to assure that case coordination protocols are included in best practice manuals and in the MC Bench book. Judicial Competency Findings: MCs employ 615 judges and 2,035 MC staff. All MCs were grandfathered into the new court system, without being required to meet any particular qualification criteria. Nearly every MC judge has attended courses at the JA on both substantive legal and procedural and administrative requirements of 9 the LoM and proposed amendments, relating to the court’s expanded jurisdiction (taxation, customs, procurement law, inspections, foreign exchange, ethics, plea bargaining, etc.)21. These initial trainings have been exported throughout the system through train-the-trainer programs in which judges attending the trainings at any of the JA’s four regional training centers have been trained on techniques for sharing what they learned with their colleagues in the field. Virtually without exception, judges who have attended the JA trainings rate the quality of the training and the increase in their knowledge substantial.22 Training modules on due process procedures, restitution and public procurement have been developed and delivered to judges of the AC; a new program on Prevention of Workplace Harassment is under development. A database of some 54,000 cases, including about 11 cases of precedential importance was established to provide full access to AC judges and redacted access to the public to the Court’s cases, decisions and sentences. Plans are underway for development of a similar module in the database of the HMC. While at one time new judges were to be appointed from the pool of judicial assistants, the JAs’ status as possible candidates has been called into question by the proposal in the National Judicial Strategy that future vacancies should be filled only from graduates of the JA’s 2-year judicial training program. JRGA has assisted the judges and staff of the MCs in the development of a Best Practices Manual; and is currently assisting the judges in the preparation of a Misdemeanor Judge’s Bench Book that will provide users with practical best practices guidance on forms and procedures needed for the effective and efficient processing of cases. Study tours to the US, Netherlands and Ireland have provided judges of the Supreme Court of Cassation, the MCs and the AC with the opportunity to learn about best practices of court systems in other EU countries in the areas of performance management, court administration, customer service and the use of technology to improve operational efficiency. JRGA is staffing an HCC committee exploring development of performance standards for new and existing judges; that committee is currently discussing criteria and qualifications. In addition, a two year training program for future judges has been established and is in place at the JA. Extensive substantive and administrative training is also being provided to judges and staff of the MCs and AC designed to assure that all judicial personnel have a complete and current understanding of the court’s legal obligations and responsibilities. JRGA has distributed more than 2000 computers, printers and related equipment throughout the MC system and, in cooperation with the JA, proactively helped prepare the MC judges and staff for implementation of MCCMS by conducting training in basic computer literacy and introduction to the use of the court system’s open source office software, LibreOffice. Conclusion: MC judges are receiving the education and training needed to serve as judicial 21 5512 person days of training were provided to HMC/MC judges and staff in substantive, administrative and computer literacy areas through 1Q Year 3, including 2552 judge days (678 male and 1854 female) and 2543 staff days (815 make and 1728 female. Every misdemeanor judge will either personally attend training on the Amendments to the LoM or attend a training in his/her court to be presented by a colleague who attended the regional training prior to the effective date of the Amendments in 2014 22 83 judges responding to the questionnaire administered at the AMJ conference in Kopaonik on October 23-25, 2013 rated the quality of training programs at the Judicial Academy between 4 and 5 (highest rating possible). See Annex III-B. 10 professionals. While the quality of training and support provided is subjectively recognized as effective, no formal standards have been established to measure judicial competence for new or permanent judges or for certification of judicial training programs. No system is currently in place to compare judicial performance of judges and staff who have received training or participated in study tours to a baseline level of competency. Although standards are not yet in place for judicial performance evaluation purposes or for formal certification of judicial training programs, extensive programs have been implemented to provide judicial staff with essential knowledge and necessary resources to ensure continuing improvement in judges’ professional competence. Recommendations: JRGA should work with the JA to provide pre- and post-training testing for attendees of all training programs including programs presented by trainers. The feasibility of developing a judicial performance evaluation program solely for internal evaluation purposes to enable the president judges to assist the judges’ direct reports in meeting jointly determined professional goals should be explored. Public Information Findings: • Facility Upgrade and Renovation: PCs promote availability of court services at one stop shop counters or kiosks and integrate informational signage and brochures in each court during renovation. • Distribution of print information materials: JRGA developed, designed, printed and distributed 92,000 copies of 7 different information brochures to all MCs in Serbia. Furthermore, 50,000 copies of an information brochure on the AC, including user-friendly descriptions of the court’s services, jurisdiction, how to initiate a case and remedies available, as well as contact information, was also developed, printed and submitted to the AC for distribution. • Distribution of court information via web portals: Court brochures are available through six of the seven Partner Court websites and all MC directories are available through the website of the HMC. JRGA facilitated development of the AC’s case law database that will be accessible by the public, and assisted the court in the development of its website. • Outreach and Communications: Technical assistance was provided to the MOJPA to include court related content on the Ministry’s website and assistance was provided to help the Ministry improve its outreach to the public. Conclusion: The courts and the public now have access to extensive information regarding court system operations, leading to increased public support for the judicial system. Recommendations: JRGA should work with PCs to explore opportunities for expanding resources to enable the public to more easily navigate court processes and identify court activities, including through the use of kiosks, bulletin boards, brochures and information centers in every location. Improved Reputation of HMC and MCs Findings: 11 • Court Facilities: JRGA designed and facilitated construction of one-stop information counters in at least three courts at which court brochures and, increasingly, court user surveys, are provided; as well as integration of user-friendly signage into the renovation process • Court initiated public information and communications activities: JRGA developed and provided brochures about the court system for all MCs, HMC and AC and helped increase public and justice community awareness of court services through website improvements and RT discussions with representatives of CSOs and other justice sector and governmental organizations and the general public. JRGA assisted the AMJ in development of a five-year strategic plan in 2013, an important provision of which addressed ways to improve MCs public outreach and communications. The AMJ’s updated user-friendly website provides information about the Association’s activities for both members and the general public. • Integrity Plans: Shortly before integrity plans for all government officials were due in 2012, JRGA facilitated modification of ACA’s generic reporting form to better reflect activities of the judicial branch in general and MCs in particular. While the ACA estimated a 40-50% compliance rate during this first year of reporting, the MCs were one of only two agencies that achieved 100% compliance. • Media: To date, 59 instances of media coverage of improved court operations have been reported, although media reporting remains largely limited to specific local events, e.g., opening of new facilities. • Court User and Staff Surveys: At least two Partner Courts routinely conduct surveys of court users on an ongoing basis and inform their operations based on the feedback received23. • School Programs: Several Partner Courts are implementing pilot programs for senior high school students to introduce them to the concept of judicial independence and courts as essential institutions of government as well as possible career options. • Reputation among other judges: MCs tend to be regarded as inferior to other courts in Serbia’s judicial system, particularly by judges of higher level courts, presumably as a result of their recent transition from the executive branch, where judges were appointed pursuant to civil service requirements rather than according to qualifications required for judicial appointment to other judicial branch courts. Although invited to attend the JAS Conferences, few MC judges attend; and most MC judges attend AMJ Conference instead. This is gradually increasing self-esteem and self-confidence as amount of training and resulting level of professional competence increases. Conclusions: Misdemeanor courts are the face of the judiciary to the vast majority of citizens. JRGA strategically increased the level of public regard for the MCs, through training that improves judges’ self￾confidence; renovation of court facilities that enhance the stature of the courthouse and instill dignity; and implementation of activities to increase public accessibility to court services, court information and justice opportunities. Nonetheless, MCs lack the respect of others within the judicial branch. Recommendations: While beyond the scope of JRGA’s authority in this project, legislative change to establish judicial salary parity is a possible solution to the problem of MC judges being viewed by the judiciary at large as less competent than judges of higher courts. Disrespect for lower court judges 23 Survey results from Zrenjanin were remarkably positive as court users expressed their great satisfaction at the possibility of accomplishing all administrative tasks in the court at one place in direct interaction with competent and kind court staff. The results of the survey in Kikinda show that court users have trust in the operations of the court that they deemed transparent and particularly positively assessed the work of court couriers with regard to service of process and enforcement. 12 within the judicial branch is reinforced by salary differences. JRGA should encourage the HCC and the MOJPA to seek to establish salary parity among all first instance judges in a future iteration of the national judicial strategy as an initiative in support of judicial reform. Salary parity would provide judicial system leaders with greater flexibility to manage judicial branch resources. Further, JRGA should offer to assist the chief justice and PJs to develop and deliver annual state of the judiciary addresses to highlight progress in achieving court system improvements, especially as defined in the National Judicial Strategy and Action Plan; and conduct annual surveys to measure public confidence in the MCs. Non-governmental Support for Rule of Law Findings: JRGA reported that 27 grants totaling $500,000 had been awarded to 20 CSOs advocating for judicial reform through Year 224, including: protection for victims of domestic violence (APPP); strengthening MCs -- including collection of selected case law for the HMC and compilation of best MC practices for Best Practices Compendium and Benchbook (AMJ); Manual for the AC (CDS); Handbook for Victims of Domestic Violence (CAFV); plea agreement training for MC judges (AMJ); assessment of MCs compliance with anti-corruption agency requirements (JA); high school program examining the ability of courts and IAs to sanction violations of anti-corruption laws (Alterfact). Other than each contractor’s report on the successful completion of their contracts, no objective evaluation system is in place to accurately assess the effectiveness of the contractors’ efforts, although one study by CSO grantee Transparency Serbia 25 (TS) found that seminars on anti-corruption presented by non￾governmental organizations were perceived to be more credible than government-sponsored trainings because CSOs are perceived as being “outsiders”, and therefore independent of the system. TS also concluded that the public’s perception of corruption in the judiciary tends to focus more on Basic Court matters where assignment of cases can be subject to manipulation, although all courts are theoretically required by the Book of Court Rules to assign cases on a random basis. Conclusions: Award of judicial reform grants to CSOs reflects JRGA’s commitment to include civil society in judicial reform issues. The increasing number of applications for grant funding reflects growing support among the non-governmental sector for improvements in the rule of law and the judicial sector. Recommendations: If the Grants Manual does not currently provide detailed guidance, JRGA should develop an assessment tool to measure the effectiveness of CSO grants. JRGA should also follow up on all grant-funded projects to assist counterparts in implementing the recommendations and solutions developed by CSOs. Q. 1 Judicial Reform Conclusions: Based on PMP analysis, interviews and review of JRGA progress reports, JRGA is on track to accomplish the objectives and results expected under Component 1. While JRGA made substantial progress in achieving these objectives, two areas where achievements lagged behind objectives were: • Facilitating development of case processing, backlog reduction, and judicial performance standards; and • Renovation of highest priority facilities. 24 The Project plans to award grants totaling about $250,000/year in Years 3-5 25 “Judiciary in the fight against corruption”, January 2013 13 Delays experienced in developing standards reflected the unwillingness of the MOJPA to transfer responsibility to the HCC and of HCC to assume responsibility for all judicial branch management functions, thereby limiting JRGA’s capacity to work through HCC to develop and establish national court performance standards. Q. 1 Judicial Reform Recommendations • Within the context of JRGA’s overall judicial reform objective to “implement activities and tasks that improve the efficiency and independence of Serbian courts”26 USAID/Serbia should facilitate HCC’s assumption of ownership over the judicial system by requiring HCC to lead implementation of the National Judicial Strategy and Action Plan. This will require that JRGA, SPP or other donors conduct strategic planning training to enable the HCC to assume responsibility for judicial branch management of facilities, IT and HR functions currently managed by MOJPA, and thereby overcome the conventional wisdom in some camps that “HCC lacks capacity”, that “judicial independence means judicial fairness, but not necessarily administrative oversight”; that “MOJPA has the requisite expertise and infrastructure in place” and that “strengthening of judicial branch management capacity would be redundant and cost￾ineffective”. • At the same time, MOJPA and the EU need to be encouraged to fund renovation and construction of highest priority MC facility relocation and renovation projects. • JRGA should facilitate the development and implementation of an HCC research and development organization/function that will eventually enable HCC to replace international advisors with Serbian resources and staff experts. Establishment of an HCC Research and Development (R&D) function will assure sustainability of JRGA initiatives to strengthen the court system, such as ongoing efforts to develop court performance standards and backlog definitions and case processing standards and development and implementation of the MCCMS. • JRGA should disseminate the results of CSO grants as well as assist judicial branch agencies in implementing grantee recommendations. Q. 1 Component 2 (Government Accountability) Findings: Q. 1 Government Accountability Findings: Under the government accountability component, JRGA is required to complete six tasks. The principal results that JRGA is expected to achieve by the end of the Project are: strengthening the ability of the ACA to effectively carry out its duties relating to conflicts of interest prevention, detection and sanctioning, and political party and election campaign financing; improving cooperation within IAs and with Courts, governmental agencies and other bodies in order to increase implementation/enforcement of IA decisions and recommendations; and increasing non-governmental support for improvements in government and official integrity, reduction of corruption, proper financing of elections and political parties and compliance with EU norms. 26 Evaluation Statement of Work, see below Annex I. 14 The following is a summary of JRGA’s performance under each task27. Task 2.2 .Assist the Anti-Corruption Agency and its Board to: a) enforce conflict of interest and property registration provisions in the Anti-Corruption Agency Act; b) detect violations of those provisions of the Act; c) initiate and conduct proceedings to establish if violations of the Act have occurred; and d) order measures under Article 51 of the Act as it read on January 2, 2010, when appropriate. During Year 1, JRGA’s Anti-Corruption Institutional expert conducted a review of the ACA’s Systems for Conflict of Interest and Income and Asset Management and offered near to medium term suggestions, in particular to prepare the ACA for the volume of work that would result from the forthcoming elections28. The JRGA expert noted that several positions at the ACA remained vacant. The ACA, with assistance from JRGA staff, implemented recommendations in connection the ACA’s responsibility to review and verify asset declarations and respond to conflicts of interest inquiries. As a result the ACA developed case management systems for recording and analyzing data, developed a Guide for New Officials and conducted training for over 800 officials, prepared templates and advised official bodies on preparing integrity plans, and adopted other policies and procedures to improve efficiency. However, the ACA did not implement all of the JRGA STTA recommendations, such as developing a risk-assessment or alert system that would improve prevention and detection of violations. The results presented in the ACA 2012 Annual Report include: fourfold increase in asset and income declarations processed and published; threefold increase in number of public authorities attending educational events; request for conflict of interest determinations doubled and 90% of cases resolved; and methodology for verification of asset declarations improved by requiring comparison with data held by other authorities such as tax authorities and business registries. A mini-survey of JRGA grantees noted moderate improvement in the ACA’s detection and sanctioning of conflicts of interest and several respondents commented that the Guide for New Officials and related training contributed to this. The ACA has applied to the EU for IPA funds to upgrade software to enhance information sharing with banks and tax authorities. At the end of Year 1, a JRGA short-term expert reviewed the ACA’s procedure for handling complaints and prepared a Strategy for Strengthening the Complaints Process in the ACA29 . The Strategy includes recommendations for streamlining the system of processing complaints, some of which were implemented, as well as a menu of options to enable the ACA to exploit the information compiled by the Complaints Department and advice on whistleblower legislation. Some recommendations for streamlining the system were implemented, but more comprehensive recommendations such as development of agency wide performance standards were not. Another JRGA anti-corruption expert provided advice on the complaints procedure at the end of Year 230 and JRGA’s Year 3, Q1 Report states that the ACA has begun to implement some of the recommendations and that JRGA will continue to provide assistance in this area. ACA staff reported that the new jobs classification system will allow the Complaints Department to designate one person to identify cases for investigation that should support system-wide improvements. The ACA has limited power to investigate potential violations of the Anti-Corruption Agency Act , which impacts its ability to detect and sanction violations. Therefore, the Agency, with support from UNDP, has considered whether its role in investigations should be expanded. JRGA contributed to this effort by bringing two well-known experts to Serbia to discuss the issue with the Anti-Corruption 27 The first task was establishing baselines and indicators for the PMP. 28 See Year 1, Q 3 Report, Annex 14. During Year 1, this activity was reported under Task 2.4i, 29 See Year 1 Annual Report, Annex Z. 30 See Year 2 Annual Report 15 Strategy working group, high-level government officials, ACA Board members and staff. An amendment to the Act to expand ACA investigative activities has been included in the new Anti-Corruption Strategy and Action Plan. Task 2.3. Assist the Anti-Corruption Agency and its Board, and any successor organization, to develop and implement investigative, monitoring and enforcement procedures governing the financing of political parties and election campaigns by the responsible body(ies). JRGA provided pre- and post-election assistance to the ACA in connection with the May 2012 national elections. Pre-election assistance included helping the ACA establish and train a network of 165 political finance monitors, develop a methodology for handling campaign expense reports, and conduct seminars for editors and journalists . However, the ACA had difficulties organizing and analyzing the high volume of data received that resulted in the ACA’s failure to issue Reports on Political Party and Campaign Financing on time31. Further, the head of the political finance department left the Agency during this period. JRGA (and OSCE) assisted in data analysis, managing election finance reports and public/media relations.32 Nonetheless, the ACA did file 2 misdemeanor cases and referred 53 violations to the appropriate bodies. Problems related to the ACA’s effective oversight of political party and campaign finance are reviewed in Transparency Serbia’s Report on Funding of Election Campaigns in Serbia. The report concludes that the ACA monitored the 2012 campaign, but failed to carry out control and assigned a rating33 of 8.1 to the legal framework and 5.5 to practice. The Report states that “this research proved once again that the implementation of anti-corruption laws in Serbia is a far greater problem than the quality of the legal provisions.” Cooperation with the ACA was difficult during the first year of the Project due in part to poor management and unclear legal provisions. The Agency was also understaffed. The first Director of the Agency was dismissed and has been replaced by the former Secretary of the ACA Board. There have also been other changes in Board membership and the Agency is fully staffed. Since the dismissal of the first ACA Director and changes in Board membership, JRGA has started to assist the ACA senior management and Board to rebuild the Agency’s reputation, improve Board-management cooperation, and improve internal systems. In that regard, JRGA organized a retreat for senior management and Board members in September 2013. The group identified some priorities, but no concrete strategy was developed at that time. ACA staff stated that improved internal communication and cooperation was needed, including better cooperation between the Spokesperson who is a member of the Director’s Office and the Public Relations Department. Staff also stated that the Agency’s Research & Development Department was planning to work with the Agency’s other departments to access needs and determine which problems are real and which are just lingering perceptions. Task 2.4. Assist Independent Agencies to achieve more timely and stringent enforcement of their decisions and implementation of their recommendations by other government agencies, and to track the relevant processes. Task 2.5. Assist relevant official bodies and actors to develop and implement procedures to increase coordination between courts and relevant government agencies that reduce the processing times of selected administrative provisions and procedures. 31 Data included Annual Reports from political parties ( 69 of 83 reported), campaign finance reports (1035 received and verified, but more than 50% of reports submitted late and many reports were incorrect, reports from monitors and information from state authorities. 32 STTA Jennifer Brunner (political party finance), Mamak Ashtari (organization of data), and Djorde Vukovic (data analysis and reporting). 33 Scoring ranges from 0 to 10. 10 indicates that a country has met all criteria expected in terms of transparency and accountability and 0 indicates that none have been met. 16 Task 2.6. Assist with improving ability of Independent Agencies to work with each other to monitor and improve administrative practices and other government operations.34 Beginning in Year 2, JRGA decided to address tasks 2.4-2.6 in a consolidated manner in order to leverage resources and address challenges of independent agencies in a holistic, systematic manner35 by promoting dialogue between independent agencies and enforcement bodies, helping independent agencies and their counterparts to prepare and respond to independent agencies’ recommendations, and addressing specific areas of government accountability. JRGA has established an informal dialogue between the IAs and enforcement bodies. This began with a series of roundtables between the SAI and MCs during Year 1 and expanded to include the ACA and CIPIPDP in Year 2. During the SAI-MC roundtables, MC judges learned about the concept of audit and SAI staff improved their understanding of evidence requirements and court procedures. As a result of these roundtables, a provision was included in a package of amendments to the Law on Misdemeanors that extended the statute of limitations for SAI cases. Roundtables with the CIPIPDP focused on issues related to the Law on Access to Information of Public Importance and Law on Protection of Personal data and the interplay between those two laws. Roundtables with the ACA dealt with the Agency’s planned filings of misdemeanor charges against political parties for failure to comply with the law on financing of political activities as well as differing amendments being proposed by both the ACA and the MCs to the Law on Misdemeanors. Due to the fact that cases initiated by the IAs are based on violations of relatively new laws that MC judges were not familiar with and that these cases can be very complex, JRGA is supporting development of modules for training MC judges at the Judicial Academy. Based on the success of the IA-MC roundtables, JRGA, in cooperation with the U.S. Department of Justice, organized a roundtable between the ACA and prosecutors dealing with corruption cases that has led to continuing exchange of information between the two sides. JRGA also facilitated a roundtable between the SAI and Public Prosecutors Association regarding the criminal procedure for cases initiated by the SAI. JRGA has undertaken targeted activities to improve the response to IA recommendations. During Year 2, a JRGA public outreach and media relations expert worked with all departments of the ACA to improve its Annual Reporting. A general template for Annual Reports was also developed and shared with the other IAs. Additional assistance was provided by JRGA’s parliamentary oversight expert in Year 3. He conducted workshops for IA staff on formulating better recommendations and on tracking and reporting on implementation. At a final joint IA-NA workshop facilitated by the JRGA expert, the NA and IAs agreed to form a working group to improve interaction. Further eight of the initiatives funded under JRGA grants program were focused on tracking the implementation of independent agencies’ recommendation at the central and local government levels (e.g. European Policy Center-- Determining Conditions for Improvement of work of the State Audit Institution of Serbia and YUCOM—Strengthening Ombudsman’s Role in Civil Society). Other activities to strengthen IA ability to perform their respective functions and promote government accountability include: • Assistance in connection with the drafting of a Whistle Blower Protection Law was provided by JRGA STTA Tom Devine, an expert on whistle blower legislation. He assisted a working group established by the CIPIPDP , whose work was supported by funding from the UK Embassy. After the May 2012 elections, the MoJPA formed a working group to revise the draft law prepared by the CIPIPDP working group and the expert returned to Belgrade to provide 34 In some reports and workplans activities discussed in this section are reported under tasks 2.4-2.6, while in other, they are reported under task 2.7. 35 See Year 2 Workplan, pp. 28-29. 17 assistance to the new MoJPA working group. The MoJPA did not include anyone from the original working group in its new working group. • Assistance to the Ombudsman’s Office in Year 1 in developing a communication strategy to raise the level of awareness of Serbian citizens of the Ombudsman’s competence and activities. During Year 2, JRGA assisted the Ombudsman’s Office in handling a continually increasing number of complaints. Training for Ombudsman staff on handling and processing complaints was delivered by two JRGA STTA, which was followed by five continuous training exercises conducted over a six-month period. JRGA STTA provided additional advice on streamlining the Office’s internal procedures relating to complaints, conducting investigations and writing reports, and identifying and reporting systematic issues. • Assistance to the CIPIPDP on the legal framework, operational procedures and practices for personal data protection designed to support implementation of the Commissioner’s recommendations. A JRGA STTA delivered training for staff of CIPIPDP and other Serbian government agencies that handle personal data in their daily work. • Support to the CPE for development of a new website and outreach materials to increase CPE visibility and improve access of vulnerable populations. • Strengthen capacities of the SAI in overseeing transparency and accountability in managing public resources by providing support for representatives of the SAI to participate in a study￾visit to the United States organized by the USAID BEP for its counterparts on the subject of ‘Transparency and Accountability in Public Financial Management: Role of the Legislature and Independent Institutions. • Assistance to the Ombudsman in developing and promoting a Code of Good Administrative Governance based on the EU model. The Ombudsman followed JRGA’s recommendation to promote the Code to units of self- government in coordination with the Standing Conference of Towns and Municipalities. • Supporting the State Secretary for Public Administration in regard to the GoS commitment to join the Open Government Partnership. • JRGA began supporting improvements in public procurement during Year 2. JRGA assisted the newly established Association of Public Procurement Professionals in organizing a conference entitled Promoting a More Efficient Public Procurement System. The PPO, SAI, ACA, judiciary, and CSOs participated in the conference, which was followed by a workshop attended by representatives of the PPO, SAI, Public Prosecutors Association and Judges Association that produced a draft amendment to the Criminal Code to ensure prosecution of violations in public procurement. JRGA also organized a retreat for the PPO and public procurement professionals and practitioners to finalize implementing regulations for the new Public Procurement Law, including establishing a “civil monitor” system. Information on this topic was developed by the Toplica Center for Democracy and Human Rights and Institute for Comparative Law as part of their JRGA-funded grant activities. Finally, JRGA organized a series of meeting with PPO, CSOs and other donors to discuss implementation of the new law and how public procurement officers will be trained. JRGA provided significant technical and logistical assistance for the development of the Anti-Corruption Strategy and Action Plan and was involved at every stage of the process. Initially JRGA supported the 18 ACA working group that prepared the first draft of the strategy. After the 2012 election, the GoS, through the MoJPA, assumed responsibility for drafting the strategy and formed its own working group. The MoJPA working group included one JRGA representative, as an observer, who provided advice on particular issues, comments from the previous ACA draft strategy and best practices in Executive Branch coordination of anti-corruption efforts. JRGA supported the MoJPA working group in organizing a public hearing for the draft Strategy, incorporating feedback from the hearing and finalizing the Strategy. JRGA also facilitated 10 workshops that served as consultation platforms for priority areas (e.g. police, judiciary, health, and media) and helped to formulate and structure Action Plan indicators. The ACA is responsible for monitoring implementation of the Action Plan and the MoJPA is responsible for coordinating CSO activities. JRGA intends to prepare a Guide to Implementation to support this process. The EU Delegation reports that the MoJPA has also requested assistance from the EU in this regard. Task 2.7. Encourage improvements in governance and official integrity, and to reduce corruption, to monitor political party financing (including financing of local and national elections),and to promote compliance with EU norms, through the provision of: Grants under Contract; Comparative information on substantive issues; and Technical assistance on the institutional development of the local civil society organizations. JRGA awarded 11 government accountability grants to CSOs during Yrs. 1 and 2 and will have two more rounds of grants. Initially grants were targeted at specific issues relevant to other JRGA activities, such as monitoring implementation of IA decisions or formulating recommendations on improving IA cooperation with other government bodies. The scope of the grants has gradually widened to cover the full spectrum of program objectives. Applications for grants have exceeded availability in each round of calls. JRGA has a clearly defined procedure for grant applications and review, including consideration by an evaluation committee.36 Although JRGA has received proposals from CSOs outside Belgrade, JRGA staff report that grants are most often made to Belgrade-based CSOs because their proposals are much stronger. In that regard, results of the SI mini-survey of JRGA grantees indicate increased NGO support for improvement in government transparency and accountability, although most respondents reported that too few CSOs are covering these issues, particularly outside of Belgrade. Conclusions: Despite difficulties cooperating with the ACA in Year 1, JRGA assistance contributed to improvements in the performance of the ACA’s duties under the Law on the Anti-Corruption Agency and the Law on Political Party Financing. Difficulties working with the ACA were due, at least in part, to relations with the Director, who was dismissed by the ACA Board in 2012. Cooperation with the new ACA Director and Board appear to be productive as evidenced by the September 2013 ACA Management retreat. Further, as a result of recent STTA assistance, the ACA is now implementing recommendations to improve performance of the Sector for Complaints, which it failed to implement before. Therefore, it appears that JRGA is on track to accomplish objectives under Tasks 2.2 and 2.3. The fact that the ACA intends for its Research and Development Department to conducts a needs assessment further supports this conclusion. Cooperation with the other IAs has been positive since the beginning of the Project. Targeted assistance has been provided to the SAI, CIPIPDP, Ombudsman and CPE to improve performance of their duties. The roundtables between IAs and MCs have been extremely productive as have joint trainings at the Judicial Academy, designed to improve enforcement of IA decisions. JRGA facilitated coordination between the IAs and Parliament designed to improve implementation of IA recommendations have also been successful. Although IAs have mechanisms to track implementation and enforcement of their decisions and recommendations, it is difficult to quantify actual improvement 36 See Grants Manual. 19 because the data is only reported once a year (once so far during the term of the JRGA project) and it is frequently reported on an inconsistent basis 37 . However, anecdotal information suggests that the situation is improving and that roundtables, training and other activities supported by JRGA are contributing to this. Therefore, it appears that JRGA is on track to accomplish objectives under Tasks 2.4- 2.6. Other JRGA activities that are providing broad-based support for government accountability and anti￾corruption efforts include: assistance to a MoJPA working group in preparation of the new Anti￾Corruption Strategy and Action Plan; support for implementation of the new Public Procurement Law; and support to the State Secretary for Public Administration in connection with Serbia’s intention to join the Open Government Partnership. The JRGA grants program is contributing to greater demand for transparency and accountability by civil society organizations, increasing citizen’s awareness of anti-corruption efforts and decreasing citizens’ tolerance of corruption. CSOs that receive grants from JRGA are conducting useful projects and several are incorporating new social media mechanisms. This indicates that JRGA is on track to accomplish objectives under Task 2.7 However, most grants go to Belgrade-based CSOs. Further these CSOs are largely dependent on international donors. Q. 1 Government Accountability Recommendations • JRGA should continue with activities as outlined in its Year 3 workplan with consideration for recommendations included under Evaluation Questions 2 and 5. • JRGA should continue to build its relationship with the revitalized ACA and follow-up on issues raised at the management retreat and by ACA staff, including reorganization of internal procedures and advanced management training for key staff. JRGA could assist the Agency’s Research and Development Department in conducting a comprehensive needs assessment. The assessment could be reviewed at an ACA management retreat to which other donors should be invited. This could be the basis for developing a 2 yr. ACA Assistance Plan that would avoid duplication of donor activities and allow for proper sequencing of activities, recognizing that EU IPA II support will not begin until 2015. Ideally this would be memorialized in a MoU signed by the ACA and all donors, although in the event that is not possible, there should at least be a MoU between ACA and JRGA clearly outlining activities and mutual duties, including availability of ACA resources that are necessary for such activities. • JRGA should pace its activities to ensure that the ACA has the capacity to absorb and act on the assistance provided. This is important because during the first two years of the program, the ACA was not able to absorb all assistance and implement recommendations provided by STTA. Accordingly, JRGA staff should continue to closely monitor and follow-up on recommendations made by JRGA STTA. • JRGA should continue its facilitation of interagency cooperation and assist IAs in publicizing improvements in enforcement and implementation of IA decisions and recommendations in order to improve public trust in the IAs. • JRGA should attempt to make grants to non-Belgrade based CSOs. To do so it might conduct grant application workshops in regions or collaborate with another program building the capacity of Serbian CSOs (e.g. CSAI if it were to continue beyond December 2013). 37 See JRGA PMP analysis. 20 EVALUATION QUESTION 2: What emerging impact has the Project had on strengthening Misdemeanor Courts and Independent Agencies in Serbia? Judicial Reform Findings: JRGA’s principal contributions to strengthening the Misdemeanor Courts have been drafting and facilitating implementation of the Amendments to the LoMs adopted in June 2013 and to become effective March 1, 2014; development of the MCCMS; strengthening the competences of MC, HMC and AC Judges; and assessment and improvement of MC physical facilities. At the national level, JRGA consulted with the HCC, HMC and MC APJs throughout the country to determine which provisions of the LoM and the Rules of Procedure most needed to be amended to enable the MCs to overcome barriers to effectiveness and judicial independence; staffed the LoM Working Group, including providing both legal and technical expert assistance; and worked with the full range of counterparts in the courts, the MOJPA and the National Assembly to resolve open issues leading up to the passage of the legislation in 2013. At the same time and following passage, JRGA worked with the HCC, the HMC, AMJ and the JA to develop and conduct trainings for all MC judges and staff regarding the challenges of implementation of the Amendments; conducted numerous roundtables among the judges and staff, court petitioners and users of court services to facilitate implementation; and led outreach efforts to make sure the bench, bar and public are aware of the changes in operations that will occur. At the same time, JRGA led the effort to develop the infrastructure needed to support changes in the Law, including development of the MCCMS; coordinated efforts to document best practices to be followed by the 45 MCs and the HMC; to eliminate conflicts between MCs and Basic courts in matters of overlapping jurisdiction, including domestic violence cases; and to facilitate coordination between petitioner agencies and the courts to effectively implement the new system. In addition, JRGA helped develop new MC procedures that will not only increase efficiency but will also reduce case processing time and potentially increase MC revenues. JRGA also drafted Omnibus Legislation to harmonize MC policies and procedures under 147 laws, conducted roundtables to develop implementation policies and procedures, established the AMJ; supported establishment of uniformity of decisions in the AC and HMC and developed comprehensive curricula for training of sitting and future judges at the JA to increase overall judicial competency. At the local level, JRGA provided hands on training and technical assistance to improve facilities, court automation and public outreach and increased access to justice through its PC Program. JRGA also facilitated adoption of operational improvements, which were developed and pilot tested in PCs; improved court facilities; developed and implemented outreach strategies, brochures and websites; and developed and conducted training about and implementation of court management systems and best practices. Conclusions: Comprehensive support provided by JRGA at both the national/policy and local/operational levels to MCs and ACs has significantly advanced prospects for institutionalization of modern judicial system management in the judicial system. Likewise, JRGA’s work with the courts and the IAs has resulted in increased coordination between and among IAs and the courts, IAs and the Assembly, IAs and the Ministry and IAs and each other. Recommendation: In addition to encouraging transition of control of judicial system resources from the MOJPA to the HCC, USAID and JRGA should assist HCC and the JA in developing a national court 21 system research and development organization to assist in the development and implementation of case management and court performance standards. Government Accountability Findings: JRGA has provided significant assistance in three overarching aspects of government accountability: development of the new Anti-Corruption Strategy and Action Plan, facilitating communication and cooperation between IAs and courts, government agencies and Parliament and among IAs, and increasing the visibility of IAs. Specific interventions are discussed under Evaluation Question 1. The new Anti-Corruption Strategy and Action Plan (2013-2018) provide a road map for increasing government transparency and accountability and fighting corruption at all levels. While all anti￾corruption donors provided support for drafting the Strategy and Action Plan, JRGA provided extensive assistance for completion of the Action Plan, including setting deadlines and establishing indicators. JRGA organized roundtables for various purposes, including interagency consultation and cooperation and legislative drafting. The roundtables are a hallmark of the JRGA Program and have led to positive results. JRGA has complete responsibility for organizing the roundtables. They are held at JRGA premises, technical support is provided by JRGA and JRGA staff even prepares and distributes notes from the meetings. During interviews, counterparts and external organizations that have participated in or observed the roundtables noted their value. JRGA supported several activities that raise the profile of the IAs, including the Good Governance Conference in 2012, Annual Anti-Corruption Day events, the public hearing on the Anti-Corruption Strategy, website development, public and media relations training and assistance in managing complaints procedures, among others. IA Annual Reports for 2012 show significant increases in website visitors38 and complaints received. According to the UNDP Seventh Public Opinion Research on Corruption, June 2013, there is growing trust among Serbian citizens in the capability of independent supervisory bodies to fight corruption. The ACA is continually rising on the list of institutions that citizens believe are capable of fighting corruption. 77% of citizens are aware of the ACA and 35% believe the ACA should be the primary force in the fight against corruption. Increased trust in the ACA is based on the belief that the ACA can control financing of political parties, which is widely perceived as one of the generators of corruption in Serbian society. 10% of citizens believe the Ombudsman should lead the fight against corruption (compared to 2% a year earlier). In the mini-survey of JRGA CSO grantees, about seven percent of respondents said there was increased visibility and public awareness of IA activities. Grantees noted SAI for its use of the media to promote its results and findings, a result of the work by JRGA grantee, Center for European Policy, with the SAI. The UNDP study also indicates that the majority of Serbian citizens’ primary source of information about corruption is the media. IA Annual Reports show that IAs continue to conduct trainings for media to ensure accurate reporting. However, IAs and civil society representatives are of the opinion that the media does not have a good understanding of anti-corruption efforts. Conclusions: One of JRGA’s most important contributions to increasing government accountability, reducing corruption and facilitating EU integration, is assistance in preparation of the new Anti￾Corruption Strategy and Action Plan. Without JRGA assistance, the Action Plan may not have been as clear in assigning responsibilities, setting deadlines and establishing indicators, all of which increase the 38 See JRGA PMP Report. 22 prospects that the Action Plan will be implemented. Although other donors have supported drafting of the Strategy and Action Plan, JRGA is a very prominent player. There may be some risk that monitoring implementation as well as specific actions to be taken by JRGA counterparts, is excessively dependent on JRGA and would not be achieved without JRGA intervention. As a result of the Roundtables that facilitate information sharing and cooperation between IAs and MC, between IAs and the NA, and among IAs, the prospect of a continuing practice of regular and periodic communications and increased cooperation is emerging. There is virtually unanimous buy-in from counterparts although they have not yet assumed ownership of a formalized, regular consultations process. Therefore, it is not clear that this practice would be sustainable without JRGA intervention. There are indications that public recognition of IAs has increased. At the same time, there appears to be increasing public awareness of the GoS’ serious anti-corruption efforts and an expectation that the GoS will take action. However, continuing training for the media is necessary to ensure that citizens receive accurate information. Recommendations: • JRGA should support the IAs in connection with implementation of the Anti-Corruption Action Plan. JRGA should also avoid activities that generate the impression that JRGA is only aligned with the GoS/MoJPA and, to the extent possible, highlight the role of the IAs. For example, if and when the Law on the ACA is amended to give the ACA some investigative powers (i.e. investigative hearings), JRGA should advise ACA as to how to present that to the public and eliminate continued confusion and excess expectations about the ACA’s role. Given that the ACA spokesperson is a staff member of the Director’s Office, not the Public Relations Department, facilitating internal coordination will be critical and could be a model for future coordination. • JRGA should try to shift some responsibility for organization of roundtables to IAs and Parliament, as appropriate. JRGA should also consider a change in venue so that not all meetings take place at JRGA, which will dispel the impression that the roundtables are a JRGA product that will ultimately disappear when the Program ends. • JRGA should monitor IA’s public outreach and media activities to ensure that messages are on point and that the quality of outputs remains high. Although educating the media about IA activities in order to obtain improved reporting has been difficult due to the low level of media competence and excessive external control, this is an area that probably merits further JRGA assistance. EVALUATION QUESTION 3: To what extent has the project design been appropriate in achieving intended objectives and results? Findings: JRGA project design merged the Mission’s primary Democracy and Governance (DG) objectives and recognized the need for technical assistance and training to help establish new judicial branch and independent agencies whose creation required both development of capacity and establishment of protocols for inter-institutional coordination. In addition, the flexibility built into the project allowed JRGA to respond easily to structural changes in government and to address targets of opportunity within the scope of the Project’s overall goals. While several donors provide assistance to the IAs, JRGA is the only program that works on IA-MC cooperation. 23 Conclusion: The project design allowed for needs assessment and consideration of previous or ongoing donor assistance. For example, after the 2012 national elections and change in government in Serbia, ACA leveraged its relationship with MoJPA established under Component 1 to assist that institution, which now has primary responsibility for implementation and monitoring of the Anti￾Corruption Strategy and Action Plan. Further, combining the two JRGA components into one program allowed for important synergies as the new MCs would have jurisdiction over adjudicating violations of some of the same laws and regulations that the IAs are responsible for detecting and preventing. As both the MCs and IAs were JRGA counterparts, JRGA could bring these two groups together. Recommendations: The principal project design recommendations during the second half of the JRGA project are to continue to expand the scope of IA-MC cooperation and focus on transition and sustainability. EVALUATION QUESTION 4: Within the current contract scope, what opportunities can be identified to expand support for Serbian judicial reform efforts? Findings: Interviews and documentary sources served to inform the Team’s professional judgment about opportunities for expanded support for Serbian judicial reform efforts. Conclusions: Targets of opportunity within the project scope include outreach to petitioners to facilitate implementation of the LoM amendments; capacity development of the HCC; capacity development of the JA in the further development of Misdemeanor Judge and staff professional training; development of a research and development function at the HCC, focusing initially on creating judicial performance standards; and increased efforts to renovate court facilities, particularly highest priority MCs in buildings shared with Basic Courts. In addition to grants to CSOs to improve the administration of justice, JRGA is also uniquely positioned to promote improvements to the judiciary among the public and to conduct surveys regarding the public’s confidence in the courts. Recommendations: Possible opportunities to expand support for judicial reform include: • Expanding assistance to the MOJPA and the MCs to renovate those facilities identified as highest priority projects by the JRGA facilities inventory, particularly those in buildings where MCs share space with the Basic Courts. • By tracking revenues attributable to MCs, help MOJPA and other donors to justify increased allocation of funds for construction and renovation of all MCs. 39. • Expanding outreach to petitioners to facilitate implementation of the LoM Amendments. • Offer assistance to the chief justice and presiding judges (PJs) to develop and deliver annual state of the judiciary addresses to highlight court system improvements, especially as defined in the National Judicial Strategy and Action Plan; • Conducting annual surveys to measure public confidence in the Misdemeanor Courts; and • Initiate discussions among counterparts regarding the feasibility of developing a research and development function at the HCC to assure long-term sustainability of judicial system reform efforts, focusing initially on the development of judicial performance and case management 39 IPA reportedly has allocated more than three million Euros per year for infrastructure investment. 24 standards, and ultimately on the development of the kinds of mechanisms and institutions deemed critical to the establishment and maintenance of judicial competence in other countries, such as the establishment of a transparent judicial discipline mechanism in which incompetent or corrupt judges could be publicly disciplined or removed. EVALUATION QUESTION 5: To what extent are Component II Government Accountability activities sufficiently focused and strategically targeted toward the broader program goal? Findings: During Year 1, JRGA focused on building relationships with counterparts, providing a variety of direct assistance to the ACA, providing public outreach assistance to the Ombudsman’s Office and building relations between IAs (SAI and CIPIPDP) and MCs. JRGA also approached other bodies (CPE, NA and PPO) to explore expanding the scope of interagency cooperation and reinforce mechanisms for implementation of IA recommendations. In Year 2, JRGA continued to provide direct assistance to the ACA (primarily related to political party and campaign financing and the 2012 election) and support for interagency cooperation, which was expanded to include the NA.40 JRGA also provided expert advice to CIPIPDP related to data protection (a new competence for the agency) and drafting of a whistleblower protection law; customer relations training to help Ombudsman Office staff deal with increased complaints; assistance to the CPE through the development of a new website and outreach materials to increase visibility; and support to the newly established Public Procurement Office and Association of Public Procurement Professionals. Importantly, during Year 2 JRGA began to focus on the preparation of the new Anti-Corruption Strategy. In Year 3, JRGA will continue activities begun or planned in Year 2. This will include strengthening IA reporting capacity; working closely with the revitalized ACA; and providing support to other counterparts on policy issues, such as implementation of the Anti￾Corruption Strategy and Action Plan, whistleblower and lobbying regulation, open government and other topics. The Year 3 Annual Workplan discusses the possibility of JRGA conducting an inventory of the facilities of IAs and other counterparts similar to its practice with the Misdemeanor Courts discussed above.41 Good governance and the fight against corruption are key issues for the GoS and donors. There are numerous activities included in the Anti-Corruption Action Plan. Despite the importance of these issues, donor coordination is conducted through an informal Anti-Corruption Donor Coordination Group co-chaired by UNDP and OSCE. Representatives of donor organizations mentioned the following planned interventions: the EU is currently conducting a needs assessment for its IPA II program and is also planning Twinning Projects for the ACA and PPO; UNDP plans to resume support for IA-NA cooperation in 2014/2015 and will continue to support the SAI and PPO. Conclusions: JRGA activities have been focused and strategically targeted toward the goal of openness and accountability of government operations – both for the sake of the people of Serbia, and to help Serbia move closer to EU accession and participation in other Euro-Atlantic institutions. JRGA followed the tasks as outlined in the RFP and conducted activities as set forth in the approved Annual Workplans and assistance largely focused on the counterparts listed in the RFP—ACA, CIPIPDP , SAI, Ombudsman, courts and civil society organizations. Assistance to the IAs focused on building institutional capacity and visibility and improving cooperation and coordination as originally planned. To the extent that JRGA 40 Work with the Parliament is intended to dovetail with support provided by OSCE’s USAID-funded program and other donor support. 41 This inventory is included in the Anti-Corruption Action Plan. 25 provided assistance to other institutions such as Parliament, CPE and the PPO, it was targeted at the broad program goal. JRGA leveraged its relationship with the Misdemeanor Courts to increase cooperation between the IAs and MCs to improve the court’s action in misdemeanor cases initiated by IAs. In light of the MoJPA’s increased role in anti-corruption efforts under the new Anti-Corruption Strategy and Action Plan, JRGA is building on the strong relationship established under Component I. JRGA’s planned activities for Year 3 build on previous years’ accomplishments, including IA – NA cooperation. This increased level of activity could take the place of assistance previously provided under the OSCE Good Governance Project that is scheduled to close at the end of 2013. The focus on the revitalized ACA provides an opportunity to build on the work of short-term experts to ensure that the ACA implements recommendations as appropriate. While policy level assistance includes areas where JRGA is already involved (e.g. implementation of the Anti-Corruption Action Plan, drafting whistleblower law and support for Open Government Partnership), it also includes new activities, such as support for preparation of a lobbying law and leaves open assistance on other issues. Further, planned activities for Year 3 include preparation of IA facilities inventories, similar to the inventories JRGA prepared for the Misdemeanor Courts, which is an activity discussed in the Anti-Corruption Action Plan. While this has the potential to strengthen the IAs, it is not clear that it is within the Project’s mandate. Recommendations: • Given that the JRGA project is at midterm, it should focus on completing ongoing activities. Project staff should continuously follow up on IA implementation of recommendations provided by JRGA short-term experts. This should include assistance drafting and advocating for adoption of amendments to the Law on the Anti-Corruption Agency to clarify provisions related to conflicts of interest and responsibilities of ACA; finalizing Whistleblower legislation, drafting a lobbying law; and supporting the MoJPA’s commitment to the Open Government Partnership. • JRGA should only initiate new activities to the extent that this will not undermine the Project’s ability to complete ongoing activities. The same is true for activities that have started but were outside JRGA’s original mandate such as assistance in implementation of the new Public Procurement Law. JRGA should limit the use of international experts and only do so to the extent that Project staff or local experts can provide follow up. • With respect to expanding activities to local self-government units (including adapting the template for integrity plans as was done for courts), JRGA should assist the ACA with this, but not do the work for them. JRGA should consider whether it would be most efficient to do this in locations where JRGA has PCs. • With respect to the IA facilities inventory, in order to build capacity JRGA should advise the appropriate bodies on conducting the facilities inventory based on the experience with the MCs rather than conducting the inventory itself. • The Mission should encourage more formal coordination among donors, particularly with respect to Component 2, Government Accountability. This cooperation could be launched through a JRGA supported high-level donor coordination meeting that includes Serbian counterparts to discuss assistance in implementing the Anti- Corruption Strategy and Action Plan. As the ACA is responsible for monitoring implementation and the MoJPA is tasked with coordinating GoS activities, these two bodies could co-chair the meeting. 26 ANNEX I: EVALUATION STATEMENT OF WORK SECTION C – DESCRIPTION / SPECIFICATIONS/STATEMENT OF WORK C.1 PURPOSE The contractor will conduct a performance evaluation of the Judicial Reform and Government Accountability (JRGA) program implemented by the National Center for State Courts (NCSC) under contract #AID-169-C-11-00001. The purpose of this performance evaluation is to conduct a full, evidence-based and independent review of USAID/Serbia’s JRGA Project, and to produce a report that provides a qualitative as well as quantitative evaluation of project activities. This will include the identification of assistance gaps and opportunities for potential new activities that could help the project accomplish its objectives and identify areas where progress is lagging or insufficient, and recommend course changes as appropriate as well as to show what has JRGA accomplished to date. C.2 BACKGROUND Currently, Serbia’s general jurisdiction courts include the Supreme Court of Cassation (replacing the Supreme Court as the highest court in the system), four courts of appeal, 26 Higher Courts, and 34 Basic Courts with outlying units. Serbia’s specialized jurisdiction courts now consist of 16 Commercial Courts, the High Commercial Court, and the Administrative Court, which is headquartered in Belgrade. The Administrative Court has chambers in Kragujevac, Nis, and Novi Sad. Appeals from the Administrative Court and the High Commercial Court are heard by the Supreme Court of Cassation. As of January 1, 2010, Serbia’s misdemeanor court system was restructured and became part of the judicial branch. Prior to 2010, these courts were offices within the executive branch. These courts do not hear any civil disputes, but they have jurisdiction over an extremely varied set of administrative and less serious criminal cases. Through amendments to the Law on Judges, all Misdemeanor Judges became permanent judges with life tenure in December 2012. There are 45 Misdemeanor Courts and one High Misdemeanor Court. Pending draft legislation will change the name of the High Misdemeanor Court to “Appellate Misdemeanor Court.” Thus far, JRGA’s assistance to Serbia’s judicial reform efforts has been limited to the Misdemeanor Courts, High Misdemeanor Court and the Administrative Court in partnership with the Ministry of Justice. JRGA’s good governance component includes providing technical assistance to Serbia’s five independent agencies. The Anti-Corruption Agency, formally known as the Agency for the Fight Against Corruption, was established by law in late 2008, and its first director was appointed in mid-2009. Parliament removed the director in November 2012 and appointed a new director shortly thereafter. The Commissioner for Information of Public Importance and Personal Data Protection was established by law in November 2004 and the Commissioner was appointed the following month. The State Audit Institution, sometimes referred to as the Supreme Audit Institution, was established by Articles 92 and 96 of the Constitution and an implementing law passed in 2005. The director was not appointed until 2007. He was reappointed for a new five year term in 2012. The Republic Ombudsman’s Office, formally known as the Republic Defender of Citizens, was established by Articles 107 and 38 of the Constitution and an enabling law passed in 2005. The Republic Ombudsman was not appointed until 2007. The office has jurisdiction over specific types of complaints although it collects 27 complaints on an ever wider range of issues, including judicial operations. The Office of the Commissioner for Protection of Equality is a national human rights institution and an equality body, established by a Serbian 2009 Law on Prohibition of Discrimination and in accordance with relevant European Union directives. It is principally tasked with prevention of discrimination and protection of equality in Serbia. The Commissioner was first appointed in May 2010. JRGA OBJECTIVES The overall objective of JRGA is to implement activities and tasks that improve the efficiency and independence of Serbian courts and Independent Agencies, promote the transparency and accountability of government institutions, improve substantive and procedural consistency, reduce opportunities for corruption and the public perception of corruption within relevant institutions, and promote gender equity. a) Judicial Reform Component Tasks and Results As the name of the project implies, its two components are judicial reform and government accountability. Under the judicial reform component, JRGA’s overall objective is to assist Serbia to make the administration of justice more efficient, transparent and responsive to the needs of their users, and to increase public awareness about the improvements. NCSC is required to accomplish the following tasks under this component: Establish baselines for indicators in the Performance Monitoring Plan(PMP); Assist the High Court Council, the judiciary and any other relevant bodies or actors to make open hearings of all the Administrative, Misdemeanor and High Misdemeanor Courts routine, dignified, accessible, safe, efficient, and promote fairness for all parties; 3. Assist the High Court Council, the judiciary and any other relevant bodies or actors reduce backlogs and improve case processing times and procedures in all the Administrative, Misdemeanor and High Misdemeanor Courts; 4. Assist relevant official bodies and actors to increase the percentage of assessed court costs and fines from all of the Misdemeanor and High Misdemeanor Courts which are collected; 5. Assist relevant official bodies and actors to increase the physical safety of victims of family and domestic cases when they appear in Misdemeanor and High Misdemeanor Court; 6. Assist relevant official bodies and actors to improve the coordination of cases involving related issues and/or parties, including those pertaining to family violence; 7. Assist relevant official bodies and actors to increase the competency of judges in all the Misdemeanor and High Misdemeanor Courts; 8. Assist relevant official bodies and actors to develop practical and user-friendly information for court users about the operations of all the Administrative, Misdemeanor and High Misdemeanor Courts and to inform the public about the availability of the information; 28 9. Assist relevant official bodies and actors to improve the reputation of all the Misdemeanor and High Misdemeanor Courts among court users and the public; and 10. Encourage reform of Serbia’s judicial system by assisting local organizations supporting and advocating for improvements in the rule of law, the judicial sector, and the culture of personal responsibility/respect for the law through the provision of: – Grants under Contract; – Comparative information on substantive issues; and – Technical assistance on the institutional development of the local civil society organizations. The expected Results are as follows: (1) Open hearings of all the Misdemeanor, High Misdemeanor and Administrative courts are the rule rather than the exception, and are dignified, fair, accessible and safe for all participants, including victims of family and domestic violence; (2) Cases of all types in all the Misdemeanor, High Misdemeanor and Administrative Courts are resolved more quickly and backlogged cases are resolved within established standards for timely case processing; (3) Court costs and fines from all types of cases in all the Misdemeanor and High Misdemeanor Courts are paid on time; (4) Courts implement coordinated policies for the handling of cases involving related issues and/or parties, including those pertaining to family violence; (5) Judges from all the Misdemeanor and High Misdemeanor Courts show improved performance against Serbian government- established competency standards; (6) Court users can easily find the information they need about the operations of the Administrative, Misdemeanor and High Misdemeanor Courts; (7) The Serbian public receives more positive information about the operations of all the Misdemeanor and High Misdemeanor Courts; and (8) There is more non-governmental support for improvements in the rule of law, the judicial sector, and/or to promote a culture of respect for compliance with the law. b) Government Accountability Component Tasks and Results The objective of the government accountability component is to assist Serbia to strengthen the capacity of Serbia’s Independent Agencies and civil society to promote open, accountable and efficient government; to fight corruption and poor administrative practices; and to generate public demand for and participation in good governance. Under this component, NCSC is required to complete the following tasks: 1. Establish baselines for indicators in the Performance Monitoring Plan; 29 2. Assist the Anti-Corruption Agency and its Board to: a) enforce conflict of interest and property registration provisions in the Anti-Corruption Agency Act; b) detect violations of those provisions of the Act; c) initiate and conduct proceedings to establish if violations of the Act have occurred; and d) order measures under Article 51 of the Act as it read on January 2, 2010, when appropriate; 3. Assist the Anti-Corruption Agency and its Board, and any successor organization, to develop and implement investigative, monitoring and enforcement procedures governing the financing of political parties and election campaigns by the responsible body(ies); 4. Assist Independent Agencies to achieve more timely and stringent enforcement of their decisions and implementation of their recommendations by other government agencies, and to track the relevant processes; 5. Assist relevant official bodies and actors to develop and implement procedures to increase coordination between courts and relevant government agencies that reduce the processing times of selected administrative provisions and procedures; 6. Assist with improving ability of Independent Agencies to work with each other to monitor and improve administrative practices and other government operations; and 7. Encourage improvements in governance and official integrity, and to reduce corruption, to monitor political party financing (including financing of local and national elections), and to promote compliance with EU norms, through the provision of: – Grants under Contract; – Comparative information on substantive issues; and – Technical assistance on the institutional development of the local civil society organizations. The expected Results are as follows: (1) The Anti-Corruption Agency effectively executes its responsibilities relating to conflicts of interest prevention, detection and sanctioning; (2) The Anti-Corruption Agency and its Board and any successor organizations effectively regulate the financing of political parties, and election campaigns; (3) The recommendations and decisions made by Independent Agencies are implemented and/or enforced by other government agencies; (4) Administrative agencies and relevant courts process and enforce laws and rules efficiently; (5) Individuals and organizations use information from or processes supervised by the Independent Agencies to monitor and improve political party financing, compliance with conflict of interest provisions, administrative practices and/or other government operations; and 30 (6) Individuals and organizations work to improve governance and official integrity, reduce corruption, monitor political party financing (including the financing of local and national elections), and to promote compliance with EU norms. c) Relationship to the Mission Strategy The overall goal of USAID assistance in Serbia is to support Serbia in its vision to be democratic, prosperous, and moving toward Euro-Atlantic integration. Through focused investments, the Mission will help strengthen democratic structures. Through our economic growth efforts toward a more competitive market economy, we will advance economic governance and inclusive, more sustainable growth. When it was designed, JRGA was designated as one of the core programs under the USAID/Serbia Strategic Objective 2.11 “Risk of Political Instability Reduced” within the Мission’s strategy for 2006 to 2010. In 2010, that strategy was amended to cover the period of 2011 to 2015, although the Мission’s overall strategic objective of supporting Serbia in its vision to be “democratic, prosperous and moving toward Euro-Atlantic Integration” remained the same. Assistance Objective 2 “Democratic structures in Serbia strengthened” (in the Amended Country Strategy 2011 - 2015) pursues two key elements: improving government operations – “Intermediate Result 2.1 Government Operations Improved”; and increasing civil society engagement in public life – Intermediate Result 2.2 Civil Society Engagement in Public Life Increased”. JRGA program supports Assistance Objective 2, Intermediate Result 2.1 and Subintermediate Result 2.1.1 “Efficient, Transparent and Accountable Provision of Government Services Enhanced”, Sub-intermediate Result 2.1.2 “ Checks and Balances Strengthened and Sub-intermediate Result 2.2.2 “Government Accountability and Responsiveness to Citizens Improved”. C.4 OBJECTIVE The objective of this performance evaluation is to conduct a full, evidence-based and independent review of USAID/Serbia’s JRGA Project, and to produce a report that provides a qualitative as well as quantitative evaluation of project activities. This will include the identification of assistance gaps and opportunities for potential new activities that could help the project accomplish its objectives and identify areas where progress is lagging or insufficient, and recommend course changes as appropriate as well as to show what has JRGA accomplished to date. The audience for this evaluation will be USAID/Serbia and, more broadly, U.S. Embassy Belgrade, USAID Washington, NCSC, and the broader democracy promotion community. C.5 DETAILED STATEMENT OF WORK a. Evaluation Questions This performance evaluation will address, at a minimum, the following questions: (1) To what extent is the program on track (i.e., meeting established milestones) to accomplish the objectives and results expected under both components? (2) What emerging impact has the project had on strengthening Misdemeanor Courts and independent agencies in Serbia? 31 (3) To what extent has the project design been appropriate in achieving intended objectives and results? (4) Within the current contract scope, what opportunities can be identified to expand support for Serbian judicial reform efforts? (5) To what extent are Component II Governmental Accountability activities sufficiently focused and strategically targeted toward the broader program goal? b. Tasks The specific tasks to be undertaken by the evaluation team to prepare the deliverables include: Review of the relevant portions of the project’s contract (original and modified.) Review of all relevant project reports, assessments, annual and life of project work plans. Review of the project’s grants documentation. Review of the project’s annual and life of project performance management plans (PMP.) Review of USAID/Serbia’s Country Development Cooperation Strategies (CDCS). Meetings with NCSC headquarters in Alexandria, Virginia (by phone.) Meetings with USAID/Serbia’s Mission Director, Democracy and Governance Office Director, program officer, project COR and Alternate COR. Meetings with NCSC staff in Belgrade. Meeting with appropriate officials of the Government of Serbia’s Ministry of Justice, Misdemeanor Courts, High Misdemeanor Court, Administrative Court, and High Court Council. Meetings with other donors, including the European Union Delegation, Untied Nation Development Programme (UNDP), World Bank, Organization for Security and Cooperation in Europe (OSCE), Embassies of U.K, Netherlands, Sweden, and others. Meetings with appropriate officials of the Anti-Corruption Agency, State Audit Institution, Commissioner for the Protection of Public Information and Private Data, Ombudsman, and Commissioner for the Protection of Equality. To support the team’s initial literature review, USAID/ Serbia will provide electronic copies to the contractor of all documents to be reviewed (listed above) the day of the award. Special attention during the review should be paid to how well the project fits into the mission’s current CDCS. All findings, conclusions and recommendations must be supported by evidence. The evaluators are encouraged to be as specific as possible in their recommendations, so as to best inform the direction of the program in the remainder of implementation. C.6 EVALUATION DESIGN AND METHODOLOGY Methodology The evaluation team will first complete a desk study that will be used to establish an understanding of JRGA’s activities and environment before arrival in Serbia. Based on this understanding, the evaluation team will prepare a work plan that will present to the mission at the evaluation in-brief. The methodology will include a mix of tools appropriate to the evaluation’s research questions. 32 Key informant interviews are expected to be a primary data source for this evaluation. This is the most appropriate method to obtain relevant data. All findings and conclusions/qualitative analysis will be supported by quantitative data, when available, and/or representative examples. The Evaluation Team Leader will provide draft of all interview protocols to USAID prior to starting interviews. USAID reserves the right to approve/suggest additional questions. The evaluation will be supported by relevant data and information gathered from meetings with program partners and all the relevant stakeholders. The evaluation team will clearly demonstrate links between the recommendations/conclusions and data available. In addition to these requirements, the offeror will provide innovative approaches to this evaluation. This additional methodology will be included in the Scope of Work (SOW). C.8 REPORTS The evaluation team is required to submit all records from the evaluation (e.g. focus groups transcripts) and all quantitative data in an organized fashion and fully documented for use by those not familiar with the project or evaluation. The evaluation report and summaries must be submitted to the evaluation COR who is responsible of uploading of the report up to Development Experience Clearinghouse (DEC) within three months of completion by the evaluation COR at http://dec.usaid.gov. Agency criteria to ensure quality in the evaluation final report are presented in Attachment I. The contractor will use all of these criteria in the preparation of the final report. C.9 EVALUATION MANAGEMENT a) Background Review and Familiarization The Evaluation team will become familiar with NCSC’s work, which includes reviewing: the Contract and all Modifications; NCSC monthly updates, quarterly and annual reports, NCSC’s work plans and Performance Monitoring Plans. The aforementioned material will be made available to the evaluation team members on the day of the award. Additional information will be made available to the team after arrival in country. USAID prefers to use Google Drive for the transfer of these project-related documents. b) In-Country Research and Evaluation The evaluation team will commence work in Serbia on or about October 13, 2013. An itinerary for the evaluation, including meetings and field visits, will be presented to the Mission upon arrival. The evaluation team will also meet with other donors, implementers and key government partners, as set forth in greater detail in the Tasks listed on page 5. The Evaluation Team will prepare the draft report while in Belgrade. The evaluation team will provide out-brief to the Mission on key findings. The Draft Report and Out-brief will be completed in 20 days from the inception of in-country research and prior to the team’s departure from Serbia. The Draft Report will be submitted to the evaluation COR for review. c) Follow-up and Final Report 33 Upon returning from Serbia, the evaluation team will receive comments on the Draft Report from USAID no later than 6 working days after Draft Report submission. The team will then incorporate these comments and provide a Final Draft Report. USAID will provide final comments to the Evaluation Team upon which the team will then submit a completed Final Report to USAID. Up to 13 work days have been allocated for this Final Report writing time period, after returning from Serbia. The Final Report must be submitted and approved by USAID Serbia by December 31, 2013. The Final Report must be submitted to the COR. STATEMENT OF WORK – ATTACHMENT 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Criteria to Ensure the Quality of the Evaluation Report The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not and why. Evaluation reports will address all evaluation questions included in the scope of work. The evaluation report will include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. Evaluation methodology must be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. Evaluation findings will assess outcomes and impact on males and females. Limitations to the evaluation will be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluation findings will be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. Sources of information have to be properly identified and listed in an annex. Recommendations have to be supported by a specific set of findings. Recommendations must be action-oriented, practical and specific, with defined responsibility for the action. 34 ANNEX II: EVALUATION METHODS AND LIMITATIONS The Evaluation Team initially reviewed Background Materials provided by the Mission, including Laws42, PMPs 43 , Work Plans 44 , JRGA Monthly, Quarterly and Annual Reports 45 , Partner Court Selection Report 46 , the original JRGA RFP 47 , the JRGA Round 3 grantee RFA48 , and the Round 3 Grants Summary49. These were supplemented during the course of the Team’s onsite visit with documents provided by JRGA, including Year 3 1st Quarterly Report (September 2013), and the JRGA Year 3 Work Plan (June 2013), as well as documents collected from interviewees and other sources, such as IA Annual Reports and independent studies.50 In order to provide meaningful answers to the Mission’s five questions, the Evaluation Team decided on a four-step process: (1) Quantitative and qualitative data collected during the evaluation would be organized according to expected results under each component, consistent with JRGA Reports and PMPs; (2) Based on an analysis of the collected data, we would draw findings regarding progress toward each result and JRGA’s overall objectives; (3) We would then draw conclusions based on those findings that provided direct or at least inferable answers to the five overarching Evaluation Questions; and (4) Where logical recommendations flowed from that analysis, including whether to continue or to change course on JRGA activities, we would propose activity-specific recommendations. Following initial consultations with Mission representatives and JRGA staff, we identified all direct and indirect beneficiaries, external sources and grantees who were likely to provide the most insight into the achievements of the project to date, and began to schedule appointments with representatives of each of the principal direct and indirect beneficiaries under each Component. Judicial Reform Component interviewees initially identified included direct beneficiaries Partner Misdemeanor Courts, HMC, AC and JA; Indirect Beneficiaries SCC, HCC, ROL grantees including JAS and APPP, MOJPA and NA Secretariat; and external organizations SPP, EU, UNDP, World Bank, OSCE and DOJ and Political Officers at USAID . (A complete list of interviewees appears below, Annex IV-B). In order to efficiently conduct interviews for the Judicial Reform Component, JRGA invited the Team to attend the Misdemeanor Court Judges Association to be held October 23-25 in Kopaonik, as virtually every Misdemeanor judge in Serbia as well as other key counterparts were scheduled to be there, and the focus of the meeting was to be on key activities being supported by JRGA, including implementation of the Law on Misdemeanors, best practices of Misdemeanor Courts and development of the MCCMS. The Evaluation Team Leader attended the AMJ Meeting and conducted meetings with APJ of the High Misdemeanor Court and APJs of two Partner Courts. In addition, the Team prepared a questionnaire that was distributed to all Conference attendees (Annex III-B). The Team also conducted site visits to four Partner Courts (Belgrade, Novi Sad, Prokupje and Zrenjanin). 42 The Law on Misdemeanors; and Draft Amendments to the Law on Misdemeanors, Law on Courts, Law on Judges,and Law on Public Prosecutor; and the Action Plan for Implementation of the National Judicial Strategy 43 Original PMP Plan (December 2011) and the Year I Results and Year 2 Targets Report (April 2013) 44 Life of Project Work Plan (August 2011), and Year 1 (June 2011) and Year 2 (June 2012) Work Plans 45 Monthly (November 2011 – August 2013), Quarterly (Year 1 Quarters 1,2 & 4 and Year 2 Quarters 1-3) and Annual (Year 1), including Annexes. 46 September 2011 47 September 2010 48 September 2012 49 June 2013 50 See Annex IV-A for a complete list of documents reviewed. 35 Governance and Accountability Component interviewees initially identified included direct beneficiaries, the ACA, CPE, SAI, Ombudsman and the CIPIPDP, as well as indirect beneficiaries the PPO, MOJPA and NA Secretariat and AC Grantees including Transparency Serbia, European Policy Center, Center for Research, Transparency and Accountability, Eutopia/Pistaljka and YUCOM; and external beneficiaries EU, UNDP, OSCE and DOJ and Political Officers at USAID . A survey of all JRGA grantees was also prepared and disseminated (see Annex III-C). The survey was designed to obtain information for purposes of assessing the grantees’ perspectives on the impact of their activities as well as their perspectives on JRGA’s overall results and impact. Semi-structured interviews questions drafted in advance of our onsite work 51 were revised on an ongoing basis to tailor the scope of the interview to the responsibilities and activities of the particular agency as well as to reflect information learned during the course of the evaluation. In addition, potential interviewees that were identified during the evaluation were incorporated into our schedule to the extent possible. We endeavored in selecting interviewees and in conducting interviews to minimize the level of selection bias due to the fact that many individuals and agencies as well as grantees were identified by JRGA staff, including Partner Courts. In order to mitigate bias, we conducted in depth interviews to probe general responses more thoroughly and verify respondent familiarity with the material discussed. In-depth interviews with external sources, including other international organizations whose work complements JRGA activities (such as EU, UNDP, and OSCE) also served to mitigate bias. With regards to gender equity, more than half of the interviewees were female, including individuals in key positions at the MC and Administrative Court and IAs. Following completion of interviews and review of as much documentation as was available, the Team prepared an Annotated Outline (preliminary analysis) of findings, conclusions and recommendations, which was submitted to USAID Serbia at the evaluation out-brief on November 5 for feedback. USAID provided comments to Social Impact on 13 November, SI submitted the first draft of the Final Report to USAID on 22 November 2013; USAID staff provided feedback on the Final Draft to the Team on December 3 and SI responded to those comments on December 10; that version was then shared with JRGA and the Mission’s and JRGA’s last set of comments was provided to SI on December___, after which SI submitted the Final report to USAID on December___. The report will be published following formal USAID approval of the final version. 51 See Annex III-A. 36 ANNEX III: DATA COLLECTION INSTRUMENTS A. EXAMPLES OF SEMI-STRUCTURED INTERVIEW QUESTIONS Component 1 Implementer Direct Participants Indirect Beneficiaries/Externals Law on Misdemeanors What was JRGA’s role in establishing the working group? What substantive assistance did JRGA provide in drafting the law? What role will JRGA take in drafting by￾laws? In drafting the Omnibus Law on changes to Rules of Court Procedure? Will recommended exceptions be implemented? Mechanism in place for obtaining feedback and revisions to training on Misdemeanor Law? Plans for training petitioners? What is timetable for preparing LoM benchbook? What assistance did JRGA provide to the working group? Why were certain provisions not included in the final law? What is procedure for regulating exceptions? What are plans for drafting regulations? What was your role in designing the four part training on Law on Misdemeanors? Will Partner Courts be involved in training for petitioners? Partner Courts/Facility Upgrades How was the architect selected? Has MoJ’s How are facility upgrades improving hearings? How are facility upgrades improving hearings? Is public aware of upgrades 37 material/financial contributions be consistent with JRGA expectations? Are other no GoS resources available to support these activities? From who? Will MoJ have funds to maintain facilities? Plans to continue renovations post JRGA? to court facilities? Attendance and media coverage of partner court presentations and outreach activities? Effect on public perception of courts? What was JRGA’s role in raising public awareness? Registry of Sanctions and Unpaid Fines/Enforcement What are relevant policy issues? Relevant policy issues? What additional assistance is required to improve enforcement? Public reaction to Voluntary Enforcement Initiative? Protection for Victims of Family and Domestic Violence/Improved Case Coordination Will inter-service cooperation continue without JRGA support? Gender balance between APP members actively participating in project? What non-JRGA support is available for assistance in this area? Were the training examples/simulations conducted by US experts at training relevant for Serbia? Are local organizations prepared to continue training? Is assistance needed? Lessons learned from pilot inter-service cooperation project in Zrenjanin? Differences in acceptance in urban/rural communities? Differences in acceptance depending on gender? Is Handbook for Victims readily available? Are victims acting on the basis of advice in the Handbook? 38 Non LoM Training Will the training on Law on Harassment prepared for AC be delivered to other courts? Was ToT conducted by JRGA useful? Had you received previous ToT? Are you prepared to conduct ToT for future trainers? Has training on Law on Public Procurement equipped you to adjudicate cases under new law? Specific example of improved capacity/engagement for advocacy? Specific example of effective advocacy campaign? Were there any specific factors that may have limited the effect of an awareness campaign? Component 2 Implementer Direct Beneficiary—IA Staff MPs Indirect/External (media, CSO, international organizations and donors) ACA Capacity Building (gap analysis; strategic planning; training; develop/implement standard procedures; staff training; awareness and training for public officials; managing and enforcing COIs; external complaint mechanism) Q1, 3 Despite difficulties in working with ACA leadership, JRGA had significant accomplishments in Yr. 1 with respect to asset disclosures, conflict of interests and political financing. The Yr. 1 Annual Report noted that the heads and staff of these departments didn’t have authority to make changes. To what extent is this still the case? Do they now have capacity and authority to continue process? Finalizing the government’s AC strategy was one How would you characterize your working relationship with JRGA? To what extent were you satisfied with the assistance provided to the ACA and working groups in preparation of the strategic plan? Is there anything that was particularly helpful or not helpful? Was the input provided by JRGA’s short term experts (Tom Devine, Mick Symons, Robert Vaughn, etc) useful? What steps have been taken to What was your opinion of the Good Governance Conference?—media, UNDP Were the comments you submitted after the Good Governance Conference adequately addressed? Was there any follow-up by the ACA? Have you reviewed the Guide for New Officials and if so, was the information clear and useful? –govt officials To what extent has UNDP cooperated with JRGA? How would you characterize your working relationship with JRGA? Has the assistance provided to the ACA been valuable? 39 important milestone. What are the next important milestones and do you think the Project is on target to achieve them? Do you foresee any obstacles at this time and if so, what are they? Are there gaps that were identified in the gap analysis that have not been addressed in the strategic plan that could be addressed during the remainder of the project? If so, what are they? Is the ACA coordination committee effective? How has JRGA’s cooperation with the USAID BEP projects, OSCE and UNDP helped in the work with Parliament? In your opinion, is the training delivered with support from JRGA sustainable? Why, why not? institutionalize best practices introduced by JRGA? What are the ACA’s plans to continue the training delivered with JRGA assistance? Was the training effective and appropriate? Is the Complaints Department functioning effectively? What additional assistance, if any, is needed? What assistance, if any, would benefit ACA Board? Q2, 5 As a result of the ACA’s recent accomplishment’s (e.g. finalization of ac strategy), are Serbian officials and citizens more committed to Are citizens satisfied with the ACA’s new complaints procedures? Do they believe the process will continue effectively? 40 and confident about ac measures? ACA regulation of political party and election financing Q 1,3 Do you think the anomalies and omissions discovered with the assistance of JRGA STTA Djorde Vukovic were the result of failures in training or something else? What additional assistance do you plan to provide to the ACA’s political finance department? How did JRGA contribute to finalization of Handbook for Political Finance Monitors? Is the Handbook useful? Would you suggest any revisions? How were the trainers selected? Was the study tour effective? What did you learn on the study tour; can you give an example of something that was applied/adapted to the Serbian elections? Did the ACA - JRGA seminars for editors and journalists contribute to the increased coverage of public finance monitoring in 2012? Q 2,5 How would you characterize the ACA’s attitude to political party/election financing? What is the status of legislative reform and amendments? Are the IA leaders committed to the reform? What were the lessons learned from the previous election? What changes to the process to you foresee as a result of the lessons learned? Do you plan to continue the publication of the Handbook for Political Finance Monitors and the training developed with JRGA support? To what extent has civil society been involved in the legislative reform to date? Enforce IA Decisions (develop legal/regulatory road map; develop/implement operational procedures (including for referrals to court); train IA and court personnel; interagency 41 cooperation; legislative reform) Q 1,3 Were the IAs cooperative in the review and analysis of procedures for enforcing decisions? Do you see enhanced cooperation between IAs? Are there particular obstacles? Can you provide an example? How would you characterize cooperation between IAs and courts in developing procedures to facilitate case referrals? Are there particular obstacles? If so, can you provide an example? Was JRGA assistance helpful in developing procedures, guidelines, mechanisms to facilitate case referrals? Provide an example of a new procedure that has been particularly effective. In your experience, has there been an improvement in the case referral system that has led to more efficient case processing? Can you give an example? Q3,5 Do you expect the improved cooperation to continue w/o JRGA involvement? Will you be able to use the skills and knowledge developed with JRGA supported training in other activities? Examples? 42 Improve IA Cooperation with other IAs and government bodies (develop mechanisms for exchange of information; develop procedure to improve public access to information) Q 1,3 Were the one-on￾one interviews an effective mechanism for gathering information? What additional assistance will JRGA provide to the IA￾CSO working group? Are the bi-annual workshops effective? Were the issues identified in your personal interviews with JRGA reflected in the inter-agency interface process? Can you give a specific example? Did JRGA provide the necessary resources to achieve meaningful cooperation? Examples? Is the IA-CSO working group effective? What was JRGA’s role in assisting that working group? How would you characterize relations between the IAs and CSOs/citizens? Have they improved as a result of JRGA supported activities Q2,5 Is the JRGA monitoring effective beyond just producing information for project? How will this monitoring be continued? Grants (award process; training in grant implementation, activities under grants and sustainability) 43 Q1,3 How did you prioritize themes for the grants program? As a result of the training provided by the expert, is the Grants Officer able to manage the grants program? What training was provided? To what extent did any of the policy debates influence party platforms? Has the training provided by JRGA increased your organization’s capacity? Have you used these skills in other projects? Examples? How has this affected your political career? What are some examples of how debated policy issues influenced your party’s platform? Is the media more effective in monitoring the government as a result of the JRGA program? Examples? Q2,5 How does the Grants Officer foresee using these skills in the future? Is the media better equipped to monitor government activities in the future? 44 B. ASSOCIATION OF MISDEMEANOR JUDGES SURVEY Attendees at the Misdemeanor Judges Association Annual Meeting in Kopaonik on 23-25 October 2013 were asked to rate the impact that each of five JRGA-sponsored activities has had on their particular court. The results of that survey appear below. The chart that follows summarizes the responses of the 16 Presiding Judges in attendance. The chart indicates that this most knowledgeable segment of the MC judicial community rated all five JRGA activities—increasing open hearings; backlog reduction; protection of victims of domestic violence; legal competency; and increased public confidence in the courts—between 3.9 and 4.4 on a scale of 5. That is, the average rating of the impact of these five activities on sixteen courts (including all Partner Courts) was in the 80%+ range, demonstrating these judges subjective assessments of the impacts of JRGA on Misdemeanor Courts. 3.6 3.7 3.8 3.9 4 4.1 4.2 4.3 4.4 4.5 Open Hearings Backlog Fam. Violence Jud Competence Pub.Confidence JRGA Results Assessment 45 1. Questionnaire (English)52 Social Impact (SI) is conducting a mid-term evaluation of the JRGA Project for USAID. This questionnaire is designed to gather feedback on JRGA’s principal goals from all misdemeanor courts. Please complete the form at your earliest convenience and turn in before leaving the Conference. A representative of SI will be here throughout the Conference to answer any questions you may have about the evaluation. Thank you for your cooperation. Please indicate your position in the court system: President Judge_________ Judge__________ Court Manager_____________ Other (Please specify):________________________________________ Please rate the following impacts of JRGA project activities on your court from 1 (none) to 5 (substantially); or check NA if you have no information on this subject. Add comments if you wish to explain your response: 1. To what extent, if at all, has the number of open hearings increased in your court as a result of training or other technical assistance provided by JRGA? ____1.____2._____3. ____4. ____5. ____NA Comments: _________________________________________________ ___________________________________________________________ 2. To what extent, if at all, has your court been able to reduce its backlog and/or improve case processing time as a result of training or technical assistance provided by JRGA? _____1. ____2.____3. ___4. ____5.____NA Comments:___________________________________________________________ _____________________________________________________________________ 3. To what extent, if at all, has there been an increase in the level of protection afforded to victims, parties or litigants in family violence cases as a result of technical assistance or training provided by JRGA? _____1. ____2.____3. ___4. ____5.____NA 52 See below, p.___ for Serbian version of questionnaire 46 Comments:___________________________________________________________ _____________________________________________________________________ 4. To what extent, if at all, has your level of legal knowledge about cases within the jurisdiction of the misdemeanor court increased as a result of technical assistance or training provided by JRGA? _____1. ____2.____3. ___4. ____5.____NA Comments:___________________________________________________________ _____________________________________________________________________ 5. To what extent has the public’s level of confidence in the court system increased as a result of public information or outreach program assistance or training provided by JRGA? _____1. ____2.____3. ___4. ____5.____NA Comments:___________________________________________________________ _____________________________________________________________________ 6. The following is a list of trainings conducted with the support of JRGA. Please check any courses you have attended: Customs Regulation______ Tax Regulations_________ Application of regulations in the area of fight against corruption______ 47 2. Questionnaire Results 16 President Judges (out of est. 45) = 33% 67 Judges (out of est. 300) = 25% 11 Other (out of est. 40) = 25% Total = 94 out of est. 385 = ca. 25% Results of Association of Misdemeanor Judges Survey Survey Respondent Title Survey Question Likert Scale Value TOTAL* Overall Assessment 1 2 3 4 5 N/A President Judges 1. To what extent, if at all, has the number of open hearings increased in your court as a result of training or technical assistance provided by JRGA? 0 0 4 1 8 3 13 4.3 2. To what extent, if at all, has your court been able ot reduce its backlog and/or improve case processing time as a result of training or technical assistance provided by JRGA? 0 0 3 5 4 4 12 4.1 3. To what extent, if at all, has there been an increase in the level of protection afforded to victims, parties or litigants in family violence cases as a result of technical assistance or training provided by JRGA? 1 1 1 2 7 4 12 4.1 4. To what extent, if at all, has your level of legal knowledge about cases within the jurisdiction of the misdemeanor court increased as a result of technical assistance or training provided by JRGA? 0 0 2 3 6 5 11 4.4 5. To what extent has the public’s level of confidence in the court system increased as a result of public information or outreach program assistance or training provided by JRGA? 1 1 2 3 6 2 13 3.9 Judges 1. To what extent, if at all, has the number of open hearings increased in your court as a result of training or technical assistance provided by JRGA? 12 2 11 4 18 20 47 3.3 2. To what extent, if at all, has your court been able to 11 4 6 12 21 13 54 3.5 48 reduce its backlog and/or improve case processing time as a result of training or technical assistance provided by JRGA? 3. To what extent, if at all, has there been an increase in the level of protection afforded to victims, parties or litigants in family violence cases as a result of technical assistance or training provided by JRGA? 9 2 8 9 13 26 41 3.4 4. To what extent, if at all, has your level of legal knowledge about cases within the jurisdiction of the misdemeanor court increased as a result of technical assistance or training provided by JRGA? 8 1 7 14 27 10 57 3.9 5. To what extent has the public’s level of confidence in the court system increased as a result of public information or outreach program assistance or training provided by JRGA? 6 1 10 19 15 16 51 3.7 Other (Judicial Assistants) 1. To what extent, if at all, has the number of open hearings increased in your court as a result of training or technical assistance provided by JRGA? 1 0 0 1 1 8 3 3.3 2. To what extent, if at all, has your court been able ot reduce its backlog and/or improve case processing time as a result of training or technical assistance provided by JRGA? 1 0 0 2 6 2 9 4.3 3. To what extent, if at all, has there been an increase in the level of protection afforded to victims, parties or litigants in family violence cases as a result of technical assistance or training provided by JRGA? 1 0 1 2 3 4 7 3.9 4. To what extent, if at all, has your level of legal knowledge about cases within the jurisdiction of the misdemeanor court increased as a result of technical assistance or training provided by JRGA? 1 0 2 3 3 2 9 3.8 5. To what extent has the public’s level of confidence in the court system increased as a result of public information or outreach program assistance or training provided by JRGA? 1 0 3 0 2 5 6 3.3 * Total responses include all likert scale values, excluding Not Applicable 49 3. Questionnaire Comments, by Respondent Title and Question President Judges 1. Open Hearings • Courtrooms have been provided 2. Backlog and case processing • The case processing time has been decreased • By acting in line with the lectures delivered by USAID experts • Due to the overburdening of the existing two judges that cover the head office of the court and its branch offices in Kladovo and Majdanpek • The court has reduced its backlog to a considerable extent only due to the efficiency of judges and court staff • Very few old cases, mostly due to the negligence of other state bodies (slow, inefficient) 3. Family violence protection • We dealt with this topic even before JRGA came • Yes, because it is easier to notice the victims of domestic violence, and abusers are usually repeat offenders • No misdemeanor charges against public order and peace with elements of domestic violence have been filed with the court (such cases are processed through the competent Prosecutor’s Office) • Open cooperation with all institutions in the system starts for the first time • Training necessary to learn different practice 4. Legal competency • A lot, due to the greater data availability and easier and faster communication; • All judges and administrative staff attended many trainings • Yes, absolutely 5. Increased public confidence • Unfortunately, very little • Development of a website and an info desk • Yes, despite disrespect of the court by some repeat offenders who are more and more insolent towards the court (judges, legal staff and civil servants) • The President Judge cooperates with media (newspapers)by giving interviews and information about the court’s work, particularly in the field of customs and foreign exchange offences which attract a lot of public attention • As a judge, together with my colleagues, we regularly emphasize at our staff meetings that we see difference compared to the period when we were administrative bodies Judges 1. Open Hearings • An opportunity has been given to the public but they do not use it as they do not have the habit of attending the open hearing • We’ve noticed improvement • It has a positive influence on the court work • Open hearings accelerate resolution of cases • Considerable improvement 2. Backlog and case processing • Case processing is more efficient and more cases are resolved • There are very few statute barred cases. They become statute barred due to the objective reasons 50 • By expedited issuance of summons, urging the applicants, etc. • It considerably reduced time of case processing • There are not many unresolved cases and the reason for them not to be resolved is that the legal assistance has not been provided or orders have not been enforced • We haven’t received equipment • Considerable improvement 3. Family violence protection • Misdemeanor court in Prokuplje acts upon domestic violence cases urgently and imposes strict punishment on abusers • Noticeable increase 4. Legal competency • The level of knowledge has been highly increased • Trainings for judges are very useful, they improve resolution of dilemmas thus increasing efficiency • Absolute increase of the level of knowledge • New computers are significant help for more efficient work 5. Increased public confidence • Questions cannot be read, therefore it is difficult to reply to them. It is necessary to separate words in a sentence, which require additional effort. That is the reason why I answered like this-with NA. • Level of confidence is highly increased as well as the respect in the new premises that you’ve renovated • A little, although without a good reason • Hearings are open for public which increases confidence in the work of misdemeanor courts. Defendants, although being punished, are aware that the punishment is reasonable and that they are convicted for a reason • Considerable increase • We still need electronic bulletin boards 51 4. Questionnaire (Serbian) Aneks F: Predlog Upitnika Aneks F1: Predlog Upitnika za prekršajne sudije i zaposlene u prekršajnim sudovima Social Impact (SI) sprovodi srednjeročnu evaluaciju JRGA projekta za potrebe USAID-a. Svrha upitnika je da se dobiju povratne informacije od svih prekršajnih sudova u vezi sa osnovnim ciljevima projekta JRGA. Molimo vas da popunite formular što je ranije moguće i dostavite nam ga pre nego što odete sa konferencije. Predstavnik SI će biti prisutan tokom trajanja konferencije i ako imate bilo kakvih pitanja u vezi sa evaluacijom, mozete mu ih direktno postaviti. Hvala na saradnji. Molimo vas da navedete svoju funkciju u sudskom sistemu: Predsednik suda_________ Sudija__________ Menadžer suda_____________ Drugo (Molimo vas da navedete):________________________________________ Molimo vas, ocenite kako su aktivnosti JRGA projekta uticale na vaš sud od 1 (bez uticaja) do 5 (značajno); ili zaokružite NA ako nemate podatke o ovom pitanju. Dodajte komentare ako želite da objasnite svoj odgovor: 1. U kojoj meri i da li se uopšte, broj ročišta otvorenih za javnost povećao u vašem sudu kao rezultat obuka ili druge tehničke pomoći koju je pružio projekat JRGA? ____1.____2._____3. ____4. ____5. ____NA Komentari: _________________________________________________ ___________________________________________________________ 2. U kojoj meri, i da li je uopšte, vaš sud uspeo da smanji broj zaostalih sudskih predmeta i/ili da ubrza vreme obrade predmeta kao rezultat obuke ili tehničke pomoći projekta JRGA ? _____1. ____2.____3. ___4. ____5.____NA Komentari:___________________________________________________________ _____________________________________________________________________ 3. U kojoj meri i da li je uopšte došlo do povećanja nivoa zaštite žrtava i/ili strana u postupku u predmetima nasilja u porodici kao rezultat tehničke pomoći projekta JRGA? _____1. ____2.____3. ___4. ____5.____NA 52 Komentari:___________________________________________________________ _____________________________________________________________________ 4. U kojoj meri i da li se uopšte, unapredio nivo vaše upućenosti u predmete u nadležnosti prekršajnog suda kao rezultat tehničke pomoći ili obuke projekta JRGA? _____1. ____2.____3. ___4. ____5.____NA Komentari:___________________________________________________________ _____________________________________________________________________ 5. U kojoj meri se nivo poverenja javnosti u sudski sistem povećao kao rezultat pomoći ili obuke projekta JRGA u oblasti javnog informisanja ili odnosa sa javnošću? _____1. ____2.____3. ___4. ____5.____NA Komentari:___________________________________________________________ _____________________________________________________________________ 6. U nastavku se nalazi spisak obuka realizovanih uz podršku projekta JRGA. Molimo da obeležite seminare kojima ste prisustvovali: Carinski propisi______ Poreski propisi _______ Primena propisa iz oblasti borbe protiv korupcije_______ Etika_______ Administrativne i operativne procedure u prekršajnim sudovima_______ Efikasna obrada predmeta i upravljanje predmetima_______ Rad na računaru/Kompjuterska pismenost__________ 53 C. QUESTIONNAIRE FOR GRANTEES, INCLUDING RESULTS Responses from 9 JRGA Grantees (Alterfact, Praxis, CERTA, Transparency Serbia, Eutopia, Center for Modern Skills, CeSid, European Policy Centre, Assn. of Public Prosecutors) Please rate the following impacts of JRGA project activities on efforts to increase government accountability and transparency and fight corruption in Serbia from 1 (none) to 5 (substantially); or check NA if you have no information on this subject. Add comments if you wish to explain your response: 1. To what extent, if at all, has the Anti-Corruption Agency’s detection and sanctioning of conflicts of interest improved? ____1.____2._(3)___3.__(3)___4. ____5. _(5)___NA Comments: Several respondents commented on value of Guide for New Officials and training supported by JRGA Notable improvements in ACA 2. To what extent, if at all, has the Anti-Corruption Agency’s regulation of political party and campaign financing improved? _____1. ____2._(5)__3. _(1)__4. __(1)___5._(2)__NA Comments: ACA contributed to quality of 2011 Law on Financing Political Activities and by-law issued by Director regulated reporting in detail. However, law and by-laws have several weaknesses. Election finance monitoring network important to more effective monitoring Significant amount of work still needs to be done New system of control by ACA more independent than previous system; although all political parties submitted 2012 Annual Report, improvement in control and sanctioning needed 3. To what extent, if at all, has there been an increase in public awareness of activities of the Anti￾Corruption Agency, the State Audit Institution or other Independent Agencies? _____1. ____2._(3)__3. __(4)__4. ____5._(2)__NA Comments: Roundtables and training for media contributed to increased public awareness CIPPID has been in public for some time. ACA and SAI have increased public presence but should increase level of coverage; grantee working on raising public awareness of ACA and SAI sanctions in misdemeanor courts and referral to public prosecutor. Increased awareness leading to more citizens willing to report corruption, but citizens need to understand what information required for IA to investigate; this has to be dealt with or it will continue to be obstacle in future. SAI visibility very high; increased in 2012 SAI recognized as a significant actor and government’s watchdog in the financial accountability system Testing Implementation of Ombudsman’s Recommendations Regarding Vulnerable People project implemented by grantee, Praxis, raised awareness of public institutions, civil servants, and local level ngos ACA more visible, SAI more visible as a result of ability to initiate misdemeanor for violation of budget law, CIPPID and Ombudsperson well known 54 4. To what extent has non-governmental support increased for improvements in government and official integrity, reduction of corruption, proper financing of elections and political parties, and compliance with EU norms? _____1. __(1)___2.____3. __(7)___4. __(1)__5.____NA Comments: JRGA support for civil society provided more opportunities to influence and monitor government and official. JRGA support increased number of CSOs involved in AC activities, but not sustainable because CSOs depend on donor funding; very few CSOs working on AC systematically rather than occasionally Only a few NGOs dealing with this and need more capacity to be effective JRGA opened channel of communications between IAs and CSOs Research indicates citizens believe corruption in political parties deeply embedded NGOs have increased support, but better coordination needed between NGOs and IAs so they have a common vision Not enough ngos active in the field, so impact as a whole is limited 5. To what extent has the public’s level of confidence in the misdemeanor court system increased? _____1. __(1)_2._(3)_3. ___4. _(1)__5._(4)__NA Comments: 55 D. PMP ANALYSIS, YEAR 1 SUMMARY Of three common, 20 Judicial Reform and 12 Government Accountability targets, 87.5% reached, principally in the areas of number of justice sector personnel trained, number of courthouses reengineered, procedures and tools made available to PCs to improve the court work flow and procedures for asset disclosure, conflict of interest and political finance monitoring in place # personnel trained: 400/637 # courts w/improved case management: 0/8 (6PCs, HMC and AC) # personnel trained in anti-corruption: 155/196 Judicial Reform 1.2 Open Hearings: Performance Standards; Facilities reengineered: 2/3 1.3 Reduce Backlogs and Improve Case Processing Times in AC, HMC and MCs: # PCs with Action Plans for timely case processing: 6/6; Backlog reduction: 15%/8.5% 1.4 Collection of fees and fines: # PC tracking: 3/2; %fines/fees paid on time: 49%/222% 1.5 Physical safety of family violence victims: # courts implementing procedures: 6/6; # PCs where litigants aware of new procedures: 12%/12% 1.6 Coordination among related cases in MC and basic courts: # courts coordinating case information: 6/6 1.7 Judicial Competency: # person-days of training: 300/1495 (640 trained, incl. 506 judges—130M, 375F; and 134 staff—37M, 97F); # new courses: 3/6 1.8 Public Information re Improved Operations: # courts implementing: 6/6 1.9 Improved reputation: # positive media reports 25/40; (websites in Years 2-5) 1.10 Increased support of ROL: # grants awarded: 3/3 Government Accountability 2.2 ACA conflict and property registration operations: % conflict of interest proceedings held: 95%/115%; # asset declarations filed: 200/586 2.3 ACA political party financing: % declarations verified: 85%/TBD; % increase in # of criminal actions filed: 5%/-50% 2.4 Enforcement of IA decisions: % of cases referred by IAs to other agencies for enforcement: 45%/56% 2.5 IA/court coordination: # agencies adopt procedures: 0/0 2.6 inter-IA coordination: # complaints registered: 201,181/332,2333 (+5%/+73%) 2.7 Improved justice: # grants awarded: 3/3 56 ANNEX IV: SOURCES OF INFORMATION A. BIBLIOGRAPHY JRGA Work Plans Life of Project Work Plan, 2011 Year 1 Work Plan, 2011 Year 2 Work Plan, 2012 Year 3 Work Plan, 2013 PMP PMP Final 2012 Results Years 1 and 2 and Targets Year 3, Report on the Performance Monitoring Plan, July 1, 2012 – June 30, 2013. Progress Reports Annual Year 1 Annual Report June 2012 • Annex A: JRGA Year 1 Budget Execution53 • Annex B: JRGA PMP Annual Report54 • Annex C: JRGA Project Staffing55 • Annex D: List of JRGA Counterparts56 • Annex E: JRGA Quarterly Report 157 • Annex F: JRGA Quarterly Report 258 • Annex G: JRGA Quarterly Report 3 w/ 21 Annexes • Annex H: Overview of the status of amendments to the Law on Misdemeanors, WG May 2012 • Annex I: Architectural Design for GENEX building, future seat of Belgrade MC (Serbian) 53 May 30, 2011: Of $4.4M budgeted, est. $4.2M to be expended through 6/30/2012 ($2.0M expended through 3/31/2012 and $2.2 projected to be expended through June 30, 2012); thus ca. 1 month behind budget schedule 54 through May 2, 2012 (July 2012): Of 3 common, 20 Judicial Reform and 12 Government Accountability targets, 87.5% reached, principally in the areas of number of justice sector personnel trained, number of courthouses reengineered, procedures and tools made available to PCs to improve the court work flow and procedures for asset disclosure, conflict of interest and political finance monitoring in place. 55 June 25, 2012 56 : Judicial Reform: MOJ (6), SCC (2), HCC, AC, JA (3), HMC, MCs (45), AMJ, JAS, APP; Government Accountability: ACA, SAI, OMB, CIPIPDP (plus other relevant organizations such as the Commissioner for Protection of Equality, the Public Procurement Office, the Parliament, and local governments); Grantees: APP, JAS, AMJ, BIRODI, YUCOM, EPC. 57 May – September 2011 Final with 2 Annexes: (1) MC Partner Court Assessment Report, comparing court leadership, administrative staff, IY infrastructure, ownerhip and condition of facilities, court size, location, cost of intervention (30 courts, 6 recommended as initial Partner Courts—Zrenjanin, Kikinda, Arandjelovac, Valjevo, Leskovac and Sjenica), September 6 2011; and (2) Grants Manual September 19, 2011) 58 October – December 2011Final with 36 Annexes, including Annex 4 Service of Process, Case Management, MC website, IT systems, CMS, Networks, Access to Courts, Judicial Training; Annex 5-7: Court Improvement Plans; Annex 8: Case Law Database for the Administrative Court in Belgrade; Annex 9: Partner Outreach Analysis and Plan; Annex 10: Court Layout Proposals for Arandjelovac and Valjevo; Annex 11 Court Improvement Plan; Annex 12-13 Service of Process and Collection of Fees and Fines; Annex 14 Case Processing Mapping in Zrenjanin; Annex 15Backlog Reduction Plan Guidelines; : Annexes 16-36 Best Practices of Partner Courts; Minutes, WG on Case Management December 14, 2011; Minutes, WG on Service of Process and Enforcement, December 9, 2011; Improving Case Coordination, Courthouse Safety and Victim Protections in the Misdemeanor Courts of Serbia Report, December 12, 2011; Judicial and non-judicial staff training report, November 18, 2011; Judicial and non-judicial staff competence presentation, October 2011; Judicial Academy Misdemeanor Courts Training Program; Rules, Procedures and Interview Schedule for Applicants to become Election Campaign Observers, Rules and Reporting Forms for Political Financial Record Keeping, all issued by the Director of ACA, November 7, 2011; Political Finance Monitoring Matrix, Workplan for Training of Political Finance Monitors and Training Schedule (October 2011); Ombudsman Outreach Proposal to increase Public Awareness; Anti-Corruption Day leaflet. 57 • Annex J: RFP for CMS for Misdemeanor Courts (MCCMS) w/ Annexes of court statistics, court register, and case process flows. • Annex K: MCCMS roundtables, Round Table, April 19, 2012 • meeting minutes • Annex L: Administrative Court Case Law Data Base, User Manual (Serbian) • Annex M: Training for Misdemeanor Judges, Customs Regulations (Serbian) • Annex N: Training for MC Judges, Administrative and Operational Procedures in the MCs (Serbian) • Annex O: Efficient Case Flow and Records Management, training materials, April 2012 • Annex P: Basic Principles of Judicial Ethics, address of Judge D. Brooks Smith, May 2012 • Annex Q: MS Office, Internet and HMC CMS training presentations (Serbian) • Annex R: Brochures for Partner Courts59 • Annex S: Association of Misdemeanor Judges, Communication and Public Outreach Strategy • Annex T: Association of Misdemeanor Judges, draft 2013 Annual Work Plan • Annex U: Guide for New Officials • Annex V: Communication Strategy for Ombudsperson’s Office • Annex W: Anti-Corruption Agency, Action Plan for Priority Area 12 Media, working papers • Annex X: SAI/MC roundtables 2 and 3, meeting minutes • Annex Y: CfI/MC roundtable 1, meeting minutes • Annex Z: Anti-Corruption Agency Complaints process, assessment • Annex AA: STTA Trip Reports • Annex AB: Grants – quarterly reports and deliverables • Annex AC: Press Clipping Year 2 Annual Report June 2013, w/ Annexes A-U • Annex B-PMP Yrs. 1-3 • Annex C—JRGA Org Chart • Annex D-List of Project Counterparts • Annex E-G—Quarterly Reports w/ annexes • Annex H—New Misdemeanor Law • Annex J-Overview of Grant Program • Annex K-Grant Reports and Deliverables • Annex L-Law on Pub Procurement training material (Serb) • Annex M-LIBRE Office training manual • Annex N-Ireland/Netherlands Study tour for Serbian court officials-agenda • Annex O-Administrative Court Brochure • Annex P-Communications training for MOJPA • Annex Q-Court User Survey • Annex R-AC Strategy Press Clippings • Annex S-Political Finance Conference Materials/Reports on 2012 Elections • Annex T-Difficult Persons Materials for Ombudsman • Annex U-Roundtable Notes (Serb) o Quarterly Reports Year 1 Q1 August 2011 Year 1 Q2 December 2011 w/36 annexes Year 1 Q3 March 2012 w/21 Annexes 59 Against Family Violence, Court Fees and Taxes, Enforcement Procedure, Get to Know Your Court, How to Obtain information on your case, How to prepare for court, Voluntary payment of misdemeanor fine (all Serbian) 58 Year 1 Q 4 June 2012 Year 2 1Q September 2012 w/21 Annexes • Annex A: Law on Misdemeanor Work Group • Annex B: Zrenjanin Misdemeanor Court Assessment • Annex C: Ruma Misdemanor Court Assessment • Annex D:Belgrade Misdemeanor Court60 • Annex E: Higher Misdemeanor Court Workload and Evaluation Report September 2012 • Annex F: Court Outreach: Brochure Final Design (Serbian); Poster final design (Serbian)) • Annex G: Zajecar Misdemeanor Court Facility Upgrade • Annex H: STTA Trip Reports61 • Annex I: Plea Agreements in Misdemeanor Courts, October 2012 • Annex J: Voluntary Collection of Fees and Fines in Misdemeanor Courts September 2012 • Annex K: Application of Tax Regulations training materials, October 2013 (Serbian) • Annex L: Ethics and Courtroom Administration Training Materials, October 2012 (Serbian) • Annex M: Strategic Planning Guide for the Association of Misdemeanor Judges, October 2012 • Annex N: Grant Evaluations62: • Annex O: Meeting Minutes, Commissioner and the Misdemeanor Courts Roundtable, September 24 2012 (Serbian) • Annex P: Assessment of Political Party Financing Oversight by the Anti-Corruption Agency during Serbia’s 2012 Elections63, • Annex Q: Letter and survey form from HMC President to HMC Divisions and to all MC president judges requesting that all misdemeanor courts file Integrity Plans with the ACA by the end of the year, August 2012. • Annex R: PPT presentations on Data Breaches Hypothetical, Data Breaches in a Global Environment and Data Privacy in a Global Era64 • Annex S: Model Regulations on Data Breach Notification and Security of Personal Data65, • Annex T: Association of Public Procurement Professionals (APPP) PPT slides (3) (Serbian) • Annex U: APPP Conference Agenda: Oversight and combatting irregularities in the system of public procurements; and Improving and increasing efficiency of the system of public procurements, September 2012 Year 2 Q2 December 2012 Year 2 Q3 March 2013 Year 3 1Q September 2013 w/Annexes A-0 Monthly Reports Nov. 2011- May 2013, July – August 2013 60 Technical Assistance and Organizational Review September 2012l Implementation Plan October 2012 61 Commissioner for Information of Public Importance and Personal Data Protection: Data Protection and Security Breach Notification, Lisa Sotto July 7-14, 2012; Anti-Corruption Agency: Political Party Financing, Jennifer Brunner, July 16, 2012; Technical Assistance and Organizational Development of the Belgrade Misdemeanor Court, Norm Meyer and Janet Cornell, September 21, 2012; High Misdemeanor Court: Service of process and enforcement, misdemeanor law amendments and judicial workload, Violaine Authemann, September 29, 2012 62 Impact Assessment of Grant to Association of Misdemeanor Judges to enhance implementation of alternative sanctions, August 2012; Progress Report by Association of Public Prosecutors of Grant to develop better protection of domestic violence victims, September 2012; Progress Report by Judges Association of Serbia on Grant to Enhance the Accountability and Administration of Justice in Misdemeanor Courts in Dealing with Illegal Migrations, August 2012; Progress Report by Bureau for Social Researches on Grant to prepare and Implement mechanisms for mapping the risk of misuse of public funds for financing of political activities, August 2012; Progress Report by Lawyers Committee for Human Rights on Grant to strengthen the Ombudsman’s Role in Serbian Society, August 2012 63Jennifer Brunner, October 2012; including nine Annexes 64 Lisa Sotto, July 2012 65 Lisa Sotto, July 2012 59 Interim Deliverables Partner Selection Courts Report Grants Request for Applications Awards Announcement Laws and Strategies Misdemeanor Law 2009 National Judicial Strategy 2013-2018 Amendments to Law on Courts 2013 Amendments to Law on Judges 2013 Amendments to Law on Public Prosecutor 2013 Amendments to Law on Misdemeanors 2013 Law on Administrative Disputes National Anti-Corruption Strategy 2013-2018 National Anti-Corruption Action Plan Law on Financing Political Activities 2011 Law on Public Enterprises 2012 Law on Public Procurement 6. Additional Information EU Serbia 2013 Progress Report: Enlargement Strategy and Main Challenges 2013-2014 Assistance provided through Instrument for Pre-Accession Assistance (IPA) Country Strategy Plan 2014-2020 being prepared (IPA II) IA Annual Reports, 2011 and 2012 • Government of Serbia Anti-Corruption Council Work Report, 2012 • Judiciary in the Fight Against Corruption, Transparency Serbia, January 2013 • Transparency in Funding of Election Campaigns in Serbia, Transparency Serbia, 2013 • Attitude of Serbian Citizens Towards Corruption, UNDP Serbia, June 2013 • The Serbia We Want-Post 2015 National Consultations in Serbia, 2013 60 B. EVALUATION INTERVIEWEES No. Name Position Org/Affil Address Phone Email 1. Milica Bozanic Head of International Cooperation Department Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 069 337 7213 milica.bozanic@acas.rs 2. Ivana Tosovic Advisor in International Cooperation Dept. Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 ivana.tosovic@acas.rs 3. Dejan Milic Head of Conflict of Interests Resolution Dept. Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 dejan.milic@acas.rs 4. Dragomir Trninic Assistant Director of the Dept. of Operational Affairs, control of officials' property and Registers Dept Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 Dragomir.trninic@acas.rs 5. Marijana Obradovic Assistant Director for Protection Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 marijana.obraovic@acas.rs 6. Natalija Ristivojevic Advisor in the Dept. for control of financing of political entities Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 natalija.ristivojevic@acas.rs 7. Natasa Jelic Complaints Dept. and District Office Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 natasa.jelic@acas.rs 8. Lidija Kujundzic PR Dept. Anti-Corruption Agency (ACA) Carice Milice 1, Belgrade 011/4149 100 lidija.kujundzic@acas.rs 9. Marina Matic Executive Director Association of Public Prosecutors and Deputy Public Prosecutors of Serbia (APP) Gracanicka 18, Belgrade 069 181 1977 marina.matic@uts.org.rs; skype. marina.matic011 10. Radenka Grubacic Counselor, Department for OMB Ombudsperson Deligradska 16, Belgrade 011 206 8100 radenka.grubacic@ombudsman.rs 61 Projects and Cooperation 11. Iva Vasilic Head of the President's Office State Audit Institution (SAI) Makenzijeva 41, Belgrade 065 493 3010 iva.vasilic@dri.rs ; ivasilic@hotmail.com 12. Nevena Ruzic Assistant Secretary General CIPIDPP Commissioner for Information of Public Importance and Personal Data Protection Bul. Kralja Aleksandra 15, Belgrade 011 340 8900; 064 8479103 nevena.ruzic@poverenik.rs 13. Emila Spasojevic Chief of Division (International Cooperation and Projects) Commissioner for Protection of Equality Beogradska 70a, Belgrade 011 243 7650; 064 8898709 emila.spasojevic@ravnopravnost.gov.rs 14. Stevan Radunovic President APPP Public Procurement Professionals Association Starine Novaka 19, 21000 Novi Sad 064 8186797; 060 3776555 stevarad@gmail.com 15. Dragomir Milojevic Acting President of SCC and HCC SCC Suprime Court of Cassation Nemanjina 9, Belgrade 011 363 5495; dragomir.milojevic@vk.sud.rs; uprava@vk.sud.rs 16. Majda Krsikapa Head of Administrative Office of HCC SCC Suprime Court of Cassation Nemanjina 9, Belgrade 063 480 780 majda.krsikapa@vrh.sud.rs 17. Sladjana Bojovic Court President Administrative Court Nemanjina 9, Belgrade 011 3635 174, 063 264 627 011 3635 174, 063 264 627 18. Borisav Knezevic Senior Adviser for EU issues and Project Management Public Procurement Office Ustanicka 130, Belgrade 064 1265 469 ada.d@orion.rs; bknezevic@ujn.gov.rs 19. Dragana Boljevic; Omer Hadziomerovic, Marko Dejanovic; Nadja Micic 1)President; 2)Deputy President; 3)Legal Assistant; 4)Head of Office JAS Judges' Association of Serbia Alekse Nenadovica 24/1, Belgrade 011 3443132 3) 064 6688501; 4)069 1719771 jaserbia@sbb.rs; mdejanovic@gmail.com; micicnadja@gmail.com 20. Milan Marinovic; Zorana Antic Court President; Court Manager Belgrade MC Timocka 14, Belgrade 011 2836 053; 069 644659 21. Danilo Nikolic State Secretary of MoJ Ministry of Justice Nemanjina 22- 011 3616 548 danilo.nikolic@mpravde.gov.rs 62 of the RS 26, Belgrade 22. Marica Milovanovic Court President MC Arandjelovac Knjaza Milosa 102, Arandjelovac 034 723 908 prek.sud-ar@open.telekom.rs 23. Nada Pandurov Court President MC Kikinda Svetozara Miletica 1, Kikinda 0230 421015 psudki@open.telekom.rs 24. Slobodan Marinkovic Court President MC Uzice Nade Matic 8, Uzice 031 512347 prekrsajnisuduzice@live.com 25. Zeljka Petrovic Court President MC Zajecar Pana Djukica ¼, Zajecar 019 422734 prekrsajnisudza@gmail.com 26. Radoslav Vujovic Deputy Secretary General NA Secretariat Trg Nikole Pasica 13, Belgrade 064 842 0223; 011 3026176 rvujovic@parlament.rs 27. Nenad Vujic Director Judicial Academy Karadjordjeva 48, Belgrade 011 2184 030; 011 2183 316; 063 1211219 akademija@pars.rs 28. Nemanja Nenadic Program Director Transparency Serbia , TS Palmoticeva 27, Belgrade 063 8898442 nemanjalaw@sezampro.rs 29. Nebojsa Lazarevic Chairman of the Management Board European Project Center, EPC Kralja Milutina 36, Belgrade 063 240 641 nebojsa.lazarevic@cep.org.rs 30. Milan Antonijevic Director YUCOM Lawyers; Committee for Human Rights Svetogorska 17, Belgade 063 234368 m.antonijevic@yucom.org.rs 31. Vukosava Crnjanski Sabovic; Bojana Milosevic; Rasa Nedeljkov Director; Development Manager; Program Manager CRTA Center for Research, Transparency and Accountability Nusiceva 7, Belgrade 065 3270442; 011 303 5014; 060 4090192; 063 356 065 vukosava.crnjanski@crta.rs; vuka@crta.rs; office@crta.rs 32. Blazo Nedic, Uros Misljenovic 1) CDCP Project Director; 2)Project Manager PDC Partners for Democratic Change Svetozara Markovica 9, Belgrade 011 3231551; 063 286 025 bnedic@partners-serbia.org 33. Vladimir Radomirovic Board President Eutopia/Pištaljka Kneginje Zorke 50/1, Belgrade, 063 770 5925 vladimir.radomirovic@pistaljka.rs 34. Vesna Stanojevic Coordinator CAFV Dalmatinska 47a, Belgrade 063 8061753 savet@eunet.rs 35. Romana 1)Head of Rule of Law OSCE Spanskih boraca 063 214 671; romana.schweiger@osce.org; 63 Schweiger; Denise Mazzolani; Mato Meyer Department; 2) Senior Coordinator for the Rule of Law and Human Rights; 3)Economic Transparency Advisor 1, Novi Beograd 011 360 6181; 063 378 160; 011 3606 156 denise.mazzolani@osce.org; mato.meyer@osce.org 36. Jelena Manic Petronikolos Programme Analyst, Good Governance UNDP Internacionalnih brigada 69, Belgrade 011 2040418 jelena.manic@undp.org 37. Srdjan Sarcev Public Sector Specialist-MDTF-JSS Coordinator World Bank Bulevar Kralja Aleksandra 86, 6th floor,Belgrade 011 3023713 ssvircev@worldbank.org 38. Mirjana Cvetkovic and Bianka Vandeputte Legal Officer; Project Manager-Operations I, Programming EU Delegation Avenija 19a, Vladimira Popovica 40/V, Novi Beograd 064 236 1370; 011 3083223; 011308 3272; 011 3083 200; mirjana.cvetkovic@eeas.europa.eu; Bianca.vandeputte@eeas.europa.eu 39. Dejan Milenkovic Executive Director NPC New Policy Center Krunska 15, Belgrade 063 355954 milenkovic.dejan1@gmail.com 40. Gordana Stamenic; Dragana Brajovic State Secretary for Public Administration; Senior Advisor MoJ Bircaninova 6, Belgrade 011 3613654 gordana.stamenic@mpravde.gov.rs 41. Zoran Pasalic Court President HMC High Misdemeanor Court; Association of Misdemeanor Judges 1. Timocka 15, Belgrade; 2.Hadzi Prodanova 11, Belgrade 011 3038 403;069 644654 visi.prek.sud.bg@gmail.com 42. Dejana Djordjevic Court President MC in Novi Sad Bulevar Oslobodjenja 58, Novi Sad 021 4896 100 prek.sud.ns@gmail.com 43. Jelena Gataric Court President MC in Zrenjanin Zitni trg bb, Zrenjanin 023 525 253, 548 452 preksudzr_pr@open.telekom.rs 44 Marija Mitic Task Officer The Delegation of the European Union to the Republic of Serbia Vladimira Popovica 40, Avenue XIXa, Belgrade 381 11 30 83 289 arija.MITIC@eeas.europa.eu 64 45 Brian LeDuc Chief of Party Separation of Powers Project Belgrade 381 112 421 663 bleduc@ewmispp.org 65 C. SITES VISITED • BEG Misdemeanor Court, October 17, 2013 • Prokupje MC, October 25, 2013 • Novi Sad MC, October 28, 2013 • Zrenjanin MC, October 29, 2013] 66 ANNEX V: EVALUATION LOGIC FLOW Evaluation Q. TASKS Findings Conclusions Recommend￾ations Evaluation Question 1: To what extent is the program on track (i.e., meeting established milestones) to accomplish the objectives and results expected under both components? Judicial Reform 1. Open Hearings On track; renovation of lower priority facilities; judicial hierarchy a barrier to implementation Unavailability of financial resources to fund priority renovations Facilitate MOJPA/EU funding of renovation 2. Case Management Partner courts adopted backlog reduction plans; LoM Amendments and MCCMS implementation on track Lack of backlog definition and case processing standards Partner courts pilot case management standards 3. Court Fees Collections Partner Courts report increase in timeliness of court fees Implementation of electronic register function of MCCMS critical Expand training of licensing agencies to facilitate implementation of MCCMS electronic register 4. Domestic Violence Public information on track; counterparts attribute limited replication of programs to need for facilities renovations Implementation of therapeutic justice programs does not necessarily require facilities renovations Assess reasons for delays in implementing domestic violence witness protection programs in Partner Courts. 5. Case Coordination LoM amended to eliminate multiple filings in MCs and Basic Courts Still lack system to identify cases arising from same incident; Include best case coordination practices in Bench Book 6. Jud. Competence Extensive training provided; unable to measure impact of training on competence Will remain unable to measure improvement in judicial competence until adoption of performance standards and metrics Develop both performance standards and metrics 7. Public Information Brochures, kiosks/counters, web portals developed; user surveys in two Lack of metrics or mechanisms to measure impact of public information services Expand functionality; develop metrics to measure level of public 67 Partner Courts understanding 8. Public Confidence Renovations, public information programs, Integrity reporting; school programs, etc. strengthen public confidence; although MCs still considered less professional by their colleagues Lack of metrics and mechanisms to measure level of public confidence State of the Judiciary addresses; surveys of public confidence; media training; explore feasibility of salary parity initiative; renovations; implement LoM Amendments, etc. 9. NGO grantees support ROL 27 grants to 20 grantees for $500,000 in projects through Year 2 and plans to fund $250,000/year for new projects Lack metrics to determine impacts of grant funded programs; lack of system to monitor implementation of CSO recommendations Develop an assessment tool to measure the effectiveness of CSO grants. Follow up on all grant-funded projects to assist counterparts in implementing the recommendations and solutions developed by CSOs; disseminate results Overall On track, but lack high priority renovations and case management and performance standards development; MOJPA unwilling to transfer and HCC unwilling to assume responsibility for management of all judicial branch functions. 1. Develop case management and performance standards; 2. Help develop HCC strategic planning capacity; 3. Facilitate funding of high priority renovations; 4. Facilitate development of R&D function to help develop case management and judicial performance standards; 5. Disseminate results, facilitate implementation of CSO grantee 68 recommendations Government Accountability Tasks Findings Conclusions Recommendations 1. ACA monitoring, prevention and sanctioning conflicts of interest Improvements in Sector for Operations-- implemented JRGA recommendations (data management) Sector for Complaints starting to implement recommendations; Prevention Sector implemented recommendations (integrity plans, Guide for New Officials) Problems due to poor ACA management and unclear legal provisions. PMP data: Policies adopted. Initial increase in % of CoI requests completed (increase in yr 1-reduced backlog; decrease in Yr. 2 due to increase in requests as a result of elections). Initial increase in # of ADs verified. Partially on track, but need improvements in Complaints and Prevention Sectors Continue assistance to ACA in implementing JRGA recommendations 2. ACA regulates political party and election finance Established system of campaign monitors Overwhelmed by quantity of data Partially on track. Assist ACA in implementing lessons learned and preparing for future elections, as necessary 69 submitted JRGA assisted ACA to manage crisis. PMP data: Policies adopted. Increase in PP/EC declarations verified NA. Increase in violations referred—lagging due to processing time. 3. Improve enforcement/imple mentation of IA decisions/recomme ndation IAs trained on better reporting; JRGA facilitating IA/NA cooperation; PMP shows slight improvement in % of recommendations implemented. Increased inter￾institutional coordination resulting in better understanding and improved procedures Continue to facilitate cooperation 4. Administrative and judicial due process Roundtables and joint trainings for MCs and ACA, CIPIPID and SAI. See Component 1— case coordination PMP: No formal policies adopted IAs and courts adopting necessary policies and procedures Continue to facilitate MC-IA cooperation 5. IAs work together to monitor and improve good governance and reduce corruption JRGA supported joint activities— Anti-Corruption Day. Ombudsperson, CIPIPDP and ACA working on whistleblower law. Continue to facilitate cooperation 6. Citizens/CSOs involved in good governance and anti-corruption initiatives 11 grants for projects that focus on specific component 2 objectives, but all Belgrade-based On target Encourage applications to non-Belgrade CSOs (if there are projects with local government units) 70 Overall On track, but ACA progress was delayed 1. Continue targeted assistance to ACA, including building management capacity. 2. Continue support for inter￾institution cooperation. EVALUATIO N QUESTION 2: What emerging impact has the project had on strengthening Misdemeanor Courts and Independent Agencies in Serbia? Judicial Reform Findings Conclusions Recommendations Drafting and facilitating the implementation of Amendments to the LoM; development of the MCCMS; strengthening the competencies of Misdemeanor Judges; and assessment and improvement of MC physical facilities. Comprehensive support provided by JRGA at both the national/policy and local/operational levels to MCs and AC has significantly advanced prospects for sustainability of modern judicial system management in these courts JRGA should assist HCC and the JA in developing a national court system research and development organization to assist in the development and implementation of court performance standards and ensure that the results of judicial system reforms initiated and implemented with JRGA assistance will be sustainable. Government Accountability Findings Conclusio ns Recommendations 3 overarching areas of assistance: AC Strategy and Action Plan; IA cooperation with other institutions; IA visibility. JRGA was extensively involved and counterparts expect continued JRGA assistance. Emerging impact: Detailed roadmap for good governance/anti￾corruption progress. Counterparts appreciate value of coordination and expect it to Continue to support initiatives but transfer responsibility to counterparts to ensure sustainability. 71 continue. Citizens increasingly aware of role of IAs and CSOs engaged in IA work. However, initiatives highly dependent on JRGA intervention. EVALUATIO N QUESTION 3: To what extent has the project design been appropriate in achieving intended objectives and results? Findings Conclusions Recommendations Merged assistance to new institutions (MCs and IAs) with common jurisdiction into one Program. Included grants to support activities linked to Program goals and objectives. Flexibility built into the Program provided space to take advantage of emerging opportunities within the scope of the project’s overall goals. As MCs and IAs were both JRGA counterparts, JRGA able to bring groups together to improve enforcement of IA decisions. JRGA used grants to directly engage CSOs in supporting JRGA JRGA used flexibility to respond to changes in government . None EVALUATIO N QUESTION 4: Within the current contract scope, what opportunities can be identified to expand support for Serbian judicial reform Findings Conclusions Recommendations Targets of opportunity within the project scope identified to date include outreach to petitioners to facilitate implementation of the LoM amendments; capacity development of the HCC and the JA; and assessment of court facility renovation requirements. Additional opportunities could include development of a research and development function at the HCC to assure long-term sustainability of judicial system reform efforts, focusing initially on the 1.Survey public confidence in the courts; 2. Expand activities to make public aware of court system improvements, including state of the judiciary addresses 3. Facilitate development of an R&D function to 72 efforts? development of judicial performance standards; expansion of outreach to petitioners to implement the LoM amendments; expanded implementation of facility assessment recommendations, particularly highest priority MCs in buildings shared with Basic Courts; and, In addition to continuing grants to civil society organizations (CSOs) to improve the administration of justice, increased efforts to promote improvements to the judiciary among the public and to conduct surveys regarding the public’s confidence in the courts. develop standards, judicial discipline mechanism, etc. Evaluation Question 5: To what extent are Component II Governmental Accountability activities sufficiently focused and strategically targeted Findings Conclusions Recommendations Built relationships with named counterparts early in program. Explored opportunities with other counterparts for inter-agency cooperation. Followed tasks outlined in RFP and approved work plans. Took advantage of opportunity to advise on key driver of good governance/anti￾Focus on completing ongoing tasks with continuing follow￾up by Program staff. Replicate successful interventions in 73 toward the broader program goal? Significant focus on development of strong, comprehensive AC Strategy and Action Plan. Year 3 workplan focuses on ongoing activities, but leaves room to pursue new policy level interventions. AC donor coordination informal. EU and UNDP currently planning for post-JRGA activities. corruption activities in the near to mid-term. Activities strategicially targeted but there is risk of becoming too thinly spread. Informal coordination may not be sufficient new areas (e.g. local government), by advising counterparts, but not doing work for them. Encourage more formal coordination. Initiate through donor coordination meeting focused on implementation of AC Strategy and Action Plan. 74 U.S. Agency for International Development/Serbia 92 Kneza Aleksandra Karadjordjevica 11040 Belgrade Serbia 75