December 2011 This publication was produced for review by the United States Agency for International Development by Ellen Kelly and Svetozara Petkova EVALUATION Formative Evaluation of Justice Sector Development Project II BOSNIA-HERZEGOVINIA Ex13 via Wikimedia Commons Formative Evaluation of Justice Sector Development Project II This publication was produced for review by the United States Agency for International Development by Ellen Kelly and Svetozara Petkova December 2011 TABLE OF CONTENTS DISCLAIMER.................................................................................................................................... iv ACRONYMS...................................................................................................................................... v I. INTRODUCTION............................................................................................................................ 1 II. EVALUATION METHODOLOGY................................................................................................... 1 III. BACKGROUND............................................................................................................................ 2 A. Country Context .................................................................................................................. 2 B. Project Background............................................................................................................. 4 IV. Component Findings and Recommendations............................................................................... 4 A. Component 1: Strengthening the Independence, Accountability and Effectiveness of the Justice Sector....................................................................................................................................... 4 Background..................................................................................................................... 4 Findings for Component 1................................................................................................ 9 Recommendations for Component 1 ............................................................................ 12 B. Component 2: Support for a Better Coordinated and More Unified System of Justice Ready for EU Accession ......................................................................................................................... 14 Background................................................................................................................... 14 Findings for Component 2.............................................................................................. 21 Recommendations for Component 2............................................................................. 24 C. Component 3: Bolstering Public Confidence in the Rule of Law........................................ 27 Formative Evaluation of JSDP II ii Background................................................................................................................... 27 Findings for Component 3.............................................................................................. 28 Recommendations for Component 3............................................................................. 30 V. Recommendations for Future Activities....................................................................................... 32 List of Appendices............................................................................................................................ 39 Formative Evaluation of JSDP II iii DISCLAIMER The views expressed in this publication do not necessarily reflect those of the United States Agency for International Development or the United States Government. Formative Evaluation of JSDP II iv ACRONYMS AC Acquis Communautaire ADI Association for Democratic Initiatives BD Brčko District BD JC Brčko District Judicial Commission BERP Balkans Enforcement Reform Project BiH Bosnia and Herzegovina CEPEJ Commission for the Efficiency of Justice COP Chief of party CSO Civil society organization DEI Directorate for European Integration DS Documentation System EC European Commission EU European Union EWMI East-West Management Institute FJP Forum for Joint Policy FWG Functional Working Group HJPC High Judicial and Prosecutorial Council JC Judicial Commission JN Justice Network JPTC Judicial and Prosecutorial Training Center JSDP I Justice Sector Development Project JSDP II Justice Sector Development Project II JSRS Judicial System Reform Strategy MOJ Ministry of Justice MPOI Model Prosecutors’ Office Initiative NGO Non-governmental organization OSCE Organization for Security and Cooperation in Europe PMP Performance Monitoring Plan RS Republika Srpska SAA Stabilisation and Accession Agreement SSPACEI Sector for Strategic Planning, Aid Coordination and European Integration SUFI System for Management of Financial Operations TC Thematic Conference UPR Universal Periodic Review US United States USAID United States Agency for International Development VAT Value-Added Tax Formative Evaluation of JSDP II v I. INTRODUCTION The United States Agency for International Development (USAID) in Bosnia and Herzegovina (BiH) retained consultants Ellen Kelly and Svetozara Petkova to conduct a formative evaluation of the Justice Sector Development Project II (JSDP II), in compliance with Section C.6 of the JSDP II contract. As set forth in the scope of work, there are two goals of the evaluation:  To provide an objective analysis of the impact and success of the first three years of JSDP II to the overall development of the BiH rule of law sector.  To examine the broad status of reforms and needs in the rule of law sector in BiH and to provide the USAID Mission with the optimal areas of involvement of JSDP II during the optional two years of the project. The evaluation team recommends that USAID/BiH exercise the option period for the contract for the following reasons:  Overall, the contractor's activities in implementing JSDP II were appropriately designed and have been well implemented;  Work under JSDP II is making an important contribution to strengthening rule of law in the country and meeting a significant part of the criteria for EU accession, which coincides with US foreign policy goals for BiH;  Most of the progress achieved so far under each of the three components of JSDP II requires additional assistance to be solidified or taken to completion. To better target JSDP II assistance during the option period and achieve the best possible results, USAID/BiH should refine some of the project activities to reflect conditions which have changed since the project began in 2009. Specific recommendations regarding how JSDP II's activities should be shaped during the option period are provided in the relevant sections dealing with each project component. The authors want to thank everyone who participated in and assisted with this report, particularly Jasna Kilalic of USAID, Emir Balic and Samir Hamzic. II. EVALUATION METHODOLOGY The evaluation team carried out a desk review of materials furnished by USAID/Bosnia prior to arriving in BiH, between November 1, 2011 and November 4, 2011. These materials included the contract with East-West Management Institute for implementation of JSDP II, quarterly and annual reports of JSDP II to date, JSDP II work plans, the project’s Model Prosecutor's Office Intervention Plan, the BiH Justice Sector Reform Strategy 2008 – 2012, the latest version of the Performance Monitoring Plan (PMP) of JSDP II and the most recent Formative Evaluation of JSDP II Page 1 European Commission Progress Report on BiH issued in October 2011. The evaluation team obtained and reviewed other materials once the field work began. The consultants had a preliminary telephone conference with USAID/BiH, and began field work on November 8, 2011. From November 8, 2011 to November 24, 2011 the evaluation team conducted interviews of officials from the judicial and executive branches, representatives of civil society, international donors, and personnel from the Departments of State and Justice and USAID at the US Embassy in Sarajevo. The consultants interviewed prosecutors in Mostar, Siroki Brijeg, Trebinje, Zenica, Banja Luka, Tuzla and Sarajevo, and cantonal Ministry of Justice officials in Mostar, Siroki Brijeg, and Tuzla. As part of the review of the Model Prosecutors’ Office Initiative within Component 1, the team asked USAID/BiH to send a questionnaire to the first 13 prosecutors’ offices receiving assistance from JSDP II. Eight of them answered the questionnaire. The evaluation team also spent approximately 14 hours in conversation with the leadership and technical staff of JSDP II, and reviewed answers to written questions the consultants had addressed to the project staff as part of the team’s field work. III. BACKGROUND A. Country Context The government and judicial structures of Bosnia and Herzegovina (BiH) are the product of the Dayton Peace Accords which formally ended three years of regional warfare in late 1995, the Constitution which was an annex to the Dayton agreement, and several later statutes. There are two entities within BiH, the Federation of Bosnia and Herzegovina (the Federation) with a largely Bosniak and Croat population, and the Republika Srpska (RS), which is primarily Serb. The country also includes the self-governing Brčko District (BD). Within the Federation, there are 10 cantons. The division of authority within BiH means there are four parallel and separate jurisdictions at the State, Republika Srpska, Federation and Brčko levels with their own court systems, 10 subsidiary cantonal jurisdictions, 13 ministries of justice (MOJs) and the Brčko District Judicial Commission (BD JC), and 14 judicial budgets. The various ministries of justice (at State, entity and cantonal level) are not linked by any vertical hierarchy. The State MOJ has coordination functions relative to the entity MOJs and BD JC, and the Federation MOJ has coordination functions relative to the cantonal MOJs. In the rule of law area, this fragmented governance system translates into discrepant application of the laws, lack of legal certainty, inefficiencies and a high cost of managing the system. The Dayton Peace Accords, as augmented by provisions known as the “Bonn Powers,” also created a High Representative to implement the civilian aspects of the Accords: the High Representative’s authority includes issuing decisions and promulgating laws which are binding on other BiH authorities, and removing public officials from office under some circumstances. In 2004 the High Representative created the High Judicial and Prosecutorial Council (HJPC), which is responsible for appointing and disciplining judges and prosecutors countrywide. It also has extensive responsibility for administering the courts, although some Formative Evaluation of JSDP II Page 2 of that authority overlaps with the authority of the State MOJ and the entity MOJs. Appendix A illustrates the structure of the court system in BiH. BiH developed a Justice Sector Reform Strategy (JSRS) with extensive donor assistance. The JSRS was adopted by the State MOJ, Federation MOJ, RS MOJ and BD JC in June 2008. It covers the period 2008 – 20121 and lists specific, actionable reforms. In December 2008, the Ministerial Conference of the Ministers of Justice in BiH, President of the HJPC and President of the BD JC adopted an action plan for implementation of the JSRS. The action plan was revised in June 2010 and again in December 2010. This document breaks down the strategic programs defined by the JSRS into concrete activities with deadlines for their implementation. The accession of BiH to the European Union (EU) is the force driving much of the judicial reform in the country. BiH and the EU signed a Stabilisation and Accession Agreement (SAA) in 2008 and BiH's progress towards accession is tracked by the European Commission (EC) in annual progress reports. Judicial reform is a key part of the political criteria for the country’s EU accession, and JSRS represents a roadmap for meeting a significant part of those criteria. Since 2008, every BiH Progress Report has followed the implementation of the Strategy. JSDP II has been operating at a time when the complicated and diffuse government structure of Bosnia and Herzegovina has been under increasing stress from the economic crisis that began in 2008, delays in the formation of a national government after elections in 2010, decreasing donor resources, and continuing international pressure for more consolidation of authority within central institutions. Political disputes between various factions within the country have made judicial reforms, including those being pursued by JSDP II, difficult to achieve and maintain. For instance, for several weeks this year the RS threatened to hold a referendum on the country’s courts and prosecution2 : the referendum was canceled in May 2011 only after the High Representative of the European Union for Foreign Affairs and Security Policy promised the European Commission would use the “Structured Dialogue” mechanism, which is part of the pre￾accession negotiation process, to provide a comprehensive review of the country’s judiciary. The first session of the Structured Dialogue was held in June 2011 and the second session was held in November 2011. 1 The Bosnia and Herzegovina 2010 Progress Report of the European Commission and the action plan for Strategy implementation refer to the JSRS as covering the period 2009 – 2013, which is in line with some statements made by interviewees in the course of this evaluation that the timeframe of the Strategy might be extended by at least a year. See p. 13, BiH 2010 Progress Report, Commission of the European Communities at http://www.europa.ba/files/docs/2010progress2.pdf. 2 The referendum would have asked RS citizens: "Do you support laws imposed by High Representative in Bosnia, in particular the laws on Bosnia's state court and prosecution?" See http://www.balkaninsight.com/en/article/catherine-ashton-to-meet-bosnian-leaders. Formative Evaluation of JSDP II Page 3 B. Project Background The main objective of the Justice Sector Development Project II is to build effective and credible justice system institutions that enable the accession of Bosnia and Herzegovina into the European Union, by strengthening the independence, effectiveness and accountability of the judiciary, supporting a better coordinated and unified justice system, and bolstering public confidence in the rule of law. USAID/Bosnia and Herzegovina awarded the cost-plus-fixed-fee contract for implementation of JSDP II to East-West Management Institute (EWMI) following a full and open competition. The contract has a three-year base period, and an option for a two-year extension. The base period of the contract began July 15, 2009 and ends July 14, 2012. The technical portions of the JSDP II contract have not been modified since the cost-plus-fixed-fee contract was awarded to EWMI: instead, any changes in project activities are recorded in the JSDP II work plans. Many aspects of JSDP II are modeled on the success of the first Justice Sector Development Project (JSDP I) which USAID/BiH implemented from March 2004 to 2009. The earlier program helped establish the High Judicial and Prosecutorial Council and the State MOJ. It also included a highly successful Model Courts Initiative, which worked with 32 of BiH’s 65 courts on various aspects of court management including records management, case processing and backlog reduction, public access, and budgeting. JSDP I also renovated some portions of selected courts and developed and installed case management software. JSDP II was designed to strengthen the ability of the HJPC and the State MOJ to manage the judiciary and lead advocacy for needed reforms. Component 1's principal counterpart is the HJPC; this component includes also a Model Prosecutor's Office Initiative based on the experience and achievements of the Model Courts Initiative under JSDP I. Component 2's main counterpart has been the State MOJ: JSDP II helps it fulfill its role as coordinator of the JSRS. Component 3 of JSDP II works with non-governmental organizations to strengthen public support for the rule of law in BiH. IV. COMPONENT FINDINGS AND RECOMMENDATIONS A. Component 1: Strengthening the Independence, Accountability and Effectiveness of the Justice Sector Background Component 1 of JSDP II provides assistance in two major areas, both of which have several subparts: 1.) Promoting independence and accountability through improved appointment and advancement processes: Formative Evaluation of JSDP II Page 4 a. Improving HJPC examinations of judicial and prosecutorial candidates; b. Developing a mechanism for measurement of prosecutors’ performances; and c. Revising judicial evaluations. 2.) Increasing the effectiveness of the judicial system through improved management and administration of courts and prosecutors’ offices: a. Implementing a Model Prosecutors’ Office Initiative in 18 prosecutors’ offices; b. Developing a matrix of standards for prosecutors’ offices; c. Developing an on-line portal for prosecutors to receive and share information, known as the T-Portal; d. Establishing case processing timeframes for courts and prosecutors’ offices; and e. Institutionalizing the updating of bench books for judges and prosecutors by the HJPC. 1.) Promoting independence and accountability through improved appointment and advancement processes. The project counterparts for this part of Component 1 are the HJPC and its Secretariat. The HJPC was created only in 2004, and its rapid progress in tackling the issues facing the BiH judicial system over the past seven years led more than one donor interviewed by the evaluation team to recommend the HJPC as the agency most likely to achieve project results in BiH. However, given the challenges facing the judicial sector in the country, not even the HJPC has been able to accomplish everything its members have wanted to do as quickly as they would like. During the first year of JSDP II, the HJPC agreed to establish seven formal working groups to implement portions of Component 1.3 Unfortunately, neither the HJPC nor its Secretariat had the capacity to provide as much leadership and technical support to the working groups as Component 1 required, so three of the groups were eliminated and one was postponed.4 3 The JSDP II Year 1 Annual Report provides a list of the goals of the seven working groups created by HJPC to implement the activities anticipated in the Year 1 work plan of JSDP II: 1) reform procedures for selection and appointment of judges and prosecutors; 2) develop measurement procedures for effectiveness of prosecutors; 3) improve budgeting capacities of courts and prosecutors; 4) develop the Model Prosecutors Office Initiative; 5) monitor implementation of case processing times for courts and develop case processing times for prosecutors; 6) review and improve the Civil Benchbook; and 7) review and improve the Criminal Benchbook. 4 JSDP II reports identify the three discontinued working groups as those dealing with budget issues, revisions of the criminal bench book and revisions to the civil bench book. The working group on developing a mechanism for measurement of prosecutors’ performance was formed in January 2010, suspended in October 2010, and had not been reactivated when the field work for this evaluation was done in November 2011. Formative Evaluation of JSDP II Page 5 Screening and selection of judges and prosecutors is one of the key duties of the HJPC. Despite the importance and visibility of this topic, the HJPC working group on improving HJPC examinations of judicial and prosecutorial candidates was not approved until January 2010, and the first meeting could not be held until April 2010 because of other commitments of some working group members. In October 2010, the HJPC removed four members of the group and appointed three new ones. The primary focus of the group has been on developing a computerized examination for judicial and prosecutorial applicants, to be used as part of the selection process. The exam would be the first of its kind in BiH, and plans call for the questions for each candidate to be drawn from a bank of approximately 3,000. In early 2011, the Norwegian government provided funds for software to be used for the exam. As of November 2011, with the assistance of experts provided by JSDP II, the working group was devoting as much time as it could to overseeing the complicated process of drafting and refining the questions. It had not yet addressed the overall design of the exam or the logistical or technical details about the actual administration of the exam, but working group members thought it would be ready to be tested by the end of the JSDP II base period. If the working group’s recommendations are approved by the HJPC as a whole, group members hoped the first use of the test for candidates could be done during the second half of 2012. However, not all HJPC members interviewed support the use of the computerized test, fearing it could eliminate too many people who could develop into independent and effective judges and prosecutors. The HJPC working group on developing a mechanism for measurement of prosecutors’ performance was appointed in January 2010. During its first meeting in March 2010, the group adopted its own detailed work plan. However, in October 2010 the HJPC voted to suspend this working group until work on performance standards for judges had been completed, and the prosecutors’ working group had not been restarted as of November 2011. At the beginning of the project, the HJPC and JSDP II agreed to collaborate on revisions to the measurement of judicial performance although no working group was formed to consider the subject. In January 2010, the HJPC told the project that JDSP II’s assistance with that work was not necessary, since the Council was still planning to pilot a new weighted caseload system to track the work of judges and judicial associates, which was originally scheduled to start in 2009. The weighted caseload analysis would have formed part of the basis for the HJPC’s work on revising judicial performance measures with JSDP II. As of October 2010, the pilot program had not started, so work on measurement of judicial performance was removed from JSDP II’s work plan. 2.) Increasing the effectiveness of the judicial system through improved management and administration of courts and prosecutors’ offices. JSDP II is the first donor activity in BiH to focus on making the administration of prosecutors’ offices more efficient and transparent, although the U.S. Department of Justice and others have provided assistance designed to improve the domestic prosecution of war crimes, Formative Evaluation of JSDP II Page 6 serious economic crimes and organized crime. With its Model Prosecutors’ Office Initiative (MPOI), JSDP II is providing assistance to all 18 of the country’s prosecutors’ offices in three waves determined by the HJPC. Although the project ultimately could not fund any renovations, JSPD II conducted an assessment of the premises of all prosecutors’ offices in October and November 2009, and provided plans and estimates for suggested changes to make the offices more efficient. In January 2010, the HJPC appointed a working group to develop a matrix of standards for prosecutors’ offices based on the results of the MPOI. In January 2010 JSPD II organized a conference of chief prosecutors and prosecutor offices secretaries to introduce the project and develop support for it. At each of the 18 prosecutor's offices JSDP II established change management teams comprised of office leadership and personnel to oversee the implementation of the MPOI. JSDP II and the change management teams developed intervention plans for each office. At the time of the evaluation field work, JSDP II had provided, or in the case of Wave 3 was preparing to provide, group training to each wave of prosecutors’ offices on strategic planning, program budgeting, public relations, archive management, and information gathering/libraries. Based on interviews done as part of the evaluation team’s field work, the review of project reports and the responses to the questionnaire received from prosecutors’ offices in Waves 1 and 2, it appears the Wave 1 offices received the most project time, in terms of developing the intervention plans and follow up to see how reforms were progressing. Offices in Wave 2 received less individual attention than the offices in Wave 1, and Wave 3 offices have received the least individual attention from JSDP II. The project organized a meeting of Wave 1 participants with their counterparts from Wave 2 offices as JSDP II began its work with Wave 2, but there was no similar event for Wave 3 offices. The project has not been able to visit Wave 2 offices to check on the outcome of its work with them.5 The work with prosecutors’ offices was supposed to be the basis for preparation of a matrix of standards for all prosecutors’ offices for the consideration of the HJPC working group on the matrix appointed in January 2010. JSDP II envisions the matrix as a compilation of best practices for prosecutorial office administration, although the draft version of the matrix seen by the evaluation team also included substantive aspects of prosecutors’ work which are not covered in the MPOI activities. As of November 2011, the project hoped the HJPC working group would be able to approve a final version of the proposed matrix, which would also be approved by the full HJPC, before the end of 2011. 5 The project’s July 2011 Model Prosecutor Office Initiative Component 1 Intervention Plan August 2011-July 2012 indicates that during the third year of the project, JSDP II would meet regularly with the change management teams within the prosecutor offices and monitor the reform activities through site visits and personal interaction with the prosecutors and the administrative staff of prosecutor offices,” but as of November 2011 those meetings and visits had not started. Interviews with JSDP II staff indicate that most of the “post-training” assistance identified in the questionnaire responses from the Wave 1 and 2 prosecutors’ offices (see Appendix B) probably represents work done by JSDP II short-term consultants as part of the training process. Formative Evaluation of JSDP II Page 7 Initially, JSDP II had agreed with the HJPC that it would help develop a series of video podcasts through which additional training could be delivered to prosecutors. The working group on the matrix and MPOI concluded it would be premature to introduce that type of technology for prosecutorial training. Instead, the working group suggested the use of an on￾line portal to deliver trainings. The resulting “T Portal” was being developed by JSDP II during the evaluation team’s field work. As envisioned in November 2011, the portal’s uses would go beyond the working group’s original suggestion. One portion of the portal would include a web resource/reference center, while another would provide access to administrative procedures and standards, books of rules, forms, templates and other local background material. For the first time, the portal would also allow prosecutors to exchange information through one or more forums. As JSDP II began, Switzerland was contemplating its own project to help BiH’s prosecutors. That project, formally known as “Strengthening the Capacity of Prosecutors in the Criminal Justice System,” currently has a budget of €3.5 million for its work from October 2010 until March 2014. The Swiss project told the evaluation team that in theory that amount could increase and the project could be extended beyond March 2014. The project comprises several activities which have also been the subject of assistance from JSDP II, including more efficient management of prosecutors’ offices, better public relations, and strengthening the capacity of the HJPC to monitor the work of prosecutors. Since its start in October 2010, the Swiss project has conducted extensive analyses. As of November 2011 the project was still completing its plans for implementation of recommendations appearing in those analyses. JSDP II and the Swiss project have done a thorough and commendable job of exchanging information and expertise that constitutes a model of donor coordination. JSDP II staff are “mandatory members” of the steering board for the Swiss project, and JSDP II apparently incorporated some of the findings of the Swiss project’s report on management practices in prosecutors’ offices into the draft matrix of standards. For its part, the Swiss project has included JSDP II recommendations in its own plans. JSDP II also has been assisting another HJPC working group to establish case processing timeframes for courts and prosecutors’ offices. This work is based on the recommendations of the task force on the time frames of proceedings of the European Commission for the Efficiency of Justice (CEPEJ). It is also based in part on work done by JSDP I, which resulted in the HJPC’s adoption of “Criteria for Performance Evaluation of Judges in BiH and Judicial Performance Evaluation Forms.” There are two types of case processing timeframes involved in this activity -- optimum and foreseeable. CEPEJ defines an optimum timeframe as the “theoretical shortest timeframe possible” to prepare and resolve a case of a particular type. The foreseeable timeframe is the time it usually takes for those cases to be completed at a given court, and considers how “the workload of the courts and judges are influenced by various (external and internal) factors, like the influx of cases, the personnel resources, working methods, level of Formative Evaluation of JSDP II Page 8 computerization, number of cases in stock, etc.”6 Computation and publication of the foreseeable timeframes for each court and each type of case would give parties realistic expectations as to the progress of their case if they decide to pursue it, and computation and publication of the optimum timeframes should encourage judges and prosecutors to come as close to them as possible. As of November 2011, the HJPC working group on case processing timeframes had been working for approximately one year to establish a list of case types and determine the optimum timeframes for them, according to one working group member interviewed by the evaluation team. Once the group finishes that process, it hopes to complete a rule book to be used by each court and prosecutors’ office as of mid-2012, to determine the foreseeable timeframes for each category of case in each separate jurisdiction. Knowing the gaps between the optimum and foreseeable timeframes in each jurisdiction will help the HJPC determine “when citizens are not well-served,” in the words of the working group member. The project’s work on institutionalizing updates of bench books for judges and prosecutors was relatively short-lived. The HJPC distributed bench books developed with the assistance of JSDP I during a conference of court presidents in May 2010. The HJPC also approved one working group to update the civil law bench book and another group to update the criminal law bench book in January 2010. By October 2010, the HJPC had decided to discontinue the two working groups, because of the demands they would have placed on the HJPC members and/or the HJPC Secretariat. Findings for Component 1 General findings: 1.) JSDP II has been flexible, appropriately persistent and responsive to the needs and requests of its counterparts in implementing Component 1, within the project limits. 2.) JSPD II’s ability to help implement reforms has been delayed by initial misunderstandings with the HJPC and the Secretariat about the type and extent of technical and financial assistance JSDP II could provide to the various working groups, and lack of capacity of the HJPC and the Secretariat. Relating to promoting independence and accountability through improved appointment and advancement processes: 3.) The HJPC working group on improving the examination process for judges and prosecutors has been very diligent about pursuing the computerized test for judicial and prosecutorial candidates. The working group has relied on experts provided by JSDP II to progress as far as it has, but realizes that it will need more outside technical assistance on the design and logistics of the exam before the computerized test could be used. 6 See the November 2007 Explanatory note to the questionnaire common case categories, timeframes and delays found at https://wcd.coe.int/ViewDoc.jsp?id=1223489&Site=COE. Formative Evaluation of JSDP II Page 9 4.) It is not likely that the computerized test will be administered to candidates before late 2012. 5.) The examination is only part of the work to be done on revising the selection process for judges and prosecutors: for instance, no decision has been made on how much weight the exam results should be given during the selection process. 6.) So far, JSDP II has made several attempts to work with the HJPC on revising measurements for judicial performance, including through the implementation of a weighted caseload system. However, the HJPC has not accepted any of those offers and it appears the HJPC has not wanted to cooperate with the project on that matter. HJPC stated it did not need any assistance for the initial stages of that process and for the subsequent stages it postponed assistance. Based on its interviews with JSDP II staff and members, the evaluation team could not determine what was driving the HJPC’s failure to accept the offer of help, but it did not seem that the HJPC’s position was due to a lack of political will to tackle the measurement of judicial performance, or any general unwillingness to work with JSDP II. Overall, the evaluation team concluded that no matter how much the HJPC would have liked to pursue all the reforms proposed by JSDP II, it does not have enough full-time members, resources or staff to handle an unlimited number of projects. 7.) Given that the HJPC has conditioned the restarting of the working group on measuring prosecutorial performance upon the completion of the significantly delayed work on the measurements for judicial performance, in the evaluation team's opinion a thorough development of a mechanism for measurement of prosecutors’ performances probably could not be completed by July 2014. 8.) JSPD II’s work with optimum and foreseeable timeframes for the work of judges and prosecutors under the second part of Component 1 could have a direct bearing on both judicial and prosecutorial performance measures. Relating to increasing the effectiveness of the judicial system through improved management and administration of courts and prosecutors’ offices: 9.) It is not clear that JSDP II will be able to verify any measurable increase in the effectiveness of prosecutors’ offices through its Model Prosecutors’ Office Initiative by July 2012. In almost every case, all of the assistance offered by JSDP II was appropriate and appreciated. However, in its visits to seven of the 18 prosecutors’ offices assisted by JSDP II, the evaluation team did not locate any before-and-after data from the prosecutors’ offices that could be used to demonstrate increased effectiveness due to the project’s efforts. 10.) Currently, most of the 18 prosecutors’ offices do not have the time, personnel or resources required to put the tools provided by JSDP II to use on their own. It would require significantly more regular contact with project staff and other experts for more reforms to take hold in those offices. Formative Evaluation of JSDP II Page 10 11.) Any matrix of standards for prosecutors’ offices should be regularly reviewed and updated to include new standards (e.g., ethics, productivity, elimination of conflicts of interest) which did not appear in the draft which JSDP II provided to the evaluation team. 12.) The only English version of the draft matrix of standards available for the evaluation team to review apparently was prepared in January 2010. The evaluation team would recommend several changes to that draft if they have not been made already. The major changes follow:  The draft matrix is so detailed that it would require much more time and staff to implement than the prosecutors’ offices the team visited have available. Therefore, the items in the matrix should be prioritized, even among those items described as mandatory.  Scoring for the matrix needs to be reconsidered. The draft matrix proposed that JSDP II staff would be responsible for scoring, but the staff of JSDP II would not have the time required to do it, given the number of the items to be reviewed in each office and the complexity of many of the factors. It is also unlikely that any other body would have the time or resources to review all of the factors for more than a few prosecutors’ offices during any one year.  There needs to be more clarity about what assistance, if any, would be available for prosecutors’ offices to achieve the standards. For example, the draft matrix includes standards for developing internal communication and case backlog reduction plans and compliance with freedom of information requirements, and JSDP II does not include work with prosecutors on these specialized subjects (although JSDP II was to score prosecutors’ offices on these standards).  The use of a particular score as the mark of a “model prosecutors’ office” should be eliminated. The draft matrix would designate a prosecutors’ office as a model office once it achieved a score of 75%, but the emphasis should be on having an office continue to progress and achieve as high a score as possible, rather than having it achieve the one target. The use of a particular score for a model prosecutors’ office designation is also confusing, since all the prosecutors’ offices participating in Component 1 of JSDP II are referred to as model prosecutors’ offices by the project once they have worked with JSDP II.  The draft matrix does not contain any reference to developing or compliance with conflict of interest rules for prosecutors, or compliance with ethics provisions. Adding such standards would improve the quality of the matrix. 13.) Many of the members of the change management teams at prosecutors’ offices interviewed by the evaluation team had vague or unrealistic expectations of the type of assistance JSDP II could provide overall, as well as of the types of assistance it could provide to prosecutors’ offices in particular. For instance, some interviewees told the evaluation team they had expected JSDP II to do or fund the renovations for which the project had developed plans and some interviewees incorrectly thought JSDP II had funded renovations in their building.7 The only interviewees within prosecutors’ offices with any information about the 7 This is consistent with some of the answers to the questionnaire drafted by the evaluation team (see Formative Evaluation of JSDP II Page 11 other, non-prosecutorial aspects of JSDP II’s work were those who were members of the HJPC. This indicates JSDP II did not manage the expectations of its prosecutorial counterparts as effectively as it might have. 14.) The portal for prosecutors could be a very valuable tool in helping prosecutors and their staffs develop practical approaches to ameliorating the effects of the professional challenges they face. 15.) Given the expected budget, resources and duration of the Swiss project and its extensive past coordination with JSDP II, now that it has staff in place and is close to finishing its general analysis of management processes in prosecutors' offices, the Swiss project should be well-placed to expand the work on administrative reforms of the prosecutors’ offices done by JSDP II. 16.) The evaluation team was asked to outline the types of assistance which would still be needed by the prosecutors after the completion of MPOI. Given the fact that prosecutors' offices have until now been largely neglected by foreign donors while their responsibilities increased exponentially, these needs are numerous. Chief prosecutors interviewed by the evaluation team would usually point to the need for facility renovations and help in implementing the JSDP II recommendations on improving the premises. Much help would also be needed in implementing the strategic plans developed in the course of JSDP II trainings for each individual PO and the determination and improvement of prosecutorial foreseeable time frames. Other important areas of work include budgeting and prosecutorial ethics/conflicts of interest, given their significance both to prosecutorial operations and as indicators of progress for the EU. If any work is being done to implement the JSRS recommendation for prosecutorial use of expert assistants or trainees to improve efficiency, the evaluation team did not come across it. See also answers No. 10 and 11 in Appendix G. At the same time, administrative reforms alone cannot achieve all the changes required to make the work of prosecutors more efficient or effective. Many of the challenges prosecutors face relate to the substantive segment of their work and need to be addressed through substantive legislative amendments (such as the alternative methods of criminal prosecution referred to in Section 1.2 of the JSRS), or subject-matter training, which generally do not fall within USAID's mandate as a development agency. Recommendations for Component 1 1.) The evaluation team recommends that JSDP II be extended for the two-year option period to continue work, at a minimum on:  helping the HJPC address judicial and prosecutorial selection, including the development and administration of any examination, Appendix B). Formative Evaluation of JSDP II Page 12  establishing foreseeable case processing timeframes for courts and prosecutors’ offices, since not all judges and prosecutors will be willing or able to put in the extra time and effort that determining foreseeable timeframes will require. The work could also include assistance with determining optimum timeframes if that work has not been finished by the end of the base period. Ideally, the option period work would also help courts, prosecutors and the HJPC determine how at least some foreseeable timeframes could be reduced even without additional resources, and  making the T-Portal as useful as possible for prosecutors. 2.) In addition to being used to measure the effectiveness of individual courts and prosecutors’ offices, the results of the analyses on optimum and foreseeable timeframes for case resolution could and should be used by the HJPC and Ministry of Justice officials at the national, entity and cantonal levels, to determine which courts and prosecutors’ offices need additional resources, in what amount and of what type, to make case processing more efficient. At the time of this evaluation, the HJPC had not drafted the guidelines to judges and prosecutors to use in determining the foreseeable timeframes for each court and prosecutors’ office. With detailed analyses done at each level of the court system, the HJPC and ministries of justice would be in a much better position to determine what procedural, legislative and regulatory changes could make the processing of cases more efficient. The results could also be used by court presidents and chief prosecutors to develop or refine their strategic plans and budgets, in order to plan for improvements and address particular shortcomings in the work of their institutions. 3.) The work with the Model Prosecutors’ Office Initiative and the matrix for prosecutors should not be extended, based on the Swiss project plans to follow-up on the work done since 2009 by JSDP II. However, before the JSDP II initiative ends, the evaluation team recommends:  USAID/BiH and JSDP II should determine if the Performance Monitoring Plan for Component 1 could be revised to reflect what improvements, if any, have been made in the administrative activities of prosecutors’ offices due to JSDP II and should be included in the final project report, and  JSDP II should sponsor a widely publicized workshop at the end of the base period to allow prosecutors and their administrators to learn more of the best practices used by their peers to improve management, highlight the cooperation between USAID, JSDP II, the HJPC and the Swiss project on the prosecutors’ offices, and to mark the handover of work with individual prosecutors’ offices on management issues to the Swiss Project. 4.) JSDP II should continue to develop the T-Portal during the option period and include information relating to the legal responsibilities of prosecutors as well as the administration of their offices. The portal could also include comparative information on how the issues facing prosecutors in BiH have been handled elsewhere. The portal and its contents would have to Formative Evaluation of JSDP II Page 13 be developed in cooperation with the HJPC. All the prosecutors interviewed by the evaluation team were very anxious for any information they could obtain affecting their roles and their jobs. Initially, the project should be able to post the training materials and recommendations developed during MPOI’s implementation on the portal for easy retrieval and reference. Other projects and agencies, including the Department of Justice, could work with JSDP II and the HJPC to contribute material dealing with the substantive aspects of prosecutors’ work. Eventually, sections of the portal might also be open to administrative staff of prosecutors’ offices. 5.) Recognizing the great importance of judicial performance measurement for both JSRS implementation and EU integration, the evaluation team recommends that JSDP II be available to provide assistance to the HJPC in this area. However, given the mixed signals that have been sent by the HJPC on that particular matter, JSDP II should only engage in such activities upon the expression of explicit and clear interest in such assistance by the HJPC. 6.) Important as the development of prosecutorial performance measurement may be, if the suspended working group is restarted by the HJPC, any JSDP II assistance in this area should proceed only after the following circumstances are considered:  First, given the postponement of this activity until after the completion of judicial performance standards, this task probably could not be completed by the mid-2014.  Secondly, such work would create a serious potential for overlap with the Swiss Project, if that project proceeds as planned. Therefore, the evaluation team recommends that any such work is undertaken with caution and understanding of these premises. 7.) Since the HJPC does not consider these activities to be among its highest priorities, the evaluation team recommends that no further assistance be provided during the option period about institutionalizing updates of the civil and criminal bench books. B. Component 2: Support for a Better Coordinated and More Unified System of Justice Ready for EU Accession Background Component 2 of JSDP II has been providing assistance in six major areas: 1.) Establishment of and assistance to the Forum for Joint Policy (FJP); 2.) Strengthening the capacity of SSPACEI to implement JSRS; 3.) Development of a Documentation System for tracking JSRS implementation; 4.) Strengthening strategic and policy capacities at entity and cantonal levels; Formative Evaluation of JSDP II Page 14 5.) Improving the required justice sector reporting to the EU; and 6.) Increasing independence and effectiveness through improved budgeting processes. Most of the activities relating to these areas are aimed at implementation of the Justice Sector Reform Strategy. The reforms in the JSRS are grouped in five pillars: 1) Judicial System; 2) Execution of Criminal Sanctions; 3) Access to Justice; 4) Support to Economic Growth; and 5) Coordinated, Well-Managed and Accountable Sector. JSDP II provides support in implementing the reforms under Pillars 1, 3 and 5. Implementation of the JSRS has been a daunting task because of the fragmentation of authority in the BiH justice sector, and the fact that the Strategy is implemented through a complex inter-institutional arrangement. JSRS prescribed the establishment of Justice Sector Ministerial Conferences, the main purpose of which is to monitor Strategy implementation, as well as provide overall political and strategic direction. Meetings of the Ministerial Conferences are supposed to occur every six months. A permanent functional working group (FWG) was established for each of the five strategic pillars, comprising senior level representatives from relevant justice sector institutions and stakeholder groups. These FWGs have responsibility for developing annual joint work plans and for taking forward all of the activities envisaged under a specific strategic pillar. Appendix C, furnished to the evaluation team by JSDP II, illustrates the structure of the implementation of the JSRS. Given its role in JSRS implementation, the State MOJ is the principal counterpart of JSDP II under this component. The overall coordination of implementation activities, including maintenance of systems for monitoring progress and provision of secretariat support to the Ministerial Conferences, is entrusted to the Sector for Strategic Planning, Aid Coordination and European Integration (SSPACEI) of the State MOJ. In addition to serving as a strategic body and technical service in organizing the Ministerial Conferences, SSPACEI provides technical support to the five FWGs. However, most of the specific actions required by the Strategy are not within the State MOJ's authority but fall under the mandate of the cantonal MOJs or entity MOJs (including the Judicial Commission of the Brčko District). The HJPC is also a stakeholder in this process, given its overall authority for judicial and prosecutorial selection, appointment, evaluation, promotion, etc. The end result of the split authorities in BiH and the complex mechanism for Strategy implementation is that progress on the JSRS has been slow and cumbersome. Progress in JSRS implementation has also been stymied by the political turmoil in BiH following the general elections of October 2010 in which the citizens of BiH voted to elect new state presidency members; state, entity, and cantonal parliaments; and the RS presidency. The BiH presidency was sworn in on November 10, 2010. The RS government was formed in December 2010, and the Federation government was formed in March 2011. Unfortunately, the formation of the State government was delayed by more that a year.8 Given the State 8The political agreement on the formation of the new state government came as late as 28 December 2012. Formative Evaluation of JSDP II Page 15 MOJ's key role in JSRS implementation, the fact that high Ministry officials were carrying out their functions only in an acting capacity amidst constant negotiations and expectations for the formation on the new cabinet, had an adverse effect on JSRS implementation. Additionally, the absence of a new government for a year after the elections, coupled with RS' constant challenges to the authority of the state, further exacerbated the sense among both BiH officials and the general public that the future of BiH as a state is questionable. This diminished the morale of everyone within the country who was engaged in policy work affecting BiH’s EU accession. In this atmosphere, the work of Component 2 of JSDP II on JSRS implementation has been extremely difficult for the project to pursue. It is only recently, with the second meeting of the Structured Dialogue on Justice and the organization of a Thematic Conference on one of the most pressing activities included within the JSRS (both discussed in more detail below under “Strengthening the capacity of SSPACEI to implement JSRS”), that it seems improvements in the BiH justice system might be within reach. A short review of the work of JSDP II in the six areas comprising Component 2 is provided below: 1.) Establishment of and assistance to the Forum for Joint Policy. JSDP II has been instrumental in the establishment of a policy-making body, the Forum for Joint Policy (FJP), which brings together high-ranking representatives from the two most important state-level institutions in the justice sector – the HJPC and the State MOJ. Before the creation of the FJP there was very little communication between these two institutions. Even though the FJP involves something as simple as getting officials from HJPC and State MOJ in the same room to talk, it took a year and a good number of meetings, exchange of documents and advocacy for the FJP to launch. Since then, JSDP II has trained FJP participants in policy proposal writing. By the end of the ninth quarter of JSDP II’s life, the FJP had produced its first policy proposal. The proposal focuses on the need for reducing the fragmentation of financing of the judicial institutions in the Federation of BiH, and provides guidance for the achievement of this goal. By the end of the base period, JSDP II plans to assist the FJP in producing one more policy proposal and to include the Justice Network, formed under Component 3 of JSDP II, in promoting some of the reforms proposed by the FJP. 2.) Strengthening the capacity of SSPACEI to implement JSRS. JSDP II assistance to strengthen the capacity of SSPACEI to implement the JSRS has been manyfold. The project has been providing guidance and support to SSPACEI in enhancing its technical capacity and its internal position within the structure of the State MOJ, as well as in organizing meetings of the FWGs and of the Technical Secretariat. JSDP II also has tried to help FWGs become more effective through the development of rules for FWG operations, and by coordinating with and providing training to individual FWGs members to ensure more productive participation in the meetings. Formative Evaluation of JSDP II Page 16 In the course of JSRS implementation, it became clear that FWGs and Ministerial Conferences had not been successful in finding policy solutions to complex or controversial issues. Therefore, JSDP II suggested conducting Thematic Conferences (TCs), which would assemble all interested parties to explore solutions to significant issues before they would be considered at the level of a Ministerial Conference. The idea was that justice sector institutions would identify activities from the JSRS action plan, analyze the difficulties and obstacles for their realization, and propose solutions. The Sixth Ministerial Conference held on July 26, 2011, made the decision to officially introduce TCs as a mechanism for the discussion of controversial issue related to JSRS implementation. The first TC was held on October 28, 2011, and was cited as a major breakthrough by all of the informants interviewed by the evaluation team. The theme of the first TC was the adoption of a single Law on the Prosecutors' Offices in the Federation. This law would eliminate differences in the operations and the status of prosecutors within the Federation, which are a consequence of the currently existing 11 laws at the cantonal and entity levels. Such streamlining of prosecutorial operations in the Federation would in turn increase the effectiveness of prosecutors. A draft of this law had been prepared as early as 2008, but it had not progressed since. The official sponsor of the first TC was the Federation MOJ. The evaluation team interviewed JSDP staff in order to assess the level of the project’s engagement in the organization of the conference. It appears that the JSDP team not only came up with the idea of the TC and was intimately involved in the technical organization of the conference, but also made efforts to ensure its quality in terms of substance. JSDP II prepared the agenda and even reviewed and suggested revisions to participants' speeches in order to manage controversies and foster a productive atmosphere. The first TC brought together all stakeholders – ministries of justice, prosecutors, and international donors. Participants and observers agreed that the conference was the first time all interested parties clearly expressed their positions on the draft law, and that the conference was an important first step in pursuing the law’s adoption: the conference ended with a plan for follow-up activities. This first TC was so successful in exploring the issues surrounding more unified prosecutor operations that the recommendations of the second Structured Dialogue, held on November 10-1, 2011, included a specific deadline of February 2012 for passage of the law.9 Interviewees for this evaluation were very optimistic that the TC mechanism could help BiH move beyond some of the many impasses in JSRS implementation. The HJPC, the State MOJ and the Republika Srpska MOJ have already identified topics for the next three TCs. They will focus on reducing budget fragmentation in BiH's judiciary, legal aid, and the creation of a forensic hospital in Republika Srpska, respectively. The first one of these TCs will be held in early January 2012. 3.) Development of a Documentation System for tracking JSRS implementation. 9 See Structured Dialogue – Recommendations from the European Commission at http://www.dei.gov.ba/aktuelnosti/vijesti/Default.aspx?id=8686. Formative Evaluation of JSDP II Page 17 In order to improve information exchange and track progress with Strategy implementation, JSDP II created a software platform known as the Documentation System (DS). The DS collects all data and documents related to the implementation of the Strategy, and allows users to track progress on various initiatives and generate reports quickly. The members of FWGs are the principal users of the DS. The system allows them to upload working documents and comment on them. It also shows the institutions responsible for each task under the action plan, the target dates for the tasks, and the extent to which the tasks have been completed. For the last year of the base period, JSDP II will work on enabling public access to the DS. Currently, read-only access is provided to five civil society organizations, which are monitoring Strategy implementation under a memorandum on the establishment of monitoring and evaluation mechanisms for the implementation of the action plan of the JSRS, signed with the chairman of the Conference of Ministers of Justice of BiH, the President of the HJPC and the Judicial Commission of Brčko District.10 4.) Strengthening strategic and policy capacities at entity and cantonal levels. Since the Strategy cannot be implemented if all the responsible institutions do not have the required capacity, JSDP II has worked to establish strategic units at the entity-level MOJs and have “strategic points of contact” identified at the cantonal MOJs. The project attempted to encourage these players to participate more actively in the work on JSRS implementation through maintaining contacts with them and organizing workshops on topics related to JSRS implementation. 5.) Improving the required justice sector reporting to the EU. The EWMI contract for JSDP II requires that Component 2 of the project work towards effective implementation of the JSRS, but the contract does not specifically provide for an activity related to justice sector reporting to the EU. The Year 1 Work Plan of JSDP II introduced “Improving the required justice sector reporting to the EU” as a separate area of work under Component 2. During its first two years, the project several times approached the permanent body under the BiH Council of Ministers tasked with co-ordinating the process of BiH's EU integration, the Directorate for European Integration (DEI)11, with specific offers of assistance. DEI did not accept the offer of help, perhaps due to a lack of capacity on its part. In its Year 3 Work Plan the project proposed development of software to track the implementation of Acquis Communautaire (AC). It is unclear from JSDP II's project documents which institution would be hosting this software. According to JSDP II's Ninth Quarterly report, SSPACEI presented the Documentation System to DEI as “a useful platform 10 This monitoring project is funded by the British Government. 11 DEI was established by the Law on Council of Ministers of BiH in 2002 and overtook the role of the former Ministry of European Integration of Bosnia and Herzegovina. For a description of DEI's functions see http://www.vijeceministara.gov.ba/stalna_tijela/dei/?id=1719. Formative Evaluation of JSDP II Page 18 for the proposed software enabling justice sector institutions to follow the progress in identification of standards, principles and laws and by-laws that need to be harmonized with the AC.” Neither the JSDP II Year 3 Work Plan nor the Ninth Quarterly Report specify what kind of DEI involvement is necessary for the implementation of this activity, but in discussing progress on the software in question, the Ninth Quarterly Report notes that “[t]he political situation and the delay in appointing a new Council of Ministers severely limited DEI’s ability to move with the planned activities related to implementing the obligations from the Stabilization and Association Agreement between BiH and the EU, particularly the establishment of working groups according to the chapters established in the accession negotiations.” No progress had been made on the AC implementation software development by the time this evaluation took place. 6.) Increasing independence and effectiveness through improved budgeting processes.12 JSDP II initially focused on two aspects of budgeting: (1) reducing budget fragmentation of the justice sector in BiH, and (2) assisting the submission of needs-based budgets by courts. A third area of work, installing financial software at selected courts, was added for the Year 3 Work Plan. Budget fragmentation is one of the Key Unresolved Strategic Issues of the Justice Sector under the JSRS. Currently, the BiH judiciary is funded through 14 different budget sources at State, entity (including Brčko District) and cantonal levels. Therefore, BiH's courts and prosecutor offices have very different levels of funding and are strongly dependent on local politicians for their budgets, which undermines both their effectiveness and their independence. The concern regarding budget fragmentation in the judicial sector has been highlighted throughout all BiH Progress Reports of the European Commission. After determining that political realities meant it was unrealistic to strive for reducing the 14 budget sources to one at the State level, the project and its counterparts started advocating for unifying judicial budgets within the Federation. If successful, this would reduce the current 14 budget sources to only four (at the State, Federation, Republika Srpska and Brčko District levels). At the outset, the project attempted to work on reducing budget fragmentation and needs￾based budgeting in the framework of a HJPC working group. The HJPC subsequently terminated the working group (in Quarter 5 of project implementation) because the HJPC Secretariat's capacity to provide support to all working groups created under JSDP II proved lower than initially expected. Since then, the project has continued to cooperate on budget issues with the Budget Department of the HJPC Secretariat and the HJPC Standing Committee on Judicial and Prosecutorial Budgets. 12 Work on budgeting was initially part of Component 1 of JSDP II; the First Annual Report of JSDP II moved budgeting to Component 2 activities based on the position that a more unified budget would support the creation of the unified justice sector ready for EU accession, which is the principal goal of Component 2. Formative Evaluation of JSDP II Page 19 The project initially was examining reducing budget fragmentation through reformation of the value-added tax (VAT) distribution formula, to allow the Federation to retain of part of its VAT allocation from the State level. These funds would be used to cover judicial funding needs before making the VAT revenue-sharing allocations to the cantons. According to the original plans, the increased VAT revenues at the Federation level would be used to cover salaries and benefits of judicial personnel, leaving cantons to fund non-judicial personnel, buildings and other areas of the justice sector on the basis of local needs and preferences. This approach was seen as a way to reduce budget fragmentation in the Federation, by taking part of this authority from the cantons and giving it to the Federation. In Year 2 of the project, consideration of the VAT distribution formula was replaced with an exploration of a wider array of options for improving justice system budgeting.13 JSDP II engaged two local experts who prepared reports covering comparative analysis of the existing systems of budgeting the judiciary in the neighboring countries, the possible use of dedicated court revenues for funding judicial operations, options for judicial budgeting without interference by the ministries of justice and finance, analysis on required changes to the existing VAT law to accommodate changes in the system of funding courts and prosecutor offices, etc. The reports were delivered to the President of the HJPC and to the HJPC Permanent Subcommittee on Judicial and Prosecutorial Budgets. The creation of these reports represented a completion of some of the specific tasks in the action plan for the JSRS. In the ninth quarter of the JSDP II, the FJP developed a policy proposal for this budget reform, which will also be the topic for the next Thematic Conference. It is not clear from JSDP II reports whether in the preparation of the policy proposal the FJP used the two reports that the project had provided to the HJPC. In the area of promoting needs-based budgets of courts, JSDP II provided assistance to the Budget Department of the HJPC Secretariat in drafting strategic guidelines and a streamlined budget format for the preparation of program-based budgets.14 According to a representative from the HJPC Budget Department, a training on the application of the guidelines at the level of individual courts had been planned but was later cancelled by the project without much explanation to the HJPC Secretariat. JSDP II’s Year 2 Annual Report states that the training was supposed to be carried out by the Budget Department itself with help from the JSDP expert, but the understaffed Budget Department was unable to provide the training. The third activity in the budget area relates to the installation of financial software at selected courts. Courts in the RS, the Federation Supreme Court and Prosecutor’s Office, and the cantonal and municipal courts in Sarajevo have direct access to their budget accounts via a software platform known as SUFI. However, most courts in BiH cannot monitor their financial accounts at the relevant Ministry of Finance directly, and this impedes the courts’ ability to manage their finances and impairs their independence and efficiency. To remedy this situation to at least some extent, for the third year of the project JSDP II proposed to 13 No explanation for this change of approach is provided in JSDP II's reports and work plans. 14 According to the EWMI contract and the quarterly reports delivered by the project, the preparation of the guidelines and the streamlined format for program-based budgeting related only to court budgets whereas the Second Annual Report refers both to courts and to prosecutor offices. Formative Evaluation of JSDP II Page 20 purchase several licenses15 and install the SUFI system at a limited number of courts in one canton. The project team views this installation process as a demonstration project to illustrate the benefit of the SUFI software, thus encouraging BiH authorities or other donors to co-finance the installation of this software at the remaining courts in the canton and possibly in some other cantons as well. Findings for Component 2 Relating to the establishment of and assistance to the Forum for Joint Policy 1.) State officials and international donors interviewed in the process of this evaluation were unanimous in considering the FJP as a positive improvement over the previous situation, in which the HJPC and the State MOJ did not discuss policy issues very frequently, if at all. JSDP II was commended for initiating this format. The meetings of the FJP are currently scheduled on an ad hoc basis. This may have been appropriate at the time when the FJP was in its nascent state, but presently seems to be a weakness which may result in the FJP being taken less seriously by some participants. It may also result in a loss of momentum for reform in the face of other pressing concerns. The JSDP II Year 3 work plan provides for the drafting of two policy proposals under the framework of the FJP for that year. The ninth quarterly report states that in that quarter, in addition to helping the FJP complete the first policy proposal, JSDP II had expected to identify the topics for the next two policy proposals, and indicates that the topics have not been identified because the first policy proposal had not yet been adopted by the State MOJ and the HJPC. The JSDP II Mid-Term Strategic Review 2011 suggests that future policy topics might be linked to the Structured Dialogue. While recognizing that ultimately the State MOJ and the HJPC will decide what substantive areas should be addressed by future policy proposals, the evaluation team finds that, at least for its internal documents and discussions with USAID, JSDP II could be more pro-active in putting forward ideas for potential topics for policy proposals. Relating to strengthening the capacity of SSPACEI to implement JSRS 2.) The documents reviewed and the interviews carried out in the process of this evaluation suggest that at this point SSPACEI has sufficient capacity to implement its principal tasks. The other institutions engaged in the implementation of the JSRS, as well as international donors, see SSPACEI as an engaged and capable coordinator of JSRS implementation. JSDP II assistance to SSPACEI in a number of areas has made an important contribution to the achievement of this result. 15 The evaluation team did not receive concrete data on the price of the SUFI license; however, the JSDP II project team indicated that it is relatively expensive, e.g. approximately $20 000 per license and therefore the project could afford to fund the purchase of only a few licenses. Formative Evaluation of JSDP II Page 21 3.) Thematic Conferences are off to an extremely promising start and may be the way to achieve further breakthroughs on complex or controversial issues. In the process of this evaluation, representatives of governmental institutions and donors, including the EU delegation, felt the TCs were an important first step in getting all parties to engage on some fundamental issues that need to be resolved. The first TC was organized when the time was ripe for consideration of the draft law on prosecutors for the Federation. However, given that only one TC has taken place so far, it is unclear whether future ones will be able to replicate its success. Plans call for various institutions to take turns in formally sponsoring upcoming TCs, and the team could not determine if future sponsors will be as open to JSDP II's substantive involvement in the organization of TCs as the Federation MOJ has been. During the evaluation team's interviews, the JSDP II team indicated they will work on organizing follow-up activities to Thematic Conferences, such as meetings with stakeholders, but neither the JSDP II Year 3 Work Plan nor the JSDP II internal Mid-Term Strategic Review 2011, which JSDP II provided to the evaluation team, specify any planned follow-up activities to Thematic Conferences. The topic of the creation of a forensic hospital in Republika Srpska, which will in all probability be the theme of one of the next TCs, falls under Pillar 2 of JSRS and therefore is outside the scope of JSDP II’s activities. At the same time, the project would like to retain its current level of involvement in the mechanism of Thematic Conferences. In light of this situation, the interview with JSDP II staff left the evaluation team with the impression that Component 2 staff was not quite sure about JSDP II’s proper level of involvement, if any, in the organization of that particular TC. Relating to the development of a Documentation System for tracking JSRS implementation 4.) The Documentation System has proven to be an excellent tool for tracking and promoting implementation of the JSRS. The system was relatively inexpensive to create and it is now fully maintained by the State MOJ rather than JSDP II. It has had a disciplining effect on FWGs members by alerting them of upcoming meetings and deadlines, and by demonstrating what each user's contribution has been in the joint preparation of reports and other documents. Furthermore, DS has made reporting much easier. Reports that took days to prepare are now generated in a matter of minutes. The evaluation team could not establish with certainty whether all users of the DS have sufficient knowledge on how to use it. The competence of DS users did not seem to be an issue mentioned in JSDP II reports, in the evaluation team's interviews of JSDP II staff or interviews with users of the system. On the other hand, civil society organizations monitoring JSRS implementation note that based on the actual contacts with members of the FWG, it is evident that there is insufficient understanding of the new system with quite a number of institutions. It must be noted that USAID JSDP II project on two occasions offered to provide training to members of all 5 FWGs, but they have not expressed interest, which Formative Evaluation of JSDP II Page 22 inevitably had an impact on (lack of) understanding of the new system and the quality of rather meagre and occasionally vague institutional reports.16 Relating to strengthening strategic and policy capacities at entity and cantonal levels 5.) Project reports indicate that as a result of continued JSDP II efforts, representatives of the Brčko District Judicial Commission have reactivated their participation in JSRS implementation activities. The level of participation by the MOJs of the Federation, its cantons and the RS remains low. During field work, the evaluation team did not see much evidence of increased capacity of strategic units and strategic points of contact within the Federation MOJ or cantonal MOJs. The Federation MOJ is in a particularly difficult position regarding JSRS implementation due to the relative weakness of its position in the Ministerial Conferences and the FWGs. Unlike its counterparts from Republika Srpska and Brčko District, the Federation MOJ does not have any decision-making mandate over most issues that are being discussed. As noted above, in most cases the decision-making authority relating to implementation of the JSRS and policy reform lies with the individual cantonal MOJs, and the Federation MOJ has only a coordination role. To date, the Federation MOJ has not been able to carry out this role effectively, which has proved to be an impediment to the overall process of JSRS implementation. By way of example, if the Federation MOJ were an effective coordinator of cantonal MOJs, it would have not taken three years and a TC organized with JSDP II help to achieve something as simple as having cantonal MOJs voice their position on the draft law for a single prosecutorial service at the Federation level. It will be even more challenging for the Federation MOJ to coordinate the cantonal MOJs' positions on a single budget for the justice sector at the Federation level. One of the reasons for the Federation MOJ's poor track record in carrying out its coordination role with cantonal MOJs is the lack of resources, both at the Federation and at the cantonal level. The strategic unit at the Federation MOJ is severely understaffed and the cantonal MOJs cannot spare either the people or the resources to travel for many if any meetings in Sarajevo. The evaluation team did not find any direct evidence that lack of political will on the part of the Federation MOJ was a reason for its failure to coordinate among cantonal MOJs. On the other hand, the very fact that the Federation MOJ has not appointed a sufficient number of people to the strategic unit despite the insistence of its international partners suggests that political will may indeed be an issue. According to some of the interviewees, the Federation Minister of Justice who took office in early 2010 may be more open to strengthening the Ministry's coordination function than his predecessor had been. Relating to improving the required justice sector reporting to the EU 16 See p. 4, Report of Civil Society Organizations (CSOs) on Implementation of the Action Plan for Implementation of the Justice Sector Reform Strategy (JSRS) of BiH, 1 January – 30 June 2011, at http://ukinbih.fco.gov.uk/resources/en/pdf/2011/sco-report-summary. Formative Evaluation of JSDP II Page 23 6.) JSPD II’s work relating to improved justice sector reporting to the EU has not made progress since the launch of the project. This is due primarily to the political situation and to the fact that the local counterpart lacked absorption capacity for such assistance. This work is not critical to the contract goal of supporting a better coordinated and more unified system of justice ready for EU accession, and it is not clear that trying to use software to solve problems that are not technological by nature, i.e. lack of sufficient capacity at DEI, is an appropriate path for the project to follow. Furthermore, there is every reason to expect that once BiH becomes a EU candidate country, the government can find the resources and the political will to develop such software on its own, or acquire a version of it from neighboring countries which are further ahead on the path to EU accession. Relating to increasing independence and effectiveness through improved budgeting processes 7.) In the area of budgeting, the decision of the HJPC to eliminate its working group on judicial budgets, and JSDP II's subsequent partnership with the Budget Department of the HJPC Secretariat and the HJPC Standing Committee on Judicial and Prosecutorial Budgets, were reasonable approaches which did not adversely affect the project. The Budget Department has been the project’s primary counterpart in the area of needs-based budgeting. Both the Budget Department and the Standing Committee were recipients of the expert reports aimed at reducing budget fragmentation. This appears to be a more appropriate approach than having one HJPC working group cover all budgeting issues. The Ninth Quarterly Report of JSDP II notes that the HJPC Budget Department and the Department for Strategic Policy and Planning have recently undergone a restructuring, delaying the creation of the plan that would have a timeframe and steps for implementing recommendations from the FJP policy proposal. 8.) Work on reducing the fragmentation of justice sector budgeting has progressed slowly. The recent FJP policy proposal and the planned Thematic Conference on this topic may provide openings for JSDP II to achieve more substantial results. 9.) The evaluation team was not able to determine if there had been any improvement in court budgeting as a result of the guidelines on program-based budgeting prepared by JSDP II for the Budget Department of the HJPC Secretariat The expert representative interviewed by the evaluation team at Budget Department also could not report whether such improvements had occurred. 10.) The purchase of the financial software SUFI, planned for Year 3 of JSDP II, does not squarely fit within the objectives and expected results of the JSDP II contract. Since SUFI already is operating in some BiH courts its benefits can be evaluated on that basis, which makes the purchase of SUFI by the project even less likely to serve a development purpose. Recommendations for Component 2 Formative Evaluation of JSDP II Page 24 1.) During the option period, JSDP II should continue to support the FJP. In particular, JSDP II should do what it can to have the FJP meet on a regular, as opposed to ad hoc basis. JSDP II should attempt to provide extensive assistance to the follow up of FJP policy proposals through activities such as help in organizing future meetings and events on the topics of the policy proposals, expert assistance, advocacy and legislative drafting support. In terms of substantive areas that could be addressed through FJP policy proposals, the evaluation team agrees that they should be linked to the recommendations of the Structured Dialogue, as suggested in the JSDP II Mid-Term Strategic Review. By way of example and based on the recommendations of the second meeting of the Structured Dialogue, these topics could include the Framework Law on Legal Aid, removing the processing of utility cases from courts, transferring many enforcement functions from the courts to private or public enforcement agencies, transferring non-contested inheritance cases to notaries, and introduction of qualitative criteria for appraisal processes in the justice system. 2.) Civil society organizations monitoring JSRS implementation, as well as the Head of the HJPC Secretariat, believe that the FJP should be extended to include the entity line ministries, the Brčko District JC and cantonal ministries of justice.17 The evaluation team does not share this view. Adding more counterparts to the forum would create a very cumbersome structure, duplicate the format of FWGs and ultimately defy the purpose of the FJP as a flexible and relatively informal policy-making body at the state level. 3.) During the option period, assistance to SSPACEI should be scaled down and include only specific activities directly related to the JSRS implementation, such as SSPACEI's participation in the FJP or organization of the Thematic Conferences. 4.) For the remaining part of the base period and during any option period, JSDP II should continue to support the organization of Thematic Conferences and their follow-up as much as possible. Project work plans should provide more specificity in respect of follow-up activities to TCs. The project should give special attention to action plans drafted at the end of each TC and the implementation of these action plans through activities such as organizing future meetings and events on the respective topic, providing expert assistance, advocacy and legislative drafting support. To the extent possible, USAID/BiH should seek the support of its US Embassy counterparts in advocating for some of the more important policy matters related to the implementation of the action plans. JSDP II also should attempt to retain a high level of substantive involvement in the preparatory work of organizing TCs. 5.) Provided that governmental officials using the Documentation System express interest in additional training, JSDP II should remain open to providing such training. The training should be tailored to to the needs of the particular users because, as indicated in the report of civil society organizations monitoring the implementation of JSRS, some members of the functional working groups still do not have sufficient knowledge on how to use the DS. Given 17 See page 5 and page 8, Report of Civil Society Organizations (CSOs) on Implementation of the Action Plan for Implementation of the Justice Sector Reform Strategy (JSRS) of BiH, 1 January – 30 June 2011, at http://ukinbih.fco.gov.uk/resources/en/pdf/2011/sco-report-summary. The Head of the HJPC Secretariat expressed this opinion during the evaluation team's meeting with him. Formative Evaluation of JSDP II Page 25 that work is underway to make the DS open to the public, at the end of the base period or the beginning of any option period, JSDP II should train interested civil society organizations to use the DS in their advocacy work. If there is interest on the part of the media, JSDP II could consider providing such training also to journalists tracking JSRS implementation. Apart from that, JSDP II does not need to provide further support relating to DS during any option period. 6.) During any option period, JSDP II should concentrate much more support on entity MOJs, the BD JC and cantonal MOJs, especially the Federation MOJ. The project should make every effort to assist the Federation MOJ to perform its coordination role in respect of cantonal MOJs. JSDP II may need the help of USAID and others at the US Embassy in order to secure political commitment by the Federation MOJ for staffing its strategic unit appropriately and pursuing its coordination role more actively. The relevant officials need to meet in person as often as possible to develop good working relationships. This could be done through day-long meetings, longer retreats, joint study tours, establishing mechanisms for information sharing such as a Federation MOJs newsletter, etc. Projects that are within everybody's manageable interest, such as work on the joint Law on Prosecutors, should be used to boost effective coordination. At the same time, JSDP II and the Mission need to be cognizant of the severe lack of human and material resources that could prevent officials from attending the events suggested above. Therefore, JSDP II should be prepared to fund the costs for some of these meetings, retreats, study tours, etc. in order to allow as many officials as possible to attend. However, rather than having officials meet only in person, to save the travel time of officials and the cost of meetings, JSDP II should also explore the possibility of using video conferences to ensure regular coordination of activities between the Federation MOJ and the cantonal MOJs. If this proves practical, the project should determine what video conferencing equipment is available at the cantonal level and, together with USAID, consider the use of limited funds to purchase any needed equipment under JSDP II. It may not be necessary for each and every cantonal government to have video conferencing equipment – several cantonal MOJ representatives could meet at one location for the coordination meetings, depending on geographical proximity and availability of equipment. 7.) JSDP II should discontinue its assistance aimed at improving justice sector reporting to the EU. 8.) During any option period, JSDP II should continue to support efforts to reduce budget fragmentation in the justice sector of BiH. Special attention should be given to follow-up activities stemming from the Thematic Conference on this topic. Hopefully, the Thematic Conference will conclude with recommendations and an action plan for implementing them. To the extent possible, JSDP II should be engaged in and provide assistance to the implementation of that action plan. The team has not seen the text of the policy proposal for reducing budget fragmentation prepared by the FJP, but since the authors of the proposal have been the HJPC and the State MOJ of BiH, it can be assumed that the proposal is sufficiently realistic and designed so as to have at least some chances of being accepted by the other governmental institutions in the country. In all probability, any efforts to reduce budget fragmentation from 14 to four budget sources will require much more high-level Formative Evaluation of JSDP II Page 26 advocacy at the cantonal level than the work on the joint Law on Prosecutors. Therefore, JSDP II and the Mission should make every effort to engage the EU Special Representative and the USG representatives in such advocacy. Assistance could be provided also in running a public awareness campaign, legislative drafting needed for streamlining justice sector budgeting, and study tours, expert consultants, roundtable discussions, etc. 9.) During any option period, JSDP II should discontinue assistance to improving program￾based budgeting in courts. 10.) JSDP II should discontinue assistance aimed at introducing financial software in selected courts. 11.) It appears that work on a follow-up Justice Sector Reform Strategy covering the period after 2013 may commence soon. JSDP II should stay engaged in this effort through activities such as facilitating and monitoring the process, providing expert assistance as needed, advocating for more specific language and timelines in sections of the new strategy that fall under JSDP II's scope of work, etc. C. Component 3: Bolstering Public Confidence in the Rule of Law Background The third component of JSDP II was designed to enhance public confidence in the rule of law by achieving the following results: 1.) Non-governmental organizations (NGOs) in BiH would have the necessary technical and organizational capacity to conduct continuous oversight, analysis and advocacy relating to justice sector operations and reform processes, and 2.) The work of the judiciary would be more transparent as a result of civil society advocacy and oversight of the justice sector and rule of law issues. To reach these results, JSDP II built activities around the creation of the Justice Network (JN), an informal group of NGOs interested in various aspects of rule of law reform. One of the innovative features of the JN is its emphasis on having traditional NGOs and professional associations collaborate on improving the justice sector in BiH. At the outset of the project, JSDP II and the Association for Democratic Initiatives (ADI), a well-established NGO in BiH, recruited members for the JN and provided strategic planning support for the establishment of the network. ADI also helped to arrange capacity-building training on strategic planning; project management; advocacy; human rights standards in international conventions, the BiH Constitution and legislation, and the Universal Periodic Review (UPR).18 18 The UPR is a human rights monitoring mechanism of the United Nations’ Council on Human Rights, which reviews the human rights records of each UN member every four years. BiH was one of the countries whose human rights record was reviewed as part of the UPR process in 2010, so JSDP II used training on the UPR, roundtables with BiH officials about on justice-related issues included in the Formative Evaluation of JSDP II Page 27 By November 2011, the Justice Network had grown to 57 members and JSPD II had awarded seven targeted and 21 competitive grants totaling $624,138.42. Of that amount, ADI received two targeted grants totalling $118,766.03, primarily for its work in organizing the JN. For Year 3 of JSDP II, all the sub-grants were awarded through a competitive process to encourage local ownership. Component 3 staff told the evaluation team that the possible subjects for the sub-grants were drawn from the JSRS, and BiH’s war crimes and juvenile justice strategies. With JSDP II in the last year of its base period, the network is in a transitional phase: ADI essentially is volunteering its services as the communications hub for members, while the Human Rights Center at the University of Sarajevo is volunteering to host JN’s website and members are discussing how the network should be organized in the future. Members are debating questions such as whether JN should introduce a membership fee to finance its Secretariat, and whether membership in the network should continue to be as open as it has been until now. All of the international organizations and donors interviewed by the evaluation team are familiar with the JN and wish it well: it was known to fewer of the government officials who met with the evaluation team. Professional associations active in the judicial sector are very familiar with the network, since the competitive grants issued by JSDP II through the network encourage activities in which a traditional NGO partners with a professional organization (for the Year 3 sub-grants, these partnerships were required). Findings for Component 3 1.) The JN is operating in crowded field of NGOs. The evaluation team was told during one interview there are more than 20,000 NGOs registered in BiH. During a second meeting the team was told there are more than 12,000 NGOs registered in the country, although fewer than 1,000 of them are active. Formal and informal networks of NGOs in BiH are not unusual. 2.) The NGO sector is very dependent on international donors for its survival. 3.) The “Justice Network” name already has a relatively high public recognition factor, despite its youth and the varied specialties, size, sophistication and capabilities of its members. 4.) Network members value their very active exchange of information on the network’s list serve: even non-members of the JN monitor it to see what is occurring on rule of law issues. UPR, and the work needed to allow members of the network appear before the UN’s Council on Human Rights as a springboard for generating interest in the JN among NGOs, justice sector officials and donors, and raising the public profile of the JN. Formative Evaluation of JSDP II Page 28 5.) Trainings for JN members may have been too advanced for some participants and too basic for others, since only one level of training on each subject was offered. 6.) Participation in grant-funded activities is still very difficult for voluntary professional associations, given their lack of staff and experience with grant procedural requirements. Working with professional associations is also challenging for their traditional NGO activity partners, since they have to act as the intermediary between JDSP II and the associations, and try to ensure the associations are meeting implementation and reporting requirements which the associations have not had to handle before. 7.) It is not realistic to expect ADI or any other organization to act as the communications center or secretariat for the JN indefinitely on a purely voluntary basis. 8.) ADI reports its staff spends considerable time on handling even fairly routine activities for the Justice Network, such as canvassing members to formulate responses to questions about the position of the network on particular topics from the media or other sources. This indicates that for the Justice Network to sustain its relatively high profile and coordinate activities among members, it will need some type of formal structure. 9.) Justice Network members cannot survive individually or as members of the network without continued access to donor funds. 10.) As of November 2011, the JN website provided links to the websites of its member organizations, but did not include any standardized information about the members’ interests, goals, strengths or past projects. 11.) It may not be possible to have consistent data for indicators relating to improved capacities for NGOs because of the expansions of members in the JN. 12.) The PMP of September 2011 does not specify who would be measuring the progress of JN members for the NGO Sustainability and Advocacy Indices factors used as indicators. 13.) The broad and positive descriptions of the goals and achievements of the JN in the JSDP II annual reports are not always consistent with the type of indicators included in the project’s Component 3 PMP. For instance, surveys conducted of JN members by JSDP II indicate training provided by JSDP II has helped individual NGOs improve their “technical and organizational capacity to conduct continuous oversight, analysis and advocacy relating to justice sector operations and reform processes,” but the PMP does not attempt to link these conclusions to any overall findings about the effectiveness of the JN as a network, nor does it have any indicators relating to how much of a difference the trainings have made for individual JN members. 14.) The only indicator in the PMP relating to NGO monitoring of justice sector reform (No. 9) relates to the UPR process, and does not include a focus on the type of individual reforms being pursued under several of the competitive sub-grants issued by JSDP II. This arguably means only monitoring by the some of the most established and experienced members of the Formative Evaluation of JSDP II Page 29 JN is being considered, and these organizations probably were capable of and pursing monitoring before the creation of JN. 15.) The goal of Component 3 is to increase public confidence in the rule of law. Presumably, JSDP II is supporting training for journalists because they are the conduit for providing information about war crime cases to the public, and better coverage would help increase public confidence in the rule of law. However, none of the PMP indicators relating to increased transparency of judicial operations consider whether the public is reading or viewing the increased coverage of war crime proceedings. For instance, Indicator No. 16 on increased awareness of court activities relating to war crimes, does not track readership of the publications being monitored or the number of viewers watching or listening to programs carrying any electronic coverage. Without this data, there is no way to determine if the public (as opposed to the media) is aware of court operations, and the project cannot definitively claim any success in increasing public interest in the rule of law, much less increasing public confidence in the rule of law. Recommendations for Component 3 1.) USAID/BiH should extend work on Component 3 through the option period, to give the Justice Network a stronger chance of sustainability. 2.) JSDP II should continue to provide competitive sub-grants during the option period, but the sub-grants should be linked only to the recommendations of the Structured Dialogue to support the goals of the other components of JSDP II, and to concentrate the interests of Justice Network members.19 The project should continue to require that at least some grants be based on partnerships between professional associations and more traditional civil society groups. 3.) If JSDP II does not receive enough credible applications relating to the Structured Dialogue recommendations during future competitive grants process to ensure that at least some civil society activities directly support the recommendations, JSDP II should consider awarding targeted grants instead. 4.) USAID/BiH and JSDP II should provide training to network members on analyzing information about implementation of the JSRS, once the public is given access to the Documentation System. 5.) To encourage greater participation by professional associations in network activities and to ease the burden on groups working with professional associations (see Finding No. 6 19 Such a donor-driven approach in the provision of grants may not be appreciated by some civil society organizations. On the other hand, it should be kept in mind that unlike USAID/BiH’s Civic Advocacy Partnership Project, JSDP II is not a general civil society project and Component 3's raison d'être is meeting JSDP II's overall objectives. Formative Evaluation of JSDP II Page 30 above), the project should provide specialized capacity-building training or other assistance on grants administration for professional associations during the base and/or option periods. 6.) Any other training to JN members offered during an option period should be based on two factors: the needs identified by JN members, and the position of USAID and JSDP II on how the implementation of the recommendations of the Structured Dialogue could be furthered. 7.) During the option period, JSDP II should again fund the costs of administering the network’s web site. JSDP II support should include funding the design of a template to be completed by each network member and posted on the public portion of the website. The template would standardize information about the interests, capabilities, track record and projects of each member, making it much easier for network members and prospective donors to identify possible partners and counterparts. One of the questions that network members should answer during the option period would be how the web site would be funded after mid-2014. 8.) To increase the chances of JN’s long-term sustainability, JSDP II should help the network establish a strong presence on social media and respected comment sites during both the base and option periods. 9). During both the base and option periods, USAID/BiH and others within the U.S. Embassy should provide more intense and continuous promotion of the network with other donors, including the National Endowment for Democracy, justice sector actors and the public. 10.) JSDP II should review the PMP for Component 3 to ensure it explains how all the required data will be gathered and by whom, and to determine whether the indicators track progress towards the desired results of Component 3. Formative Evaluation of JSDP II Page 31 V. RECOMMENDATIONS FOR FUTURE ACTIVITIES Under its Scope of Work, the evaluation team was tasked with making recommendations for future Mission involvement in the rule of law sector with a specific focus on areas where USAID programs can have the greatest impact. On the basis of the written materials reviewed and the meetings conducted in the framework of the evaluation, the team has identified eight areas of possible future Mission involvement in the justice sector. A short overview of each one of these areas is provided below. Recommendations are presented in order of priority based on the evaluation team's assessment of the extent to which each of the activities would be in USAID's manageable interest. It is understood that while many of the potential programs outlined below would not be appropriate for implementation by USAID, they would enhance rule of law in BiH and thus could be considered by either other donors or the government. In accordance with USAID’s instructions, the team has suggested programs the team thinks would produce the most improvement to judicial operations overall, without regard to the length of any program or its cost. In examining the recommendations for future activities, the Mission should keep in mind that all meetings held by the evaluation team and most of the written materials reviewed were directly linked to the JSDP II Scope of Work. Therefore, any conclusions regarding future programming needs in the rule of law sector are based on incidental evidence rather than on in-depth analysis. Furthermore, due to the JSDP-related focus of the meetings conducted in the course of the field work, the evaluation team was not able to do much to verify its findings on possible future areas of work with local stakeholders. For these reasons, the implementation of any of the following recommendation warrants an in-depth preliminary assessment with a view to the current state of affairs in the field, possible involvement of other donors, and the feasibility of the proposed program. 1.) Reforming enforcement of civil claims. Effective enforcement of civil claims is a key element of a well-functioning justice system. Moreover, it directly contributes to the citizens’ right to have their legal rights and obligations decided upon in a fair manner and in reasonable time, as guaranteed in Article 6 of the European Convention for the Protection of Human Rights and Fundamental Freedoms. In BiH, enforcement of civil claims is perceived as extremely inefficient. Pillar 1 of the JSRS points to enforcement of civil claims as one of the reforms which would contribute to raising the efficiency and effectiveness of the justice system. The JSRS notes that around 56% of the backlog of all cases consists of execution cases for small-value claims. The EU Progress Reports on BiH also highlight this issue. Enforcement of civil claims in BiH includes both execution of civil judgments issued by courts and execution based on so-called “authentic documents” (such as promissory notes or utility bills), which is currently done through the court system. This approach is very similar to the system most of the countries in the region used to have. Recently, many of these countries Formative Evaluation of JSDP II Page 32 such as Bulgaria, Macedonia, Serbia, Albania and Montenegro, have reformed this area. Croatia is currently working to introduce such a reform. In reforming enforcement of civil claims, all of these countries looked into different options but ultimately decided to free court resources by assigning enforcement to legal professionals outside the judicial system. The model chosen in all of these countries is based on the experience of the Netherlands, where enforcement is being done by private enforcement agents who are members of a professional association and are appointed and closely controlled by the Ministry of Justice. This has been an extremely popular reform. Notably, out of 27 member states of the European Union, 19 have adopted this institution, and out of 12 member states which joined the EU in 2004, 10 had introduced it before their accession. In many of the countries in the region which reformed enforcement of civil claims, USAID led the effort. This has been the case in Bulgaria, Serbia, Macedonia. In Albania and Croatia, EU funded this reform. Another donor interested in the topic is the Government of the Kingdom of the Netherlands through its Balkans Enforcement Reform Project – BERP (covering Albania, Bosnia-Herzegovina, Croatia, Kosovo, Macedonia, Montenegro and Serbia).20 BERP has traditionally funded smaller-scale training and public information activities rather than large￾scale reform advocacy and legislative drafting. The project began in early 2009 and ended in mid-2011. In considering future programming in the Rule of Law field in BiH, the evaluation team recommends that USAID look into the reform of enforcement of civil claims for several reasons:  This reform has the potential of achieving a significant impact on the functioning of the judiciary, on meeting the criteria for EU accession and on improving the business climate in the country.  Experience in neighboring countries demonstrates that in the presence of sufficient political will, such reform legislation can be drafted, advocated for, adopted and implemented in a relatively short period of time (e.g. five years).  USAID has experience and expertise in implementing this reform in the region. Should USAID/BiH choose to launch a program on enforcement of civil claims, the program could, very roughly, comprise the following activities:  Assessment of the options for reforming enforcement of civil claims, including the introduction of private enforcement agents and additional legislative solutions to secure enforcement rights.  Formation of a working group to develop reform legislation;  Providing technical assistance to the working group and potentially organizing a study tour to one or more countries, which have reformed enforcement pursuant to the selected reform option;  Reform advocacy;  If a new law on enforcement is adopted, providing legislative drafting assistance for 20 See homepage of the Balkan Enforcement Reform Project at http://www.berp.info/1/about-berp. Formative Evaluation of JSDP II Page 33 the development of all accompanying rules and regulations;  Assistance in implementing the law, including through training, institution-building support to any new bodies that have been created, outreach, etc. 2.) Reforming enforcement of utility cases. The second area of future Rule of Law involvement that the Mission could consider relates specifically to the enforcement of utility cases. In terms of backlog and social significance, they are perhaps the most important category of enforcement cases. JSRS, the structured dialogue recommendations and the EU Progress Reports on BiH all emphasize the seriousness of the problem stemming from utility cases clogging the court system. If enforcement of civil claims is reformed and taken out of the justice system, this would naturally lead to relieving the courts from utility cases as well. On the other hand, the urgency of the problem with utility cases and its social implications may warrant a special, and perhaps accelerated solution to this particular issue. This evaluation team recommends that USAID/BiH explore the possibility of launching a program devoted to resolving the issue with utility cases backlog for several reasons:  A successful program in this area would have a very visible impact, with very strong positive implications for the functioning of the BiH Judiciary and for BiH's performance from the viewpoint of the EU and the Council of Europe.  A program on utility cases can in all probability be implemented and achieve results more quickly than the program on enforcement of civil claims proposed above.  A dedicated program can give more consideration to the social issues tied to utility cases as compared to the general program on enforcement of civil claims proposed above. If USAID/BiH decides to launch a program on reforming enforcement of utility cases, it could comprise the following activities:  Assessment of the options for reforming enforcement of utility cases, including consultations with interested agencies, businesses and NGOs and court management teams which have had success in reducing the backlog of utility cases in BiH;21  Formation of a working group to develop reform legislation;  Providing technical assistance to the Working Group;  Reform advocacy;  Assistance in reform implementation, including through drafting regulations, training, outreach, etc. 3.) Improving the performance of courts by enhancing their administration. 21 According to USAID/BiH, the Sarajevo Municipal Court President has been particularly successful in this regard. Formative Evaluation of JSDP II Page 34 The third area of future Rule of Law involvement that the Mission could consider is the improvement of court management. Generally, court (and prosecutor office) management is not well developed in the entire Balkan region. Court presidents come to office with no training on how to perform their management functions. There is no tradition of specialized personnel responsible for managing the operations of judicial institutions similar to court managers in the USA. The closest to this type of function in BiH is the position of court secretary, but they do not receive court management training either. In the course of this evaluation, representatives of the judiciary shared that the newly introduced computerized case management system is able to generate numerous types of reports but the people responsible for managing courts usually cannot use the reports to improve operations. The same observation was made during the discussion of the optimal and predictable timelines: interviewees felt that court presidents did not have the capacity to use those as a tool for speeding up proceedings and reducing backlog. Similarly, chief prosecutors who were beneficiaries of JSDP II assistance on strategic planning, need-based budgeting, etc., have put the information they received to very different uses, as demonstrated in the responses to the questionnaire sent on behalf of the evaluation team, mostly depending on their personal style and energy levels. The above examples indicate that there is a need for the development of the managerial function in justice institutions, including through the introduction of a comprehensive training program and a career for professional court administrators/managers. Pillar 1 of JSRS also includes ongoing training in management for managerial staff in the institutions of the BiH judiciary as one of the actions to improve effectiveness and efficiency in justice sector operations.22 This evaluation team recommends that USAID/BiH explore the possibility of launching such a program for the following reasons:  There is a demonstrated need for such program.  This activity would contribute to meeting some of the goals of the JSRS.  The program could use some of the lessons learned from JSDP I and JSDP II's activities with respect to Model Courts and Model Prosecutor Offices.  The program could contribute to judicial efficiency in general and backlog reduction in particular, which are priorities for JSRS and EU accession.  The US experience in court management is particularly positive, and many US court administrators also have extensive experience in working with courts in other countries.  It might be possible to draw on the expertise of court administrators working at the Administrative Office of the US Courts or members of the International Association of Court Administrators at reduced cost. 22 See p. 20, point 1.2.6. of the Justice Sector Reform Strategy 2008 – 2012 at http://mpr.gov.ba/userfiles/file/Projekti/24__SRSP_u_BiH_-_EJ.pdf. Formative Evaluation of JSDP II Page 35 The action plan for implementation of the JSRS is quite specific in terms of activities directed at management training. One of its Strategic Programs is “Constantly improve training in the area of management for managerial staff in the institutions of the BiH judiciary”23 and it includes a list of four concrete activities, which would contribute towards implementing such a program:  Ensure that Judicial and Prosecutorial Training Centers (JPTCs) have a mandate to provide management training that meets court administration needs;  Upgrade the existing manuals for managerial staff in courts, in terms of adjusting to the needs of chief prosecutors;  Improve training in the field of leadership in the judicial institutions of Bosnia and Herzegovina;  Supervise the implementation of established programs and analyze effects of training. A potential USAID/BiH program on improving court and prosecutor office management could build on the JSRS action plan and comprise the following activities:  Assessment of the needs of the justice sector institutions in terms of management and administrative training, including interest in and benefit from the introduction of the position of court manager (the Open World Program of the Library of Congress, a country-specific International Visitor Program or even the State Department’s Speaker Program could be used to introduce BiH judges to use of court administrators in the United States even before any new program began);  Development of a specialized job description and career path for court managers/administrators and any corresponding regulations;  Development of comprehensive training for the managerial staff in the justice sector institutions (including court presidents, court secretaries, court managers or other staff, as appropriate);  Delivery of the training, evaluation, and inclusion of the course in the curriculum of JPTCs;  Assistance to the appropriate justice sector institutions in establishing the procedures for recruiting court managers/administrators, once such a position has been introduced. 4.) Continued work on reducing foreseeable time frames. As the work of CEPEJ demonstrates, courts and prosecutor’ offices should always be concerned about the foreseeable time frames of the cases they handle. The more sophisticated the analysis of the reasons for backlog and delays, the more useful the data will be in designing legislative, regulatory or other procedural solutions to the problems. 23 See p. 14, point 1.2.6. of the Second Revised Action Plan for the Implementation of the Justice Sector Reform Strategy in BiH at http://mpr.gov.ba/userfiles/file/Strateško %20planiranje/06_4%20Drugi%20revidirani%20AP%20SRSP%20u%20BiH%20-%20EJ.pdf. Formative Evaluation of JSDP II Page 36 This type of analysis is just starting in BiH and no baseline information has been developed. Given the number of actors who will have to participate in setting the foreseeable baselines, it is unlikely that courts and prosecutors will be able to make much headway in reducing these timelines by mid-2014. 5.) Supporting the establishment of an Appellate Court in BiH. The Court of BiH was formally established by a Decision of the High Representative dated 8 May 2002. The Court of BiH has limited criminal, administrative and appellate jurisdiction. Within its criminal jurisdiction, the Court of BiH tries war crimes, organized crime, economic crime and corruption cases. The Court of BiH also has jurisdiction over the criminal cases punishable by the laws of entities and the Brčko District of BiH, under the conditions specified and stipulated by law. Administrative jurisdiction of the Court of BiH covers complaints against decisions issued by BiH institutions as part of their public functions. Additionally, the Court of BiH tries property disputes between the state of BiH, entities and the Brčko District, as well as other property disputes when the Court’s jurisdiction is stipulated by the BiH laws or an international treaty. As part of its appellate jurisdiction, the Court of BiH rules on appeals from decisions issued by the Criminal Division or Administrative Division, and rules on complaints pertaining to violations of the Election Law of Bosnia and Herzegovina. The Court of BiH does not act on appeals from decisions issued by entity courts. Appeals for cases in the Court of BiH are handled by appellate division of the same court which issued the initial decision. This situation creates a perception of a compromised second instance hearing and is contrary to the principle of judicial independence. A proposal was developed in 2008 to create an Appellate Court for BiH but there has been no further progress in this area; therefore, the second Structured Dialogue recommendations state that the European Commission “[expects] that the proposals adopted in 2008, also reiterated by the HJPC at its session on 9 November, be put into Parliamentary procedure before the next meeting of the structured dialogue.” If the decision to establish an Appellate Court of BiH is made, a comprehensive program providing assistance to this court would be needed for several reasons:  The Appellate Court of BiH would be one of the most important judicial institutions in the country;  If the Appellate Court is established as quickly as recommended by the Structured Dialogue, it would need assistance soon and compared to other donors, USAID can launch a program quite quickly;  The establishment of the Appellate Court is important for meeting the political criteria for EU accession, which coincides with US foreign policy goals for BiH. Formative Evaluation of JSDP II Page 37 The program supporting the establishment of an Appellate Court for BiH should include activities related to court administration, technical assistance and training, drafting of rules and regulations and equipment purchases. 6.) Introducing public education on issues related to the court system in schools. In all countries in the region, including BiH, there is a serious lack of understanding by the general public of the work of the judicial system. This is part of the cause for the very low level of trust in the judiciary. The only public education campaigns or work with students that are being done in this respect are on ad hoc basis, including public education carried out through JSDP II-provided sub-grants. BiH's younger generation and ultimately the entire society would benefit greatly from the introduction of a course in primary and/or high schools which teaches the basics of the court system. The benefits of such a program are manyfold:  Over the long-term it would raise society's trust in its judiciary;  It would demonstrate the benefits and mechanisms of civilized dispute resolution;  It would encourage judges and prosecutors, perhaps through their professional associations, to have more interaction with the public. A public education program on the court system could include the following activities:  Assessment and development of a policy proposal that includes the modalities of such education (i.e. what particular topics should be covered at which levels, which grade shall take the course(s), how many hours per year the course shall cover, etc.)  Advocacy for the introduction of public education on the court system as a (preferably) mandatory course in BiH schools;  Developing the course materials and training teachers about their use;  Engaging judicial and prosecutorial professional associations in the delivery of the course. 7.) Improving service of process. The delivery of court notices to parties and witnesses should be a routine matter in most cases, but in much of the region service of process is so inefficient it creates significant delays in court proceedings. Countries in the region trying to eliminate these problems generally have examined only ways to make service of process more efficient (e.g., ensuring better rates of return of service of process notices by the postal service). However, for a much greater impact on the efficient handling of cases, BiH should consider more fundamental changes to its rules (e.g., fining or otherwise penalizing parties and witnesses who do not provide current addresses to the court, and making service the responsibility of the parties rather than the court). Work on service of process could be a relatively small stand-alone project, or part of a larger project dealing with improvement of court operations. Formative Evaluation of JSDP II Page 38 8.) A follow-up to the Model Courts Initiative Finally, a donor assistance program could consider revisiting USAID's Model Courts Program implemented under JSDP I. This work was praised several times by counterparts interviewed by the evaluation team. At the same time quite a few interviewees expressed regret that the program had not been successfully rolled out to the rest of the courts as originally planned. A new program could improve the operations of those courts which were not included in JSDP I, and introduce further improvements in court operations and administration for those courts which had been included. List of Appendices Appendix A – Structure of the court system in BiH Appendix B – Answers to the MPOI questionnaire drafted by the evaluation team Appendix C – Structure of the implementation of the Justice Sector Reform Strategy Appendix D – Scope of work for the current evaluation Appendix E – List of interviews conducted by the evaluation team Appendix F – List of materials reviewed by the evaluation team Appendix G – Answers to specific questions contained in the Scope of Work Formative Evaluation of JSDP II Page 39 Formative Evaluation of JSDP II Appendix B Page 1 Appendix B Summary of Prosecutor's Offices' Answers to a Questionnaire Prepared by the Evaluation Team East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 1. What type of equipment or materials has your office received from the JSDP II project (e.g., furniture computers, copy or fax machines, filing materials)? 2 computers, 2 copying machines and 2 fax machines Fax and a copying machine So far we haven't received anything. We opted for computer equipment and provided the necessary equipment specification. 8 Armchairs, 3 air conditioners, 11 archival shelves, 1 fax machine, 2 copiers , shelves for the TCMS files, TCMS file folders, computers, reconstruction of the clerk's office. Note: computers, shelves for TCMS files and TCMS file folders and reconstruction was done through the HJPC. From JSDP II the PO obtained computers with printers, scanner, copy machine, 2 fax machines, still￾picture camera and movie-camera. Camcoder FS305 - 1 pc., Copying machine Canon iR2520 - 3 pcs.., Camera Canon A800 - 2 pcs, Canon Fax L 140 - 3 pcs The PO, with financial assistance JSDP II provided furniture (cabinets and shelves) Sarajevo Cantonal PO received technical equipment in the form of digital cameras, and in terms of office furniture we have received a larger amount of TCMS office cabinets. Formative Evaluation of JSDP II Appendix B Page 2 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 2. Has your office received architectural plans or designs for improvements to the premises of your office from JSDP II? If the answer is yes, has your office been able to put them to use? How? No No, because it was not necessary. No. Cantonal Prosecutor's Office has received the proposal of construction work to be done to improve the conditions of prosecutors in BiH with the assessment of expenses.Plan not used because of high costs of approximately 77,000.00 KM Yes. Received a proposal for construction activities, which would improve the working conditions of the prosecution with cost estimates, but it has not been implemented by JSDP II. Yes, it has not been used yet because we do not have the funds. No. This Report made by JSDP II shows that the premises and equipment are fully functional and the PO does not need any interventions in this area (The proposal to further equip the detentions room has not been implemented because this space has been made into a library) . Yes, we received from representatives of JSDP 2 a document that is an overview of the state of the space with suggestions for improvement, and cost estimates totaling $ 426,000,00 KM; we engaged an expert committee of the Construction Bureau of Sarajevo Canton, whose estimate was higher by 22,000,00 KM; we are currently seeking donations because of the current financial inability to finance this project by the Government of Sarajevo Canton. Formative Evaluation of JSDP II Appendix B Page 3 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 3. Please describe what remodeling, if any, was done to your offices by JSDP II. No, but it promised that, if finances permit it, they would renovate the clerks office in Visegrad. Photos were taken of the clerks offices of Visegrad and East Sarajevo where conditions of work are poor and don't comply with minimum standards. No, but it was also not necessary. No. Reconstruction of the clerks office was performed on the basis of an open public procurement procedures issued by the HJPC, which also selected the best suppliers and works have been completed No. No. No. See answer 2. No. Formative Evaluation of JSDP II Appendix B Page 4 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 4. Did your office develop any portion of a manual case management system with the help of JSDP II ? If the answer is yes, please give a short description of the JSDP II assistance and how it has helped your office? No No. No. No. No. No. Yes, we have procedures for the management of case files and archives and for providing access to documents from the archives. JSDP IIhelped through education in the field of archival operations, which has significantly assisted in the establishment of the procedure. No. 5. Does your office use case movement control cards? If the answer is yes, have any of those cards been provided by JSDP II? No No. No. No. No. No. No. We do not understand what are the cards that you have in mind; we use the TCMS as all other Pos. Formative Evaluation of JSDP II Appendix B Page 5 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 6. What assistance or advice relating to manual case management has your office received from JSDP II since the formal training on this subject? Who provided the assistance or advice, and (approximately) how often was it offered? As far as we know, we haven't received any such assistance after the training. No. No. We did not receive any advice or training on this topic. We did not get any help or advice from JSDP II related to manual case management since the prosecution cases are managed in electronic form, i.e. there is automatic case management of all cases in Prosecutors offices. HJPC is providing training in electronic case management 3-4 times a year. Management of cases in the District Court of Banja Luka, is carried out in accordance with the Regulations on the automated management of cases in the POs (TCMS), which adopted by HJPC. See answer 4. With the introduction of TCMS the prosecution moved to an electronic case management system, for which the HJPC provided the necessary training to including in respect of the manual case management to the extent that it is required by this system. No. Formative Evaluation of JSDP II Appendix B Page 6 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 7. Did your office have a strategic plan before the strategic planning training by JSDP II? Yes - The strategic plan was aimed at making important decisions and determining the most important activities and indicates what the organization does, and to plan its future activities, determine priorities and efficiently allocate resources and monitor progress against strategic goals. No. No. No. No. No. No. Yes. Formative Evaluation of JSDP II Appendix B Page 7 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 8. Please describe what benefit resulted for your office from the training on strategic planning provided by JSDP II. This PO is able to better meet its duties under the law and other regulations and everybody within POs office is striving towards achieving these goals. We learned of the importance of strategic planning in our work. The training on strategic planning will greatly facilitate the process of developing the budget of the PO. Strategic planning identified a number of problems which is in some ways good, but unfortunately we are unable to make the necessary changes, so we expect your full support With help from JSDP II this PO adopted a three￾year strategic plan, which this Office continues to update each year so as to have a strategic plan for every next three years. Training on strategic planning gave us an opportunity to present our experience, exchange opinions, and through practical training together with expert team develop the strategic plan of our PO, which helped us plan our activities, determine priorities, mobilize available resources to achieve stated objectives. We learned about the methodology of strategic planning, and adopted an institutional strategic plan for the period from 2011th -2014. Primarily in terms of strategic planning we confirmed the existing way of work; however, it is necessary to emphasize that all strategic planning as a rule remains an expression of the desire to continually improve the quality of work and better work results; the sole problem being the lack of material funds. Formative Evaluation of JSDP II Appendix B Page 8 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO - 9. Please provide the name of the Point of Contact for strategic planning within your office. Vera Cosovic – Head of the financial department The position is currently vacant. The head of the financial department Lilija Trivun. Suada Begić and Admir Arnautović Kristina Nogalo – Secretary of the PO The Chief District Prosecutor Strategic planning team comprises: Chief Prosecutor Zekerija Mujkanović, Deputy Chief Prosecutor Slavo Lakić;, Secretary of the PO Samir Beganovic; Head of the Judicial Finance Commission Zeljko Lozić, Chief of the PO Administration Amir Imamović Chief Cantonal Prosecutor Nives Kanevčev with the deputies and the Secretary. 10. What participation has the person who serves as the Point of Contact had with the JPF or the HJPC Secretariat relating to strategic planning? Sends budget requests, i.e. the plans determined by the budget framework document and created within the framework of given limitations; Sends the budget requests with the real needs of the PO and HJPS provides objections and suggestions on the request. Since the person who performed these functions left the Republican Prosecutors Office, we cannot answer this question with any degree of specificity. Contacts with the HJPC related to the development of the budget. So far we haven't had contacts with HJPC in respect of strategic planning. Good. In the process of adoption of the Plan we did not have contacts with HJPC but we took into account the strategic plan of the HJPC as well as other strategic documents regarding justice in BiH as a whole. None. Formative Evaluation of JSDP II Appendix B Page 9 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 11. What assistance or advice relating to strategic planning has your office received from JSDP II since the formal training on strategic planning? Who provided the assistance or advice and (approximately) how often was it offered? Training regarding the conceptual phase of the creation of the plan, strengthening and support to the efficiency of the PO and the reduction of backlog A draft strategic plan for the Republican PO was developed. Help consisted of developing a strategic plan for the Prosecution, and this type of assistance was provided through JSDP II's external consultants Ranko Šapina and Hrvoje Batinic Joško Mandić, Ermin Sarajlija, Hrvoje Šapina, Ranko Batinić, Sanela Tunović After completing training on strategic planning by JSDP II, PO received assistance in strategic budget planning and PR. Help was provided by JSDP II consultants H. Šapina, Ranko Batinić for strategic budget planning and Sanela Tunović for PR. Help on finalizing the strategic plan and a visit from JSDP II. After the adoption of the plan it was submitted to JSDP but there were no further contacts in this area. There was no need because the Sarajevo Cantonal Prosecutor's Office had used this methodology for many years already. 12. Please describe what benefit, if any, your office saw from the any of the budget training your personnel received from JSDP II. The development of the plan has been simplified and adapted to our core work. Better budget development. Benefits reate to the adoption of programs aimed at achieving strategic goals and operational objectives and determining the necessary resources to achieve those goals; The benefits of training are numerous, but little could be changed because of the way of work of the Ministry of Finance and the Ministry of Justice in the Tuzla Canton Training resulted in the development of a three-year strategic budget plan that we submitted to the MOJ and the government of the West Herzegovina Canton in order to facilitate implementation in the coming budget year. To more easily identify our priorities. Program budgeting is in line with the Strategic Plan, which should allow easier presentation of budget requests and justification thereof, and thus contribute to elimination or reduction of the risk of disapproval. We have been using program based budgeting for the last 7 years. Formative Evaluation of JSDP II Appendix B Page 10 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO - 13. Is your office preparing program-based budgets now? If the answer is yes, please describe what benefits, if any, your office has seen from using this type of budgeting. Only partially, because of the limitations of the budget guidelines of the MOF in RS Yes, as mentioned, a more realistic and better budgeting Yes, because now the budget is made on the basis of specific programs We are preparing our budgets as per the guidance of the Ministry of Finance of the Tuzla Canton. Yes, benefit has been the expansion of the systematization for working places some of which are already filled, and some will be completed as planned in the coming budget year. We prepare our budget on the basis of program budgeting. Due to the limited budget we didn't get understanding with regard to all our proposed programs and the funds we were provided with were based on last years budget. Yes, the benefits have been described in the answer to question 12. This was answered to in the previous questions. 14. Was your office preparing program-based budgets before you started working with JSDP II? Yes, based on the instructions of the MOF RS. No. No. No. Yes. Yes. Yes, during budget preparation phase. Yes. 15. What benefit, if any, has your office seen from the program-based budgeting information provided by JSDP II? Education, simplification, motivation, realistic planning. Benefits of the training in program budgeting are mainly the proper determination of program priorities in order to achieve strategic and operational objectives The benefit was the increase in the systematization of working places. To have a better understanding of the purpose and ways of budget planning. See answer 12. It confirmed the appropriateness of the current methods of work. Formative Evaluation of JSDP II Appendix B Page 11 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 16. What budget-related advice or assistance has your office received from JSDP II since the formal budget training provided by JSDP II? Who provided the assistance or advice and (approximately) how often was it offered? No. We haven't asked for additional assistance but we are sure that your consultants Šapinić and Batinic would have been very responsive. No. No. Assistance provided to us consisted of education and training seminars that took place within the projects, as well as visits by Hrvoje Batinic Šapina and Ranka Batinica, and the fact that we were provided with a draft strategic plan. No. No. Formative Evaluation of JSDP II Appendix B Page 12 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 17. Does your office have a librarian? If the answer is yes, please describe what non-library duties, if any, that person has. No, currently this function is being carried out by the Secretary of the PO and the Technical Secretary of the Chief Prosecutor in the PO. The technical secretary is doing that. No. No. No. The Rulebook on the internal organization of prosecutors offices in the RS does not specify in its systematization a librarians position; the librarian duties in the District Court of Banja Luka, are being carried out by the head of the Department of Personnel who also does other support work, in addition to his regular duties. No. The Cantonal PO of Sarajevo has an independent position of a Higher Independent Referent – librarian. With regard to the question on non￾librarian tasks, the employee monitoring of new laws, copying and distribution depending on the specific needs of the case and most frequently for the Department of Economic Crime and the Office of the Chief Prosecutor. Formative Evaluation of JSDP II Appendix B Page 13 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 18. What benefits has your office received from the library training provided by JSDP II? It was beneficial but we did not receive the promised assistance (books and library shelves) Expert help and advice regarding the establishment of libraries and information on the possibilities of obtaining the necessary data; We do not have a librarian's office. We saw the need for the improvement of libraries through the establishment of T-Portal, through which we would be able to quickly and easily get the information we need. The analysis of the state of library￾information services in our Prosecutor's Office with conclusions and recommendations for improvement done by an expert team of the project gave us a chance to reflect on the current situation, identifying the needs and where possible, taking measures to improve the situation and the work of the library. This training has contributed to a stronger focus on improving the prosecutorial administration of internal information services and linking to external databases for institutional needs. During the first meeting we stressed that regardless of the excellent quality of the suggestions of the JSDP experts, we will not be able to implement them because of the lack of resources. Our suggestion is to broaden the already existing project under HJPC (Center for Court Documentation) because if the Cantonal PO in Sarajevo cannot afford to hire a professional librarian, then probably other Pos are not able to do so either. 19. What library-related assistance or advice has your office received from JSDP II since the formal library training? Who provided the assistance or advice and (approximately) how often was if offered? None. No. No. No. No. No. No. No. Formative Evaluation of JSDP II Appendix B Page 14 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 20. What benefits has your office received from the archival training provided by JSDP II? We haven't had but hope to have. The common conclusion after the training for archival operations was that in addition to the application of laws and regulations, appropriate accommodation capacity is necessary, special staff involvement in these affairs and regulardisposal of worthless archived materials; We do not have an archives person to benefit from this training. Even before the training our archive was good. Based on the analysis of the state of the archives and the data collected through workshops and in the office, the expert team proposed, as a possible solution, a draft ordinance on archival operations, a template of archival books, a list of categories of documentary material, which can help us in organizing our internal operations. The Ordinance on archival operations of the PO was adopted and a List of the Categories of registered material with the time periods for safe￾keeping was developed which regulated the corresponding internal procedures and records, as the formal pre￾conditions for the lawful and regular implementation of archival operations. Due to the unresolved financial needs as a precondition for the realization of the archive issues in the Sarajevo Cantonal Prosecutor's Office, we were not able to start addressing this issue in 2011 due to lack of funding despite the fact that two years ago, we demanded that the Government approve funding for this activity, which has not been done. Also, JDSP II project did not have the necessary funds. 21. What archival-related assistance or advice has your office received from JSDP II since the formal archival training? Who provided the assistance or advice and (approximately) how often was it offered? None. No. A visit of JSDP representative in late 2010. No. No. No. No. No. Formative Evaluation of JSDP II Appendix B Page 15 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 22. Which person in your office is responsible for archival issues and operations? Please describe what non-archival duties, if any, that person has. We do not have such a position, the chief of the clerk office and the clerk are performing these functions. The clerk is doing this. No, the clerks are performing these functions. We do not have an archives office. No. The District PO of Banja Luka has systematized and filled position of archivist, who is exclusively responsible for archival operations. These activities will have to be assigned to one or more employees, who work with case files while performing office management tasks such as reception and entry of cases, taking of items out the evidence room, etc. Yes, this person manages the archive of the PO, receives archived files and keeps an archive book, publishes documents from the archives of which he keep records, makes preparation for writing off and destroying old archives, destroys records past their retention period, and performs other activities on behalf of Chief Cantonal Prosecutor and Registrar. Formative Evaluation of JSDP II Appendix B Page 16 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 23. Which person in your office is responsible for making sure that standard information is available to the public about your office operations on a routine basis (e.g. information on web pages or in pamphlets)? What other duties, if any, does that person have? Neven Kramer - Secretary of the PO - According to the Law on the Prosecution and the Regulations on internal organization of the prosecution, the prosecution Secretary manages, oversees and coordinates all services within the administrative departments of the prosecution. Also, in charge of public relations and external communication with other authorities. Currently this position is vacant. The Secretary of the PO. Expert Advisor for Public Relations Admir Arnautović is responsible for media relations and specialized questions that the media can have, and is addressing request for access to information in accordance with the Law on Access to Information. We have not specifically appointed a person responsible for public relations, so that the chief prosecutor and the other prosecutors inform the public about the criminal proceedings which they are in charge of, while the chief prosecutor and the secretary of the PO are in charge of the web page. The new systematization provides for a position for an expert associate for public relations, which is not filled. The activities of public relations in the Banja Luka District Prosecutor's Office are performed by the information official Đaković￾Maja Vidovic. The main tasks of the information officer are in accordance with the Regulations on internal organization and operations of the POs in RS, as well as with the adopted strategy of public relations, and the Rules of Procedure of the web editors of the District PO of Banja Luka. These are performed by the Secretary, in cooperation with the Judicial Commission as joint spokesman for the judicial institutions of the Brcko District. Basic responsibility of the Secretary is the execution of strategic programs of the PO relating to prosecutorial administration (labor relations, business archives, protection of personal and confidential data, information, security, registration, legislative affairs, etc.). The person in charge of public relations and public information is Jasmina Omićević. Her primary responsibilities are the development of analysis reports based on appropriate data and dispatching press releases in accordance with the instructions and with the prior permission of the Chief Prosecutor, processes requests for free access to information and render decisions in accordance with the Act. Formative Evaluation of JSDP II Appendix B Page 17 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO - 24. If the person responsible for providing standard information is different from the person responsible for responding to requests for specific information from the media or the public, what other duties, if any, does the person responsible for answering specific requests have? No, the Secretary is doing this. No, all information is provided by the Secretary. Same as the answer to question 23. Just as we do not have a person responsible for PR, we also do not have a person responsible for our relationship with the media. The information official is responsible for both public relations and media relations. The spokesman, as a person authorized to directly communicate with the media has an obligation to obtain information and PO's permission to publish the information and to coordinate and cooperate with the Secretary. The spokesman shall inform the PO with the help of the Secretary about all media requests that should be answered on behalf of the PO. At the same time the spokesman shall inform the PO of all media reports about the institution, and shall propose measures for improvement. The current situation is that the decisions under the Freedom of Information Act are announced by the Secretary, while all other tasks are performed by Ms. Jasmina Omićević. Formative Evaluation of JSDP II Appendix B Page 18 East Sarajevo PO Republika Srpska Bijeljina District PO Tuzla Cantonal PO Shiroki Brijeg PO Banya Luka District Brcko District PO Sarajevo Cantonal PO 25. What assistance or advice about public information or public relations has your office received from JSDP II since the formal training on those subjects? Who provided the assistance or adv ice and (approximately) how often was it offered? Specific assistance was related to the provision of the PR Strategy and a Manual for media relations in the model prosecutors' offices. This strategy and the manual are very helpful for training our staff on the way of communicating with the media. Since the person who performed these functions left the Republican Prosecutors Office, we cannot answer this question with any degree of specificity. Since June 2011 the gudelones on the creation of an online library have been delivered. Provided useful information and exchanged experiences in dealing with the media and the public in general. Generous assistance was provided by Amela Tunović, during a visit to the Prosecutor's Office and the training. Sanela Tunović provided assistance to develop the Strategy for Public Relations of the Cantonal Prosecutor's Office and assistance for the creation and broadcasting of radio programs together with JSDP II and the Media Centre Sarajevo. With the help of JSDP we developed a PR Strategy which assists us in our contacts with the media. The training contributed to the development of specific skills in managing the relationships with the public. Consultative assistance by Sanela Tunović when developing the strategy for public relations for Banja Luka District Prosecutor's Office. The Help of JSDP II consisted primarily of the presentation and submission of a draft strategy for public relations, which was taken into account when developing the strategy for public relations of the PO and the institutional strategic plan. No further concrete assistance has been provided by JSDP in this area. There was no help and any help would be very important. Appendix C – JSRS Structure provided by JSDP II Monitoring implementation of the JSRS, and providing general political and strategic guidelines. Duties Minister of Justice of BiH HJPC President JCBD President Ministers of justice in BiH Members Ministerial Conference Technical Secretariat Members Ministry of Justice of BiH HJPC JCBD Ministry of Justice of RS Ministry of Justice of FBiH Duties Prepares proposal for annual calendar of activities of functional working groups and MC  Prepares and convenes MC meetings, including preparation of agenda, analysis and drafting of consolidated institutional reports on implementation of the JSRS in BiH – quarterly, semiannual, three‐quarterly, and annual, and timely distribution of materials for MC;  Organizes, prepares, and conducts meetings for briefing of donors in the justice sector in BiH FWG 1 Judiciary FWG 2 Enforcement of Criminal Sanctions FWG 3 Access to Justice FWG 4 Support for Economic Growth FWG 5 Coordinated, Well Managed, and Accountable Sector Members: Members: Members: Members: Members: MoJ BiH (Milana Popadić) Chairperson MoJ BiH (Mustafa Bisić) Chairperson MoJ BiH (Nikola Sladoje) Chairperson MoJ BiH (Fatima Bašić) Chairperson MoJ BiH (Jusuf Halilagić) Chairperson HJPC (Amra Jašarević) HJPC / HJPC (Haris Grizović) HJPC (Admir Suljagić) HJPC (Katarina Peroš) MoJ FBiH (Hidajet Halilović) MoJ FBiH (Miralem Duranović) MoJ FBiH (Maja Kulenović) MoJ FBiH (Sanin Paćariz) MoJ FBiH (Entoni Šeperić) MoJ RS (Nikola Kovačević) JCBD M(DPamHjNanK Kaurinović) MoJ CS (Edo Hadžović) MoJ HNC (Jasminka Bratić) MoJ RS (Duško Šain) JCBD (Smajila Kikić) MoJ ZE‐DO (Vesna Ajanović) MoJ C10 (Zdravko Božić) MoJ RS (Snežana Marjanac) JCBD (Director of Legal Aid Office) MoJ TC (Ruzmira Bajrić) MoJ PC (representaed by MoJ TC) MoJ RS / JCBD (Jadranko Grčević) MoJ BPC (Emir Oković) MoJ WHC (Denis Primorac) MoJ RS (Nikola Panić) JCBD (Slobodan Zobenica) MoJ CBC (Renata Vrhovac) MoJ USC (Vahid Čoralić) List of acronyms: FWG – Functional Working Group MoJ BiH – Ministry of Justice of Bosnia and Herzegovina HJPC – High Judicial and Prosecutorial Council of Bosnia and Herzegovina MoJ FBiH – Federal Ministry of Justice MoJ RS – Ministry of Justice of Republika Srpska JCBD – Judicial Commission of Brčko District MoJ CBC – Ministry of Justice of Central Bosnia Canton MoJ CS ‐ Ministry of Justice of Canton Sarajevo MoJ HNC ‐ Ministry of Justice of Herzegovina‐  Neretva Canton MoJ ZE‐DO ‐ Ministry of Justice of Zenica‐Doboj Canton MoJ TC ‐ Ministry of Justice of Tuzla Canton MoJ PC – Ministry of Justice of Posavina Canton MoJ BPC ‐ Ministry of Justice of Bosnia‐Podrinje Canton MoJ WHC ‐ Ministry of Justice of West‐  Herzegovina Canton MoJ USC ‐ Ministry of Justice of Una‐Sana Canton MoJ C10 – Ministry of Justice of Canton 10 – Livno SCOPE OF WORK Mid-Term Evaluation of USAID/BiH’s Justice Sector Development Project II I. Purpose of Evaluation The purpose of this evaluation is twofold. First, USAID in Bosnia-Herzegovina (BiH) desires an objective analysis of the impact and success of the first three years of the Justice Sector Development Project II (“JSDP II”) to the overall development of the BiH rule of law sector. The second objective is to look at the broad status of reforms and needs in the rule of law sector in Bosnia and Herzegovina (BiH) and provide the Mission with the optimal areas of involvement of JSDP II during the optional two years of the project. II. Background Establishment of the rule of law through credible, effective and independent justice sector institutions capable of upholding the law and protecting the rights of all citizens is a critical ingredient in the development of a peaceful and stable Bosnia-Herzegovina. Strong justice institutions, especially at the State level, are crucial for t he development of an effective state. As BiH strives to move closer to E U accession, a stronger justice system is necessary to harmonize the legal framework with EU standards, and to enforce those standards at all levels of society. More effective courts and prosecutors that can resist political pressure are especially needed to overcome the on-going challenge of combating organized crime, and achieving justice for war crimes. A more efficient justice system will also underpin economic growth, by ensuring that citizens and businesses have a credible recourse to enforce contracts and resolve disputes. Greater citizen confidence that their justice system can protect the rights of all citizens and hold individuals accountable for crime and abuse is also a necessary foundation for national reconciliation and the development of a culture of tolerance. Despite considerable progress, the justice system in BiH is still too weak and divided to adequately address these challenges. The Dayton Peace Accords, which ended the war in Bosnia in 1995, put i n place a complex system of divided and decentralized governmental structures that include two entities, the Federation of Bosnia and Herzegovina and the Republika Srpska, as well as the Brcko District. A state constitution was adopted as part of the Dayton Accords, and each Entity created its own constitution. The constitution of the Federation further divides the Entity into ten cantons, each of which has broad governing powers and institutions of their own, including individual Ministries of Justice and court systems. Separate criminal codes and codes of criminal procedure exist in each Entity, as well as in the Brcko District. The sum of this complex structure therefore includes four parallel and separate jurisdictions at the State, Republika Srpska, Federation and Brcko levels; ten subsidiary cantonal jurisdictions, each with its own set of institutions and budgets; directives issued by fourteen Ministries of Justice; and four court systems and codes of criminal law and procedure. This fragmented and weak legal system has resulted in inconsistent applications of t he law, high rates of reversal in major criminal cases, an d widely varying performance among courts and prosecutors throughout the country. Formative Evaluation of JSDP II Appendix D Page 1 Over the last decade, significant steps have been taken to overcome this fragmentation by strengthening state institutions and promoting the rule of law. A High Judicial and Prosecutorial Council (HJPC) was created in 2004 and charged with regulating the judiciary, including the appointment and discipline of judges and prosecutors, as well as the administration of the courts. A new Ministry of Justice was established at the State level, and the State Court, with an international registry as well as international judges and prosecutors, is vested with increased authority to prosecute complex and high-level crimes. New laws were adopted reforming the procedures for criminal, civil and administrative matters throughout BiH. International assistance has provided extensive training, technical assistance and budgetary support to build these new institutions and improve the performance of judges and prosecutors. All of these steps have contributed to the emergence of a m ore centralized and coherent legal authority that can eventually underpin a consistent rule of law throughout the country. USAID has played a central role in these achievements. Through the Justice Sector Development Program I (JSDP I), USAID helped create the HJPC and the State MOJ from scratch, helping to endow these new institutions with core organizational structures and developing their capacity to fulfill their roles established by law. Perhaps the most visible success of the JSDP I has been the Model Courts Initiative. This initiative worked with 32 of BiH’s 65 courts to introduce modern records management strategies; improve court management and administration; increase public access and information systems; reduce case backlog; introduce budget planning procedures; and renovate selected courts. Despite these considerable successes achieved through 2009, a number of challenges remain for the BiH justice sector and its still fledgling state-level institutions. The fragmented legal system characterized by multiple codes of law and procedure has contributed to continued inconsistent application of the law. While new codes of criminal, civil and administrative law and procedure have helped raise the system to a common standard, without a State Supreme Court, there is no authoritative body that can resolve legal conflicts as they emerge. The lack of clear lines of authority or division of responsibilities between the State Ministry of Justice and those at the entity and canton levels, as well as between the MOJs and the HJPC, create numerous opportunities for executive interference. The absence of a unified budget leaves many courts drastically under-funded and dependent on local politicians for basic resources, creating further opportunities for political pressure. These challenges also affect the efficiency of the courts, as the inability to overcome legal and procedural obstacles or to secure adequate resources or personnel for court operations continue to fuel excessive case delay and inconsistent outcomes, further undermining public confidence in the justice system. Even in the absence of constitutional reforms needed to address the issue of complicated governance structure, a considerable space to en able the still fledgling state-level justice institutions to exert their existing authorities, protect the integrity of the judicial process, and advocate for greater independence is seen. M any of the functions of the HJPC, such as overseeing the budgets of courts and prosecutors, managing the selection and promotion of judges and prosecutors, and maintaining and enforcing performance and training standards, are not yet fully developed. While significant improvements have been made in court administration, additional assistance seems needed to implement new time standards and Formative Evaluation of JSDP II Appendix D Page 2 other targeted measures to reduce high case backlog, particularly for utility cases. The management and oversight of prosecutors is largely neglected by assistance efforts, leaving critical gaps in the functioning of the justice system. Greater capacity in these areas would lead to more effective court operations and enhance judges’ and prosecutors’ abilities to withstand political pressure, even in the absence of constitutional or legal changes. Addressing these challenges is also seen as critical for BiH to continue moving toward E U accession. Since a Stabilization and Association Agreement (SAA) between BiH and the EU was signed in June of 2008, BiH is officially on t rack to join the EU, though not without problems. BiH achieved a major milestone toward EU accession in adopting the Justice Sector Reform Strategy (JSRS) in 2007. Developed with intensive donor assistance to the State MOJ, the JSRS is a blueprint for specific reforms that would strengthen the justice sector and help it address many of the conditions for EU accession. The JSRS includes five pillars, each with a number of specific actions to be completed: 1) Judicial System, 2) Execution of Criminal Sanctions, 3) Access to Justice, 4) Support to Economic Growth, and 5) Coordinated, Well Managed and Accountable Sector. The State MOJ is expected to serve as the coordinator for the implementation of the JSRS and as the primary interlocutor for meeting EU requirements in the justice sector. Yet this weak institution lacks sufficient capacity to coordinate the numerous state, entity and canton justice institutions or to harmonize the fragmented legal framework. The absence of clear roles or lines of authority among these institutions threatens to undermine efforts by the State MOJ to move toward EU accession. To respond to the sector needs USAID designed a follow-on to JSDP I with the aim to deepen the technical capacity of t he HJPC to manage complex responsibilities on its own, further develop its policy-making capacity thus positioning itself as leader in advocating for greater independence through constitutional, legal and policy changes when the opportunities arise. Also, based on USAID’s strong track record in this sector, the State MOJ requested USAID assistance to coordinate among these numerous actors as a means to implement the JSRS and fulfill EU accession requirements. Finally, USAID was aware that this assistance had to be balanced with a concerted effort to build capacity within BiH society to advance the rule of law. While significant progress has been achieved in civil society development in BiH, with a number of organizations playing an active role in monitoring the executive and legislative branches and advocating for improved performance, the capacity to monitor the operations of the justice sector or advocate for concrete policy reforms in this sector remains weak. Supporting the involvement of citizens is crucial to achieving sustainable improvements in the justice sector. A. Project Information USAID’s Justice Sector Development Project II (JSDP II) was designed to build on prior achievements to promote credible justice sector institutions that operate effectively and uphold the rule of law – even in the face of political challenges and reduced international involvement. The program focuses on three primary objectives: 1) Strengthen the independence, accountability and effectiveness of the judiciary; Formative Evaluation of JSDP II Appendix D Page 3 2) Support a better coordinated and more unified justice system ready for EU accession; 3) Bolster public confidence in the rule of law. The program targets the State MOJ, the HJPC and civil society toward the achievement of these objectives. By assisting the State MOJ to coordinate among justice sector institutions, political actors and donors, the program helps the State MOJ implement the JSRS and meet EU accession requirements. Targeted support to the HJPC enables it to implement the elements of the JSRS that are under its authority, thereby strengthening the effectiveness of the judiciary. Support to the HJPC and to civil society enhances judicial independence and accountability, bolsters public confidence and access to justice, and ensures sustainability of reforms. Assistance primarily targets these state-level institutions to shore up a unified legal authority that can resist political pressure. However, it also benefits the justice system and citizens throughout the country since these institutions manage and oversee all justice sector operations in BiH. Activities include technical assistance to strengthen the core technical capacities of t he HJPC and State MOJ that enable them to fulfill their oversight and management roles, and enhance the operations of courts and prosecutors throughout the country. Assistance also focuses on strengthening the policy-making role of these institutions, to increase their ability to successfully exert their authority and advocate for greater independence for the judiciary. Support to these institutions is complemented by targeted technical assistance to civil society, to strengthen citizen oversight and sustain the momentum of reforms. 1) Strengthen the independence, accountability and effectiveness of the judiciary. The activities build the capacity of the HJPC to promote the independence of the judiciary and enhance the credibility and responsiveness of individual courts and prosectors. JSDP II moves beyond broad organizational development to strengthen key technical functions that are necessary for the HJPC to fulfill its responsibilities. Emphasis is on increasing budgetary independence, raising performance standards for judges and prosecutors, and improving the efficiency and effectiveness of courts and prosecutors to reduce excessive case backlogs and improve public confidence. Technical assistance is provided primarily to the HJPC, focusing on such key departments as the Budget and Statistics Department, the Selection an d Promotion Department, and the Court Administration Department. The program also works with these departments to develop and implement standards and management practices, to implement them in courts and prosecutors’ offices throughout the country, and to monitor performance according to these standards. Targeted assistance is provided to individual prosecutors’ offices through a model prosecutors approach, replicating the successful model courts initiative. In addition to building technical capacity, the program attempts to enhance the HJPC’s policy-making role and provides targeted assistance to enable it to advocate for legal, constitutional, policy and budgetary changes that enhance its authority and independence, and reduce the high backlog of utility and other cases. 2) Support a better coordinated and more unified justice system ready for EU accession. As the designated coordinator for JSRS implementation, the State Ministry of Justice required assistance to set up effective coordination structures and to provide oversight and analysis of progress. The JSRS envisioned MOJ taking the lead in coordinating the JSRS through inter-Ministerial Conferences of entity and canton MOJs. Formative Evaluation of JSDP II Appendix D Page 4 However the division of roles in implementing the strategy, especially between MoJ and HJPC, as well as priorities and timelines, were not yet defined during the design of the project. The State MOJ still lacks the capacity to formulate actionable plans or to mobilize resources to implement them. JSDP II therefore works to strengthen the capacity of t he State MOJ Sector for Strategic Planning, Aid Coordination and European Integration (SPACEI) to coordinate implementation of the JSRS. 3) Bolster Public Confidence in the Rule of Law. This component builds partnerships with citizens and organizations aimed at advocating for stronger justice institutions and increasing public awareness and engagement in p romoting the rule of law. The primary focus of assistance is on building the technical capacity of existing domestic organizations to conduct oversight and advocacy on rule of law issues, including collecting and analysing data on the justice system, monitoring court operations and judicial selection, and advocating for key reforms. Support also enables civil society actors to raise public awareness on judicial processes, legal and policy reform issues, and citizen rights and responsibilities in upholding the rule of law. III. Statement of Work The Evaluation team is asked to provide the Mission with the following: 1. Evaluation of the three Components of the project. 2. Outline of the general needs of the justice sector in Bosnia and Herzegovina (BiH) recommendations for future Mission’s involvement in the rule of law sector. and 1. 3. Review of selected management issues Evaluation of the project Components Component 1 - JUDICIAL INDEPENDENCE, ACCOUNTABILITY AND EFFECTIVENESS Activities under this component aim at improving the appointment and advancement process for judges and prosecutors, and improving the effectiveness of the judicial system through improved management and administration of courts and prosecutors’ offices. JSDP II provides assistance to prosecutors’ offices through the Model Prosecutors Office Initiative, base d in part on the successful Mode l Courts/European Standards program implemented under JSDP I. JSDP II also works with relevant counterparts to improve the judicial budgeting process, by helping prosecutors’ offices develop need-based program budgets and increasing budget autonomy for the judiciary. On the policy development side, the USAID￾funded Joint Policy Forum (JPF) is tasked with development of policy proposals related to development of judicial independence, transparency and accountability. Additional information on the JPF is noted below under component 2. Working groups, consisting of HJPC members and local professionals, were established to implement a number of activities JSDP II oversees with the HJPC. These working groups are assisted by JSDP II experts in subject matters while logistical support is provided by the HJPC Secretariat. Implementation of these activities was delayed during the first year of Formative Evaluation of JSDP II Appendix D Page 5 the project due to problems with the HJPC forming these working groups. Further, once established, the HJPC failed to provide necessary logistical support to all working groups, and denied payment of honoraria to working group members. As a result, some of the working groups have ceased or postponed their work. The most visible part of this component is the Mo del Prosecutor Office Initiative (MPOI). Although the role of prosecutors changed dramatically in the past five years as a result of a 2003 reform that eliminated investigative judges as well as assign ed that role to prosecutors and eliminated municipal prosecutors’ offices (POs), POs in general have been neglected by international donors. POs often operate in cramped, antiquated facilities with poor working conditions and have only been offered limited training, both in substantial investigation skills as well as case management. JSDP II’s Model Prosecutors’ Office Initiative (MPOI) works with prosecutor offices on management, administrative practices, backlog reduction, public outreach, strategic and operational planning, financial management, and improvement of IT services and PO’s premises. MPOI is scheduled for implementation during the first three years of the project in all 18 prosecutors’ offices throughout BiH, with exception to State prosecutor’s Office ( three waves of POs, six per year). Results achieved so far under this component are as follows:  Working Group on Strengthening Selection and Appointment Procedures for Judicial and Prosecutorial candidates made a draft of Book of Rules on written testing and prepared 3000 questions.  Working Group on Developing Optimal and Foreseeable Deadlines for Cases before Courts and POs developed tables with optimal deadlines for first instance case types before courts and POs in the country.  MPOI has been implemented in thirteen out of eighteen model prosecutor offices. Real effects of the MPOI in terms of increased efficiency that led to higher number of prosecutions are not known yet. The Evaluation Team is asked to address the following questions, not limited to:  Overall, how has JS DP assistance improved the HJPC’s concrete functions e.g. court administration, budget execution, performance measurements, appointment and selection? Which activities are or are not improving institutional capacity of HJPC?  What is the capacity of HJPC to embrace the recommendations of existing working groups and roll them out to BiH judiciary?  Is the end result of the HJPC as a self-sufficient state level institution realistic and achievable by June 2014?  Can anything be done to secure commitment of the HJPC Council and the secretariat for timely implementation of the project activities? Where are the real problems for this? Formative Evaluation of JSDP II Appendix D Page 6  Is lack of political will an issue for working groups?  Is there a need for further USG involvement in strengthening the HJPC beyond JSDP II base period?  Will HJPC need assistance in their joint efforts with the State MoJ to implement the justice sector reform strategy? What is the role of the HJPC in the Joint Policy Forum (JPF)?  What actions are required from JSDP II to ensure that reforms introduced are fully implemented and sustainable? How is the HJPC demonstrating its commitment to reform?  What effects to overall POs operations has the MPOI had thus far?  How do the improvements in the POs administration improve the delivery of justice and increase customer satisfaction? Has there been evidence of such increase?  Is there a need for further USG involvement in POs administration in the optional period, either separately or in coordination with the HJPC, and if so in what way?  Does the scope of Swiss–funded project Strengthening Institutional Capacities of Prosecutor Offices in BIH pose a risk of potential overlap? Can this be overcome and how? Component 2 - A COORDINATED AND MORE UNIFIED JUSTICE SECTOR The overall objective of Component 2 is to improve coordination among justice sector institutions, develop more unified and adequate court budgets that will enhance the independence of the judiciary, and implement the Justice Sector Reform Strategy (JSRS). JSDP II works closely with the MOJ, who is the designated coordinator of t he JSRS implementation, to help it track and enhance the progress of implementation. In this process, JSDP II also engages entity and cantonal authorities. Despite the political pressures from the executive branch Component 2 activities were met during year 1 of the project. However, in year two, implementation of activities under Component 2 has been difficult to achieve due to the deteriorating political situation which is negatively influencing the work of all justice sector institutions. Political attacks on the judiciary have consequently exhausted some of the resources of key partner institutions1 . In addition progress toward European integration has been delayed, in most part due to the delay in government formation2 . Formative Evaluation of JSDP II Appendix D Page 7 Under such circumstances, a number of donors involved in justice sector reform, decided to either decrease or seize their assistance projects. EC’s project designed to assist the MOJ’s Sect or for Strategic Planning, Aid Coordination and European Integration (SSPACEI) in coordination of the JSRS was first suspended and then terminated earlier this year due to lack of political will, primarily with the lower level Ministries. USAID at this point remains one of the most significant donors in this area. Regardless of the problems and implementation gridlocks, Component 2 progressed in certain areas. The Joint Policy Forum (JPF), a policy body consisted of representatives of Mo J and HJPC and tasked with preparation of policy proposals related to judicial system is “ up and running”. The capacities of the MOJ and HJPC for creation of strategic policy decisions have been strengthened. The Sector for Strategic Planning, Aid Coordination and European Integration (SSPACEI) of BiH MOJ strengthened to the certain extent its position both internally, within BiH MOJ, and externally, as a leading coordinating point in the implementation of the JSRS. Structural changes have been introduced in some institutions, with introduction to Strategic Units within these institutions, in charge for Strategy coordination. The Documentation System (DS), software designed to help with JSRS implementation, has been in full use and upgraded by its main users. Two analyses of the strategic importance for judicial budgets have been completed and delivered to HJPC. Despite the progress, a number of issues remain. Strong political influence of th e executive branch and complex governance structure of the country create a n inefficient and fragmented justice system. These make the implementation of the JSRS as a precondition for the EU accession process troublesome. Further, political pressure on the judiciary continues. The authorities of the HJPC, the State Court and the Prosecutor's Office have often been questioned by political leaders. A general lack of political will has translated into lack of progress in implementation of the JSRS for 2009-13. In particular, authorities at Entity and Cantonal levels have failed to integrate the strategy into their work plans. Limited human resources and poor coordination hamper implementation. The work of the five working groups for implementation of the strategy has been undermined by the lack of a quorum. Due to these problems, USAID would like to examine if the programmatic approach under Component 2 of JSDP II is th e best use of resources. USAID would like the evaluation team to respond to the following, but not limited to, questions:  Overall, has the assistance provided to the State MoJ improved the institution’s ability to coordinate JSRS implementation? 1 The most significant pressures are coming from the RS Entity which is openly questioning the need for State Court, State Prosecutors Office and High Judicial and Prosecutorial Council. There are also attempts to abolish the Council at the state level and create separate councils at the entity levels. Further, the RS recently prepared the draft Amendments to the Law on Courts in the RS with number of articles that directly interfering into the HJPC’s authorities (court budgeting, selection of sitting and reserve judges, judicial associates, etc.) Formative Evaluation of JSDP II Appendix D Page 8 2 Although the General Elections were held in October 2010, the Government at the State level has not been formed yet due to lack of consensus of leading political parties.  Is there a need for any further capacity building assistance to the State MoJ’s SSPACEI? Is there a demand for this assistance?  Is there a need for any assistance to the State MoJ in the implementation of the JSRS beyond JSDP II base period?  Is the JSDP II approach the right one?  Are there specific activities that are no t currently being implemented that should be in order to strengthen the Ministry? What are the key program areas that this activity should focus on during the remainder of the contract, if any?  Given that there is no or little inter-branch dialogue with the judiciary and no formal mechanism to support inter-branch dialogue on judicial independence and accountability, how successful can the JPF be under the current circumstances? Component 3 - INCREASED PUBLIC CONFIDENCE IN THE RULE OF LAW Improving the engagement of civil society in developing the rule of law and increasing public confidence in the delivery of justice in BiH are essential to lasting judicial reform. In year 1 JSDP II established the Justice Network (JN), a network of NGOs and professional associations, and helped them develop effective advocacy and public awareness campaigns and monitoring programs. As part of this support, JSDP II established a sub grants program to provide financial support to network members. JSDP II supports efforts by civil society to develop tangible policy and legislative reform initiatives to improve the administration of justice as well as supports journalists in their court monitoring efforts. In year 2, the Justice Network expanded from 47 to 57 professional associations and NGOs. Some benefits for network members were recognized immediately after its creation with increased access to information, expertise and financial resources, increased efficiency, solidarity and support, and increased visibility of issues, best practices, and underrepresented groups. With JSDP II support, the JN advocated for juvenile justice, access to justice by marginalized groups such as women, minorities and the poor, the efficiency of courts through the reduction of backlog through increased use of mediation in dispute resolution, the independence of judiciary by improving its funding and appointment process, and the transparency of courts. Since the penetration to the justice sector by local organizations is not easy, JSDP II aim s at establishment of partnerships of CSOs with courts, prosecutors, Ministries of Justice and the High Judicial and Prosecutorial Council. This partnership already resulted in the BiH Ministry of Justice’s invitation to the JN to participate in the government led working group on the development of the Strategy for Sustainable Development of CSOs. Formative Evaluation of JSDP II Appendix D Page 9 USAID would like the evaluation team to respond to the following, but not limited to, questions:  How successful was the JSDP II approach to creation of the JN?  Was the selection of policy topics on target?  Was the decision to switch from targeted to competitive grants correct?  What are the opportunities, if any, for JN to reach sustainability in the near future?  Is there a need to further support JN in the optional period?  What is the level of coordination of JSDP II Component 3 with other USAID civil society projects? 2. Outline of the general needs of the justice sector in Bosnia and Herzegovina (BiH) and recommendations for future Mission’s involvement in rule of law sector. BiH is at the early stage of the EU accession process. Reform of the justice system represents the important segment of that process. Currently, this process is burdened with a number of obstacles which result in delays in the development of an independent and effective judiciary in line with European standards. There is a unified opinion that significant progress has been achieved but that further efforts are needed to ensure implementation of the justice sector reform strategy and of the national war crimes strategy. G i v e n t h e po l i ti cal env i r o n m ent a n d f in d i n gs o f yo u r a s sess m en t , USA I D wo u l d li k e t o ga i n a be tt er u n d e rstan d ing o f t h e re m ai nin g n ee d s of t h e ju stice s ector in BiH, with specif i c focus on t h e a r eas where U S A I D p ro g r a m s can have the greatest i m p a ct. 3. Review of selected management issues USAID would like the evaluation team to look at the structure of the project team and to advise:  Whether the project is properly staffed?  What is the COP’s role in policy issues that project is undertaking?  Has JSDP II coordinated its activities with other donors?  What, if any, a re the comparative advantages of USAID’s JSDP II approach to rule of law reform in BiH?  What is the perception of USAID JSDP II by local counterparts? IV. Evaluation tasks The Team Leader will be responsible for leading the evaluation team and will be the principal contact with USAID. The Team Leader will also be responsible for coordinating the writing of th e final assessment report with an executive summary; major findings and conclusions; a description of the evaluation methodology, and specific program recommendations. The team member will be responsible for contributing his/her portion by writing a sound methodological plan and drafting a comprehensive evaluation report with key recommendations and findings for his/her respective area under the overall direction of the Team Leader. Formative Evaluation of JSDP II Appendix D Page 10 Task 1: Desk Review This review shall include reading all relevant background materials (attachment A), background interviews and at least one conference call with staff from USAID/BiH’s Democracy Office. The conference call is important to ensure that the Mission and the e valuation team are in agreement on the scope and purpose of the evaluation. Task 2: Conduct a Field Evaluation The evaluation team will spend no more than three weeks in-country to conduct the evaluation. Fieldwork will commence beginning on/about November 7 and end on /about November 23, 2011. While in BiH, the team will meet with key ROL representatives, representatives of the international community, USAID staff, USG implementing partners, and others (see Attachment B). Also, the team is expected to make at least two visits outside of Sarajevo, one to Banja Luka to meet the major rule of law stakeholders and one combined visit to at least three Cantons and their major rule of law stake holders. These visits should include, but not limited to, prosecutors’ offices that are partners of the project. Task 3: Debrief Embassy and Mission Before departing the country, the evaluation team will d ebrief interested personnel from USAID and Embassy on the team’s preliminary findings and recommendations. V. Deliverables A. Rough Draft Assessment Report due prior to departure. B. Draft Assessment Report due within 30 calendar days following completion of fieldwork. C. Final Draft Evaluation Report due not more than 7 calendar days after receipt of USAID/BiH’s comments on the draft evaluation report. The final draft evaluation report will include, at minimum, an executive summary; major findings and conclusions; a description of the evaluation methodology; a review of JSDP II’s key efforts and the overall impact of these efforts, and specific program recommendations. The final evaluation report will be submitted to Jasna Kilalic, Deputy Democracy Officer, in electronic format within 5 business days after receiving USAID/BiH’s final written comments and/or questions. Recommended composition of the assessment team The Assessment Team will consist of two members: Team Leader: This will be an international consultant with at least 10 years of relevant experience, an understanding of rule of law development programs preferably in Balkans countries (and ideally a specific understanding of these issues in BiH) or experience working on rule of law efforts in Eastern Europe and/or the former Soviet Union, and excellent organizational, analytical, and writing skills. Formative Evaluation of JSDP II Appendix D Page 11 Team Member: This will be an international consultant with at least 5 years of relevant experience, an understanding of rule of law development programs in transition countries (and ideally a specific understanding of these issues in BiH), and excellent organizational, analytical, and writing skills. Logistical support USAID/Bosnia and Herzegovina will provide logistical assistance to the tea m members with scheduling meetings, computer access, and office supplies. Technical direction during the performance of this delivery order will be provided by the USAID/Democracy Office Team. USAID/Bi H will also arrange for an interpreter for in-country meetings and transportation outside of Sarajevo. Period of performance The period of performance for this activity will b e from about November 1, 2011 and ending on/about January 7, 2012. Fieldwork should be conducted during the period from November 7, 2011 to November 23, 2011. LEVEL OF EFFORT The Team shall provide 53 days of personnel effort in performance of the evaluation, in the following illustrative breakdown: Tasks to be performed Team Leader Team Member Desk review 4 4 Field Evaluation* 15 15 Draft assessment report 6 3 Finalization of the report 4 2 TOTAL 29 24 *Six-day workweek is authorized while performing the assessment on the field. Formative Evaluation of JSDP II Appendix D Page 12 Appendix E List of Interviewees Field work of the JSDP II Formative Evaluation November 8, 2011 – November 24, 2011 ` US Government : Jasna Kilalic, Deputy Democracy Office Director, USAID Selma Sijercic, Project Management Specialist, USAID Nathan Petterson, Resident Legal Advisor, US Department of Justice - OPDAT Gregg A. Marchessault, Resident Legal Advisor, US Department of Justice – OPDAT Hasan Pleh, US Department of Justice – OPDAT Alica Lejlic, Political Officer, US Embassy JSDP II: Nicolas D. Mansfield, Director of Legal Programs, EWMI Gerald Meyerman, Chief of Party, JSDP II Ermin Sarajlija, Component 1 Leader, JSDP II Josko Mandic, Component 1 Program Coordinator, JSDP II Muhamed Susic, Component 2 Leader, JSDP II Nedim Daul, Component 2 Program Coordinator, JSDP II Elmerina Ahmetaj Hrelja, Component 3 Leader, JSDP II Dzenan Trbic, Component 3 Program Coordinator, JSDP II International Donors: Una Kelly, Task Manager for Judiciary, Delegation of the European Union in BiH Julien Berthoud, Senior Policy Advisor, EU Special Representative in BiH Anne Macleod, Head of Political, Press and Programme Team, British Embassy Sarajevo Dalida Tanovic, Project Manager, British Embassy Sarajevo Jonathan Francis, Embassy of Sweden in BiH Meagan Hrle, Head of Judicial and Legal Reform Section, Oorganization for Security and Cooperation in Europe (OSCE) BiH Dagmara Pilaszek, Legal Advisor, Judicial and Legal Reform, OSCE BiH Sanela Paripovic, Project Manager, Access to Justice, UNDP BiH Mervan Mirascija, Open Society Fund BiH High Judicial and Prosecutorial Council: Formative Evaluation of JSDP II Appendix E Page 1 Milorad Novkovic, HJPC President Admir Suljagic, Chief of Cabinet of the HJPC President Enisa Adrovic, HJPC Vice President, Chair of the Working Group for Model Prosecutor Offices Zdravko Knezevic, HJPC member, Main Federal Prosecutor at the Federation BiH Prosecutor’s Office Slavica Curic, HJPC member, Chair of the Working Group for Strengthening Selection and Appointment Procedures Enisa Adrovic, HJPC BiH Vice President, Chair of the Working Group for Model Prosecutor Ljiljana Filipovic, FBiH Supreme Court Judge, Member of the HJPC working group on optimal and predictable timeframes Muhamed Tulumovic, Director, HJPC Secretariat Damir Balic, Head of Budget Department, HJPC Secretariat Haris Grizovic, Analyst, Department of Judicial Administration, HJPC Secretariat Amila Rahic, Project Manager of the Swiss project “Strengthening the Capacity of Prosecutors in the Criminal Justice System” at the HJPC Ministries of Justice: Srdan Arnaut, Deputy Minister of Justice, Ministry of Justice of BiH Niko Grubesic, Assistant Minister, Head of Sector for Strategic Planning, Aid Coordination and European Integration (SSPACEI), Ministry of Justice of BiH Entoni Seperic, Expert Advisor, Sector for Strategic Planning, Federation MOJ Ruzmira Bajric, Deputy Minister, Tuzla Cantonal MOJ Jasminka Bratic, Assistant Minister of Justice, Herzegovina-Neretva Cantonal MOJ Denis Primorac, Assistant Minister of Justice , West Herzegovina Cantonal MOJ Prosecutor Offices: Nives Kanevcev, Chief Prosecutor, Cantonal Prosecutor’s Office, Sarajevo Canton Nijaz Mehmedbasic, Chief Prosecutor, Cantonal Prosecutor’s Office, Herzegovina- Neretva Canton Vesna Cavar, Chief Prosecutor, Cantonal Prosecutor’s Office, West Herzegovina- Canton Slobodanka Gacinovic, District Chief Prosecutor, Trebinje Vesna Kaknjo, Chief Prosecutor, Cantonal Prosecutor’s Office, Zenica-Doboj Canton Sesenam Cosic, Chief Prosecutor, Cantonal Prosecutor’s Office, Tuzla Canton Vitomir Soldat, District Prosecutor, RS Prosecutor’s Office Amor Bukic, Prosecutor at the RS Prosecutor’s Office Non-governmental Organizations: Snezana Ivandic Ninkovic, Director, Association for Democratic Initiative Sarajevo Formative Evaluation of JSDP II Appendix E Page 2 Anisa, Suceska Vekic, Director, Balkan Investigative Reporting Network, BiH Ivica Cavar, Monitoring Team Coordinator, Center for Civil Initiatives Jasmina Mujezinovic, Director, Foundation for Local Democracy Hajrija Hadžiomerović-Muftić, President, Association of Prosecutors in FBiH Adisa Zahiragic, President, Association of Women Judges in BiH Formative Evaluation of JSDP II Appendix E Page 3 Appendix F List of Reading Materials Materials produced by USAID/BiH : East West Management Institute Contract for JSDP II Materials produced by JSDP II : JSDP II Year 1 Work Plan JSDP II Year 2 Work Plan JSDP II Year 3 Work Plan JSDP II First Quarterly Report JSDP II Second Quarterly Report JSDP II Third Quarterly Report JSDP II Year 1 Annual Report JSDP II Fifth Quarterly Report JSDP II Sixth Quarterly Report JSDP II Seventh Quarterly Report JSDP II Year 2 Annual Report JSDP II Ninth Quarterly Report JSDP II Performance Monitoring Plan as revised in August 2011 JSDP II Mid Term Strategic Review 2011, Reviewing and Planning for Success and Sustainability Model Prosecutor Office Initiative, Intervention Plan, July 2010 – July 2011 Model Prosecutor Office Initiative, Intervention Plan, August 2011 – July 2012 Report on Survey Results „Expectations and Level of Satisfaction with Performance of the Justice Network in BiH“ Report on Survey Results „Assessment of Resources and Needs of the BiH Justice Network Member Organizations” Concept Paper, Development of T-Portal Performance Standards for Model Prosecutor Offices, Draft, 24 January 2010 Public Opinion Polling in Bosnia and Herzegovina, How Citizens Perceive Judiciary System, May/June 2010 Formative Evaluation of JSDP II Appendix F Page 1 Public Opinion Polling in Bosnia and Herzegovina,The Inside View of Judicial System, June 2010 Materials produced by the BiH Government: Justice Sector Reform Strategy, 2008 – 2012, Sarajevo, 2008 Second Revised Action Plan for the Implementation of the Justice Sector Reform Strategy in BiH for the period 2009 – 2013, Sarajevo, December 22, 2010 Memorandum of Cooperation between MOJ of BiH and HJPC of BIH in Establishment of Joint Policy Forum EU-Bosnia and Herzegovina, SAA Structured Dialogue on Justice, Technical Information Requested by the European Commission, Sarajevo, August 2011 Recommendations from the First Thematic Conference in BiH on the Topic of the Law on Prosecutors' Offices in Federation of Bosnia and Herzegovina Materials produced by the European Union: Progress Report 2011 Bosnia and Herzegovina, Brussels, 12 October 2011 Progress Report 2010 Bosnia and Herzegovina, Brussels, 9 November 2010 Progress Report 2009 Bosnia and Herzegovina, Brussels, 14 October 2009 Progress Report 2008 Bosnia and Herzegovina, Brussels, 5 November 2008 Progress Report 2007 Bosnia and Herzegovina, Brussels, 6 November 2007 Recommendations from the European Commission, Second meeting of the “Structured Dialogue on Justice between the European Union and Bosnia and Herzegovina”, Sarajevo (Bosnia and Herzegovina), 10-11 November 2011 Functional Review of the BiH Justice Sector, Funded by the European Commission, March 2005 Other Materials: Report of Civil Society Organizations (CSOs) on Implementation of the Action Plan for Implementation of the Justice Sector Reform Strategy (JSRS) of Bosnia and Herzegovina, Reporting period: 1 January – 30 June 2011, Date of the Report: 22 July 2011 Reaction to the 2011 Progress Report on B&H, signed by six networks of NGOs, Sarajevo / Banja Luka / Mostar, November 10, 2011 NGO Networks: Strength in Numbers?, Adam Abelson, Office of Private and Voluntary Formative Evaluation of JSDP II Appendix F Page 2 Cooperation, Bureau of Democracy, Conflict, and Humanitarian Assistance, U.S. Agency for International Development, July 2003 Internet Resources: Council of Ministers of BiH homepage, Directorate for European Integration at http://www.vijeceministara.gov.ba/stalna_tijela/dei/Default.aspx?id=1719 Ministry of Justice of BiH, Justice Sector Reform Strategy documents at http://mpr.gov.ba/en/str.asp?id=434 Court of BiH homepage at http://www.sudbih.gov.ba/?opcija=sadrzaj&kat=3&id=3&jezik=e Delegation of the European Union to BiH key documents at http://www.europa.ba/?akcija=clanak&CID=39&jezik=2&LID=57 Justice Network homepage at http://www.mrezapravde.ba/mpbh/english/txt.php?id=15 Formative Evaluation of JSDP II Appendix F Page 3 Appendix G Responses to Specific Questions Posed in the SOW Component 1 1.) Overall, how has JSDP assistance improved the HJPC’s concrete functions, e.g. court administration, budget execution, performance measurements, appointment and selection? Which activities are or are not improving institutional capacity of HJPC? JSDP II’s work with the HJPC has centered on the working groups established by the HJPC and described in the body of the evaluation report. These working groups are still focusing on their assignments. Once those assignments, which involve analyses, planning or drafting are completed, the full HJPC will have to decide whether to accept the groups’ recommendations. Assuming the recommendations are approved, they still would have to be implemented before it could be said that JSDP had improved the concrete functions of the HJPC. JSDP II’s work with the HJPC has focused only on individual working groups, rather than the institutional capacity of the Council as a whole. 2.) What is the capacity of HJPC to embrace the recommendations of existing working groups and roll them out to BiH judiciary? The HJPC’s decision to have its members closely involved with project was part of the reason for the initial delay in the appointment of the working groups working with JSDP II. This decision ensures that the HJPC has a direct role in setting policies for the judicial system, but it also means members of the HJPC have relatively little time to spend on any one issue: it is common for HJPC members to serve on several working groups simultaneously. The reforms being considered by the working groups all would require significant amounts of time from HJPC members and extensive work with judges, prosecutors and their staffs throughout BiH. If the HJPC intends to implement most of them at the same time, it would probably not have the capacity to do so on its own. 3.) Is the end result of the HJPC as a self-sufficient state level institution realistic and achievable by June 2014? Assuming the question refers to the Secretariat as well as the HJPC, in some respects the answer is yes. The Council will be well-equipped to do detailed strategic planning and assuming appropriate data is available, budget analysis. Due to the scope and number of problems the HJPC has to address, as illustrated by the November 2011 Formative Evaluation of JSDP II Appendix G Page 1 recommendations of the Structured Dialogue, the Council will not be able to reach self￾sufficiency by June 2014. Its progress towards self-sufficiency will continue to suffer as long as the HJPC has to contend with serious domestic political challenges to its authority. 4.) Can anything be done to secure commitment of the HJPC Council and the secretariat for timely implementation of the project activities? Where are the real problems for this? Is lack of political will an issue for working groups? The evaluation team does not see the problem as one of commitment or lack of will, but as of capacity. Many of the problems facing the HJPC and the Secretariat are caused by factors outside the control of those agencies, USAID and JSDP II. Most members of the HJPC are not full-time on the Council, and must serve on several working groups considering complicated issues, and the reforms within the scope of JSDP II are not the only ones the HJPC is trying to develop or implement. Several Council members interviewed by the evaluation team mentioned the “misunderstanding” at the outset of JSDP II about the type and assistance the project could provide, indicating they had unrealistic expectations about JSDP’s II design. One Council member told the team the HJPC had been “afraid” to turn down assistance offered by USAID, which contributed to the misunderstandings. Given the delays in implementation of JSDP II attributable to the HJPC, the best approach would be a frank discussion with the HJPC about its priorities and how they relate to the scope of JSDP II. However, the members of the HJPC interviewed for the evaluation certainly recognize that there could be a lack of political will within other agencies, or at other levels of government, so other offices or agencies would not be willing or able to implement HJPC policies. Overall, the evaluation team concluded that no matter how much the HJPC would have liked to pursue all the reforms proposed by JSDP II, it does not have enough full-time members, resources or staff to handle an unlimited number of projects – no institution does – even though some donors might disagree. The political and economic context for the HJPC’s activities and the role of its Secretariat changed between the time JSDP II was designed and project implementation, which left the HJPC with less time and fewer resources generally. The implementation of many reforms supported by the HJPC also requires the cooperation and work of other agencies, such as ministries of justice and finance at different levels, with their own priorities and political concerns and which do not always match the priorities of the HJPC. 5.) Is there a need for further USG involvement in strengthening the HJPC beyond JSDP II base period? Yes. Please see the detailed Component 1 recommendations in the body of the report. 6.) Will HJPC need assistance in their joint efforts with the State MoJ to implement the justice sector reform strategy? What is the role of the HJPC in the Joint Policy Forum (JPF)? Formative Evaluation of JSDP II Appendix G Page 2 Yes. Please see the detailed Component 2 recommendations in the body of the report. The Component 1 recommendations are also relevant here, since the reforms included in that section of JSDP II are based on the JSRS. The interviews conducted by the evaluation team suggest that the HJPC is an equal counterpart of the State MOJ in the Joint Policy Forum. 7.) What actions are required from JSDP II to ensure that reforms introduced are fully implemented and sustainable? How is the HJPC demonstrating its commitment to reform? To answer the second question first, the HJPC can be considered a victim of its own good intentions. The Council’s insistence on a greater role of its members on setting policy and monitoring of judicial operations means its members have to fill many roles and are stretched quite then. The evaluation team concluded most members of the HJPC are committed to reform, but there are not enough full-time members of the HJPC to cover all the Council’s work. The leadership of JSDP II (and probably the leadership of USAID) needs to have frank, well-informed and regular discussions with the HJPC leadership about practical solutions to the political as well as technical problems faced by the HJPC. 8.) What effects to overall POs operations has the MPOI had thus far? The evaluation team did not identify any overall effect on the operations of prosecutors’ offices due to the MPOI. Several prosecutors interviewed by the evaluation team said their public relations skills had improved due to JSDP II assistance, the prosecutor in Siroki Brijeg used the plans and estimates provided by JSDP II to raise funds for renovation of those premises, and the answers provided for the questionnaire drafted by the evaluation team indicate the strategic planning and budget training provided by the project have helped some offices do a better job of analyzing their needs. 9.) How do the improvements in the POs administration improve the delivery of justice and increase customer satisfaction? Has there been evidence of such increase? The evaluation team could not identify any data indicating that, so far, the improvements have improved the delivery of justice or customer satisfaction. Without developing much more targeted indicators and conducting much more targeted customer satisfaction surveys, JSDP II will not be able to demonstrate such improvements even if they have occurred. 10.) Is there a need for further USG involvement in POs administration in the optional period, either separately or in coordination with the HJPC, and if so in what way? The base period work of JSDP II covers many of the initial steps required to improve the operations of prosecutors’ offices identified in the JSRS, but several more years of sustained effort is required for the reforms identified in the strategy to become Formative Evaluation of JSDP II Appendix G Page 3 entrenched in the country’s judicial system. Further work with strategic planning and budgeting, as noted in Section 1.2 of the JSRS, and greater exchange of ideas among prosecutors in different parts of the country, including through the T-Portal, would help prosecutors learn how to accomplish more with the limited resources at their disposal. Work on strategic planning relates directly to the determination and setting of foreseeable timeframes for case processing, which is about to begin, and to selection and performance standards for prosecutors. There is also substantial work to be done before prosecutors’ offices would meet the “care of court users” objectives within the JSRS, although the work done on improved public relations by JSDP II falls within that part of the strategy. It would be appropriate for USAID to have JSDP II provide technical assistance on any of these topics if those assistance needs are not already being met. JSDP II probably could provide more sustained assistance to development of the T-Portal without risk of overlap with any other programs. If the working group on prosecutorial performance measurement is restarted, USAID could consider providing assistance to it. However, this should be done with a clear understanding that this work would in all probability not be completed by mid-2014, and that there is a serious risk of overlap with the Swiss Project in this area on this subject in particular. 11.) Does the scope of Swiss–funded project Strengthening Institutional Capacities of Prosecutor Offices in BIH pose a risk of potential overlap? Can this be overcome and how? Both JSDP II and the Swiss project have worked hard to avoid overlap so instead of overlap the two projects have produced coordination. However, by July 2012 the Swiss project should have completed its planning and started implementation of its activities. At the time of the evaluation, the Swiss-funded project was planning to provide all the assistance on the administration of prosecutors’ offices that the offices will be able to absorb within the next two years. If those plans hold, overlap would be unavoidable if JSDP II continues to work on PO administration during any option period (see answer to No. 10 as well). Component 2 1.) Overall, has the assistance provided to the State MoJ improved the institution’s ability to coordinate JSRS implementation? Yes. Please, see the background and the findings section on assistance to SSPACEI in the Component 2 section of the main body of the report. 2.) Is there a need for any further capacity building assistance to the State MoJ’s SSPACEI? Is there a demand for this assistance? Any of the justice institutions in BiH could use more capacity building assistance but compared to the other institutions engaged in JSRS implementation, the State MOJ's SSPACEI is a relatively adequate and competent coordinator of JSRS activities. For this Formative Evaluation of JSDP II Appendix G Page 4 reason, the evaluation recommends scaling down assistance to SSPACEI to specific activities directly related to the JSRS implementation, such as SSPACEI's participation in the Forum for Joint Policy or organization of the Thematic Conferences. SSPACEI is interested in JSDP II's continued assistance. 3.) Is there a need for any assistance to the State MoJ in the implementation of the JSRS beyond JSDP II base period? Yes. To the extent the State MOJ is engaged in some specific activities related to JSRS implementation, e.g. the organization of Thematic Conferences, participation in the FJP or coordination with the entity and cantonal MOJs, JSDP II should continue to provide assistance for such activities during the option period. 4.) Is the JSDP II approach the right one? JSDP II's approach in implementing Component 2 and supporting all the players engaged in Strategy implementation has been appropriate. After three years of implementation of JSDP II, Federation MOJ and the Cantonal MOJs continue to have very low capacity for strategy implementation. Therefore, assistance to them needs to be stepped up. JSDP II's approach to go beyond the implementation mechanisms identified in the Strategy itself and introduce the Documentation System and Thematic Conferences has been very appropriate. 5.) Are there specific activities that are not currently being implemented that should be in order to strengthen the Ministry? What are the key program areas that this activity should focus on during the remainder of the contract, if any? The main concerns regarding JSRS implementation do not stem from State MOJ's capacity but rather from entity and cantonal MOJ's capacities. For the optional period, Component 2 should focus mainly on strengthening capacities at entity and cantonal levels, assisting Strategy implementation through the organization of and follow-up to Thematic Conferences and work to reduce the budget fragmentation of the justice sector. 6.) Given that there is no or little inter-branch dialogue with the judiciary and no formal mechanism to support inter-branch dialogue on judicial independence and accountability, how successful can the JPF be under the current circumstances? The Forum for Joint Policy actually represents a mechanism for inter-branch dialogue with the judiciary, albeit an informal one. This is already a small success for the forum. The expectations for the forum should not be too great but if it continues to exist, manages to develop adequate policy proposals on important topics and, with JSDP II assistance, follows up on at least some of them successfully, this could be regarded as an important achievement. Formative Evaluation of JSDP II Appendix G Page 5 Component 3 1.) How successful was the JSDP II approach to creation of the JN? It was very successful. JN has grown rapidly, far beyond 10-15 traditional NGO members JSDP II expected when the project was designed. JN has a high public profile and donor-recognition factor for such a new organization, based at least in part on its work with the UPR. The high volume of information exchanged on the JN’s electronic mailing list also demonstrates the enthusiasm members have for the network. 2.) Was the selection of policy topics on target? Yes. They not only reflect the contents of the JSRS, war crimes and human rights strategies, but they match the interests of the NGOs identified in surveys conducted by JSDP II. However, most of the grants awarded to date do not correspond to the Component 1 and 2 activities or the November 2011 recommendations of the Structured Dialogue. 3.) Was the decision to switch from targeted to competitive grants correct? Yes. If JN members as a whole had not had a chance to compete for sub-grants from JSDP II, many NGOs probably would have lost interest in JN membership. Competitive grants also allowed NGOs to develop partnerships with professional associations in more creative ways and on topics of interest to BiH citizens, as those interests have been identified in JSDP II’s surveys. However, the evaluation team did not identify any reason why a combination of competitive and targeted grants generally would not be appropriate. 4.) What are the opportunities, if any, for JN to reach sustainability in the near future? The chances of survival for any network – for any domestic NGO – are not high. But the chances for the JN to reach sustainability will be higher if members continue to see benefits from JN membership, e.g., a competitive advantage in applying for funding from donors because of the JN’s public profile; greater access to policy makers and the media as members of the JN; and access to information and contacts with other organizations that would not be as easily available outside the network. If JN can pick one or two high-profile rule of law issues as the center of campaigns during the next two years and produce well-publicized wins, it will greatly increase the chances of JN’s sustainability. 5.) Is there a need to further support JN in the optional period? Yes. Please see the Component 3 detailed recommendations in the main body of the report. 6.) What is the level of coordination of JSDP II Component 3 with other USAID civil society projects? Formative Evaluation of JSDP II Appendix G Page 6 High. They are in regular communication and have complementary rather than competing or overlapping programs and activities. Review of selected management issues 1.) Whether the project is properly staffed? EWMI can be commended on the excellent qualifications of the local staff and the level of responsibility they have been given. A team of two professional local employees is working on each one of the three project components. Also, according to an organizational chart of the project provided in the Second Annual Report, Components 1 and 3 each have one administrative assistant. For Component 1 this assistant is also a driver. This strict division of responsibilities has probably contributed to fostering a sense of ownership over each respective activity on the part of the local staff. At the same time, the fact that there is an equal number of professional staff engaged in each component despite the different levels and type of effort required for the three components, means that some JSDP employees may have been overburdened while others had less work. Specifically, during the base period and especially during Year 1, Component 1 had a disproportionately large amount of organizational work on its plate with the seven HJPC working groups, the hiring of numerous experts and the implementation of the Model Prosecutors Office Initiative. Therefore, during the base period, Component 1 seems to have been understaffed. During the option period, the evaluation recommends significant scaling down of activities under Component 1. If this is done, the employees currently engaged in it could be reassigned. Regarding Component 2, during the base period it may have had some quite low periods in terms of workload due to the slow pace of few of the reforms it has been working on and the low absorption capacity of some of its counterparts. For the option period, the evaluation recommends scaling down some and stepping up other activities of this component. The appropriate level of staffing for Component 2 for the option period cannot be determined at this point since it will depend on the course some of its policy activities will take in the future. As regards Component 3, a large part of the work on the formation of the Justice Network has been completed, so it may not require the same level of staffing in the future. In short, this evaluation's recommendation to terminate or scale down some activities under each of the project components during the option period, may warrant an adjustment in the number of local staff. If the project reduces the number of staff significantly, it may consider retaining the strict division of staff between components only for the Component Leaders and having a more flexible assignment of duties for the rest of the staff. 2.) What is the COP’s role in policy issues that project is undertaking? Formative Evaluation of JSDP II Appendix G Page 7 There was a change of the Chief of Party (COP) in the spring of 2011. The evaluation team was left with the impression that the new COP was not as conversant or knowledgeable about the particulars of the project or the context as might be expected, given that he had already been in that position for half a year at the time of this evaluation. On a number of occasions, the information on important project or rule of law issues provided by the COP to the evaluation team was vague or simply incorrect. The project staff did not attend the first interview the team had with the COP, but in follow-up meetings they provided precise answers to the evaluation team's questions. If the project’s counterparts perceive the COP as being misinformed on critical policy or factual issues, they probably do not have much confidence in his opinions or advice. The evaluation team could not determine the COP's role in the policy issues that the project is addressing. From the one meeting that the COP had with the evaluation team (he was on leave for most of the evaluation period), it seemed that the COP had a strong interest in the BiH political situation as a whole and a relatively lower level of knowledge on policy issues pertaining to the justice sector. He had managed to meet with some of the project counterparts and other donors. He was having regular contacts with the international member on the HJPC. The HJPC Component 1 working groups focusing on examination/selection procedures for judges and prosecutors and case processing timeframes expect to be ready to move on to implementation of their polices and recommendations by the end of the JSDP II base period. Assuming the project is extended into an option period as recommended by the evaluation team, Component 1 would be providing technical assistance on these topics. The work on timeframes especially would benefit from some court administration expertise (especially to reduce the foreseeable timeframes), but JSDP II does not have any expatriate staff qualified to provide a fresh perspective on the issues the courts and prosecutors’ offices would face. Similarly, the implementation of Component 2 activities recommended for the option period requires a great deal of networking with local counterparts and other donors on topics related to JSRS implementation. It would therefore be very helpful if the COP has the ability to understand thoroughly the complex legal issues involved in JSRS implementation. Overall, the leadership of the project could be stronger. 3.) Has JSDP II coordinated its activities with other donors? JSDP II has coordinated its activities with other donors. Representatives of the EU Delegation were particularly knowledgeable about the project and referred to the content of its components with remarkable ease. The section of the main evaluation report dealing with Component 1 of JSDP II already commended JSDP II for the excellent coordination with the Swiss project working with prosecutors. Other donors, for instance the UK Embassy, the Swedish Embassy and the Open Society Foundation were also quite knowledgeable about JSDP II activities. The representatives of the Organization for Security and Cooperation in Europe and the United Nations Formative Evaluation of JSDP II Appendix G Page 8 Development Program had less information but were still aware of at least some of JSDP II's activities. 4.) What, if any, are the comparative advantages of USAID’s JSDP II approach to rule of law reform in BiH? The most significant comparative advantage of the JSDP II's approach to rule of law reform in BiH is that, as most USAID projects, its scope of work allows for much greater flexibility as compared to other donor projects. This means that it can relatively quickly adapt to changed circumstances and accommodate requests for assistance falling under its general objectives. Another comparative advantage of this project is that in helping the implementation of the Justice Sector Reform Strategy it works not only with one institution (e.g. the State MOJ as the coordinator of Strategy implementation) but at the state, entity (including Brcko District) and cantonal levels. This allows the project to design more comprehensive activities and have a wider impact. 5.) What is the perception of USAID JSDP II by local counterparts? Overall, JSDP II local counterparts appreciate highly the assistance they have been provided with. Some of the prosecutors’ offices have been particularly appreciative, given that USAID is the first donor to give attention to the strengthening of their administrative operations. The level of knowledge about JSDP II of local counterparts varied. Interviewees at the HJPC Secretariat and the State MOJ's SSPACEI who are used to working with donors and conversant in development terms, had a very high level of understanding regarding JSDP II activities. Institutions with less experience in working with donor agencies, like prosecutor offices and cantonal MOJs, had little information about the project and its goals. Overall, each of the interviewed counterparts had limited knowledge of the particular activity he or she had been engaged in, but no information on any of JSDP II’s other activities. Most of the counterparts did not even realize how the work they were doing might fit within the JSDP II component they were engaged in. Below, the evaluation team finds it appropriate to briefly discuss some additional management issues identified in the process of the evaluation, which were not specifically raised by the Scope of Work for this evaluation. 6.) Reporting. The evaluation team examined all work plans, quarterly reports and annual reports prepared by JSDP II. It is unclear what the process of report-preparation has been but there are clear discrepancies in the quality of the reports. Formative Evaluation of JSDP II Appendix G Page 9 The quarterly reports are generally easy to follow and use the same model throughout the project life. They do not always contain details about who did what, when, why or how, but they do provide a good general outline of project activities. On the other hand, the quality of the two annual reports is relatively low. The structure of these reports does not track the structure of the quarterly reports, and the two annual reports do not use the same format. Neither annual report provides a full account of the activities and events that took place during the year; in order to get a better picture of what took place, the reader needs to carefully review all of the quarterly reports. The Year 2 annual report is particularly poorly drafted. Editorially, it abounds in misspellings, incomplete sentences and grammatical mistakes. In terms of content, the Year 2 annual report does not clearly distinguish between activities which took place during that year and activities which occurred during Year 1. It also contains general discussions about the importance of some project activities instead of concentrating on what the project has accomplished and what remains to be done. For example, the entries on the importance of manual case management systems and case management control cards received so much emphasis the consultants were concerned they had misunderstood how those subjects fit within JSPD II’s work with the prosecutors’ office. Therefore questions on these subjects were included in the survey sent to the Wave 1 and 2 prosecutors’ offices. If the quality of reporting for JSDP II improves, it will make it simpler for USAID to demonstrate the importance of the project goals, challenges and activities to the public, other donors and Embassy counterparts. The poor reporting is a management issue, which needs to be addressed at the appropriate managerial level. 7.) Performance monitoring. As part of the desk review, the evaluation team examined the Performance Monitoring Plan for JSDP II, as revised in August 2011. According to the project's COP, the EWMI contract did not require formal USAID approval of the PMP. According to the PMP reviewed by the evaluation team: EWMI did not seek “approval” and USAID did not consider that approval was a requirement under the terms of the Contract with EWMI. Unlike other project, policy focused projects present difficulties to more traditional methods of data gathering monitoring and evaluation techniques. Rather than extensive quantitative data, policy work requires evaluation of work by examining the professional rigor, a well thought out and designed policy modification process, transparency of objectives, and resulting systemic learning. As a result, the PMP had apparently been revised on several occasions but had never been formally approved by USAID. Absent a formal approval, the indicators envisioned by the PMP can only be viewed as discretionary guidelines, rather than mandatory criteria for contract performance. The team’s second general concern with the PMP is that so many of the indicators in the PMP are outputs rather than impact-based. While some aspects of JSDP II involve Formative Evaluation of JSDP II Appendix G Page 10 the relatively intangible nature of policy work, some of the indicators could and should be based on results. For example, there are only output-based indicators for Component 1 even though the Model Prosecutor's Office Initiative implemented under this component provided opportunities for defining impact-based indicators, at least in areas like public relations. Given the lack of baseline data for any impact-based indicators for MPOI, even if the PMP is revised, it may be unable to capture improvements in the work of prosecutor offices. The only indicator under Component 2 which could be seen as impact-based is “Improved budget proposals submitted by courts,” but the project has not identified a reliable mechanism to measure such improvement. Some of the indicators for Component 3 go more towards results, e.g., the inclusion of measurements based on the NGO Advocacy Index and the NGO Sustainability Index (although see comments about the indices in the Findings section for Component 2). However, with regard to the high-level impact-oriented Component 3 indicator “Increased public understanding of rights and responsibilities and of justice sector reform issues”, it is not clear from the PMP or from survey “How Citizens Perceive Judiciary System” what the source is for the 5% result for Year 2 of the project and the targeted 10% result for Year 3. 8.) Outreach to US Government Agencies A project may be achieving excellent results but if it does not have appropriate outreach activities, these results will remain unrecognized. This holds true to an even greater extent for rule of law projects, which are often quite technical in nature and difficult for non-lawyers to appreciate. During field work, the evaluation team gave special attention to the level of knowledge that counterparts, US Government agencies and other donors had about JSDP II. Above, the evaluation team has discussed JSDP II's relationship with local counterparts and with other donors. However, we believe that JSDP II's relationship to other US Government agencies also warrants attention. US Government agencies (US Department of Justice and US Department of State) had very low levels of knowledge of JSDP II’s work, even with regard to activities that fell within the respective agency's portfolio. This was particularly striking given the high level of knowledge other donors had on those same activities. For example, staff from the US Department of Justice had only learned of the MPOI because the Ambassador asked them to attend an MPOI event. They had no knowledge of the Thematic Conference on the Law on Prosecutors for the Federation, or indeed that the law had been drafted. Similarly, prior to the publication of the Structured Dialogue Recommendations, the US Department of State did not seem to appreciate JSRS's importance for BiH's EU accession and JSDP II's role in assisting Strategy implementation. 9. Branding. Formative Evaluation of JSDP II Appendix G Page 11 Each USAID project is required to follow branding guidelines. In recent years USAID has stepped up efforts to use consistent and visible branding across the globe. The use of the Graphic Standards Manual for USAID is compulsory for all contractors. Even though this was not one of the tasks of this evaluation, the evaluation team notes that it did not see evidence of consistent branding of the equipment provided to prosecutor offices under the MPOI. Project staff has commented that in some cases boxes, as opposed to the equipment they contained, may have been branded. This is a shortcoming that can easily be remedied. Formative Evaluation of JSDP II Appendix G Page 12