MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE II) MIDTERM PERFORMANCE EVALUATION OF THE JUDICIARY AGAINST CORRUPTION ACTIVITY (JACA) October 2022 MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE II) MIDTERM PERFORMANCE EVALUATION OF THE JUDICIARY AGAINST CORRUPTION ACTIVITY (JACA) Final Report October 2022 Prepared under the USAID Bosnia and Herzegovina Monitoring and Evaluation Support Activity (MEASURE II), Contract Number AID-167-I-17-00004, Task Order Number 72016819F00001 Submitted to USAID/Bosnia and Herzegovina, October 2022 Contractor: American Institutes for Research (AIR) ABSTRACT This midterm performance evaluation of the $7.9 million United States Agency for International Development in Bosnia and Herzegovina (USAID/BiH)-funded Judiciary Against Corruption Activity (JACA), implemented by Development Professionals, Inc., examines its progress toward contractual objectives in the first two and a half years of its implementation. This report will inform USAID/BiH’s decision-making about potential adaptations of the Activity during the remaining period of its implementation. The midterm performance evaluation was conducted between May and August 2022 and arrived at the following conclusions: (1) JACA technical assistance (TA) is both well designed and highly rated; (2) the support of local judicial institutions to frontline prosecutors and judges working on high-level corruption and organized crime (HCOC) cases remains generally insufficient, and JACA’s TA succeeded in filling many of the resulting gaps; (3) JACA was instrumental in forging a highly appreciated informal network of HCOC prosecutors; (4) the operational environment for HCOC prosecutors and judges is highly complex, and performance quotas constitute a major problem; (5) JACA’s interventions strengthened the capacity of the BiH judiciary to deal with integrity and ethics issues, but it continues to need support; (6) the degree of commitment to effective HCOC processing by frontline prosecutors on the one hand and their managers on the other varies even from one partner prosecutor office to another, but all JACA partner institutions should continue to receive support; (7) JACA should continue to use predominantly domestic senior experts as trainers; and (8) cooperation between USAID anti￾corruption Activities, as well as with other United States Government agencies can and should be strengthened. Contents ABBREVIATIONS ix EXECUTIVE SUMMARY 1 INTRODUCTION AND BACKGROUND 1 METHODOLOGY 1 FINDINGS AND CONCLUSIONS 1 RECOMMENDATIONS 5 INTRODUCTION 6 COUNTRY CONTEXT 6 PROGRAM DESCRIPTION AND THEORY OF CHANGE 6 EVALUATION PURPOSE AND QUESTIONS 7 EVALUATION PURPOSE 7 EVALUATION QUESTIONS 8 EVALUATION METHODS AND LIMITATIONS 8 BRIEF OVERVIEW OF METHODOLOGY 8 FINDINGS AND CONCLUSIONS 9 EQ 1 FINDINGS 9 EQ 1 CONCLUSIONS 17 EQ 2 FINDINGS 18 EQ 2 CONCLUSIONS 21 EQ 3 FINDINGS 21 EQ 3 CONCLUSIONS 24 EQ 4 FINDINGS 24 EQ 4 CONCLUSIONS 26 EQ 5 FINDINGS 27 EQ 5 CONCLUSIONS 28 RECOMMENDATIONS 29 ANNEX I: STATEMENT OF WORK 30 PURPOSE OF THE ASSIGNMENT 30 SUMMARY INFORMATION 30 BACKGROUND 31 COUNTRY CONTEXT 31 JACA DESCRIPTION AND THEORY OF CHANGE 31 JACA MONITORING, EVALUATION, AND LEARNING PLAN 33 EVALUATION QUESTIONS 39 EVALUATION DESIGN AND METHODOLOGY 39 EVALUATION LIMITATIONS 41 DELIVERABLES AND REPORTING REQUIREMENTS 41 TEAM COMPOSITION 42 SCHEDULE 43 ANNEX II: DETAILED METHODOLOGY 44 DATA COLLECTION APPROACH, METHODS AND EVALUATION MATRIX 44 DATA ANALYSIS AND UTILIZATION 46 EVALUATION LIMITATIONS 47 ANNEX III: LIST OF ACTIVITY AND SECONDARY DOCUMENTATION REVIEWED 48 THIRD ANNUAL REPORT ON JUDICIAL RESPONSE TO CORRUPTION: THE IMPUNITY SYNDROME, OSCE, NOVEMBER 2020 49 ANNEX IV: LIST OF KEY INFORMANTS AND FOCUS GROUP PARTICIPANTS 50 ANNEX V: INTERVIEW GUIDES 51 USAID 51 JACA 53 HJPC CO1 55 HJPC CO2 57 CO1 PROSECUTORS/POS 59 JPTCS 61 INL/OPDAT/US EMBASSY 63 DONORS/INTERNATIONAL COMMUNITY 64 ANNEX VI: FOCUS GROUP GUIDES 67 CO1 JUDGES/COURTS 67 CO2 POS/COURTS 68 ANNEX VII: PHOTOGRAPHS OF EQUIPMENT PROVIDED BY JACA DURING THE COVID PANDEMIC 71 ANNEX VIII: JACA SCORECARDS 72 ANNEX IX: EXAMPLES OF MEDIA ATTACKS ON HCOC PROSECUTORS 74 ANNEX X: LIST OF HCOC CASES INITIATED OR CONCLUDED BY A PROSECUTORS’ OFFICE 75 ABBREVIATIONS ACFC Assistance to Citizens in Fight Against Corruption Activity AIR The American Institutes for Research BiH Bosnia and Herzegovina CCJE Consultative Council of European Judges CCEP Consultative Council of European Prosecutors CDCS Country Development Cooperation Strategy CLA Collaboration, Learning, and Adapting CoI Conflict of interest COP Chief of Party CSO Civil society organization DKB Digital knowledge database DPI Development Professionals, Inc. EQs Evaluation questions ET Evaluation team EU European Union FBiH Federation of Bosnia and Herzegovina FG Focus group FGD Focus group discussion HCOC High-level corruption and organized crime HJPC High Judicial and Prosecutorial Council of Bosnia and Herzegovina HO Home Office IJP Investigative Journalism Program INL Bureau of International Narcotics and Law Enforcement Affairs IP Implementing Partner IR Intermediate Result ITP Individual Training Program for Prosecutors JACA Judiciary Against Corruption Activity JPTC Judicial and Prosecutorial Training Center JEI-BiH Judicial Effectiveness Index of Bosnia and Herzegovina KI Key informant KII Key informant interview LEA Law enforcement agency LoA Life of Activity LoE Level of Effort MEASURE II Monitoring and Evaluation Support Activity MEL Monitoring, Evaluation, and Learning NSCP National Survey of Citizens’ Perceptions OCCRP Organized Crime and Corruption Reporting Project OPDAT Office of Overseas Prosecutorial Development, Assistance, and Training OSA Intelligence and Security Agency (orig. “Obavještajno-sigurnosna agencija”) PC Partner court PPE Personal protection equipment PO Prosecutor’s office PPO Partner prosecutor’s office RS Republika Srpska SCJPEII Standing Committee for Judicial and Prosecutorial Ethics, Independence, and Incompatibility SIPA BiH State Investigation and Protection Agency SPJ Specialized Program for Judges SoW Statement of work TCMS Prosecutors’ Case Management System USAID/BiH United States Agency for International Development BiH Mission USG U.S. Government USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 1 EXECUTIVE SUMMARY INTRODUCTION AND BACKGROUND Upon request of the United States Agency for International Development Mission (USAID) in Bosnia and Herzegovina (BiH), the Monitoring and Evaluation Support Activity (MEASURE II) conducted a midterm performance evaluation of the Judiciary Against Corruption Activity (JACA) in BiH to provide an evidence-based and independent review of the Activity’s implementation to date. JACA is a $7.9 million USAID/BiH-funded Activity implemented by Development Professionals, Inc. (DPI), launched in September 2019 and scheduled to close in September 2024. JACA provides policy, institutional capacity, and legislative assistance, which aims to enhance the BiH judiciary’s ability to successfully deal with high-level corruption and organized crime (HCOC) cases. The performance evaluation focused on the Activity’s design and progress toward expected results. The Mission and the implementing partner (IP) will use the evaluation results to make informed programmatic decisions and take midterm corrective actions in the Activity design and implementation arrangements. This document outlines the rigorous evaluation design and implementation plan, which ensured that the evaluation findings, conclusions, and recommendations would be credible and relevant. METHODOLOGY The JACA evaluation took place between May and August 2022. The evaluation team (ET) applied the mixed-methods data collection approach and triangulated data to develop credible findings, which were the basis for the team’s development of conclusions and recommendations. The ET used techniques that included document review (e.g., Activity documents and secondary documentation) and quantitative data analysis (High Judicial and Prosecutorial Council of Bosnia and Herzegovina [HJPC] administrative data and JACA survey data), and 26 key informant interviews (KIIs) and six focus groups (FGs) with 20 participants. Evaluation team members also observed four JACA events in which more than 50 JACA beneficiaries were present (see Annex II for details of the methodological approach). FINDINGS AND CONCLUSIONS The evaluation questions (EQs), and corresponding findings and conclusions are present below. EQ 1: WHAT PRIMARY FACTORS HAVE CONTRIBUTED TO THE SUCCESS OF OR PRESENTED SPECIFIC CHALLENGES IN JACA IMPLEMENTATION WHEN IT COMES TO IMPROVED ADJUDICATION OF HIGH￾PROFILE CORRUPTION AND ORGANIZED CRIME CASES (75 PERCENT OF JACA LOE)? JACA’s interventions related to improving the processing and adjudication of HCOC cases are well designed and highly rated by beneficiaries. Additional efforts to mitigate contextual issues are still necessary for achieving more tangible results, but JACA has made processing of HCOC cases the focus of many discussions and activities within the BiH judiciary. The support by local judicial institutions to frontline HCOC judges and prosecutors remains generally insufficient. JACA is filling many of these gaps with its expertise and timely assistance: delivering high￾quality training, organizing inter-institutional meetings, facilitating operational implementation of the new HCOC case designations as the basis for the specialization of prosecutors (and judges), tracking the processing of these cases, and supporting the provision of additional resources. In addition, during COVID-19, JACA helped partner institutions ensure continuity of their work on HCOC cases. Through its Individual Training Program for Prosecutors (ITP), JACA created an informal network of HCOC prosecutors who exchange experience and information related to processing HCOC cases even outside JACA events. This network is important because there are no formal meetings in which frontline prosecutors can discuss issues related to processing HCOC cases and no established formal channels for proactive communication and consultation with higher level prosecutors’ offices (POs). Despite the highly-rated JACA technical assistance (TA), HCOC prosecutors and judges work in a challenging environment characterized by clashing priorities, unclear guidelines, and contradictory 2 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION incentives, all of which are outside JACA’s control. Case resolution “quotas”1 used in performance evaluation of judges and prosecutors are the central problem for all interviewed judges and prosecutors because they are misaligned with prioritizing HCOC cases in POs and courts. Individual prosecutors working on HCOC cases encounter numerous operational issues (i.e., working on multiple cases of various complexity in parallel), as well as frequent unavailability of support staff and resources. This prevents the prosecutors from focusing on HCOC cases, which weakens the credibility of the standing official pronouncement that the HCOC cases are the priority for the judiciary. Similarly, the assignment of multiple cases that judges are supposed to work on in parallel prevents judges from organizing trials of HCOC cases in continuity and deciding these cases quickly. Prosecutors working on HCOC cases are exposed to indirect pressure, mainly as targets of politically orchestrated media attacks and malicious criminal and disciplinary reports filed against them and the law enforcement agencies’ (LEA) frequently dithering cooperation causes the prosecutors considerable frustration. All the above issues combine to make the frontline HCOC prosecutors feel unsupported and unprotected by the judicial system. EQ 2: WHAT PRIMARY FACTORS HAVE CONTRIBUTED TO THE SUCCESS OR PRESENTED SPECIFIC CHALLENGES IN JACA IMPLEMENTATION WHEN IT COMES TO PREVENTION OF CORRUPT BEHAVIOUR IN THE JUSTICE SECTOR (25 PERCENT OF JACA LOE)? Beneficiaries find that JACA’s technical assistance (TA) for preventing corrupt behavior has been well designed, of high quality, timely, and tailored to their needs. In the absence of other donors’ funding, the HJPC and other judicial institutions need continued JACA support of the same scope as JACA has provided so far. The implementation of an integrity-related agenda in the judiciary remains in its early stages. The HJPC’s Secretariat only recently established its Integrity Department, and JACA has been the main partner in strengthening its capacity to coordinate and oversee implementation of integrity-related activities of all judicial institutions. JACA has assisted in the implementation of a range of interventions that have helped the HJPC to connect different elements into one coherent system for improving ethics and integrity in the judiciary. Moreover, expertise and knowledge of integrity- and ethics-related issues is not widely available in the judiciary. JACA has therefore filled these knowledge gaps and assisted the HJPC in responding to recommendations of confidential counseling provided by the Groupe d’Etats contre la corruption, or Group of States Against Corruption (GRECO), and other international organizations, In addition, JACA’s contribution to implementation of integrity plans of judicial institutions was particularly valued by the beneficiaries. JACA’s TA did not focus only on partner POs (PPOs) and partner courts (PCs) but included a broader range of institutions. JACA assisted in establishing a network of integrity coordinators, providing them with training and guidance and thereby ensuring the harmonized and timely response to the requirements of the HJPC Secretariat’s Integrity Department. Judicial officials’ understanding of standards of behavior, acts of misconduct, and associated sanctions is still underdeveloped. There is a need for JACA to continue to proactively work with the HJPC on dissemination of disciplinary practice and opinions of the Standing Committee on Ethics, Integrity and Accountability of Judges and Prosecutors. 1 In estimating its productivity in terms of the number of resolved cases, the BiH judiciary mainly relies on the “orientation/individual/collective quota” metric (widely referred to as the quota). The orientation quota refers to the number of cases a judge or a prosecutor is expected to resolve in a year (as set by the High Judicial and Prosecutorial Council [HJPC] regulations). In a calendar year, a judge/prosecutor can work on various combinations of case types and cases of different complexity. The total number of resolved cases (by a judge/prosecutor) at the end of the year is compared with the number prescribed by the orientation quota, and the percentage of fulfillment of the quota requirement is calculated. That number constitutes the individual quota of a judge/prosecutor. For example, a prosecutor can decide to work on a complex case but risks not having any results over a prolonged period and at the end of the year. Alternatively, the same prosecutor can work on many simpler cases, show good results throughout the year, and achieve a good annual result at the end of the year. The result at the end of a year is one of the criteria for the career advancement of judges/prosecutors. The average value for all judges in one court (or prosecutors in one PO) represents the collective quota for that court (or PO). The managers of judicial institutions are interested in achieving good results in the collective quotas because that is one of the criteria for their own advancement. Consequently, the managers of judicial institutions are not happy to have prosecutors/judges with no results. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 3 EQ 3: IS THERE ROOM FOR ADJUSTMENTS IN PROVIDING JACA’S TECHNICAL ASSISTANCE TO PARTNER INSTITUTIONS, E.G., FOR STARTING TO PROVIDE SUPPORT TO NEW PARTNER INSTITUTIONS AND/OR WITHDRAWING SUPPORT FROM SOME OF THE CURRENT ONES, AND WHY? EQ 3.1: TO WHAT EXTENT DOES JACA HAVE THE SAME COMMITMENT TO THE IMPLEMENTATION OF ITS INTERVENTION FROM THE MANAGEMENT/PROSECUTORS/ JUDGES OF INITIALLY SELECTED PARTNER INSTITUTIONS (POS AND COURTS) FOR PROCESSING/ADJUDICATION OF HCOC CASES AND THE HJPC FOR COMPONENT 2 INTERVENTIONS? Concerning JACA’s partner institutions performance in processing HCOC cases, key informants (KIs) hold diverse, even conflicting, views on whether to include new partner institutions in JACA’s partnership programs. It is evident that KIs have their own criteria for what constitutes demonstrated commitment and results of individual courts and POs. Nevertheless, arguments provided for including any new POs in the JACA partner program or for excluding current JACA partner institutions were unconvincing. For example, some key informants were proposing inclusion of the BiH PO because of its jurisdiction in processing HCOC cases, while others disputed this proposal because, despite considerable resources placed at its disposal, this PO consistently lacks results in processing HCOC cases. Furthermore, the ET found considerable variation in POs’ commitment to processing HCOC cases and interactions between the management and frontline HCOC prosecutors. The ET grouped these interactions in three different categories. The first type is POs with only declarative support for processing HCOC cases while having no results. The second type is POs in which there is synergy between the PO management and frontline prosecutors, resulting in tangible results at the local level. The third type is POs in which frontline prosecutors lack the support of their management and believe that their superiors are under political influence. The ET found that, when frontline prosecutors encounter issues in their POs, these prosecutors do not have available proactive channels for communicating these issues to higher-instance bodies (i.e., the HJPC or higher-level POs). Communication between frontline prosecutors and representatives of higher-instance bodies occurs only informally, at the margins of conferences and seminars, or formally when issues escalate and higher-instance bodies retroactively start to use their formal authority2 and get involved in monitoring of a particular case(s) or work of a PO. Taking these findings in consideration, the ET concluded that JACA should continue working with the same partner institutions because these institutions remain most likely to have HCOC cases (the PPOs’ have jurisdiction over the strongest local economies in the country), while individual prosecutors involved in JACA’s activities in the future may receive promotions to higher or managerial positions, which would take the benefits of the knowledge acquired within JACA interventions to the next level. Nevertheless, within partner institutions, JACA’s support needs to be refocused to emphasize assistance and support to individual frontline prosecutors/judges who directly work on prosecuting/adjudicating HCOC cases and directed toward enabling proactive channels for their communication with the higher-level instances. 2 The Law on the Prosecutors’ Office of FBiH, https://portalfo2.pravosudje.ba/vstvfo-api/vijest/download/3288 Article 9: (1) The Chief Federation Prosecutor shall supervise the performance of the cantonal Prosecutors’ Offices in order to guarantee the legality and efficiency of proceedings. Article 20: (1) Within the scope of his or her authority as defined by Article 9 of this Law, the Chief Federation Prosecutor may issue general or individual mandatory instructions to cantonal Prosecutor’s Offices and carry out criminal investigation and prosecution in cantonal Courts and Municipal Courts, whenever the Chief Federation Prosecutor has reason to believe that the cantonal Prosecutor’s Offices have failed to implement the criminal law of the Federation or that the prosecution of criminal acts cannot be carried out efficiently under the jurisdiction of a cantonal Prosecutor’s Office. (2) The Chief Federation Prosecutor may entrust some cases or actions falling within the competencies of a cantonal Prosecutor’s Office to another cantonal Prosecutor’s Office. Regardless of the competencies of a cantonal Prosecutor’s Office, the Chief Federation Prosecutor may also entrust individual cases to individual Chief Prosecutors, Deputy Chief Prosecutors, Federation Prosecutors or cantonal Prosecutors. 4 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION There is no alternative to JACA’s continued work with the HJPC. Through work with the HJPC and HJPC’s issuance of guidelines and instructions, JACA simultaneously conveys its TA to all judicial institutions. EQ 4: WHAT, IF ANY, ARE THE NEEDS OF THE BENEFICIARIES IN ADJUSTING THE CURRENT LECTURER/TRAINER SELECTION MODEL? EQ 4.1: WHAT LECTURER/TRAINER SELECTION MODEL DOES JACA APPLY? EQ 4.2: WHAT IS THE BENEFICIARIES’ ASSESSMENT OF THE MODEL CURRENTLY IN USE? The Activity’s beneficiaries have rated highly JACA’s two specially designed multiyear training programs: ITP and Specialized Training Program for Judges (SPJ), endorsed by the HJPC and included in the Judicial and Prosecutorial Training Centers (JPTCs) training programs. For local trainers, JACA typically uses senior judges from the highest level courts and prosecutors of equivalent stature. For specific topics, JACA has used regional and, where particularly appropriate, international experts. Since the start of the ITP and SPJ, JACA has also recognized the ability of some participants to share valuable knowledge and has promoted them to become trainers. JACA beneficiaries find that this current model of trainer selection is most suitable for their needs and the BiH context. Most local experts who participate in delivering JACA’s trainings are JPTCs’ certified trainers, and they already provide training within JPTCs’ curricula. Despite current limitations of JPTCs’ programs, mainly caused by inadequate organizational capacity and restrictive budgets, most of our interviewees find the idea of transferring the ITP and SPJ programs to JPTCs acceptable. This transfer could be viable over JACA's remaining life of the Activity, provided that USAID’s branding is retained and JACA remains involved in a mentoring and supervisory role to ensure quality. In addition, under such an arrangement JACA would need to continue to fund some aspects of training that JPTCs currently cannot cover. EQ 5: IN WHAT WAYS COULD COOPERATION/COLLABORATION WITH OTHER USAID EFFORTS AND PROGRAMS BE MORE EFFECTIVE? There is room for improving both the regular exchange of information and the coordination of interventions among anti-corruption USAID Activities, such as the Assistance to Citizens in Fight Against Corruption (ACFC) and Investigative Journalism Program (IJP), and U.S. Government (USG) agencies, primarily the Bureau of International Narcotics and Law Enforcement Affairs (INL) and Office of Overseas Prosecutorial Development, Assistance, and Training (OPDAT). The cooperation among JACA, ACFC, and IJP mainly takes the form of JACA’s inviting these other Activities to its events. JACA is not familiar with any interactions the ACFC/IJP have with courts/judges or POs/prosecutors emerging from court cases or investigations based on ACFC/IJP￾initiated activities. JACA only finds out about ACFC-supported civil-society-organization (CSO) activities from the media. There is neither systematic coordination nor any joint planning. On the other hand, prosecutors stated that, while various CSOs kept reporting the HCOC cases they uncovered, for the most part these reports were not of sufficient quality for investigative and prosecutorial purposes. Similarly, investigative journalism articles typically do not provide prosecutors with sufficient information that can help in conducting effective investigations. Prosecutors believe that some guidance and targeted education about what prosecutors need and can use effectively could help selected CSOs deliver more actionable inputs. On the other hand, improved communication between IJP-assisted investigative journalists and JACA’s beneficiaries, possibly through participation in trainings intended for CSOs, might mitigate the existing animosity between journalists and prosecutors. Adapting JACA’s quality control checklist developed for LEAs for use by CSOs could be an entry point for strengthening cooperation between selected CSOs and JACA-assisted prosecutors in the future. JACA and the INL’s Senior Anti-Corruption Advisor Program have cooperated in the past, but there is room for more cooperation, particularly in adaptation of the quality control checklists originally developed for LEAs for use by anti-corruption offices that are currently being created in several USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 5 cantons. The existing coordination between JACA and OPDAT covered topics and dates of their respective training events, but there is potential for closer synchronization and development of joint activities. Because the target audiences for both programs are very similar, both the beneficiaries and the programs would benefit from creation of a single unified list of participants, a schedule of trainings, and regular exchange of event attendance lists to minimize overlaps and manage demands on participants’ time. Also, because not all participants attend all trainings, a unified knowledge management platform for trainings organized by both programs would be beneficial. RECOMMENDATIONS The recommendations here are presented in an abbreviated form, in two groups: the overall recommendation for JACA’s further work, and a set of specific recommendations. OVERALL RECOMMENDATION: To ensure continuity, further reinforce processing and adjudicating HCOC cases, and help prevent corrupt behavior in the BiH justice sector, JACA should continue to support frontline judges and prosecutors from its current partner institutions by implementing all its planned interventions, taking into account adjustments proposed in the Specific Recommendations. After the draft JACA midterm evaluation report was delivered to the Mission for review, MEASURE II organized a post-evaluation workshop and presented findings, conclusions, and recommendations to representatives of the Mission and JACA. After the presentation, the participants accepted three specific recommendations as viable for implementation in the remaining part of JACA under its current contract. SPECIFIC RECOMMENDATIONS: R 1. Continue with the ITP as planned, extend the SPJ until the end of JACA, and organize more joint training sessions with prosecutors and judges. R 2. Retain the current lecturer selection model within the ITP and SPJ and involve regional and international trainers to cover complex and emerging legal and high-tech topics of relevance for processing HCOC cases. R 3. Increase practical exercises and hands-on training in JACA’s training sessions. Remaining recommendations that the Mission may consider in its future decision-making processes are delivered to the Mission in a separate document. 6 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION INTRODUCTION COUNTRY CONTEXT Bosnia and Herzegovina (BiH) has a complicated government structure that was created through the Dayton Peace Agreement, which ended the war in 1995. The country consists of two entities, the Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska (RS), and one self-governing unit, the Brčko District. Each entity has its own constitution, president, government, and parliament. The FBiH is further divided into 10 cantons, each having a considerable level of autonomy. The organization of the judiciary mirrors the country’s complex and fragmented internal structure. There are four separate judicial systems in the structure of the State, resulting in differences in substantive and procedural law. Furthermore, there are 15 laws governing the work of 20 prosecutorial units (cantonal, district, entity, Brčko District, and BiH POs). Despite some improvements in the justice sector over the past 20 years, corruption continues to impede social and economic development, as well as the accession of BiH to the European Union (EU). The criminal justice system in BiH fails to combat serious crime and corruption. None of the four existing criminal justice jurisdictions is performing adequately. Transparency International’s 2021 Corruption Perception Index ranks BiH 110 of 180 countries. In the 2021 National Survey of Citizens’ Perceptions (NSCP-BiH), more than a third of respondents (38 percent) viewed the court system as extremely affected by corruption, a substantial increase compared with previous years and an all-time record high since the inception of this survey in 2015. On the positive side, the preliminary results of the 2021 Judicial Effectiveness Index of Bosnia and Herzegovina (JEI-BiH) show some improvements in processing cases in the BiH judiciary, including better processing of corruption cases. Nevertheless, sustained and significant efforts to support the fight against corruption need to continue to further strengthen the effectiveness, independence, and professionalism of the justice sector, thus bringing BiH closer to meeting the EU accession criteria. PROGRAM DESCRIPTION AND THEORY OF CHANGE JACA aims to strengthen selected justice sector institutions to combat corruption and economic and organized crime. Exhibit I, below, provides basic information on the JACA Activity. Exhibit 1: Basic information on the JACA Activity Activity Name Judiciary Against Corruption Activity (JACA) USAID Office USAID/BiH Democracy Office Implementer Development Professionals, Inc. (DPI) Contract number 72016819C00001 Total estimated cost $7,996,582 (exclusive of fixed fee: $7,543,945 fixed fee: $452,637) Life of Activity September 19, 2019, to September 18, 2024 (5 years) Active geographic Across Bosnia and Herzegovina Target groups Prosecutors, judges, and other relevant individuals from the partner courts and partner prosecutor’s offices. CDCS Intermediate Result DO1: Accountability of Governance to Citizens Strengthened IR 1.2: Government effectiveness in targeted areas strengthened Sub-IR 1.2.1: Corruption in targeted areas reduced Required evaluation Yes External or internal External Note: DO1 - Development Objective 1; IR 1.2 – Intermediate result; USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION JACA envisions that, if prosecutors and judges are appropriately trained and equipped to investigate and adjudicate corruption cases and if preconditions are created by the High Judicial and Prosecutorial Council (HJPC) to allow for efficient work of the justice sector in combating corruption and enhancing its integrity, then we can expect justice actors (judges and prosecutors) to deal with high-profile corruption and organized crime cases more effectively and efficiently. JACA aims to achieve the following results: Activity Goal: More effective, independent, and accountable justice actors Purpose: Selected justice sector institutions strengthened to combat corruption, and economic and organized crime • Component 1: Processing and adjudicating of the most complex high-profile corruption and organized crime (HCOC) cases in selected prosecutors’ offices (POs) and courts improved o Outcome 1.1: POs and courts have the necessary information and tools to plan, budget, prioritize, manage, and monitor resources to support investigation, prosecution, and adjudication of HCOC cases. o Outcome 1.2: Specialized training, new tools and resources, and technical assistance for prosecutors and judges of partner POs (PPOs) and partner courts (PCs) enhance their overall capacity to investigate, prosecute, and adjudicate HCOC cases. o Outcome 1.3: Strengthened quality of prosecutors’ and judges’ reasons for decisions, indictments, collection, and presentation of evidence, and quantification of damages in HCOC cases reduces errors and improves prosecutor and judge performance. o Outcome 1.4: POs and courts uphold public trust and integrity by increasing predictability and equality before the law; unifying institutional standards and processes; and harmonizing interpretation and application of law, procedure, and judicial practice in HCOC proceedings, decisions, and sentencing. o Outcome 1.5: PPOs and PCs establish effective public communication that increases confidence in the judicial system. • Component 2: Corrupt behavior in the BiH justice sector prevented o Outcome 2.1: Model guidance on ethics and conduct for court and prosecutor’s office is developed and adopted. o Outcome 2.2: Judges, prosecutors, and judicial system personnel share a common understanding of and commitment to HJPC-driven integrity principles, professional ethics, conflict of Interest (CoI) standards, and asset declaration requirements, and have access to resources and tools to comply with these standards and requirements. o Outcome 2.3: The HJPC uses effective tools to monitor compliance with regulations and guidance on judicial ethics and integrity and to deter violations of ethics, CoI, and asset declaration requirements. o Outcome 2.4: Integrity plans are fully implemented and regularly updated in PCs and PPOs. EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE The midterm performance evaluation of JACA examined the results achieved during the first two and a half years of implementation regarding prosecuting and adjudicating the most complex HCOC cases in selected POs and courts and preventing corrupt behavior in the BiH justice sector. This midterm performance evaluation will provide the Mission with actionable findings and information to enable decision making for the rest of the JACA’s implementation period to improve the prospects of achieving the intended Activity results. USAID/BiH and the implementing partner (IP) will use the 7 8 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION evaluation results to make midcourse adjustments to the Activity designs and/or implementation practices. EVALUATION QUESTIONS EQ 1: What primary factors have contributed to the success of or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behaviour in the justice sector (25 percent of JACA LoE)? EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? EQ 4.1: What lecturer/trainer selection model does JACA apply? EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? EVALUATION METHODS AND LIMITATIONS BRIEF OVERVIEW OF METHODOLOGY The ET employed a mixed-method data collection approach and triangulated data to assess the efficiency of JACA interventions and activities. The methodology ensured systematic and efficient collection of data from the following sources: • Review of Activity documents, including Activity Award; Monitoring Evaluation and Learning (MEL) Plan; work, annual and quarterly progress reports; Activity reports; documents (strategies, models, guidelines) produced by the Activity, and secondary documentation relevant to the anti-corruption topics (including relevant laws, regulation, surveys, assessments, and reports from local and international organizations). • Key informant interviews (KIIs) with 26 participants including JACA’s partner institutions/individuals (i.e., POs/prosecutors, courts/judges, and the HJPC and its bodies), and international organizations. • Six focus groups (FGs) with 20 JACA beneficiaries, including prosecutors and judges from the PCs and PPOs in which JACA is working on the implementation of measures for preventing corrupt behavior, and members of the HJPC working on Component 1 and Component 2 with JACA. • HJPC administrative data and JACA survey data covering progress in processing HCOC cases and perceptions of the HCOC prosecutors related to the evaluation of JACA’s TA, contextual factors, and performance in processing HCOC cases. USAID.GOV 9 JACA MIDTERM PERFORMANCE EVALUATION • Observation of the four JACA-organized events that provided the ET with direct insights into participants/beneficiaries’ ability to absorb JACA’s TA, the relevance of events’ topics for them, JACA’s adaptability to meet their needs for the TA, and participants/beneficiaries’ overall response to the events. Limitations of this evaluation include recall and response bias of some beneficiaries, and interviewer/FG moderator bias. To mitigate these, whenever possible, the ET compared the inputs provided by implementors with information from beneficiaries and other sources to corroborate the findings. The ET systematically advised respondents about the importance and value of their frank and honest opinions and assured them of the confidentiality of any information they were providing. Finally, the interviewers and focus group discussion (FGD) moderators were coached to ask questions in a nonleading way and to avoid body language that could be construed as eliciting a specific response from the KIs (see Annex II for a detailed overview of evaluation methods and limitations). FINDINGS AND CONCLUSIONS In this section, the evaluation questions (EQs) are introduced first, and their corresponding findings and conclusions follow. EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high￾profile corruption and organized crime cases (75 percent of JACA LoE)? EQ 1 FINDINGS Finding 1: JACA beneficiaries find that JACA interventions are of good quality, timely, and adequate to their needs. The most-quoted successful activities include training programs, JACA analyses, provision of legal associates, introduction and implementation of HCOC definition in POs, and inter-institutional meetings. At the same time, all JACA beneficiaries rated the interventions successful and welcome. Without exception, beneficiaries assessed JACA’s interventions very favorably in terms of quality, relevance, and timing and often stated that the interventions’ focus on HCOC cases targeted the segment in which judicial institutions have the greatest need for assistance. The ITP has been one of JACA’s most successful interventions because the targeted participants were consulted about the curriculum in advance, trainings were tailored to what HCOC prosecutors asked for, and partner institutions have found the content very suitable for practical application in real situations when dealing with HCOC cases. The specialized training for judges (SPJ) started a year later and has already been enthusiastically welcomed by participating judges. Both ITP and SPJ were approved by the HJPC and included in the Judicial and Prosecutorial Training Centers’ (JPTCs’) programs for 2022.3 The calendar of events was also published, and events were scheduled to take place throughout the year. Trainings were also very positively assessed for organization, selection of lecturers and topics, and use of practical exercises. Because of the adequacy of this training model for prosecutors’ needs, prosecutors have consistently shown 3 Judicial and Prosecutorial Training Program for 2022, FBiH JPTC, pp. 92–98. Let me be brutally honest: JACA is maybe the only project which, through its interventions, shows that they care about improving the response to this type of crime. – Prosecutor “A major corruption case was launched on the basis of these trainings. Without consultation with the lecturers and without using the teaching materials, I could not have done it!” – Prosecutor 10 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION considerable interest in participation in JACA’s trainings. The approach that JACA applies in designing the ITP aligns with the Opinion No. 14 of the Consultative Council of European Prosecutors (CCEP), on the role of prosecutors in fighting corruption, which underlines the importance for HCOC prosecutors to “undergo regular in-service training, as tailor-made as possible, responding to their specific needs.”4 JACA’s scorecards on timeliness/adequacy/quality of JACA’s ITP, obtained through surveys of judges and prosecutors (see Annex VIII), confirm data obtained through KIIs. Representatives of international organizations and donors regularly found JACA’s analytical work very useful for expanding their understanding of certain segments of the work of the BiH judiciary and, thus, very suitable to support their high-level decision-making processes. JACA’s PPOs and PCs very frequently praised the support in the form of provision of legal associates who assisted these institutions by performing legal research, which freed judges and prosecutors to focus on more demanding procedural and legal matters in HCOC cases. JACA was instrumental in supporting the HJPC in introducing the HCOC definition. JACA monitored its implementation and published the report on the application of the HCOC definition, which included recommendations on uniform interpretation and improvements. JACA assisted in preparing the draft proposal for amendments to consider the recommendations given in the report, which the HJPC adopted, with5 the criteria for the categorization of HCOC cases and clear guidelines for their application. JACA helped with the integration of the HCOC definition into the Case Management System for POs (TCMS), but POs then encountered considerable challenges with initial application of the HJPC’s definition of HCOC cases, which made many POs hesitant to apply the HCOC designation to cases and affected eligibility of these cases for priority treatment (see Finding 3). JACA developed guidelines that helped clarify this problem and ensured easier identification of cases that should be treated as HCOC cases. Once the difficulties with proper designation of HCOC cases were removed, JACA provided additional targeted TA, which enabled POs to obtain TCMS reports on these cases and better track their progress. JACA regularly monitors the application of the definitions of HCOC cases and has suggested measures for improvement. This aspect of JACA’s support was fully embraced by the HJPC, as reflected in its Reform Program.6 Evaluation participants indicated that JACA￾organized inter-institutional meetings (both between judges and prosecutors generally and between court presidents and chief prosecutors) and joint trainings were extremely beneficial because local judicial institutions simply do not organize events that bring judges and prosecutors to the same table. For participating prosecutors, these meetings are useful as opportunities for judges and prosecutors to resolve organizational issues, with the aim of improving efficiency of HCOC processing and adjudication. Joint training enabled the two groups to align their understanding about certain judicial situations, which will help prevent future misunderstandings and divergence in jurisprudence. 4 The role of prosecutors in fighting corruption and related economic and financial crime, Opinion No. 14 (2019), CCEP, para. 53–57. 5 The new definitions were adopted by the HJPC at its session held on June 23 and 24, 2021. 6 HJPC Reform Program 2021–2023, p. 16. The manual on the HCOC case designation is very useful to me in my everyday work, particularly as the HJPC did not respond to my written inquiries. – Prosecutor When JACA enabled meetings with judges, which otherwise happen rarely, if ever, in practice, discussion about general legal challenges which follow high-corruption cases took place, helping to prevent problems in the courtrooms in the future. – Prosecutor USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 11 Even when asked directly, not one beneficiary or stakeholder assessed any of JACA’s intervention as unsatisfactory or anything but well received. Instead, they brought up additional positive aspects or examples of benefits of JACA’s interventions, such as JACA’s online platform, that offer easy access both to past training materials and to HCOC international and national case law. In a few individual cases, responses took the form of suggestions for JACA’s work in the future, e.g., to ensure the right balance of theoretical and practical segments of a training or to provide legal associates with a broader range of specialties, in addition to law. Prosecutors from partner POs who were interviewed individually or who took part in the FGD with prosecutor trainees mentioned that the two-year training program also allowed them to establish direct acquaintance and connections with other prosecutors working on HCOC cases. These contacts permitted direct consultations about the work on HCOC cases between ITP participants even outside JACA￾organized events and were often significant in overcoming concrete situations individual prosecutors encountered in their HCOC cases. Finding 2: JACA’s trainings imparted knowledge relevant for starting investigations or overcoming certain challenges in HCOC cases (e.g., use of video recordings of corruption crimes not obtained with prior court decision). Other concrete examples of JACA’s direct assistance that enabled trials in HCOC cases to continue during the COVID period included providing personal protective equipment and equipping courtrooms with the necessary equipment for enabling trials with 50 or more indictees. Inadequacy of the domestic legal framework and the lack of case law to deal with technological advances (e.g., encrypted text messages or audio/video recordings from smart phones, which have become a part of daily life) had created a legal environment in which, e.g., a phone video clip, recorded without a court’s prior authorization, could not be used to open an HCOC case, even though it unambiguously showed that a crime was perpetrated. An ITP program participant used the learning material on European case law for situations involving “unauthorized” video recordings in the context of the European Convention on Human Rights (balancing the right to privacy and the right to information) was able to easily access in an indictment, which the court ultimately confirmed. Without this material, the HCOC case in question, involving a very high-level public official, could never have been launched. The COVID-19 outbreak, which occurred in Year 1 of JACA’s implementation, initially led to a complete lockdown and subsequently required observance of strict social-distancing measures, which caused major difficulties in normal functioning of POs and courts. JACA was flexible and prompt in responding to these emerging needs by providing initial quantities of personal protective equipment, which helped ensure continuity of courts’ and POs’ operations. This form of support was highly appreciated and often highlighted in the KIIs (see Annex VII for a photograph of USAID￾branded transparent plexiglass partitions in a courtroom). Furthermore, a court faced a particular challenge of conducting trials in HCOC cases with 50 or more indictees, because the existing courtrooms could not accommodate so many participants. Using its special fund, JACA was able to provide audiovisual equipment, which was installed in an adjacent room to allow an additional number of participants in the given case to attend the trials, thereby helping overcome this unique challenge (see Annex VII). Finding 3: JACA’s Monitoring, Evaluation, and Learning (MEL) systems are designed properly and adequately and reflect the results of interventions in the course of the The network of people who these days communicate and talk to each other was created thanks to USAID. If you spend two years attending a specific training program, you will naturally establish a bond with other people in the same program. Truly, USAID “mended” certain omissions which are the job of competent institutions. – Prosecutor USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 12 implementation of the Activity. In Year 2, JACA did not reach its targets on three MEL indicators, but this was caused by technical or contextual reasons beyond the Activity’s influence. The JACA MEL Plan includes a total of 15 indicators (see Annex I, Exhibit 2), including one Activity Goal indicator, six Activity Purpose indicators, and eight Activity Sub-Purpose and Outcome/Output indicators (five for Component 1 and three for Component 2). As part of its MEL system, JACA established a Score Cards Monitoring System, which originally envisaged three scorecards that tracked the results of TA provided to PPOs. Two more score cards that tracked the support to PCs were added to the System in 2021. The score for JACA TA monitoring for PPO prosecutors was 80.4 percent, confirming that JACA’s TA was both applicable and adequate, which is in line with the independent findings of the ET. Similarly, the Composite Context Indicators CC 1 and CC 2 look at the factors beyond the control of both judicial institutions and JACA for prosecutors and judges, respectively, and the results in Year 2 signaled that the contextual conditions for processing HCOC cases were unfavorable. These results were also corroborated by the findings the ET generated on the basis of other sources. JACA met all its targets in Year 1 and even managed to reach more training participants than originally planned. According to its Year 2 actuals, JACA met or exceeded its targets for all except three of its indicators. Of these three indicators that track HCOC indictments and investigations, for the indicator AP 1—number of indictments filed for HCOC cases by PPOs—the ET found that the actual for Year 2 was negatively affected because the newly introduced definition of HCOC cases made POs uncomfortable about their application and very conservative in assigning the HCOC designation, as previously presented in Finding 1. As a result, the number of HCOC investigations was somewhat lower than predicted, which caused the values for this indicator to miss the target slightly. The situation was similar to the CBLD 9 indicator, which tracks HCOC investigations. In Year 2, the number of new investigations was 10 (one fewer than the target), while three PPOs (of the targeted four) delivered the projected number of new investigations. In the year when the aftereffects of COVID protection measures were still felt, the actual for indicator DR 2.4-1, number of government officials receiving USG-supported anti-corruption training reached 95 percent of its Year 2 target; during the interviews, it was explained that some of participants’ last-minute cancellations could probably be attributed to COVID-risk aversion. Finding 4: Underlying factors for the success of JACA interventions can be summed up as inadequacy of local institutions’ response to HCOC cases and JACA’s expertise and commitment to address needs of HCOC judges and prosecutors. Almost all evaluation participants complained that judicial institutions are not fulfilling their role. Some specific examples of supporting actions that clearly fall within the power and authority of judicial actors but are not happening include organizing periodic meetings on HCOC between courts and POs, taking initiative to harmonize case law and sentencing policy, and spreading awareness of (declared) top-priority of HCOC cases. JACA’s interventions have been filling many of these gaps. Some direct examples included JACA’s assistance in introduction of the HCOC designation in POs, organizing meetings between courts/judges and POs/prosecutors, and providing specialized HCOC training to prosecutors and judges. More indirectly, JACA’s trainings provided the venues in which prosecutors and judges got an opportunity not only to learn but to communicate with their peers. It is important to note that this “privilege” is limited to prosecutors and judges who are enrolled in the ITP and SPJ). When it comes to fighting HCOC, it seems that POs do not have real support. Put simply, you feel that you are on your own. That you are fighting a hopeless battle. – Prosecutor USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 13 On the other hand, KIs and FGD participants assessed JACA’s expertise and professionalism very positively, noting in particular the relevance of training topics, quality of trainers, and value and timeliness of JACA’s analytical work and materials, which were developed to meet specific needs of the judiciary. Beneficiaries also praised JACA’s responsiveness to and interactivity with its training participants (enabling online access to training materials and creating a dedicated database of HCOC case law and professional legal papers and articles). Finding 5: While some of the impediments in JACA’s highly challenging environment can be attributed to the lack of political will, the main issue is the absence of local “judicial will”—reflected in the judges’ reluctance to make decisive use of tools and powers already at their disposal for achieving better results in processing HCOC cases. KIs recognized that a number of external factors constricted the effectiveness of JACA’s TA. Some interview participants perceived that a lack of political will, exemplified by weak hierarchical links among POs in various jurisdictions in the country, budgetary dependence on the executive branch of government, and the delay in the adoption of the new amendments to the law on the HJPC7 —all of which were undermining the judiciary’s ability to act adequately in HCOC cases. Prosecution of HCOC cases is also complicated by poor cooperation with LEAs, which tend to be highly susceptible to political influence.8 Still, the interviewees focused even more on the failings of the judiciary itself. According to evaluation participants, especially those within the judiciary, using quotas as main performance metric is the major problem.9 Court/PO-level (collective) quotas encompass cases of all types, and this drives institutions’ managers to spread their resources to meet their quotas and creates incentives for individual judges/prosecutors to focus on resolving the required number of cases rather than take on important and complex ones. Following the introduction of the HCOC definition, the HJPC improved its regulations related to performance evaluation of prosecutors10 to allow assigning greater (quota) weights to HCOC cases. This approach had previously been successfully used to address the problem of war crimes in the BiH judiciary. Nevertheless, chief prosecutors and court presidents, who face a number of competing requirements and priorities imposed by the HJPC, such as resolving backlogs, war crimes, and bankruptcy cases, are frequently reluctant to give precedence to HCOC cases over these equally prioritized cases by the HJPC. The report of the Parliamentary Committee stated, “The current system, in which the performance of judges and prosecutors is primarily assessed on the basis of quota fulfillment, encourages focusing on less demanding indictments for more benign criminal acts and separation of proceedings, while in judges it removes the motive for putting extra effort in justifying their decisions.”11 In practice, this means that individual prosecutors, even if they specialize in HCOC cases, need to handle other types of cases and other duties and are often left under-resourced and with limited support when dealing with their HCOC cases. The few committed HCOC prosecutors are deeply frustrated with the environment they work in. They feel unsupported and unprotected by the system and often consider quitting their line of work and becoming judges or lawyers. Nevertheless, some prosecutors believe that, if HCOC cases were assigned top priority, if HCOC prosecutors were 7 Bosnia and Herzegovina 2021 Report, European Commission, October 2021, pp. 15–24. 8 Bosnia and Herzegovina 2021 Report, European Commission, October 2021, p. 5. 9 “Quota” system is explained in detail in Footnote 1. 10 Rule Book on General Measurements of Performance of Prosecutors in BiH, BiH Official Gazette, no. 6/22, June 2022. 11 Report on the state of the BiH judiciary, Interim Investigative Committee, Parliamentary Assembly of Bosnia and Herzegovina, June 2022, p. 104. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 14 assigned to work on HCOC cases only, and if they received other needed support, then the new quota system could still yield improvements in HCOC processing. However, this is not the case now. As Finding 1 showed, the integration of the HCOC definition in the TCMS was implemented with JACA’s TA. Unfortunately, integration of the HCOC definition in the Case Management System for Courts (CMS) has not been implemented, which makes reporting on and monitoring of the work of courts on HCOC cases impossible within the CMS.12 In addition, the result of the HCOC definition’s not being applied in the CMS is that SPJ judges do not get HCOC cases despite having the necessary specialization for this type of cases. Integration of the HCOC definition into the CMS would enable better planning of work on these cases and monitoring of the execution. As the work of POs on HCOC cases increases, the courts will soon see an increased inflow of these cases and, without the introduction of the HCOC definition in the CMS and without changing the quota regulations, the courts will become the next stumbling block in seeing successful HCOC adjudication. Finding 6: In terms of contextual factors, the situation is not the same in all PPOs. There is an issue with the responsiveness of LEAs, but while some POs claim that this is the major issue that stops them from having any HCOC cases, some others successfully overcome this issue by their proactivity and by using all legal tools at prosecutors’ disposal. In proactive PPOs, there is some progress in processing HCOC cases at the local level. As described in the Expert Report on Rule of Law Issues in Bosnia and Herzegovina, the challenges to cooperation between POs and LEAs, stemming from the lack of cooperation and coordination and the lack of proactivity on the part of LEAs, lead to deep fragmentation of LEAs.13 A judge of the BiH Court summed up the way politics infiltrates and controls LEAs: “In every post-election sharing of offices after ruling coalitions are formed, the offices of the ministers of interior affairs, heads of SIPA [BiH State Investigation and Protection Agency] and OSA [Intelligence and Security Agency, originally, Obavještajno-sigurnosna agencija], and ministers of finance are the most coveted. The ministers of interior affairs and heads of LEAs are the key levers of power because their task is to protect their political bosses from accountability for fraud and organized crime. In this political game a prosecutor is a player of lesser importance because the police, who are under control of the politics and of the minister, will determine through their executive apparatus what PO’s will and what they won’t prosecute.”14 All evaluation participants in one or way or another confirmed that there were issues with cooperation of LEAs in HCOC investigations. However, some of JACA’s PPOs have dealt with the above difficulties more proactively than others. While the problem of cooperation with the police is generally recognized as affecting the work on HCOC cases, some prosecutors described the way proactivity (in the form of working with different LEAs on different segment of investigations) and use of available powers (using prosecutorial legal authority to either exercise powers vested in prosecutors themselves or to command greater police involvement) allowed them to circumvent initial lack of enthusiasm by the LEAs. Such proactivity made it possible for some PPOs to show results in processing HCOC cases at the local level in the face of all the above challenges (see Annex X). Finding 7: Prosecutors and judges working on HCOC cases face external and internal challenges aiming to distract them from working on HCOC cases. Because of the lack 12 CMS functionalities enable many shortcuts in tracking cases, and some of these are possibly applicable to HCOC cases. Nevertheless, there are “hidden” software solutions available to a few system users only. This approach cannot be compared with visibly marking HCOC cases with a special case type that would allow them to be easily spotted and tracked by all. 13 Expert Report on Rule of Law issues in Bosnia and Herzegovina, Brussels, 5 December 2019, pp. 44-46. 14 Judiciary in BiH: Current Status and Prospects, Transparency International, 2019. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 15 of protection by the system and a lack of incentives to take on HCOC cases, prosecutors consider leaving POs to take other positions. None of our evaluation participants personally experienced or were aware of direct pressure on judicial officials in the form of personal demands by Government officials or political figures or any other external persons or entities to suppress HCOC investigations. Still, all participants mentioned media attacks, public protests, obstruction by LEAs, and filing disciplinary and occasionally criminal reports against prosecutors and judges as the most frequent forms of indirect pressure that are typically related to processing of HCOC cases. Both prosecutors and other consulted stakeholders expressed frustration because of intensive attack and smear campaigns on judicial officials (and members of their families) acting in prominent HCOC cases.15 Attacks of online media are often openly partisan, and on social media they tend to be even more personal and slanderous. Moreover, in the survey conducted by the European Network of Councils for the Judiciary, of 392 BiH judges that took part in this survey, 23.7 percent believed that the media had affected judicial decisions, confirming the influence of the media on judicial actions or decisions.16 The HJPC has not issued any guidelines that take into consideration the way these factors affect the overall trust in the judiciary or suggest any actions judicial institutions could take. In the absence of specific guidelines on the way to act in cases of media and other attacks on prosecutors and judges, an individual PO or court must deal with such attacks on its own. Courts and POs often do not even report these cases to the HJPC. In certain cases, media attacks were combined with and led to orchestrated public protests in front of official judicial buildings. A more recent format of attacks involves filing of often multiple disciplinary and even criminal reports against prosecutors pursuing HCOC cases, with the intention of discouraging them or at least diverting their attention.17 HCOC prosecutors and judges work in an environment in which public trust in the BiH judiciary is very low. In the 2021 Balkan Barometer, 82.7 percent of participants in BiH found the judiciary to be the most corrupt sector.18 According to the 2021 NSCP-BiH: “Trust in public institutions remained low, with citizens having the lowest levels of trust in the judiciary, government institutions, and political parties since 2016.”19 Baseless media attacks on judicial officials do not contribute toward increasing public support for the judiciary. When there is public support, it significantly contributes to the independent judiciary: “The greater the public support, the more likely it is that the judges and prosecutors will resist the political branch.”20 To remedy some of these pressures, JACA developed the Model of Crisis Communication Strategy for PCs and PPOs and a Guide to Crisis Communications, which subsequently became integrated as annexes into the HJPC’s Communication Strategy, adopted in December 2021. The BiH Analytical Report stated, “In practice, judges and prosecutors are vulnerable to pressures, including through self-censorship. Judges and prosecutors have been subject to politically motivated threats.”21 The Expert Report on Rule of Law Issues in BiH noted that the pressure was observed in the work of prosecutors and judges and manifested itself as interference in ongoing cases, pressure, threats, and intimidation.22 15 Examples of media attacks on prosecutors working on HCOC cases are attached in Annex IX. 16 Independence, Accountability and Quality of the Judiciary BiH 2018–2020, Netherlands Council for the Judiciary and the Norwegian Court Administration, p. 32. 17 A New Judicial Problem: The Accused and Suspected Report the Prosecutors Investigating Them, Žurnal, 3, August 2022. 18 Balkan Barometer 2021, Regional Cooperation Council, p. 123. 19 The 2021 National Survey of Citizen’s Perceptions, p. 67. 20 The Blindfolding Justice in Bosnia and Herzegovina? State Capture of Bosnia and Herzegovina Judiciary and Public Prosecution, Open Society Fund, January 2021, p. 18. 21 Bosnia and Herzegovina Analytical Report 2019, European Commission, Brussels, May 2019, p. 33. 22 Expert Report on Rule of Law Issues in Bosnia and Herzegovina, Brussels, December 5, 2019, p. 49. And there is this aspect which we already made into a joke: the number of criminal reports against prosecutors is the measure of the quality of our work! – Prosecutor USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 16 In addition to challenges related to the quota system explained earlier, internal challenges within the judiciary can also take the form of expanding the HCOC prosecutors’ workload with additional cases; assigning them to cover their colleagues’ cases; serve as “duty prosecutors” for cases that emerge after working hours (typically unrelated to HCOC), which they then have to follow; and withholding or reallocating support and resources. Faced with heavy pressures and disappointed by the lack of recognition and support in their institutions, individual prosecutors working on HCOC cases often feel frustrated and lose motivation. In the current circumstances, HCOC prosecutors often find themselves either applying for judgeships or leaving to become lawyers. Finding 8: The frontline prosecutors/judges offered proposals on types of support they need to improve processing of HCOC cases. Some possibilities include providing them access to available databases in BiH and identification of illegally obtained assets through proactive data mining and provision of advisors of different expertise. Despite all the challenges discussed above, some frontline prosecutors and judges are still keen to improve the work on HCOC cases, and they offered some viable ideas on how to improve HCOC processing. One suggestion involved extensive modification or full exemption of quotas for HCOC prosecutors and judges and specialization of prosecutors and judges for HCOC cases. Another idea was to organize and hold periodic meetings of frontline prosecutors in cantonal POs with the FBiH Chief Prosecutor to discuss operational problems and potential solutions.23 The third proposal was to facilitate access to existing official BiH databases, which currently are not accessible (i.e., company registers, banking databases, land registers, motor vehicle database, securities register), mostly for bureaucratic or technical reasons. In combination with this proposal, it may be more efficient and cheaper to grant access to one institution, rather than to multiple ones with a similar mandate, e.g., to enable access to the PO FBiH, which would then, through developed protocols, share data with 10 cantonal POs. This proposal is fully in line with the position of the Consultative Council of European Prosecutors, which noted, in its Opinion No. 14: “Prosecutors, when they conduct or supervise the investigation, must have, subject to, where appropriate, judicial authorization, effective access to all relevant sources of information, often stored in public or private databases. It is furthermore decisive to ensure the prosecution service’s access to registers of property and interests or asset declarations regularly provided by public officials and other persons in accordance with national law, in order to deter potential perpetrators from committing an act of corruption and to prosecute them when such an act is committed. This is an essential tool for uncovering an existing system of corruption or associated criminal structures. Access to bank records or tax information is also of the utmost importance for effective prosecution of corruption cases.”24 Prosecutors would also benefit from access to existing international public databases, in addition to domestic ones; prosecutors could, for example, use the Organized Crime and Corruption Reporting Project’s (OCCRP) Aleph investigative data platform to check whether people implicated in HCOC cases own property or hold funds abroad. Finally, access to advisors with a broader range of expertise and data-mining skills would also ease the burden on HCOC prosecutors in building complex cases. 23 See the Law on the Prosecutors’ Office of FBiH: Article 9 and Article 20. https://portalfo2.pravosudje.ba/vstvfo-api/vijest/download/3288 24 The role of prosecutors in fighting corruption and related economic and financial crime, Opinion No. 14 (2019), (CCEP), para 28. People working on corruption cases en masse apply to become judges or leave to start their own legal practices. They are running because they are aware that, when you start working on a big case—you yourself and your family become targets ... – Prosecutor 17 EQ 1 CONCLUSIONS JACA’s interventions related to improving the processing of HCOC cases are well tailored to the needs of the beneficiaries and highly rated by them. JACA’s support covers an extensive spectrum of equally important interventions needed to improve processing and adjudication of HCOC cases (starting from the quality of HCOC criminal reports filed with POs, through all necessary steps in successful prosecutors’ investigations, up to adjudication and sentencing in HCOC cases), all of which are needed to achieve progress in processing and adjudicating HCOC cases. It will still take some time and additional Activities effort, mainly in mitigating contextual issues, until stronger results are produced, but it is evident that JACA brought processing of HCOC cases into the focus of many discussions and activities within the BiH judiciary. Unfortunately, engagement of local judicial institutions in providing institutional support to the frontline HCOC judges and prosecutors (ranging from simple organization of judge–prosecutor meetings, up to creating an efficient operational setup for processing HCOC cases in courts and POs) is generally insufficient. JACA succeeds in filling many of these gaps by providing expertise and timely assistance, including delivery of high-quality trainings, development of knowledge management databases, organization of inter-institutional meetings, facilitation of operational implementation of the new, more precise designations for identification of HCOC cases as the basis for the specialization of prosecutors (and judges), management tracking of processing these cases, and support in provision of additional human resources (i.e., legal associates) and assorted IT and office equipment. In addition, during COVID-19, JACA responded to emerging challenges in partner institutions and helped them ensure continuity of their work on HCOC cases. Through ITP activities, JACA created an informal network of HCOC prosecutors, who took part in specialized training for the past two years. These prosecutors exchange information related to processing HCOC cases on an ad hoc and as-needed basis in one-on-one communication even outside JACA events, which should be seen as an element of the sustainability of this JACA’s intervention. However (outside JACA’s current training program), there are no formal meetings of frontline prosecutors (those who work only or mostly on HCOC cases) in which they could discuss factual and legal challenges, as well as potential solutions pertaining to processing of HCOC cases. In particular, frontline HCOC prosecutors do not have a formal channel for proactive communication and consultation with the higher-level POs (which will act in the appeal cases, but in many instances that will be too late to resolve issues not correctly addressed in the first phases of prosecution). Among themselves, frontline prosecutors have the opportunity to discuss operational needs and problems solely during or on the margins of JACA’s training sessions. HCOC prosecutors and judges work in a highly unsettled environment. The issue of quotas in performance evaluation is one of the central problems. While HCOC cases have been assigned some extra weighting under the HJPC regulation that governs performance measurement of prosecutors, those provisions are not at all aligned with other provisions that set targets (following all concurrent HJPC strategic priorities) for the overall performance of POs/courts, heads of institutions, and judges/prosecutors when working on other case types. This creates major problems in the practical implementation of prioritizing work on HCOC cases in POs and courts. POs/courts (and individual prosecutors/judges) are effectively left to work independently make trade-offs between achieving institutions’ (and individual prosecutors/judges) quotas and successful processing of HCOC cases. Individual prosecutors who commit to working on HCOC cases face numerous operational issues in seeking the opportunity to concentrate on HCOC cases, which frequently drives them to consider giving up the work on HCOC cases and focusing on cases that are easier to deal with (as the HJPC regulation leaves room for it). With all these issues unsorted, a formal pronouncement that HCOC cases are the priority for the judiciary is diluted and disappears at the operational level. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 18 A further proof of the absence of adequate management support for frontline HCOC prosecutors at the operational level is evident in the failure to assign support staff to work with these prosecutors, in frequent shortages of equipment and/or adequate workspace, and in the lack of funding for emergent needs related to abrupt changes (emerging opportunities) in dealing with HCOC cases. In addition, the prosecutors who continue to work on HCOC cases are exposed to various indirect forms of pressure, mainly in the form of politically orchestrated attacks on them and their family members by politically captured media (mainstream and online) and/or filing of malicious criminal and disciplinary reports. These prosecutors are also greatly frustrated with the absence of cooperation with LEAs, which almost all informants find to be highly susceptible to various political influences. All the above-mentioned issues, coupled with an inadequate quota system and the lack of HCOC specialization and prioritization, impede the prosecutors’ work on HCOC cases. In sum, all these issues combine to make the frontline HCOC prosecutors feel unsupported and unprotected by the judicial system. The remaining enthusiastic HCOC frontline prosecutors should be urgently supported before even they decide to leave the judiciary for good. The conclusions offered here mainly relate to prosecutors’ work because they are the ones who have to successfully initiate HCOC cases before such cases can even arrive at the courts. As there are not many HCOC cases filed with courts, the attention should be first on the frontline prosecutors, but judges who work on adjudication of HCOC cases face the same challenges. EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behavior in the justice sector (25 percent of JACA LoE)? EQ 2 FINDINGS Finding 9: JACA beneficiaries found that JACA interventions raised visibility and traction of integrity issues and that Component 2 interventions are of good quality, timely, continuous, and not supported by other donors. The most quoted interventions in this segment are the support to the HJPC’s Integrity Department, support for the implementation of integrity plans, and confidential counseling. Evaluation participants praised interventions conducted under JACA’s C2 component as essential for raising awareness of judicial professionals about the importance of better understanding issues of ethics and integrity. Beneficiaries value JACA’s contribution under this component for its quality, continuity, and responsiveness, and they recognize that, without JACA’s support, the progress achieved in domain of ethics and integrity could not have occurred. The representatives of the relatively recently formed HJPC’s Integrity Department stated that the Department received considerable JACA assistance in establishing its main functions. JACA supported the establishment of a network of integrity coordinators, who received training and ongoing support. In this segment, JACA worked not only with PPOs and PCs but with all judicial institutions across the board. In its partner institutions, JACA followed up and, where necessary, assisted in preparation of annual reports on the implementation of integrity plans, which were submitted to the HJPC’s Integrity Department by given deadline. JACA also supported development of the Integrity Communication Strategy and Action Plan, which were to be incorporated into the HJPC overall communication strategy. 19 JACA supported the HJPC in defining a model of confidential counseling for the BiH judiciary, which was another of JACA’s contributions that beneficiaries specifically mentioned. Confidential counseling is a forum in which judges and prosecutors who have ethical dilemmas can seek advice through a confidential channel with appointed advisors for the judiciary. JACA helped the HJPC develop the roadmap and recommendations for confidential counseling, and it is now up to the HJPC to decide whether this forum will sit in the HJPC, in individual judicial institutions, or in a combination of both. This is in line with the GRECO recommendation on strengthening and further developing confidential counseling and dedicated training of a practical nature on issues of ethics and integrity.25 KIs also mentioned the comparative analysis of the integration of background checks into the appointment process of judges and prosecutors. The importance of introducing integrity criteria into the BiH judicial appointment process was recognized in the Open Society Fund report, which underlined that “critical problems are related to the fact that the criteria of merit, ethics, and integrity are dominant neither in the initial recruitment process nor in the career advancement of judges and prosecutors.”26 Finding 10: Some JACA-supported interventions under C2 resulted in the formal adoption of integrity policies by the HJPC and relevant conclusions were incorporated into the proposed amendments to the HJPC law,27 which ensures sustainability of JACA interventions in a long run. Well-informed evaluation participants stated that JACA’s support on integrity and ethics issues included assistance in the preparations of the HJPC’s official policy documents,28 notably the Model Code of Ethics for court and PO employees and the new list of integrity risks, which were formally endorsed by the HJPC.29 The HJPC distributed the Code of Ethics to all courts and POs, with the recommendation for its adoption and with an accompanying proposal for a tool to monitor the implementation of the Code. The HJPC adopted the new list of integrity risks, for the next four-year cycle, which will be integrated into integrity plans of all judicial institutions in BiH. The list includes the risks related to the operation of POs and courts, integrity and ethics for judicial and non-judicial staff, security of information, and CoI.30 The formal adoption of judicial guidance and regulations related to ethical conduct and their implementation are of paramount importance. In its Opinion No. 21, the CCJE stressed that a lack of such guidance can lead to “judges becoming indifferent to the requirements of objective and impartial justice.”31 Finding 11: JACA prepared summaries of disciplinary case law and Integrity Committee’s opinions on ethical questions judicial office holders had, but these have not yet been made public on the HJPC website. One of JACA’s interventions to spread awareness and increase understanding of integrity and ethics issues was to prepare summaries of the disciplinary case law and the Integrity Committee opinions on ethics questions submitted by judges and prosecutors. This activity was endorsed by the HJPC.32 25 Second Compliance Report Bosnia and Herzegovina, GRECO, adopted on September 25, 2020, p. 10. 26 The Blindfolding Justice in Bosnia and Herzegovina: State Capture of Bosnia and Herzegovina Judiciary and Public Prosecution, Open Society Fund, January 2021, p. 113. 27 Draft Law on Amendments to the Law on the High Judicial and Prosecutorial Council of Bosnia and Herzegovina. 28 HJPC Reform Program 2021–2023, p. 11. 29 The HJPC session held on March 24 and 25, 2021. 30 The HJPC session held on February 9 and 10, 2022. 31 Preventing Corruption among Judges, Opinion No. 21 (2018), Consultative Council of European Judges, November 2018. 32 The HJPC session held on February 9 and 10, 2022. Technical assistance about the principles of integrity is recognized as one of the more significant contributions to the development of the judiciary in recent years. – International organization representative USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 20 The disciplinary case law is now available in a user-friendly format, containing disciplinary offense, function of judicial official, institution, adjudicated disciplinary measure, and summary of the case. Summaries provide easy-to-search information, and users can obtain required information quickly. This material was submitted to the HJPC, but this and other essential integrity materials, documents, and policies have not yet appeared on the official HJPC website. Finding 12: Underlying factors for the success of JACA regarding prevention of corrupt behavior in the justice sector can be summed up as lack of technical knowledge of local institutions for implementation of ethical principles, absence of other donors in this segment, and JACA’s availability, expertise, and promptness to respond the needs of HJPC and beneficiaries in courts and POs. The factors that contributed to the success include USAID/BiH’s long-term presence, commitment, and experience in this segment of work; JACA’s team professionalism and expertise, as well as local institutions’ needs for TA, and absence of other donors supporting similar interventions. JACA has built on progress achieved under the previous USAID/BIH-funded justice project to prevent corrupt behavior within the justice sector. USAID/BIH carefully planned the next phase (JACA) on the basis of insights, knowledge, and data gained from implementation of the previous project to ensure progress toward sustainability. Participating prosecutors and judges noted that interventions were well pitched and that technical assistance was provided in areas that the local institutions needed. Because of its long-term presence, systematic approach to the issue, and responsiveness to the local institutions’ needs, USAID/BiH is perceived as a trustworthy partner. While some assistance has been provided by other donors, such as the Organization for Security and Co-operation in Europe, UK government, and EU, no other assistance has been as comprehensive and long lasting, so USAID/BiH is perceived as the major donor in this segment of work by both local and international organizations. KIs found that JACA’s team professionalism, expertise, and responsiveness contributed to the Activity’s success. JACA’s start coincided with the establishment of the HJPC Secretariat’s Integrity Department, enabling JACA to deliver the necessary TA promptly. JACA built trusting relationships with the HJPC and other judicial institutions that rest on openness, collaboration, and adaptiveness. All these factors were important in achieving durable results, which resulted in the formal adoption and implementation of integrity-related policy documents. Evaluation participants noted one example of JACA’s prompt response to the HJPC’s request to produce and present the analysis of public statements and use of social media by judges and prosecutors. The request came at the time when some HJPC members excessively used social media to communicate with a wider public33 and when there were reports on judges who were perceived, on the basis of their views expressed on social media, to be biased.34 33 HJPC members comment on public events and their colleagues without compunction, detektor.ba, 12 February 2020. 34 The judge who freed the attacker on Hadžifejzović previously badmouthed Face TV: Schizophrenia is a cold in comparison, Raport.ba, 31 May 2021. JACA colleagues have been working on the judiciary long enough, so they were competent in formulating the project’s focus, which means that the selected interventions were quite relevant. – Focus group participant 21 EQ 2 CONCLUSIONS As in Component 1, JACA’s interventions related to preventing corrupt behavior in the justice sector are well designed, timely, and tailored to the beneficiaries’ needs. Continuity in USAID/BiH’s support in this area, professionalism and quality of JACA’s team, and absence of other donors were recognized as factors contributing to the success of its interventions. JACA’s interventions helped the HJPC connect different elements in a single system for improving ethics and integrity in the judiciary. JACA’s interventions included developing a Model Code of Ethics and Guidelines for Employee Conduct; an online training program on ethics and CoI, supporting and training the HJPC’s Integrity Department, developing the list of integrity risks and its eventual integration and implementation in the integrity plans of individual institutions, creating a network of integrity coordinators to conducting research and analysis, providing TA on various integrity and ethics issues, including background checks when assessing candidates’ integrity in the appointment process, and assisting the HJPC in responding to recommendations of confidential counseling. JACA provided immediate support to the HJPC Secretariat’s Integrity Department when it was established and is recognized as a reliable partner. The documents produced with JACA’s assistance are used as a basis for setting the HJPC’s integrity agenda and raising awareness of its importance across the judiciary. JACA’s support in creating the new list of integrity risks, which will be used for the next four-year cycle of implementation of integrity plans, ensured that the list would include the most serious integrity risks, as well as measures to address them. JACA’s work on supporting the drafting of a Model Code of Ethics and Conduct for Court and PO Employees ensured adoption of the Code by sixteen individual judicial institutions so far, which goes beyond JACA’s partner institutions. The HJPC has a need for JACA’s continued support, including dissemination of disciplinary practice and opinions of the Standing Committee on Ethics, Integrity and Accountability of Judges and Prosecutors. This will enable judicial employees to understand what standards of behavior are expected from them, what constitutes acts of misconduct, and what sanctions can be imposed. Due to absence of other donor support in this area and the ongoing initiatives in which JACA is involved, JACA’s support to the HJPC and to individual judicial institutions will need to continue at the same pace. EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/ adjudication of HCOC cases and the HJPC for Component 2 interventions? EQ 3 FINDINGS Findings 4–7 presented within EQ 1 (e.g., findings dealing with contextual factors for the judiciary or individual POs related to processing HCOC cases) feed into findings related to EQ 3 and are the starting point for examining EQ 3. Finding 13: There are diverse, even conflicting, views on whether new partner institutions (i.e., any new cantonal/district POs, BiH PO, and corresponding courts) should be added to JACA interventions but no convincing arguments that any new judicial should be added to the JACA partner institutions. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 22 The first set of disagreements is related to the fact that some POs do not have HCOC cases, and some argue that with JACA’s support such cases might be initiated. The interviewees were divided about the need and feasibility of expanding the network of JACA’s partner institutions for the remainder of its life of Activity. Some JACA beneficiaries considered it beneficial to include other cantonal and some district POs, while others found it unrealistic at this stage of the Activity or unnecessary because any potential new partners were small and with a negligible number of potential HCOC cases. The other set of issues relates to the inclusion of the BiH PO because of its jurisdiction over prosecuting HCOC cases; many disagreed with including the BiH PO because of its lack of results (relative to available resources) in prosecuting HCOC cases. Some evaluation participants were in favor of including the BiH PO in some format, or at least including some of its individual prosecutors, while others believed that the BiH PO did not need the support provided by JACA and would not be willing to put it to use. Individual participants also suggested including the Court of BiH, other courts, and other agencies, such as LEAs and anti-corruption offices. Finding 14: There are conflicting views about continued commitment and results in processing and adjudicating HCOC cases in JACA partner institutions, as among KIs there are different viewpoints and uneven criteria for assessing commitment and results for processing and adjudicating HCOC cases. Perceptions of our interviewees differed widely about the commitment and performance of individual JACA’s partner institutions. Some evaluation participants felt that all partner institutions showed progress, while other participants were more discerning; some spotlighted one PO for its outstanding results in processing HCOC cases, and others mentioned another PO as a success story. In the case of the third JACA PPO, some of the interviewed participants perceived progress, while others detected stagnation or slowdown in its processing of HCOC cases. While the importance and complexity of the HCOC work of this PO was specifically recognized and its results were praised by some, other participants saw this PO’s recent performance as mediocre, or even poor. One JACA PPO is predominantly seen as underperforming, but at least some of our interviewees were positive about its results. Some beneficiaries and other stakeholders found it necessary to highlight the importance of enthusiasm and dedication of individual HCOC prosecutors. On the other hand, several reports covering processing of corruption cases in the BiH judiciary noted some progress in 2021. The HJPC stated in its annual report that the number of indictments for corruption increased by 87 percent in 2021 relative to 2020.35 The 2021 JEI-BiH report also captured some progress in processing corruption cases: the number of resolved cases rose by 26 percent and the average duration of unresolved corruption cases was cut by 27 percent.36 Nevertheless, the number of indictments for high-profile corruption is still low, and only three percent of a total of 235 indictments for corruption were for high-profile cases.37 Finding 15: Commitment to processing HCOC cases and interactions between the management and frontline HCOC prosecutors in POs varied considerably. The ET categorized these interactions in three different types. In the current operational context of POs, after analyzing the KIs’ inputs, the ET found that there are three types of interactions between management and frontline HCOC prosecutors. The first type includes declarative prioritization of HCOC processing, but POs in this group find that numerous challenges beyond their control, such as lack of cooperation with LEAs, lack of corruption criminal reports being filed, the age structure of prosecutors, budgetary limitations, and frequent sick 35 HJPC 2021 Annual Report, April 2022. 36 2021 JEI-BiH Report, July 2021, p. xii. 37 HJPC 2021 Annual Report, April 2022. 23 leave absences of prosecutors, prevent results. The POs in this group have practically no concrete results in processing HCOC cases. The second type is characterized by alignment between the PO management and HCOC prosecutors, which produces tangible results at the local level, but there were few POs of this type. In those POs, prosecutors recognized that they had the same powers as LEAs to investigate HCOC cases and collect evidence. They worked proactively with multiple law enforcement agencies to get individual pieces of evidence that they put together into a bigger picture. These POs had tangible results in processing HCOC cases at the local level, and their communities recognized these results by submitting increasing numbers of criminal reports as the basis for prosecutors to launch subsequent investigations. An example of results of a PO of this type is presented in Annex X. The third type when enthusiastic and dedicated HCOC frontline prosecutors did not have support of their management. The frontline prosecutors in this last type of POs faced the greatest number of challenges and operational obstacles in processing HCOC cases in practice. In these POs, managers are perceived to be under political influence. Such managers use their authority to distract committed frontline HCOC prosecutors from focusing on HCOC cases by assigning them less important cases, dispatching them to represent other cases in trials, or keeping them on a roster of duty prosecutors. In such POs, frontline prosecutors mistrusted their superiors, which caused them to withhold information and avoid seeking legal advice from their superiors in challenging legal situations. Frontline prosecutors who encounter issues in their POs when working on HCOC cases do not have available proactive channels for communicating these issues to higher-level instances (i.e., the HJPC or higher-level POs). The law on the PO FBiH, in its Article 21, Paragraph 1, stipulates: “The Chief Federation Prosecutor may issue general or individual mandatory instructions to cantonal POs and carry out criminal investigation and prosecution in cantonal Courts and Municipal Courts, whenever the Chief Federation Prosecutor has reason to believe that the cantonal POs have failed to implement the criminal law of the Federation or that the prosecution of criminal acts cannot be carried out efficiently under the jurisdiction of a cantonal PO.” However, this authority is used only retroactively. In cases when issues in a PO escalate to become unmanageable in a PO, the FBiH PO acts, using the authority provided in Article 9 of the Law on the PO FBiH, which reads: “The Chief Federation Prosecutor shall supervise the performance of the cantonal Prosecutors’ Offices in order to guarantee the legality and efficiency of proceedings.” Currently, this authority vested in the FBiH PO is not used proactively, and the only way of communication between frontline prosecutors and representatives of higher level instances is informal conversation that occurs at the margins of conferences/seminars. Finding 16: When it comes to JACA’s assistance with preventing corrupt behavior in the judiciary, the HJPC is committed to implementing the integrity agenda under Component 2. All evaluation participants found the continuation of work on the integrity and ethics agenda to be important and indispensable. Whether direct beneficiaries or not, they all stated that they wished and hoped for JACA’s continued support. Specific topics for this continued support included strengthening the capacity of the HJPC to manage integrity issues of the judiciary, raising awareness of the issues related to integrity and ethics for judicial and non-judicial staff, proactively disseminating disciplinary case law, achieving higher standards in implementation of integrity plans in judicial institutions, supporting the HJPC in introduction of new institutes and practice (confidential counseling, background checks in appointment process, etc.), and ad-hoc support that the HJPC will need in new developments. The evaluation participants noted an increased commitment to integrity of the HJPC in the past two years, demonstrated by establishment of the Integrity Department within the HJPC’s Secretariat, the adoption of a set of tools that assisted in raising awareness of USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 24 integrity and ethics in the judiciary, and more willingness of the new HJPC members to work on this agenda. One stakeholder noted poor communication between the HJPC and its constituents (judges and prosecutors) and questioned the integrity of the HJPC’s members. EQ 3 CONCLUSIONS Key informants offered neither sufficient agreement nor arguments for inclusion of any new institutions into the JACA partner program or exclusion of any current JACA partner institutions. All current JACA’s partner institutions can remain JACA partners and continue to receive general support of JACA and to build their capacity as immediate operational resources available to process HCOC cases if contextual factors are mitigated through JACA’s context-centered interventions. In addition, in the future, individual prosecutors involved in JACA’s activities may be promoted to higher or managerial positions and take the benefits of the knowledge acquired within JACA interventions to the next level, which is a desirable outcome. Abandoning these prosecutors while they are still in the capacity-development stage would not be beneficial in the long run. While maintaining the general course with the current partner institutions, there is a need to narrow down the focus of JACA’s support to further emphasize assistance to individual frontline prosecutors who show commitment to prosecuting HCOC cases and achieving tangible results. As explained above, the local judiciary found JACA’s interventions that dealt with integrity issues useful and wish such TA to continue. In terms of JACA’s work with its partner institutions on integrity issues, the top-down approach should be maintained, considering that everything that is achieved with JACA’s assistance (e.g., the Code of Ethics and list of major integrity risks) can be immediately replicated in all other judicial institutions in the country. Although the HJPC still lacks a clear priority agenda and its institutional memory is poor, there is no alternative to JACA’s continued work with the HJPC on TA covering integrity issues. EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? EQ 4.1: What lecturer/trainer selection model does JACA apply? EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? EQ 4 FINDINGS To identify findings related to the EQ 4, the ET first examined two sub-questions to establish facts about the identification of JACA’s current training model and the beneficiaries’ assessment of JACA’s training programs. Finding 17: JACA selects trainers on the basis of their expertise; they are mostly senior local judges and prosecutors, or equivalent regional experts, while beneficiaries perceive only a limited role for international experts. JACA training participants are prosecutors and judges from partner POs and courts who have expressed interest in HCOC specialization. JACA’s beneficiaries are very satisfied with the quality of lecturers and perceive that their local expertise and experience are what make JACA’s training events so useful in practice. JACA’s lecturers are typically high-level local senior judges, prosecutors, or practitioners who bring practical, applicable experience to their teaching. For Networking with other colleagues is an additional benefit of JACA trainings. Being a part of JACA’s network of participants is an indication that you are reliable, that you can be trusted! – JACA training participant 25 specific topics, JACA brings in regional experts and specialists from outside the judicial field (e.g., a presentation on the use of open-source information was delivered by a journalist from Serbia). Since the start of the ITP and SPJ, JACA also promoted some of its participants into trainers, as their ability to share valuable knowledge with others was recognized. Some trainings were also delivered by international experts, but the beneficiaries found that solutions offered by international experts were not sufficiently customized and applicable to the BiH context. Still, JACA training participants recognized that international input would be useful for topics beyond local and regional expertise, such as accessing information from different encrypted messaging platforms (Sky, WhatsApp, Viber, etc.), seizure of cryptocurrencies, or similar issues emerging from use of high-tech in committing HCOC crimes. JACA’s training program has been building a cadre of well-trained legal professionals (41 prosecutors and 25 judges from partner POs and PCs) who can deliver justice for society by acting with the required professional knowledge and skills to interpret and apply law, as well as judicial practice, to the facts. The ITP and SPJ training programs are based on voluntary participation; participants individually choose to take part in JACA’s training activities and actively participate in identification and selection of topics. Audiences at the training sessions might comprise only prosecutors, only judges, or mixed groups of judges and prosecutors. Both training programs directly contributed to the measure from the HJPC’s Reform Program for 2021–2023, “implementation of specialist training for holders of judicial functions.”38 Each ITP participant had his or her own individual training plan, against which JACA measured and monitored individual results. JACA is in regular contact with participants and is able to respond to the needs of individual prosecutors. ITP participants include Chief Prosecutors, Deputy Chief Prosecutors, senior- and midlevel prosecutors, and prosecutorial associates. The ITP aligns with the recommendations on training provided by CCEP Opinion No. 14,39 on the role of prosecutors in fighting corruption, stating that training should be “as tailor-made as possible, responding to their specific needs” and “should cover legislative and case law developments, both the domestic developments, and the jurisprudence of international courts.” Both the ITP and SPJ were initially created with the assistance of senior national and international experts and were tailored to address the weakest areas in prosecutorial and judicial capacities. Prior to launching the SPJ, a training needs analysis of HCOC judges was conducted and major obstacles to understanding crimes of corruption and the efficient processing of HCOC cases were identified. The SPJ is divided into two major modules that directly respond to the needs of judges. Participants in JACA’s trainings value the opportunity to network with like-minded colleagues. Both groups periodically identify topics that interest them, which JACA aggregates annually and incorporates into the ITP and SPJ curricula. Finding 18: Training programs delivered by JACA are valued by beneficiaries for the high quality of preparation and organization, continuous consultations about beneficiaries’ needs and opinions, quality of trainers, availability of training materials on Digital Knowledge Database (DKB), and professionalism of JACA’s team. All JACA’s beneficiaries40 found that JACA trainings were high quality, comprehensive, relevant, and current. They appreciated the chance to choose the topics for their own training programs, including some they did not expect to be offered, e.g., public relations or rhetorical skills. Beneficiaries 38 HJPC Reform Program 2021–2023, p. 8. 39 The role of prosecutors in fighting corruption and related economic and financial crime, Opinion No. 14 (2019), CCEP, pp. 53–57. 40 In the first two years of JACA implementation 703 judicial personnel were trained, and 260 government officials received anti￾corruption training. I find it to be excellent! JACA’s lecturers are excellent practitioners, which is crucial! – JACA training participant USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 26 particularly praised joint sessions of prosecutors and judges. Participants also found practical exercise segments in JACA trainings useful, engaging, and applicable in real-world cases. Relying on their experience, lecturers discussed concrete problems and base learning on the search for practicable solutions. While participants could not attend all training sessions organized by JACA within ITP and SPJ, they valued the access to training materials on the DKB. They also commented that organization of trainings was impeccable with pre- and post-follow-up. Trainers are asked not only to deliver the training but to prepare user-friendly materials and articles that would then be uploaded to the DKB. Some participants wished that certain trainings had been better attended because they were so beneficial and timely. Finding 19: JPTCs lack the know-how and/or funding to prepare and deliver training to JACA’s standards, but to promote sustainability, beneficiaries would welcome a transfer of methodological and technical expertise for ITP and SPJ training from JACA to JPTCs. The JPTCs have faced challenges in trying to address numerous needs and target groups to support the improvement of the overall functioning of the judicial system. Addressing gaps for judges and prosecutors handling complex cases, including HCOC cases, is especially important. Beneficiaries believed that transferring JACA’s organizational and training methodologies to JPTCs, including the FBiH JPTC itself, would be conducive to ensuring sustainability of training programs developed by JACA while increasing the capacity of the centers for providing other trainings that JPTCs organized regularly within their current mandate. In the absence of a supreme judicial body guaranteeing a unified approach and consistency in the decision-making process of judges and courts, dissemination and sharing of case-law, as well as training, are the key prerequisites to the successful harmonization of court practice and equality before the law. Transferring JACA’s trainings and the DKB will serve to harmonize the practical implementation and interpretation of relevant HCOC laws and to improve quality of judicial acts. Evaluation participants found that JACA’s trainings and other tools should be transferred to JPTCs so that they would continue to be available to prosecutors and judges beyond the duration of JACA. Continuous professional judicial and prosecutorial training activities of JPTCs are funded from local budgets and by international donors. In 2017, the local budget covered 63 percent of planned training activities of the FBiH JPTC and 20 percent of training activities in the Republika Srpska JPTC.41 Stakeholders also recognized that JPTCs operate under greater capacity and budget constraints and suggested the need for a period in which JACA would gradually take a mentoring role, while continuing to perform the tasks that are beyond the current capacity of JPTCs. EQ 4 CONCLUSIONS JACA’s two specially designed multiyear training programs for prosecutors (ITP) and judges (SPJ), endorsed by the HJPC and included in JPTCs’ training programs, are very highly rated by the Activity’s beneficiaries. Most trainings focus on the application of substantive and procedural criminal laws of relevance for processing and adjudicating HCOC cases. The current model of trainer selection, which primarily relies on local senior experts but is complemented by regional and international experts for specific topics, is most appropriate for participants’ needs and the BiH context. 41 Study on the Existing Systems of Judicial Training in the Western Balkans, Regional Cooperation Center, December 2017, p. 24. 27 Most local experts who participate in delivering JACA’s trainings are JPTC certified trainers who already provide trainings within the JPTCs’ curricula. Gradually transferring JACA’s organizational and training methodologies to the JPTCs is the best way to ensure sustainability of JACA training programs. The transfer process will also ensure the increase of the capacity of the centers to deliver all their trainings to a higher standard. Transferring the ITP and SPJ programs and other JACA trainings to the JPTCs should be conditional upon the retention of JACA’s branding and involvement of JACA in a mentoring and supervisory role to ensure quality. However, the JPTCs face restrictions in hiring regional and international lecturers, so this is an additional aspect in which JACA will need to remain involved. EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? EQ 5 FINDINGS Finding 20: Cooperation between JACA and other USAID-funded activities (ACFC, IJP) involved in anti-corruption efforts occurs sporadically, and interventions of these Activities are not fully coordinated and synchronized. Information provided by civil society or investigative journalism is often not suitable for prosecution purposes. Selected CSOs and investigative journalists would benefit from improved communication with POs, and targeted capacity-strengthening to enhance their contribution to the fight against corruption is desirable. The current cooperation mainly takes the form of inviting the ACFC and IJP to JACA events but without a wider agenda for joint actions. JACA and the ACFC organized a joint conference, “We Monitor Tenders,” which was attended by the HCOC Chief and line prosecutors from PPOs. No follow-up meetings were organized. On the other hand, the ACFC has invited representatives of the HJPC and some judges and prosecutors to some of the events it has organized, e.g., the “Path of Justice” Presentation in Banja Luka (November 2021), but JACA’s representatives were not invited. Similarly, JACA is not familiar with any interaction the ACFC and IJP have with courts/judges or POs/prosecutors emerging from court cases or investigations based on ACFC/IJP-initiated activities (i.e., the ACFC supported CSOs/groups filing criminal reports, libel cases, and complaints against CSOs for compensation of damage, issues in enforcement cases in which the ACFC-supported CSOs/groups won cases, assistance to courts in implementation of whistleblower legislation). JACA only finds out about ACFC-supported CSO activities from the media. Two PPOs confirmed that they recently received by email materials from pratimotendere.ba (ACFC), but these initiatives were not coordinated with JACA. The ET learned through interviews that some prosecutors were not satisfied with the quality of information provided by CSOs and that they would seek to follow up with pratimotendere.ba, while others stated that they based some of their work on the information provided by pratimotendere.ba. Beneficiaries stated that the quality of reports received by CSOs was mainly inadequate for POs’ processing, while most investigative journalism articles offered little information that could be useful to prosecution. However, individuals admitted that both these sources had occasionally been useful. For some, information supplied by CSOs was rarely sufficient and adequate to lead to an indictment, while five interviewees found that even most investigative journalism was more interested in populism and less willing to conduct thorough research and verification of what they published. Still, JACA’s partner prosecutors would want to see improved communication among POs, CSOs, and investigative journalists; these prosecutors suggested various forms of targeted capacity-building that would improve the ability of selected CSOs and selected investigative journalists (CIN, Balkan USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 28 Investigative Reporting Network) to contribute to the fight against corruption. A specific suggestion was to adapt JACA’s quality control checklist, developed for LEAs, to CSO needs for streamlining preparation of effective criminal reports. Finding 21: While JACA is already in communication with the INL, there is room for more cooperation, notably when it comes to supporting new anti-corruption offices. The INL Senior Anti-corruption Advisor Program is working with cantonal anti-corruption offices in BiH, some of which have already been established, some in process of being formed, and others expected to emerge in the near future. JACA could work with the INL to pave the way for future cooperation of anti-corruption offices and POs. A possible first step would be to consider whether the quality control checklist that JACA developed for use by LEAs, to facilitate streamlined production of more robust HCOC criminal reports, could be adapted to the needs of anti￾corruption offices. Also, the INL is close to completing the INL Forensic Accounting Program, which will be continued by Center for Forensic Accounting in Sarajevo. The continuation of this program by a local institution presents an opportunity for further improvements in JACA’s training programs, possibly in the form of including some of the INL Forensic Accounting Program lecturers/participants among JACA trainers. Finding 22: Both JACA and OPDAT deliver trainings on similar topics for the same audience, but there is no unified list of trainings offered and the same participants often participate in trainings by both JACA and OPDAT. Representatives of JACA and OPDAT meet periodically to coordinate their training programs, mainly to avoid overlap. As the target audiences for both programs are very similar, both the beneficiaries and the programs would benefit from creation of a single, unified list of participants; a schedule of trainings; and frequent exchange of updated beneficiaries’ presence at training. Also, as not all participants attend all trainings, a unified knowledge management platform for training organized by both programs is currently needed. While this EQ explored cooperation among USAID-activities, the good cooperation between JACA and the Organization for Security and Co-operation in Europe was acknowledged because it helped involve the LEAs in implementation of the quality control checklist developed by JACA. This cooperation complemented the work JACA does with the prosecutors and is a good example of cooperation between different projects operating in the same field. EQ 5 CONCLUSIONS There is room for further improvement in both regular exchange of information and coordination of interventions among anti-corruption USAID Activities and USG agencies. Because the support of LEAs to the work of HCOC prosecutors is viewed as inadequate, all additional options to assist frontline HCOC prosecutors need to be considered and implemented when viable. On the other hand, the quality of information currently gathered and provided by citizens and CSOs needs to be improved. A better approach would include coordination among the ACFC, IJP, and JACA, which could take the form of guidance from prosecutors about the type and format of information to be provided by CSOs. JACA’s contacts in the judiciary and the tools JACA developed could be used to fine-tune and adapt ACFC and IJP interventions so that they are more relevant for prosecutors processing HCOC cases. There is also a need to educate CSOs to improve the quality of criminal reports that they file. Adapting the quality control checklist for use by CSOs could be an entry point for strengthening cooperation between selected CSOs and JACA-assisted prosecutors in the future. At the same time, there is a clear need to improve communication between IJP-assisted investigative journalists and JACA’s beneficiaries, which could potentially mitigate the existing animosity between journalists and prosecutors. 29 Participation in trainings organized for CSOs could provide investigative journalists with a solid understanding of the aspects of information essential for viable prosecution. The INL Senior Anti-corruption Advisor Program and JACA have cooperated in the past, but the chance to further adapt the quality control checklist for use by anti-corruption offices creates a window of opportunity for greater cooperation. JACA should communicate with the INL’s Forensic Accounting Program and seek opportunities for cooperation between lecturers/participants in this program and frontline HCOC prosecutors. While JACA and OPDAT already coordinate their trainings on a basic level, there is room for improvement and further synchronization and creation of joint activities, such as the compilation of a joint list of training events to be offered by JACA and OPDAT in a calendar/fiscal year with regular quarterly updates, exchange of lists of participants in those trainings, and joint consultation on study visits and participants. This increased cooperation effort should help mitigate some of the contextual factors that currently impede greater results in processing and adjudication of HCOC cases. RECOMMENDATIONS The recommendations here are presented in two groups: the overall recommendation for JACA’s further work, and a set of specific recommendations. OVERALL RECOMMENDATION: To ensure continuity, further reinforce processing and adjudicating HCOC cases, and help prevent corrupt behavior in the BiH justice sector, JACA should continue to support frontline judges and prosecutors from its current partner institutions by implementing all its planned interventions, taking into account adjustments proposed in the specific recommendations. After the draft JACA midterm evaluation report was delivered to the Mission for review, a post-evaluation workshop was organized, and findings, conclusions, and recommendations were presented to representatives of the Mission and JACA. After the presentation, three specific recommendations were found viable for implementation in the remaining part of JACA under its current contract. SPECIFIC RECOMMENDATIONS: R 1. Continue with the ITP, as planned, extend the SPJ until the end of JACA, and organize more joint training sessions with prosecutors and judges. R 2. Retain the current lecturer selection model within the ITP and SPJ and involve regional and international trainers to cover complex and emerging legal and high-tech topics of relevance for processing HCOC cases. R 3. Increase practical exercises and hands-on training in JACA’s training sessions. Recommendations that the Mission might consider in its future decision-making processes will be submitted to the Mission in a separate document. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 30 ANNEX I: STATEMENT OF WORK PURPOSE OF THE ASSIGNMENT The United States Agency for International Development Mission (USAID) in Bosnia and Herzegovina (BiH) has requested its Monitoring and Evaluation Support Activity (MEASURE II) to conduct a midterm performance evaluation of the Judiciary Against Corruption (JACA) Activity in BiH. The main purpose of a performance evaluation under this SoW is to provide USAID/BiH with evidence-based and independent review of USAID/BiH’s JACA since its start of implementation until the initiation of this evaluation. This performance evaluation will provide the Mission with credible and useful insights to make informed programmatic decisions and potential adaptations for the remainder of the Activity, maximizing the likelihood of achieving the desired results. The primary audience for this evaluation is the USAID/BiH. The Mission and the implementing partner (IP) will use the evaluation results to take midterm corrective actions (if needed) in the Activity design and/or implementation. SUMMARY INFORMATION JACA is a $7.9 million USAID/BiH-funded Activity implemented by Development Professionals Inc., (DPI). This Activity contributes to Development Objective 1: “Accountability of Governance to Citizens Strengthened.” The performance evaluation will focus on analyzing the Activity’s design and progress toward expected results. The evaluation intends to utilize rigorous methods and design to obtain high-quality data and produce credible findings, conclusions, and recommendations. Activity details are presented in Exhibit I. Exhibit 1. Basic Information about the JACA Activity Activity Name Judiciary Against Corruption Activity (JACA) USAID Office USAID/BiH Democracy Office Implementer Development Professionals Inc., (DPI) Contract Number 72016819C00001 Total Estimated Cost $7,996,582 (exclusive of fixed fee: $7,543,945; fixed fee: $452,637) Life of Activity September 19, 2019, to September 18, 2024 (5 years) Active Geographic Region Across Bosnia and Herzegovina Target Groups Prosecutors, judges and other relevant from the partner courts and partner prosecutor’s offices. CDCS Intermediate Result DO1: Accountability of Governance to Citizens Strengthened IR 1.2: Government effectiveness in targeted areas strengthened Sub-IR 1.2.1: Corruption in targeted areas reduced Required evaluation Yes External or internal evaluation External 31 BACKGROUND COUNTRY CONTEXT BiH has a complicated government structure that was created through the Dayton Peace Agreement that ended the war in 1995. The country consists of two entities, the Federation of BiH (FBiH) and Republika Srpska, and the self-governing Brčko District. Each entity has its own constitution, president, government, and parliament. The FBiH is further divided into 10 cantons, each having a significant level of autonomy. The judicial system mirrors the country’s complex and fragmented internal structure. There are four separate judicial systems in the structure of the State, resulting in differences in substantive and procedural law. This results in a significant lack of harmonization of legislation and judicial decision making which results in a lack of legal certainty for citizens. There are 15 laws governing the work of 20 prosecutorial units (cantonal, district, entity, BD and BiH POs). They operate separately with no firm functional links between them. Despite some improvements in the justice sector over the past 20 years including the establishment of state-level judicial institutions, specifically the Court of BiH and the Prosecutor’s Office (PO) of BiH; the establishment of the High Judicial and Prosecutorial Council (HJPC); restructuring of courts and prosecutorial services; re-appointment of judges and prosecutors, a number of major challenges still remain. Corruption continues to impede social and economic development, as well as accession by BiH to the EU. The criminal justice system in BiH fails to combat serious crime and corruption. None of the four existing criminal justice jurisdictions is adequately functioning.42 Transparency International’s 2021 Corruption Perception Index ranks BiH 110 of 180 countries. In the Balkan Barometer 2021, an annual public and business opinion survey by the Regional Cooperation Council, 82.7 percent of participants in BiH find their judiciary to be the most corrupt sector.43 Moreover, in the 2021 National Survey of Citizens’ Perceptions44 (NSCP-BiH), more than a third of respondents (38 percent) viewed the court system as extremely affected by corruption, a substantial increase compared to previous years and an all-time record high since the inception of this survey in 2015. On the positive side, the preliminary results of the 2021 Judicial Effectiveness Index of Bosnia and Herzegovina45 (JEI-BiH), a measure of the BiH’ judiciary effectiveness (analyzed through the HJPC administrative data on the processing of cases in BiH courts and POs, and data on perceptions of public, and judges and prosecutors of efficiency, quality, transparency, accountability, capacity, resources, independence and impartiality of the BiH judiciary), show some improvements in processing cases in the BiH judiciary including better processing corruption cases. Nevertheless, sustained and significant efforts to support the fight against corruption need to continue to further strengthen the effectiveness, independence, and professionalism of the justice sector, thus bringing BiH closer to meeting the EU accession criteria. JACA DESCRIPTION AND THEORY OF CHANGE JACA aims to strengthen selected justice sector institutions to combat corruption and economic and organized crime. JACA envisages that if prosecutors and judges are appropriately trained and equipped to investigate and adjudicate corruption cases and if preconditions are created by the HJPC to allow for efficient work of justice sector to combat corruption and its integrity enhanced then we 42 Expert Report on Rule of Law issues in Bosnia and Herzegovina, Brussels, 5 December 2019. Available at: https://www.transparency.org/en/cpi/2021/index/bih 43 Balkan Barometer 2021—Public Opinion, Regional Cooperation Council, 24 June 202. Available at: https://www.rcc.int/pubs/122/balkan￾barometer-2021--public-opinion 44 Source: MEASURE II. 45 Source: MEASURE II. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 32 can expect justice actors (judges and prosecutors) to deal with high-profile corruption and organized crime cases more effectively and efficiently. JACA aims to achieve the following results: Activity Goal: More effective, independent and accountable justice actors Purpose: Selected justice sector institutions strengthened to combat corruption, economic, and organized crime • Sub-Purpose 1.1: Processing and adjudicating of the most complex high-profile corruption and organized crime cases in selected POs and courts improved o Outcome 1.1: POs and courts have the necessary information and tools to plan, budget, prioritize, manage and monitor resources to support investigation, prosecution and adjudication of HCOC cases o Outcome 1.2: Specialized training, new tools and resources, and technical assistance for prosecutors and judges of PPOs and PCs enhance their overall capacity to investigate, prosecute and adjudicate HCOC cases o Outcome 1.3: Strengthened quality of prosecutors’ and judges’ reasons for decision, indictments, collection and presentation of evidence, and quantification of damages in HCOC cases and thereby reduce errors and improve prosecutor and judge performance o Outcome 1.4: POs and courts uphold public trust and integrity by increasing predictability and equality before the law by unifying institutional standards and processes, and harmonizing interpretation and application of law, procedure and judicial practice in HCOC proceedings, decisions and sentencing o Outcome 1.5: PPOs and PCs establish effective public communication that increases confidence in the judicial system • Sub-Purpose 1.2: Corrupt behavior in the BiH justice sector prevented o Outcome 2.1: Model guidance on ethics and conduct for court and prosecutor’s office developed and adopted o Outcome 2.2: Judges, prosecutors and judicial system personnel share a common understanding of and commitment to HJPC-driven integrity principles, professional ethics, CoI standards, and asset declaration requirements, and they have access to resources and tools to comply with these standards and requirements o Outcome 2.3: The HJPC uses effective tools to monitor compliance with regulations and guidance on judicial ethics and integrity and to deter violations of ethics, CoI and asset declaration requirements o Outcome 2.4: Integrity plans are fully implemented and regularly updated in PCs and POs USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 33 JACA MONITORING, EVALUATION, AND LEARNING PLAN JACA is tracking 12 indicators to measure progress in meeting Annual and Life of Activity targets. In addition, JACA tracks two context indicators (see Exhibit 2). Exhibit 2. Activity indicators, with relevant baseline values, targets, and actuals Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved Activity Goal: More effective, independent and accountable justice actors 1.1-b Judicial Effectiveness Index score Score (Number) 57.3 60 57.4 60 56.5 60.5 61 61.5 61.5 Activity Purpose: Selected justice sector institutions strengthened to combat corruption, economic, and organized crime AP 1 – Number of indictments filed for HCOC cases by PPOs46 Number None 0 10 10 6 5 9 11 13 49 30.6% AP 2 – Result on Score Card for applicability/ adequacy/ timeliness of JACA TA - ITP47 for prosecutors Percentage None 0% \0 0% 65% 80% 70% 75% 80% 80% N/A 46 JACA’s partner POs 47 Individual Training Program USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 34 Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved AP 3 – Result on Score Card for applicability/adequacy /timeliness of JACA TA - specialized training program for judges Percentage None 0% 0% 0% 0% 0% 65% 70% 75% 75% N/A AP 4 - Number of tools developed for better quality of adjudication and HCOC cases management in PCs48 Number None 0 0 0 0 0 1 1 1 3 0 CCI49 1– Composite Context Indicator Result on Score Card for prosecutors/PPOs - for factors beyond control and influence of judicial institutions and JACA Percentage None CCI 1: 44.82% Context indicator – Triggers set in CIRS 48 JACA’s Partner Courts. 49 Composite Context Indicator. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 35 Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved CCI 2 – Composite Context Indicator Result on Score Card for judges/PC - for factors beyond control and influence of judicial institutions and JACA Percentage None CCI 2: 45.63% Context indicator – Triggers set in CIRS Activity Sub-Purpose 1: Processing and adjudicating of the most complex high-profile corruption and organized crime cases in selected POs and courts improved SP 1-1 Percentage of prosecutors from PPOs that demonstrated satisfaction of received individualized training for processing of HCOC cases Percentage None 0% 0% 0% 60% 83% 70% 75% 80% 80% N/A 36 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved SP 1-2 Percentage of judges from PCs that demonstrated satisfaction of received specialized training for processing of HCOC cases Percentage None 0% 0% 0% 0% 0% 60% 70% 75% 75% N/A CBLD-9 Percent of JACA-assisted justice sector institutions with improved performance Percentage PPOs and PCs 0 66.7% 4/6 PPOs 66.7% 4/6 PPOs 66.7% 4/6 PPOs 50% 3/6 PPOs 66.7% 4/6 PPOs 6/10 PCs 66.7% 4/6 PPOs 6/10 PCs 66.7% 4/6 PPOs 6/10 PCs 66.7% 4/6 PPOs 6/10 PCs 35% Activity Outcome/Output: 1.1-1.5 1.1 Number of tools developed for better quality of investigation, prosecution and HCOC cases management in PPOs Number None 0 1 1 1 1 1 1 1 5 40% USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 37 Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved 1.2 (DR 1.3-1) Number of judicial personnel trained with USG assistance Number Gender 0 50 M 25 W 25 204 M 98 W 106 450 M 225 W 225 499 M 256 W 243 500 M 250 W 250 450 M 225 W 225 300 M 150 W 150 1750 M 875 W 875 40.2% Activity Sub-Purpose 2: Corrupt behavior in the BiH justice sector prevented SP 2-1 Percentage of judges, prosecutors and judicial personnel from PCs and PPOs that believe that necessary anticorruption measures have been taken to prevent corrupt behavior in judicial institutions where they work Percentage None 45% 0% 0% 0% 0% 55% 65% 65% N/A USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 38 Indicators Unit of Measure Disaggregation Overall Activity Baseline Target Year 1 Actual Year1 Target Year 2 Actual Year 2 Target Year 3 Target Year 4 Target Year 5 Life of Activity Cumulative Target* % Target Achieved Activity Outcome/Output: 2.1-2.4 2.1 Number of JACA-supported specific measures from IP Risk Management Plans implemented in PCs and PPOs Number None 0 0 0 1 1 2 2 1 6 16.6% 2.2 (DR 2.4-1) Number of government officials receiving USG￾supported anti￾corruption training Number Gender 0 50 M 25 W 25 70 M 30 W 40 200 M 100 W 100 190 M 56 W 134 300 M 150 W 150 370 M 185 W 185 340 M 170 W 170 1260 M 630 W 630 20.6% USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 39 EVALUATION QUESTIONS The ET will assess the Activity’s work to-date along the following evaluation questions (EQs): EQ 1. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE), EQ 2. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behavior in the justice sector (25 percent of JACA LoE)? EQ 3. Is there room for adjustments in providing JACA’s technical assistance to partner institutions, for example, for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? EQ 3.1. To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? EQ 4. What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? EQ 4.1. What lecturer/trainer selection model does JACA apply? EQ 4.2. What is the beneficiaries’ assessment of the model currently in use? EQ 5. In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? EVALUATION DESIGN AND METHODOLOGY The ET will employ a mixed-method data collection approach and triangulate data to assess the efficiency of JACA interventions and activities, utilizing the following data sources: 1. Activity documents (including but not limited to the: Activity Award; Monitoring, Evaluation, and Learning Plan; work plans; annual and quarterly progress reports; Activity reports; documents (strategies, models, guidelines) produced by the Activity, and lists of trainers, experts, and other stakeholders involved in Activity implementation. 2. Secondary documentation relevant to trust and reconciliation topics (i.e., MEASURE-BiH/MEASURE II National Survey of Citizens’ Perceptions (NSCP) and the Judicial Effectiveness Index of Bosnia and Herzegovina (JEI-BiH), the EU relevant reports; the evaluation report of former USAID’s judiciary-related intervention (Justice Activity (JA)) and sector assessments; documents developed by government institutions; international organizations and civil society organizations (CSOs) of relevance for the JACA interventions. 3. Key informant interviews (KIIs) with USAID/BiH and JACA implementing partner (IP) and subcontractors, U.S. Embassy, INL/OPDAT, prosecutors and judges from the partner POs and courts, other JACA beneficiaries and stakeholders (i.e., the High Judicial and Prosecutorial Council (HJPC), HJPC’s Standing Committees, HJPC Secretariat, the Judicial and Prosecutorial Training Centers (JPTCs)), relevant USAID/BiH Activities (i.e., ACFC, IJP), USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 40 international donors and representatives of international organizations relevant for JACA interventions, and independent judiciary or rule-of-law experts. The full list of key informants will be presented in the Evaluation Work Plan and subject to USAID/BiH approval. 4. Focus groups (FGs) with JACA beneficiaries, which may include prosecutors and judges from the partner courts and partner prosecutor offices. Draft FG guide(s) will be presented in the evaluation work plan and subject to USAID/BiH approval. 5. HJPC administrative data and JACA survey data covering progress in processing HCOC cases and perceptions of the HCOC prosecutors related to the evaluation of the JACA TA, contextual factors, and performance in processing the HCOC cases, The HJPC data might include, but is not limited to, the number of filed criminal reports, investigations opened, investigations terminated, indictments filed, and results of the adjudication process in partner courts and POs. 6. Observation of the JACA-organized events will provide the ET with direct insights into participants/beneficiaries’ ability to absorb JACA’s TA, the relevance of events’ topics for participants/beneficiaries, and JACA’s adaptability to meet the participants/beneficiaries’ needs for the TA. An ET member will participate in and observe a couple of JACA-organized events in agreement with JACA. The ET will use a mixed-method triangulation approach in data analysis. Exhibit 3 presents the evaluation matrix outlining the methodology to be employed to address each evaluation question and sub-questions. The ET will collect data by conducting a desk review of the Activity and secondary documents, coding KII, FG, and MEASURE II observation notes and conducting a descriptive analysis of the administrative and survey data. The ET will start the analysis by reviewing secondary data on the BIH judiciary in general. The team will then review Activity documents to learn about the outcomes of different interventions. Subsequently, the team will conduct KIIs and FGDs to further explore topics of relevance. The team will triangulate all findings with available HJPC administrative data and JACA survey data and accompanied by the ET’s notes collected through observation of the JACA-organized events. The team will compare data from all sources and further explore any discrepancies to ensure that the evaluation findings, conclusions, and recommendations are high-quality, valid, credible, and reliable. Exhibit 3. Evaluation matrix EVALUATION QUESTION (EQ) DATA SOURCES/ DATA COLLECTION METHODS EQ 1. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? Activity and secondary documents on the BiH judiciary, KIIs and FGDs with relevant stakeholders, HJPC administrative data, JACA survey data, and notes from observation of JACA-organized events. EQ 2. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behavior in the justice sector (25 percent of JACA LoE)? Activity and secondary documents on the BiH judiciary, KIIs and FGDs with relevant stakeholders, and JACA survey data. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 41 Exhibit 3. Evaluation matrix EQ 3. Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? EQ 3.1. To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? Activity and secondary documents on the BiH judiciary, KIIs and FGDs with relevant stakeholders, and JACA survey data. EQ 4. What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? EQ 4.1. What lecturer/trainer selection model does JACA apply? EQ 4.2. What is the beneficiaries’ assessment of the model currently in use? Activity and secondary documents on the BiH judiciary, KIIs and FGDs with relevant stakeholders, JACA survey data, and notes from observation of JACA-organized events. EQ 5. In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? Activity and secondary documents on the BiH judiciary, KIIs and FGDs with relevant stakeholders, and notes from observation of JACA-organized events. EVALUATION LIMITATIONS • Recall bias. Some beneficiaries who participated in interventions over two and a half years before data collection may have difficulties remembering the interventions. The ET members will review all Activity documents and prepare themselves for the interviews, search for the KIs who continuously participated in the JACA TA delivery, and, when needed, remind the KIs about interventions to help them recall their experiences and impressions. • Response bias. Implementors may overstate the outcomes of JACA’s interventions in which they engage. Whenever possible, the ET will compare the information obtained from implementers with information from beneficiaries and other sources, to verify the credibility of the findings. The ET will make sure that respondents understand that their true opinions are the most appreciated. The ET will also ensure that respondents are aware of confidentiality of any information they provide. DELIVERABLES AND REPORTING REQUIREMENTS All deliverables will be submitted electronically and in English. The deliverables will include: 1. Detailed evaluation work plan and data collection instrument(s) The evaluation work plan will include: (1) a detailed evaluation design matrix (including the key questions, methods, and data sources used to address each question and the data analysis plan for each question); (2) draft data collection instruments (interview guides, and focus group discussion guides) (3) the list of potential interviewees (without personal USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 42 information); (4) known limitations to the evaluation design; (5) the anticipated schedule and logistical arrangements; and (6) a list of the members of the evaluation/ team, delineated by roles and responsibilities. 2. Presentation of preliminary findings/Briefing for the Mission A presentation of preliminary findings to USAID/BiH will include a summary of preliminary findings and recommendations to USAID/BiH. 3. Draft evaluation report The draft evaluation report will be consistent with the USAID Evaluation Report Requirements (ADS REFERENCE 201MAH) and USAID Evaluation Policy (October 2020) and take into account the Criteria to Ensure the Quality of the Evaluation Report (ADS REFERENCE 201MAA), a mandatory reference for ADS Chapter 201 – Program Cycle Operational Policy (ADS Chapter 201). 4. Final evaluation report Once USAID’s comments on the initial draft are provided to the ET, the team will address comments and submit a revised final report within ten calendar days. The final report will be up to 30 pages long, excluding any annexes. 5. Evaluation follow-up workshop Upon the Mission’s approval of the final report, MEASURE II will organize a follow-up workshop to discuss the utilization of evaluation findings and conclusions, as well as the application of recommendations to ongoing and/or future USAID/BiH development programming. The workshop will strengthen the use of evidence and facilitate improved collaborating, learning, and adapting (CLA) practices for USAID/BiH. TEAM COMPOSITION The ET is expected to include five members. All tasks will be coordinated by the Team co-leads and team members. The tentative key staff and their qualifications are shown in Exhibit 4. Additional MEASURE II staff research analysts will also support this evaluation as team members. Exhibit 4. Key team members and their qualifications POSITION KEY QUALIFICATIONS Team co-lead (MEASURE II staff member) Project management skills; expertise in evaluation methodologies and USAID’s evaluation requirements; familiarity with the JACA Activity and ongoing reforms in the BiH justice sector. Team co-lead (Local Consultant, Subject-matter Expert) Subject matter expertise in RoL and acquaintance with ongoing activities and reforms in the BiH justice sector Team members (MEASURE II Research Analysts) High-level of technical expertise in research and ability to adapt rigorous methodologies to evaluation research. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 43 Support for the ET will include: • Home Office (HO) and Field Office support (FO) in reviewing the evaluation deliverables and conducting general oversight of the evaluation process • HO CLA experts who will contribute to the application of CLA principles throughout the evaluation process • Transcribers experienced in transcribing audio recordings from KIIs and FGDs • An Office Manager who will provide logistical support to contracting, payments, and field work. The team composition and level of effort will be finalized in the Evaluation Work Plan. SCHEDULE The overview of the tentative evaluation timeline is provided in Exhibit 5. Exhibit 5. Tentative evaluation timeline TENTATIVE DATES TASKS AND DELIVERABLES June 13, 2022 Finalize the Statement of Work June 20, 2022 Finalize the Evaluation Workplan June 22 – July15, 2022 Data collection July 11-22, 2022 Data analysis Week of July 25, 2022 Briefing for the Mission August 15, 2022 Submission of the draft evaluation report TBD Submission of the final evaluation report TBD Evaluation follow-up workshop USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 44 ANNEX II: DETAILED METHODOLOGY DATA COLLECTION APPROACH, METHODS AND EVALUATION MATRIX Five key methods were used to collect data: document review, key informant interviews (KIIs), focus groups (FGs), analysis of quantitative data (HJPC administrative data and JACA’s beneficiary survey data), and observation of JACA-organized events. The evaluation team (ET) employed a mixed-method data collection approach and triangulated data to assess the efficiency and effectiveness of JACA interventions. From the launch of the evaluation and throughout data collection and drafting, the ET conducted document review. Quantitative data (HJPC administrative data and JACA survey data) were obtained early in the course of the evaluation, while KIIs and FGs were organized midway through the evaluation. JACA events were observed as opportunities arose. 1. Documentation review encompassed Activity documents (including but not limited to: the Activity Award; Monitoring, Evaluation, and Learning Plan; work plans; annual and quarterly progress reports; Activity reports; documents (strategies, models, guidelines) produced by the Activity, and lists of trainers, experts, and other stakeholders involved in Activity implementation (see Annex III). 2. Secondary documentation relevant to the anti-corruption topic included relevant local laws and regulation, MEASURE-BiH/MEASURE II National Survey of Citizens’ Perceptions (NSCP) and the Judicial Effectiveness Index of Bosnia and Herzegovina (JEI-BiH), relevant EU reports; the evaluation report of USAID’s earlier judiciary-related intervention (Justice Activity - JA) and sector assessments, documents developed by government institutions, international organizations, and civil society organizations (CSOs) of relevance for the implementation of JACA (see Annex III). 3. Key informant interviews (KIIs) were conducted with USAID/BiH, and JACA implementing partner (IP) and its subcontractors, U.S. Embassy, INL/OPDAT, prosecutors and judges from the partner POs and courts, other JACA beneficiaries and stakeholders (i.e., the High Judicial and Prosecutorial Council (HJPC), its Standing Committees, and the HJPC Secretariat), international donors and representatives of international organizations (i.e., the Office of the High Representative, Organization for Security and Cooperation in Europe and the European Union Special Representative’s Office) relevant for JACA interventions. The full list of key informants is presented in Annex III. All KIs were offered an interview in person or online, on a suitable Internet communication platform, in accordance with their preferences. Several interviews were organized in person, while MEASURE II provided all technical support (including purchasing and setting up the Webex online platform, the only platform that the BiH judiciary is officially permitted to use) when KIIs were organized online. 4. Focus groups (FGs) were conducted with JACA beneficiaries, including prosecutors and judges from the partner courts and partner prosecutor offices involved in JACA’s training programs as trainees or trainers, and representatives of the Judicial and Prosecutorial Training Centers (JPTCs). FG guides were included in the evaluation work plan and approved by USAID/BiH and attached to this Report in Annex VI. All focus groups were held online via the Webex Internet platform. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 45 5. HJPC administrative data and JACA survey data covering progress in processing HCOC cases and perceptions of JACA’s beneficiaries about JACA’s TA, contextual factors, and performance in processing HCOC cases. The HJPC data included the number of criminal reports filed, investigations opened, investigations terminated, indictments filed, and results of the adjudication process in partner courts and POs. 6. Observation of JACA-organized events provided the ET with direct insights into participants/beneficiaries’ level of interest and their ability to absorb JACA’s TA, the relevance of training topics, and JACA’s adaptability to the participants/beneficiaries’ needs. An ET member participated in and observed four JACA-organized events at the invitation of the JACA team: 1. Training “financial investigations and special characteristics of money laundry crimes”, Neum, 12-13 May 2022 2. “Thematic meeting of the extended collegium of the Chief Prosecutors in FBiH”, Igman, 26 April 2022 3. Workshop “Status conference”, Bihac, 27 May 2022 4. Training “Proactive investigations”, Sarajevo, 24 May 2022 The ET used the mixed-method triangulation approach in data analysis. The evaluation matrix links evaluation questions with data sources, data collection and analysis methods used to collect evidence used to address the EQs. Exhibit 2 below presents the evaluation matrix for the JACA evaluation, including a detailed description of data sources, data collection methods, and analytical approaches for each evaluation question. The ET started the analysis by reviewing secondary documentation and data on the BiH judiciary in general. The team next reviewed Activity documents to find out about the outcomes of its various interventions. The next stage involved conducting the KIIs and FGDs which made it possible to delve deeper into identified topics of relevance to the EQs. The ET then compared all findings generated up to this point with available HJPC administrative data, JACA survey data, and the notes collected during observation of events organized by JACA. The ET triangulated data from all sources and clarified all perceived discrepancies which ensured the highest possible quality, validity, credibility, and reliability of the evaluation findings, conclusions, and recommendations. Exhibit 2: Evaluation Matrix EVALUATION QUESTION (EQ) DATA SOURCES/ DATA COLLECTION METHODS DATA ANALYSIS APPROACH EQ 1. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 1. Activity and secondary documents on the BiH judiciary, 2. KIIs with relevant stakeholders, 3. FGDs with relevant beneficiaries, 4. HJPC admin data, 5. JACA survey data, and 6. Notes from JACA-organized events’ notes. 1 – Desk review 2, 3, 6 – Notes coding 4, 5 – Quantitative analysis EQ 2. What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behavior in the justice sector (25 percent of JACA LoE)? 1. Activity and secondary documents on the BiH judiciary, 2. KIIs with relevant stakeholders, 3. FGDs with relevant beneficiaries, 4. JACA survey data, and 1 – Desk review 2, 3 – Notes coding 4 – Quantitative analysis USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 46 EVALUATION QUESTION (EQ) DATA SOURCES/ DATA COLLECTION METHODS DATA ANALYSIS APPROACH EQ 3. Is there room for adjustments in providing JACA’s TA to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? EQ 3.1. To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/ judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 1. Activity and secondary documents on the BiH judiciary, 2. KIIs with relevant stakeholders, 3. FGDs with relevant beneficiaries, 4. JACA survey data, and 1 – Desk review 2, 3 – Notes coding 4 – Quantitative analysis EQ 4. What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? EQ 4.1. What lecturer/trainer selection model does JACA apply? EQ 4.2. What is the beneficiaries’ assessment of the model currently in use? 1. Activity and secondary documents on the BiH judiciary, 2. KIIs with relevant stakeholders, 3. FGDs with relevant beneficiaries, 4. JACA survey data, and 5. Notes from JACA-organized events’ notes. 1 – Desk review 2, 3, 5 – Notes coding 4 – Quantitative analysis EQ 5. In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 1. Activity and secondary documents on the BiH judiciary, 2. KIIs with relevant stakeholders, 3. Notes from JACA-organized events’ notes. 1 – Desk review 2, 3 – Notes coding DATA ANALYSIS AND UTILIZATION ET co-leads guided and managed the systematic analysis of qualitative and quantitative data which applied rigorous methods, including data triangulation, to cross-check results and ensure the soundness and quality of evidence underpinning the evaluation’s findings and conclusions. The ET members took detailed notes of KIIs and FGs and took particularly great care, as it was decided not to record any interviews or FGs to encourage the greatest possible candor on interlocutors’ part. The notes were added to a single online document which was accessible to all team members. The interviews were planned and scheduled to ensure that, in addition to the lead interviewer, each interview is attended by two note-takers. Comparison and cross-referencing of these two sets of notes considerably strengthened the accuracy and quality of collected data. ET held regular internal debriefings several times a week during fieldwork to discuss progress, evidence collected, and to detect any possible need for adjustments to the evaluation schedule, but only minor scheduling changes were necessary. The themes and patterns that emerged were incorporated into the coding matrix. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 47 ET developed data collection protocols from the single question bank, which permitted selection of most appropriate questions for any given KII stakeholder group while facilitating comparability between different groups. This approach improved the effectiveness of data triangulation as each method/stakeholder group dealt with a subset of the same EQs and these respective inputs were validated or refuted by insights generated by other methods or from other stakeholders. The ET found it easier to verify accuracy of data collected and to detect and prove/disprove either expected or unexpected findings even when one method or one stakeholder group yielded less conclusive results than obtained through another method or stakeholder group. The quantitative analysis of the HJPC administrative data used descriptive statistics to generate aggregate and average values related to processing HCOC cases, including specifically the number of criminal reports filed, investigations opened, investigations terminated, indictments filed, and results of the adjudication process in partner courts and POs. The same approach was used with JACA’s survey data to show the distributions in responses provided by judges and prosecutors regarding timeliness/adequacy/quality of JACA’s TA, contextual factors, and performance in processing the HCOC cases. ET employed analytical triangulation in generating its findings and conclusions. Triangulation was also used for cross-checking the findings to validate responses and identify linkages between different findings. The ET compiled the findings, conclusions, and recommendations (FCR) in an Excel-based matrix structured around EQs. The matrix was useful in identifying and capturing topics that emerged from KIIs and included metadata, such as respondent type or interview type (KII or FGD). The matrix helped the ET prepare a systematic and comprehensive response to each EQ, identify any gaps that required additional clarification or analysis , and constituted the basis for developing the evaluation report. EVALUATION LIMITATIONS • Recall bias. As it was expected that some beneficiaries who participated in interventions over two and a half years before the time of this evaluation could have difficulties recalling specific details, the ET members prepared themselves by reviewing all Activity documents , but also emphasized identification of the KIs who continuously participated in JACA’s TA delivery, and, when it was needed, reminded the KIs about interventions to help them recall their experiences and impressions. • Response bias. To compensate for the likelihood that implementors may overemphasize the outcomes of JACA’s interventions which they conducted, the ET made sure to cross-check the information provided by implementors with inputs collected from beneficiaries and other sources. The ET made all respondents understand that their honest opinions were important and valuable and ensured that respondents were informed that any information they provide would be held in confidence. • Interviewer/FG Moderator bias: To minimize the chance that interviewers’ conduct and actions might influence KIs’ or FGD participants’ responses, interviewers and FGD moderators were trained to ask questions in a non-leading way and to refrain from suggestive comments or body language. The interviewers and FGD moderators were also instructed to ask all questions included in the data collection instruments, and to ask questions in the stated order as much as possible. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 48 ANNEX III: LIST OF ACTIVITY AND SECONDARY DOCUMENTATION REVIEWED No. Title 1 2021 Judicial Effectiveness Index of Bosnia and Herzegovina, USAID MEASURE II, July 2022 2 2021 National Survey of Citizens’ Perceptions in Bosnia and Herzegovina, USAID MEASURE II, July 2022 3 A Model of Crisis Communication Strategy for courts and POs and accompanying Guide to Crisis Communications, Yr2 4 Assessing Integrity of Candidates for Judicial Office in Bosnia and Herzegovina, Yr2 5 Balkan Barometer 2021, Regional Cooperation Council, 24 June 2021 6 Bosnia and Herzegovina 2021 Report, European Commission, October 2021 7 Bosnia and Herzegovina Analytical Report 2019, European Commission, Brussels, May 2019 8 Corruption Trial Monitoring Report, Transparency International BiH, October 2021 9 Draft Law on Amendments to the Law on the High Judicial and Prosecutorial Council of Bosnia and Herzegovina 10 EU support for the rule of law in the Western Balkans: despite efforts, fundamental problems persist, European Court of Auditors, January 2022 11 Expert Report on Rule of Law issues in Bosnia and Herzegovina, Brussels, 5 December 2019 12 HJPC’s Annual Reports for 2020, High Judicial and Prosecutorial Council, March 2021 13 HJPC’s Annual Reports for 2021, High Judicial and Prosecutorial Council, April 2022 14 Independence, Accountability and Quality of the Judiciary BiH 2018 -2020, Netherlands Council for the Judiciary and the Norwegian Court Administration 15 Individualized Training Program for Prosecutors, Yr1 and Yr2 16 Judicial and Prosecutorial Training Program for 2022, FBiH Judicial and Prosecutorial Training Center 17 Minutes from relevant HJPC sessions from 2019-2021 18 Model Code of Ethics and Conduct for Court and POs Employees, Yr2 19 Pravilnik o orijentacionim mjerilima za rad tužilaca u BiH, Službeni list BiH, number 6/22, June 2022 20 Pravosuđe u BiH: Stanje i perspektive, Transparency International BiH, 2019 21 Preventing Corruption among Judges, Opinion No. 21 (2018), Consultative Council of European Judges, November 2018 22 Quality Control Checklist, Yr2 23 Rapid Enhanced Weighted Caseload Study, Yr2 24 Rapid HCOC Case Processing Assessment, Yr1 25 Reformski program Visokog sudskog i tužilačkog vijeća Bosne i Hercegovine za period 2021–2023. godine 26 Report on Application of Definitions for Classifying Cases as HCOC in Courts and POs, Yr1 and Yr2 27 Report on the state of the BiH judiciary, Interim Investigative Committee, Parliamentary Assembly of Bosnia and Herzegovina, June 2022 28 Rule of Law Index 2021, World Justice Project, October 2021 29 Second Compliance Report Bosnia and Herzegovina, GRECO, adopted on 25 September 2020 30 Specialized Training Program for Judges, Yr1 and Yr2 31 Srednjoročni plan rada VSTV-a BiH za period 2022-2024, adopted on the HJPC session held on 8 and 9 September 2021 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 49 No. Title 32 Study on the Existing Systems of Judicial Training in the Western Balkans, Regional Cooperation Center, December 2017 33 Survey on Youth Emigration in Bosnia and Herzegovina, UNFPA, August 2021 34 The Blindfolding Justice in Bosnia and Herzegovina: State Capture of Bosnia and Herzegovina Judiciary and Public Prosecution, Open Society Fund, January 2021 35 The role of prosecutors in fighting corruption and related economic and financial crime, Opinion No. 14 (2019), Consultative Council of European Prosecutors, 22 November 2019 36 Third Annual Report on Judicial Response To Corruption: The Impunity Syndrome, OSCE, November 2020 37 USAID JACA 2020 Annual Report 38 USAID JACA 2020 Q1 Quarterly Report 39 USAID JACA 2020 Q2 Quarterly Report 40 USAID JACA 2020 Q3 Quarterly Report 41 USAID JACA 2020 Work Plan 42 USAID JACA 2021 Annual Report 43 USAID JACA 2021 Q1 Quarterly Report 44 USAID JACA 2021 Q2 Quarterly Report 45 USAID JACA 2021 Q3 Quarterly Report 46 USAID JACA 2021 Work Plan 47 USAID JACA 2022 Q1 Quarterly Report 48 USAID JACA 2022 Q2 Quarterly Report 49 USAID JACA 2022 Q3 Quarterly Report 50 USAID JACA 2022 Work Plan 51 USAID JACA Monitoring, Evaluation and Learning Plan 52 USAID JACA Survey/poll of judges participating in JACA specialized training for judges of high-profile corruption and organized crime cases 53 USAID JACA Survey/poll of prosecutors participating in JACA specialized training for prosecuting of high-profile corruption and organized crime cases USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 50 ANNEX IV: LIST OF KEY INFORMANTS AND FOCUS GROUP PARTICIPANTS KII OR FG STAKEHOLDER GROUP NAME OF INSTITUTION/ORGANIZATION NUMBER OF INDIVIDUALS INTERVIEWED KII INTERNATIONAL DONORS USAID 2 KII INTERNATIONAL DONORS US EMBASSY 1 KII INTERNATIONAL DONORS INL 1 KII INTERNATIONAL DONORS OPDAT 1 KII INTERNATIONAL DONORS OHR 1 KII INTERNATIONAL DONORS OSCE 1 KII INTERNATIONAL DONORS EUSR 1 KII JACA IP DPI 5 KII ENTITY POs FBiH PO 1 KII JACA PARTNER POs PO ZENICA-DOBOJ CANTON 1 KII JACA PARTNER POs PO TUZLA CANTON 1 KII JACA PARTNER POs PO RS 1 KII JACA PARTNER POs PO UNA-SANA CANTON 2 KII JACA PARTNER POs PO HERCEGOVINA-NERETVA CANTON 1 KII JACA PARTNER POs PO SARAJEVO CANTON 2 KII JACA PARTNER COURTS CANTONAL COURT SARAJEVO 1 KII JACA PARTNER COURTS MUNICIPAL COURT SARAJEVO 1 KII JUDGES DISTRICT COURT BANJA LUKA 1 KII HJPC C1 HJPC 1 FGD ITP PARTICIPANTS PO RS, PO UNA-SANA CANTON, PO HERCEGOVINA-NERETVA CANTON 3 FGD C1 EXPERTS PO BiH, SUPREME COURT OF FBiH 4 FGD JPTCs JPTC FBiH 3 FGD HJPC C1 HJPC 3 FGD HJPC C2 HJPC 3 FGD JUDGES SUPREME COURT FBiH; CANTONAL COURTS: ZENICA, BIHAC; MUNICIPAL COURT TUZLA 4 TOTAL KIIs: 26 TOTAL FGD PARTICIPANTS: 20 GRAND TOTAL: 46 USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 51 ANNEX V: INTERVIEW GUIDES USAID DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 1. In your opinion, were JACA's activities tailored to needs of beneficiaries and how would you rate JACA's delivered assistance so far? 2. In your opinion, which JACA's interventions were best received by prosecutors/judges/other beneficiaries and why? 3. In your opinion, which JACA's interventions were not so successful or well-received and why? 4. To what extent did JACA’s technical assistance help JACA’s beneficiaries in processing HCOC cases? 5. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 6. In the period October 2020 – October 2021, administrative data show a small number of indictments and investigations for HCOC cases in some partner POs, what are the reasons for that? 7. In your opinion, what were beneficiaries’ biggest challenges in processing HCOC cases? 8. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 9. What is missing or what else do prosecutors/judges need to do to make progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behavior in the justice sector (25 percent of JACA LoE)? 10. In your opinion, what is the level of understanding among judges, prosecutors, and judicial staff about integrity principles, professional ethics, Conflict of Interest, and asset declaration requirements? Are resources and tools on how to comply with these standards adequate? 11. To what extent are judges, prosecutors, and judicial staff committed to complying with these standards? 12. To what extent did JACA’s technical assistance contribute to the prevention of corrupt behavior in the BIH judiciary? 13. What more could be done for the judiciary to embrace the integrity and ethics principles? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 52 14. Did you observe any significant change in partner institution that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 15. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? 16. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 17. In your opinion which other institutions should be included in JACA activities? And why? 18. In your opinion, who is leading efforts in the BiH judiciary in preventing corrupt behavior, including offers related to CoI, asset declaration, implementation of IPs and could you recall and describe some related activities? (PROBE: What about the role of HJPC in these activities, and how would you rate HJPC’s efforts) 19. Do you notice any progress in preventing corrupt behavior, e.g., related to CoI, asset declaration, and implementation of IPs in the BiH judiciary? Please provide illustrative examples if you can recall them? 20. In your opinion, is it worth continuing JACA’s support for these activities? If Yes, what else is needed to limit opportunities for corrupt behavior in the BiH judiciary? Who should lead those efforts? If No, please explain? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 21. To your best knowledge, how are JACA’s trainings designed in terms of selection of topics, participants, and trainers? (PROBE: how are the topics selected, who are the participants and how are they selected, what is the attendance rate of participants at trainings, who are trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 22. How would you rate the quality, adequacy, and timeliness of the training content? 23. How would you rate competence and ability of trainers to transfer their knowledge? What did you like and what did you not like in the trainings you participated in? 24. Have you attended training programs other than JACA that were delivered by international experts? What do you think worked well and what did not in those trainings? 25. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what can local/regional trainers bring that international ones cannot? 26. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 27. Do you know any other USAID-funded Activity interacting in any way with POs/courts/HJPC, except JACA? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between JACA and these activities? 28. In addition to law enforcement and other government agencies, do you recognize any CSOs, or citizens supported by CSOs that file criminal reports or in some other ways help prosecution of HCOC cases? If yes, what are CSOs’ experiences in interacting with POs or courts? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 53 29. In your opinion, what CSOs need to improve in order to be more helpful to the prosecution of HCOC cases? 30. How can POs encourage CSOs to provide more support and get more involved in HCOC investigations? 31. In your opinion, to what extent is investigative journalism helpful to the prosecution of HCOC cases? 32. Do you see a need to better coordinate and educate CSOs and investigative journalists as the “supply side” for the POs in investigating HCOC cases? If no, please explain. If yes, in your opinion, what could be the steps to initiate better cooperation between POs and CSOs/investigative journalists? DO NOT READ: CQ: CLOSING QUESTIONS 33. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 34. Do you have any questions for us? JACA DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 1. In your opinion, were JACA activities tailored to needs of your beneficiaries and how would you rate JACA's assistance delivered so far? 2. In your opinion, which JACA's interventions were best received by prosecutors/judges/ other beneficiaries and why? 3. In your opinion, which JACA’s interventions were not so successful or well-received and why? 4. To what extent did JACA technical assistance help you/JACA beneficiaries in processing HCOC cases? 5. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 6. In the period October 2020 – October 2021, administrative data show a small number of indictments and investigations for the HCOC cases in some partner POs, what are the reasons for that? 7. In your opinion, what were your/beneficiaries biggest challenges in processing HCOC cases? 8. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 9. What is missing or what else do prosecutors/judges need to make progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behaviour in the justice sector (25 percent of JACA LoE)? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 54 10. In your opinion, what is the level of understanding among judges, prosecutors, and judicial staff about integrity principles, professional ethics, Conflict of Interest, and asset declaration requirements? Are resources and tools on how to comply with these standards adequate? 11. To what extent are judges, prosecutors, and judicial staff committed to complying with these standards? 12. To what extent did JACA’s technical assistance contribute to the prevention of corrupt behaviour in the BIH judiciary? 13. What more could be done for the judiciary to embrace the integrity and ethics principles? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 14. Did you observe any significant change in partner institution that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 15. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? 16. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 17. In your opinion which other institutions should be included in JACA activities? And why? 18. In your opinion, who is leading efforts in the BiH judiciary in preventing corrupt behaviour in the judiciary, including offers related to CoI, asset declaration, implementation of IPs and could you recall and describe some related activities? (PROBE: What about the role of HJPC in these activities, and how would you rate HJPC’s efforts) 19. Do you notice any progress in preventing corrupt behaviour, e.g., related to CoI, asset declaration, and implementation of IPs in the BiH judiciary? Please provide illustrative examples if you can recall them? 20. In your opinion, is it worth continuing JACA’s support for these activities? If yes, what else is needed to limit opportunities for corrupt behaviour in the BiH judiciary? Who should lead those efforts? If no, please explain? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 21. How were JACA’s trainings designed in terms of selection of topics, participants, and trainers? (PROBE: how are the topics selected, who are the participants and how are they selected, what is the attendance rate of participants at trainings, who are trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 22. What was s beneficiaries’ assessment of the training delivered so far? 23. What are further needs of the beneficiaries in terms of training? 24. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what can local/regional trainers bring that international ones cannot? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 55 25. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 26. Do you know any other USAID-funded Activity interacting in any way with POs/courts/HJPC, except JACA?? If yes, which ones and what are they about? Is there any cooperation or synchronization between JACA and these activities? 27. In addition to law enforcement and other government agencies, do you recognize any CSOs, or citizens supported by CSOs that file criminal reports or in some other ways help prosecution of HCOC cases? If yes, what’s CSOs experience in interacting with POs or courts? 28. In your opinion, what CSOs need to improve in to be more helpful to the prosecution of HCOC cases? 29. How can POs encourage CSOs to provide more support and get more involved in HCOC investigations? 30. In your opinion, to what extent is investigative journalism helpful to the prosecution of HCOC cases? 31. Do you see a need to better coordinate and educate CSOs and investigative journalists as the “supply side” for the POs in investigating HCOC cases? If no, please explain. If yes, in your opinion, what could be the steps to initiate better cooperation between POs and CSOs/investigative journalists? DO NOT READ: CQ: CLOSING QUESTIONS 32. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 33. Do you have any questions for us? HJPC CO1 DO NOT READ: SQ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you briefly name JACA’s activities that you were involved in? No need to go into details now, we will explore them during the conversation a. (PROBEs: Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases.)? DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. Were JACA’s activities tailored to needs of beneficiaries and how would you rate JACA's assistance that you were involved in so far? 3. In your opinion, which JACA's interventions were best received by prosecutors/judges/other beneficiaries and why? 4. In your opinion, which JACA's interventions were not so successful or well-received and why? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 56 5. To what extent did JACA’s technical assistance help you/JACA’s beneficiaries in processing HCOC cases? 6. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 7. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 8. What is missing or what else do prosecutors/judges need to do to enhance visibility of the progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 9. Did you observe any significant change in partner institutions that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes, please provide us with some details? 10. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases, and if yes, please provide us with some details? 11. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 12. In your opinion, which other institutions should be included in JACA activities? And why? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 13. To your best knowledge, how are designed JACA trainings in terms of selection of topics, participants, and trainers? a. (PROBE: how the topics are selected, who are the participants and how they are selected, what’s the attendance rate of participants at trainings, who are trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 14. How would you rate the quality, adequacy, and timeliness of the training content? 15. How would you rate competence and ability to transfer the knowledge of trainers?? What did you like, and what did not you like in the trainings you participated in? Have you attended training programs other than JACA that were delivered by international experts? What do you think worked well and what did not in those trainings? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 57 16. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what can local/regional trainer bring that international ones cannot? 17. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 18. Do you know any other USAID-funded Activity interacting in any way with POs/courts/HJPC, except JACA? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between JACA and these activities? 19. In addition to law enforcement and other government agencies, do you recognize any CSOs or citizens supported by CSOs that file criminal reports or in some other ways help prosecution of HCOC cases? If yes, what has been CSOs’ experience in interacting with POs or courts? 20. In your opinion, what CSOs need to improve in to be more helpful to the prosecution of HCOC cases? 21. How can POs encourage CSOs to provide more support and get more involved in HCOC investigations? 22. In your opinion, to what extent is investigative journalism helpful to the prosecution of HCOC cases? 23. Do you see a need to improve the coordination and education of CSOs and investigative journalists as the “supply side” for the POs in investigating HCOC cases? If no, please explain. If yes, in your opinion, what could be the steps to initiate better cooperation between POs and CSOs/investigative journalists? DO NOT READ: CQ: CLOSING QUESTIONS 24. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 25. Do you have any questions for us? HJPC CO2 DO NOT READ SQ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you briefly name JACA’s activities that you were involved in? No need to go into details now, we will explore them during conversation a. PROBE: CoI, asset declaration, implementation of IPs)? DO NOT READ EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behaviour in the justice sector (25 percent of JACA LoE)? 2. In your opinion, what is the level of understanding among judges, prosecutors, and judicial staff about integrity principles, professional ethics, Conflict of Interest, and asset declaration requirements? Are resources and tools on how to comply with these standards adequate? 3. To what extent are judges, prosecutors, and judicial staff committed to complying with these standards? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 58 4. Could you recall any JACA’s interventions in this segment of work in the judiciary and which are those? 5. Were JACA’s activities tailored to your needs/needs of beneficiaries and how would you rate JACA's assistance that you were involved in so far? 6. In your opinion, which JACA's interventions were best received by prosecutors/judges/other beneficiaries and why? 7. In your opinion, which JACA's interventions were not so successful or well-received and why? 8. To what extent did JACA’s technical assistance contribute to the prevention of corrupt behaviour in the BIH judiciary? 9. What more could be done for the judiciary to embrace the integrity and ethics principles? DO NOT READ EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 10. In your opinion, who in the BiH judiciary is leading efforts on preventing corrupt behaviour, e.g., related to CoI, asset declaration, implementation of IPs and could you recall and describe some related activities? 11. Do you notice any progress in preventing corrupt behaviour, e.g., related to CoI, asset declaration, and implementation of IPs? Please provide illustrative examples if you can recall them? a. PROBE: How do you rate the HJPC’s involvement in these activities? 12. In your opinion, is it worth continuing JACA’s support for these activities? If yes, what else is needed to limit opportunities for corrupt behaviour in the BiH judiciary? Who should lead those efforts? If no, please explain? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? 13. Which JACA’s training sessions/events did you attend? How do you rate them? 14. Did you attend all training/events sessions that you were invited to? 15. How do you rate JACA training(s) you attended? 16. Do you have a channel of communication with JACA for informing them about your further training needs? a. PROBE: Is JACA responsive? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 17. Are you involved in (or in contact with) any other USAID-funded activities, except JACA, related to issues of judicial integrity? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between these activities? 18. What about other donors? DO NOT READ: CQ: CLOSING QUESTIONS USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 59 19. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 20. Do you have any questions for us? CO1 PROSECUTORS/POS DO NOT READ: SQ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you name JACA activities that you were involved in? No need to go into details now, we will explore them during conversation (PROBEs: Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases.)? DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. Were JACA activities tailored to your needs and how would you rate JACA's assistance that you were involved in so far? 3. In your opinion, which JACA's interventions were best received by prosecutors/judges/other beneficiaries and why? 4. In your opinion, which JACA's interventions were not so successful or well-received and why? 5. To what extent did JACA technical assistance help you in processing HCOC cases? 6. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 7. Was there political or any other pressure put upon your regarding HCOC processing and how did it manifest in your work? 8. What is missing or what else do prosecutors/judges need to make progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 9. Did you observe any significant change in your institution that demonstrate increasing commitment to process HCOC cases and implement of techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 10. Did you observe any significant change in your institution that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 60 11. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 12. In your opinion which other institutions should be included in JACA’s activities? And why? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 13. To your best knowledge, how are JACA’s trainings designed in terms of selection of topics, participants, and trainers? (PROBE: how are the topics selected, who are the participants and how are they selected, what is the attendance rate of participants at trainings, who are trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 14. How would you rate the quality, adequacy, and timeliness of the training content? 15. How would you rate competence and ability of trainers to transfer their knowledge? What did you like, and what did not you like in the trainings you participated in? 16. Have you attended training programs other than JACA that were delivered by international experts? What do you think worked well and what did not in those trainings? 17. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what can local/regional trainers bring that international ones cannot? 18. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 19. Do you know any other USAID-funded Activity interacting in any way with POs/courts/HJPC, except JACA?? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between JACA and these activities? 20. In addition to law enforcement and other government agencies, do you recognize any CSOs, or citizens supported by CSOs that file criminal reports or in some other ways help prosecution of HCOC cases? If yes, what are CSOs’ experiences in interacting with POs or courts? 21. In your opinion, what CSOs need to improve in to be more helpful to the prosecution of HCOC cases? 22. How can POs encourage CSOs to provide more support and get more involved in HCOC investigations? 23. In your opinion, to what extent is investigative journalism helpful to the prosecution of HCOC cases? 24. Do you see a need to better coordinate and educate CSOs and investigative journalists as the “supply side” for the POs in investigating HCOC cases? If no, please explain. If yes, in your opinion, what could be the steps to initiate better cooperation between POs and CSOs/investigative journalists? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 61 DO NOT READ: CQ: CLOSING QUESTIONS 25. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 26. Do you have any questions for us? JPTCS DO NOT READ: SQ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you briefly name JACA’s activities that you were involved in? No need to go into details now, we will explore them during the conversation a. (PROBE: Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases.)? DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. In your opinion, were JACA activities tailored to the needs of your beneficiaries and how would you rate JACA's assistance delivered so far? 3. In your opinion, which JACA's interventions were best received by prosecutors/judges/other beneficiaries and why? 4. In your opinion, which JACA's interventions were not so successful or well-received and why? 5. To what extent did JACA’s technical assistance help you/JACA beneficiaries in processing HCOC cases? 6. In your opinion, what were your/your beneficiaries’ biggest challenges in processing HCOC cases? 7. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 8. What is missing or what else do prosecutors/judges need to do to enhance visibility of the progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 2: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to prevention of corrupt behaviour in the justice sector (25 percent of JACA LoE)? 9. Do you work with JACA on judicial integrity-related trainings? IF “No”, SKIP QUESTIONS BELOW AND START AT EQ 3 10. In your opinion, what is the level of understanding among judges, prosecutors, and judicial staff about integrity principles, professional ethics, Conflict of Interest, and asset declaration requirements? Are resources and tools on how to comply with these standards adequate? 11. To what extent are judges, prosecutors, and judicial staff committed to complying with these standards? 12. To what extent did JACA’s technical assistance contribute to the prevention of corrupt behaviour in the BIH judiciary? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 62 13. What more could be done for the judiciary to embrace the integrity and ethics principles? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 14. Did you observe any significant change in partner institutions that demonstrate increasing commitment to process HCOC cases and implement of techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 15. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? 16. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 17. In your opinion, which other institutions should be included in JACA activities? And why? 18. In your opinion, who in the BiH judiciary is leading efforts on preventing corrupt behaviour, e.g., related to CoI, asset declaration, implementation of IPs and could you recall and describe some related activities? (PROBE: What about the role of HJPC in these activities, and how would you rate HJPC’s efforts) 19. Do you notice any progress in preventing corrupt behaviour, e.g., related to CoI, asset declaration, and implementation of IPs in the BiH judiciary? Please provide illustrative examples if you can recall them? 20. In your opinion, is it worth continuing JACA’s support for these activities? If yes, what else is needed to limit opportunities for corrupt behaviour in the BiH judiciary? Who should lead those efforts? If no, please explain? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 21. How are JACA’s trainings designed in terms of selection of topics, participants, and trainers? (PROBE: how are the topics selected, who are the participants and how are they selected, what is the attendance rate of participants at trainings, who are the trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 22. How would you rate the quality, adequacy, and timeliness of the training content? 23. How would you rate competence and ability of trainers to transfer knowledge? What did you like and what did you not like in the trainings you participated in? Have you attended training programs other than JACA’s that were delivered by international experts? What do you think worked well and what did not in those trainings? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 63 24. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what local/regional trainers can bring that international ones cannot? 25. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 26. Are you involved in (or in contact with) any other USAID-funded activities, except JACA, related to work or interaction with the judiciary? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between these activities? 27. What about other donors? DO NOT READ: CQ: CLOSING QUESTIONS 28. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 29. Do you have any questions for us? INL/OPDAT/US EMBASSY DO NOT READ: SQ: STARTING QUESTIONS 1. Do you know what JACA is doing, and did you have any interaction with them so far? If yes, please tell us something about those interactions? (If needed, say: Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases, etc.) DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. (Skip - if the answer on the first question does not provide grounds for this question) Did you hear of any JACA's interventions that were well received by prosecutors/judges /other beneficiaries and why? 3. (Skip - if the answer on the first question does not provide grounds for this question) Did you hear of any JACA’s interventions were not so successful or well-received and why? 4. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 5. In your opinion, what were the biggest challenges in processing HCOC cases? 6. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 7. What is missing or what else do prosecutors/judges need to do to make progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 64 DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 8. Did you observe any significant change in partner institution that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 9. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? 10. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 11. In your opinion which other institutions should be included in JACA activities? And why? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? 12. Did you participate (in any capacity) in any JACA training session and what are your observations regarding selection of topics, participants, and trainers? a. PROBE: From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what local/regional trainer can bring that international ones cannot? b. PROBE: In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 13. Are you involved in (or in contact with) any other USAID-funded activities, except JACA, related to work or interaction with the judiciary? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between these activities? 14. Do you have your own projects interacting or which might be related to work of prosecutors and courts? If yes, what are they? Any interaction between your projects and JACA? DO NOT READ: CQ: CLOSING QUESTIONS 15. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 16. Do you have any questions for us? DONORS/INTERNATIONAL COMMUNITY DO NOT READ: SQ: STARTING QUESTIONS 1. Do you know what JACA is doing and did you have any interaction with them so far? If yes, please tell us something about those interactions? (If needed, mention: USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 65 Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases, etc.) DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. Did you hear of any JACA's interventions that were well received by prosecutors/judges/ other beneficiaries and why? 3. Did you hear of any JACA's interventions that were not so successful or well-received and why? 4. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 5. In your opinion, what were biggest challenges in processing HCOC cases? 6. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 7. What is missing or what else do prosecutors/judges need to do to make progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 8. Did you observe any significant change in partner institutions that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes, please provide us with some details? 9. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases, and if yes, please provide us with some details? 10. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? 11. In your opinion which other institutions should be included in JACA’s activities? And why? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? 12. Did you participate (in any capacity) in any JACA’s training session(s) and what are your observations regarding the selection of topics, participants, and trainers? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 66 13. Do you have any projects of your own that interact or might be related to the work of prosecutors and courts? If yes, what are these projects? Any interaction between your projects and JACA? DO NOT READ: CQ: CLOSING QUESTIONS 14. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 15. Do you have any questions for us? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 67 ANNEX VI: FOCUS GROUP GUIDES CO1 JUDGES/COURTS DO NOT READ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you briefly name JACA’s activities that you were involved in? No need to go into details now, we will explore them during the conversation a. (PROBEs: Individualized Training Plans for prosecutors, specialized training for judges, improvements in operational management related to processing HCOC cases.)? DO NOT READ: EQ 1: What primary factors have contributed to the success or presented specific challenges in JACA implementation when it comes to improved adjudication of high-profile corruption and organized crime cases (75 percent of JACA LoE)? 2. Were JACA’s activities tailored to your needs and how would you rate JACA's assistance that you were involved in so far? 3. In your opinion, which JACA's interventions were best received by prosecutors/judges/ other beneficiaries and why? 4. In your opinion, which JACA's interventions were not so successful or well-received and why? 5. To what extent did JACA technical assistance help you in processing HCOC cases? 6. In general, how would you rate the progress in processing HCOC cases in the BiH judiciary? 7. Was there political or any other pressure put upon prosecutors/judges regarding HCOC processing and how did it manifest in their work? 8. What is missing or what else do prosecutors/judges need to do to enhance visibility of the progress in processing HCOC cases? a. PROBE: What do you think JACA should prioritize to boost the processing of HCOC cases? b. PROBE: Based on your experience with JACA, what do you think JACA should do less of? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 9. Did you observe any significant change in partner institutions that demonstrate increasing commitment to process HCOC cases and implement techniques or tools obtained through JACA’s training or other technical interventions, and if yes please provide us with some details? 10. Did you observe any significant change in partner institutions that demonstrate decreasing commitment to processing HCOC cases and if yes, please provide us with some details? 11. Could you recall any other judicial institution(s) very important for processing HCOC cases in BiH? Why are they important? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 68 12. In your opinion, which other institutions should be included in JACA activities? And why? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? DO NOT READ: EQ 4.1: What lecturer/trainer selection model does JACA apply? 13. To your best knowledge, how are JACA trainings designed in terms of selection of topics, participants, and trainers? a. (PROBE: how are the topics selected, who are the participants and how are they selected, what is the attendance rate of participants at trainings, who are the trainers and how are they selected?) DO NOT READ: EQ 4.2: What is the beneficiaries’ assessment of the model currently in use? 14. How would you rate the quality, adequacy, and timeliness of the training content? 15. How would you rate competence and ability to transfer the knowledge of trainers?? What did you like, and what did not you like in the trainings you participated in? Have you attended training programs other than JACA that were delivered by international experts? What do you think worked well and what did not in those trainings? 16. From your point of view, what do you see that international trainers bring that local/regional cannot, and vice-versa, what can local/regional trainer bring that international ones cannot? 17. In your view, what is the best formula for selection of trainers that JACA should pursue in providing its training in the future? DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 18. Are you involved in (or in contact with) any other USAID-funded activities, except JACA, related to work or interaction with the judiciary? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between these activities? a. PROBE: What about CSOs? b. PROBE: What about investigative journalism? DO NOT READ: CQ: CLOSING QUESTIONS 19. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 20. Do you have any questions for us? CO2 POS/COURTS DO NOT READ: SQ: STARTING QUESTIONS 1. How long have you been involved/cooperating with JACA and can you name JACA’s activities that you were involved in? No need to go into details now, we will explore them during the conversation a. (PROBE: CoI, asset declaration, implementation of IPs)? DO NOT READ: EQ 2: What primary factors have contributed to the success or USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 69 presented specific challenges in JACA implementation when it comes to the prevention of corrupt behaviour in the justice sector (25 percent of JACA LoE)? 2. In your opinion, what is the level of understanding among judges, prosecutors, and judicial staff about integrity principles, professional ethics, Conflict of Interest, and asset declaration requirements? Are resources and tools on how to comply with these standards adequate? 3. To what extent are judges, prosecutors, and judicial staff committed to complying with these standards? 4. Could you recall any JACA’s interventions in this segment of work in the judiciary and which are those? 5. Were JACA’s activities tailored to needs of beneficiaries and how would you rate JACA's assistance that you were involved in so far? 6. In your opinion, which JACA's interventions were best received by prosecutors/judges/ other beneficiaries and why? 7. In your opinion, which JACA's interventions were not so successful or well-received and why? 8. To what extent did JACA’s technical assistance contribute to the prevention of corrupt behaviour in the BIH judiciary? 9. What more could be done for the judiciary to embrace the integrity and ethics principles? DO NOT READ: EQ 3: Is there room for adjustments in providing JACA’s technical assistance to partner institutions, e.g., for starting to provide support to new partner institutions and/or withdrawing support from some of the current ones, and why? DO NOT READ: EQ 3.1: To what extent does JACA have the same commitment to the implementation of its intervention from the management/prosecutors/judges of initially selected partner institutions (POs and courts) for processing/adjudication of HCOC cases and the HJPC for Component 2 interventions? 10. In your opinion, who in the BiH judiciary is leading efforts on preventing corrupt behaviour, e.g., related to CoI, asset declaration, implementation of IPs and could you recall and describe some related activities? 11. Do you notice any progress in preventing corrupt behaviour, e.g., related to CoI, asset declaration, and implementation of IPs? Please provide illustrative examples if you can recall them? a. PROBE: How do you rate the HJPC’s involvement in these activities? 12. In your opinion, is it worth continuing JACA’s support for these activities? If yes, what else is needed to limit opportunities for corrupt behaviour in the BiH judiciary? Who should lead those efforts? If no, please explain? DO NOT READ: EQ 4: What are the needs of the beneficiaries in adjusting the current lecturer/trainer selection model, if any? 13. Which JACA’s training sessions/events did you attend? How do you rate them? 14. Did you attend all training/events sessions that you were invited to? 15. How do you rate JACA’s training(s) you attended? 16. Do you have a channel of communication with JACA for informing them about your further training needs? a. PROBE: Is JACA responsive? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 70 DO NOT READ: EQ 5: In what ways could cooperation/collaboration with other USAID efforts and programs be more effective? 17. Are you involved in (or in contact with) any other USAID-funded activities, except JACA, related to issues of judicial integrity? If yes, which ones and what are they about? Do you notice any cooperation or synchronization between these activities? 18. What about other donors? DO NOT READ: CQ: CLOSING QUESTIONS 19. Do you want to tell us anything else related to the topics we discussed that we have overlooked? 20. Do you have any questions for us? USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 71 ANNEX VII: PHOTOGRAPHS OF EQUIPMENT PROVIDED BY JACA DURING THE COVID PANDEMIC Exhibit 3: Plexiglass partitions in a courtroom Exhibit 4: Audio/visual equipment funded by JACA in an adjacent courtroom USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 72 ANNEX VIII: JACA SCORECARDS Exhibit 5: Score Card results for applicability/adequacy/timeliness/ of JACA’s ITP for prosecutors and SJP for judges * The question did not apply because the intervention did not start or had already finished in the given year. ** The question did not apply because the intervention was relevant solely for prosecutors The data presented in Exhibit 5 represents the average of individual grades by JACA training participants expressed in percentages (The best is 100 percent). USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 73 Exhibit 6: Factors beyond the control and influence of judicial institutions and JACA in prosecuting HCOC cases * The question did not apply because the intervention did not start in the given year. ** The question did not apply because the intervention was relevant solely for prosecutors The data presented in Exhibit 6 represents the average of individual grades by JACA training participants expressed in percentages (The best is 100 percent). USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 74 ANNEX IX: EXAMPLES OF MEDIA ATTACKS ON HCOC PROSECUTORS In 2021 and 2022, the portal Normala.ba published several articles criticizing the work of Adnan Tulić, Prosecutor with the Una-Sana Canton (USC) Prosecution Office (PO). In their article of 4 May 2022, the portal reported that the USC PO dropped the investigation against the Police Commissioner of the USC Ministry of Interior, on whom Prosecutor Tulić had been trying to “hang” at least one indictment and that the “war” between this prosecutor and the Commissioner went on for years. The portal again covered the prosecutor’s attempts to indict the Police Commissioner on 23 May. The portal depicted Tulić as an incorrigible careerist and “American prosecutor” and declared that the former, “with monstrous and inexplainable (political?) motivation seeks to suspect and accuse the USC Police Commissioner at any cost to bring about its suspension and resignation. This tug of war to show ‘whose balls are bigger’ has lasted for three years.” The articles do not show whether the portal asked Prosecutor Tulić to comment. The Bosnia Times Portal, in its article published on 11 May 2022, reported that the Chief Prosecutor of the Tuzla Canton appointed as his Deputy Prosecutor Dražen Miličević, who “on its Facebook profile insulted generals of the Army of Bosnia and Herzegovina, referring to one of them as ‘Piggy’, and claiming that the BiH Armed Forces were Islamized, and that this prosecutor uses the same Facebook profile to sell Chetnik pamphlets as well as assorted remotely operated model toy cars. So, the HJPC finds such a man fit to serve as a Deputy Chief Prosecutor! You can be proud, o ye executioners of the justice system, you are leading this country to its doom!” The article also criticized the system of appointment of judicial office holders, referred to kinship connections of a member of the HJPC, and questioned the origins of assets of the Chief Prosecutor of the Tuzla Canton, whom it accused of taking bribes to stop investigations. The article does not describe the sources of this information, nor whether the author contacted the persons mentioned in the article to obtain their reaction. Since early December 2021, when Zenica’s Mayor Fuad Kasumović was arrested, the media covered the work of the cantonal PO, its Chief Prosecutor, and the Prosecutor on the Case. Some of the coverage is palpably biased and full of general statements in the attempt to discredit the PO, its manager and the Prosecutor on the Case, and failure to secure the response of the parties covered in the article. Two days after Kasumović’s arrest, the portal Ekran.ba published an article reporting that “Kasumović’s arrest is the consequence of (Chief Prosecutor) Vesna Kaknjo’s obedience to the policies of the SDA and HDZ. It has long been known that Kaknjo, as well as many other prosecutors in BiH, through their work created legal insecurity in BiH. For such performance, she has recently been rewarded by another term in office.” On 6 December 2021, the portal Dnevno.ba published the following article: “Fuad Kasumović is a victim of SDA-controlled Zenica PO: Chief Prosecutor Kaknjo and Judge Ljevaković are close to the SDA party circles in Zenica!” which described alleged links of the Chief Prosecutor and the judge with Nezir Pivić, SDA’s Mayoral Candidate whom Kasumović defeated in the most recent elections. The article, among other things, stated that the PO intentionally omitted to interview 11 witnesses to ensure that the court would approve their request to hold Kasumović in custody. On 8 December 2021, the portal Bosnia Times announced that the prosecutor on this case would be promoted to a higher office and insinuated that she would have the support of the Chairman of the HJPC, for whom she had worked as an intern. The article also stated that the prosecutor obstructed an investigation against Sanin Bogunić, a prosecutor from Sarajevo, who is a family friend. The portal also alleged that the prosecutor “was willing to entertain her colleagues by dancing and playing tambourine.” This article was carried by other portals as well. USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 75 ANNEX X: LIST OF HCOC CASES INITIATED OR CONCLUDED BY A PROSECUTORS’ OFFICE Public officials indicted by a Prosecution Office, 2017–2022 • Cantonal Prime Minister (2x) • Advisor to a Cantonal Prime Minister (3x) • Cantonal Minister of Interior • Cantonal Minister of Economy (2x) • Cantonal Minister of Education, Science, Culture, and sports (2x) • Speaker of a cantonal Parliament • Member of a cantonal Parliament (5X) • Member of an Entity Parliament (4X) • Mayor of the City (2x) • Advisor to the Mayor of the City • Head of Urban Planning, City • Head of the Property Management Department, City • Mayor of a Municipality (2x) • Advisor to the Mayor of a Municipality • Head, Urban Planning Department, a Municipality • Mayor of a Municipality • Mayor of a Municipality (2x) • Chief, Entiry Police Directorate • Chief, a Police Station • Assistant Chief, a Police Station • Chief, a Police Station • Chief, a Police Station • Chief, a Police Station (2x) • General Manager, a Cantonal Forest Company and five members of senior management (organized crime) USAID.GOV JACA MIDTERM PERFORMANCE EVALUATION 76 • Head, a cantonal Forestry Directorate • Head, a cantonal Health Insurance Institute • General Manager, a Public Enterprise • Head of Department, a cantonal Forestry Directorate • Seven physicians and 49 instructors (55 indictees) in a organized crime case • A High-profile case: indictment on 90 counts of corruption • A High-profile organized crime case • Head, Employment Bureau, together with 12 other indictees • Inspectors of a Entity Inspection Department (2x) • Chief Inspector, a cantonal Inspection Directorate • Inspectors of the Inspections Directorate (2x) • Entity inspectors • Director, a Public Health Care Center (2x) • Head, Transportation Department, Canton Ministry of Education, Science, Culture, and Sports (2X) • General Manager, a Public Water Utility Company (6x) • General Manager, a Municipal Services Company • General Manager, a Municipal Services Company • General Manager, Canton Roads Directorate • General Manager, a Municipal Services Company • Head, a Public Health Care Center • High-profile privatization cases • Secretaries, department and division heads and other public administration officials • Heads of public companies and public institutions MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE II) Fra Anđela Zvizdovića 1 UNITIC Tower B, Floor 21 71000 Sarajevo Bosnia and Herzegovina Phone: + (387) 033 941 676 contacts@measurebih.com www.measurebih.com