JUNE 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Democracy International, Inc. under Order No. AID-624-TO-17-00001, Contract No. AID-OAA-I-15-00017 CÔTE D’IVOIRE JUSTICE SECTOR STRENGTHENING PROGRAM MIDTERM PERFORMANCE EVALUATION FINAL REPORT DISCLAIMER This is an external report. The views expressed in this document are the authors' and do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Submitted to: USAID/Côte D’Ivoire Prepared by: Renaud Beauchard, Team Leader Magda Raupp, Senior Evaluation Expert Pierre-Olivier Lobé, Local Expert Contractor: Democracy International, Inc. 7600 Wisconsin Avenue, Suite 1010 Bethesda, MD 20814 Tel: 301-961-1660 www.democracyinternational.com CÔTE D’IVOIRE JUSTICE SECTOR STRENGTHENING PROGRAM MIDTERM PERFORMANCE EVALUATION FINAL REPORT JUNE 2017 TABLE OF CONTENTS EXECUTIVE SUMMARY ...........................................................................................................................................................................1 BACKGROUND ...........................................................................................................................................................................................1 PURPOSE OF THE EVALUATION AND EVALUATION QUESTIONS ...........................................................................4 METHODOLOGY ......................................................................................................................................................................................5 FINDINGS .................................................................................................................................................................................................... 11 CONCLUSIONS ....................................................................................................................................................................................... 30 RECOMMENDATIONS ........................................................................................................................................................................ 34 ANNEX A: REFERENCES .................................................................................................................................................................. A-1 ANNEX B: STATEMENT OF WORK AND EVALUATION ACTIVITIES .................................................................... B-1 ANNEX C: EVALUATION MATRIX ............................................................................................................................................ C-1 ANNEX D: LIST OF PROJECT AND PERFORMANCE INDICATORS ....................................................................... D-1 ANNEX E: LIST OF PEOPLE INTERVIEWED ........................................................................................................................... E-1 ANNEX F: QUESTIONNAIRE FOR STAKEHOLDER INTERVIEWS .............................................................................. F-1 ANNEX G: EXIT POLL SURVEY QUESTIONNAIRE ........................................................................................................... G-1 ANNEX H: EXIT POLL SURVEY RESULTS (CREFDI) ......................................................................................................... H-1 ANNEX I: TETRATECH STATEMENT OF DIFFERENCES ................................................................................................... I-1 LIST OF ACRONYMS AFD Agence Française de Développement (French agency of int’l development) BAJ Bureau d’assistance judiciaire (Office of judicial assistance) BOP Bureau d’ordre penal (Criminal Registry Office) CMS Case management system CNO Centre Nord Ouest CREFDI Centre de recherché et de formation sur le développement intégré CSO Civil Society Organization DiD Difference in Difference DOPS Document opérationnel de Politique sectorielle (government’s strategic document) EJ Examining judge (juge d’instruction) GI Group Interview GoCI Government of Côte d’Ivoire IDI In-depth interview JPO Judiciary Police Officer JSSP Justice Sector Support Program M&E Monitoring and evaluation MoJHR Ministry of Justice and Human Rights NGO Non-Governmental Organization PAD Project Appraisal Document PAPS Plan d’action de politique sectorielle (multi-years sector policy action plans) PJP Pôle judiciaire penal (Criminal Justice Unit) PR Procureur de la République (head Prosecutor) SIGP Système d’information pour la gestion des procédures judiciaires et des actes administratifs SNDI Société Nationale de développement informatique STTA Short Term Technical Assistance TPI First Instance Court (Tribunal de première instance) UCQG Unité de contrôle de qualité et de gestion (Quality Control and Management Unit) USAID United States Agency of International Development COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I EXECUTIVE SUMMARY The Justice Sector Strengthening Program (JSSP) was designed to improve the Government of Côte d’Ivoire (GoCI) delivery of justice services, especially criminal justice which collapsed as a result of a prolonged crisis spanning more than a decade and which remains, as several studies show, widely perceived as corrupt, biased, inefficient, punitive and controlled by politics.1 JSSP is based on the following theory of change: "IF the supply of justice services in Côte d’Ivoire are improved through capacity building, case management strengthening, transparency and accountability activities at the local level and leadership development and institutional strengthening at the national level; and IF demand for more fair, professional and accessible justice is fostered through legal awareness campaigns and legal aid strengthening activities, THEN there will be significant improvement in the administration of justice and service delivery and increased usage of justice sector services,” JSSP aims to address two main objectives: (1) improve the administration of justice and (2) increase access to justice. PURPOSE OF THE EVALUATION AND EVALUATION QUESTIONS The primary objectives of the mid-term performance evaluation are: (1) to assess and to document the extent to which the project is achieving its objectives; and (2) to make justified recommendations to improve the efficacy of the project, thereby contributing to the ambitious but vital goal of improving justice sector effectiveness, transparency, and accountability. The evaluation findings could be a learning tool that allows informed decisions about adjustments necessary to achieve the overall goal of the JSSP activity. The evaluation findings can also be used to inform a new Project Appraisal Document (PAD) in the Democracy sector and for future programs. This evaluation responds to the five key evaluation questions USAID formulated in this solicitation: 1 To what extent did JSSP interventions in the pilot courts lead to improvements in case management? 1a What change, if any, was noted in the case management capacity in each court? Was it increased? 2 To what extent did JSSP interventions with non-governmental organizations and civil society organizations promote access to justice for the general public and outreach to the Office of Legal Aid? 1 According to Transparency International, the justice system is still seen as dangerous partial as impunity remains generalized throughout the country. Similarly, according to the last Afrobaromenter survey in Côte d'Ivoire, many Ivorians distrust the formal court system and perceive the justice system to be biased. See http://www.afrobarometer.org/publications/ad142-building-legal-system-citizens-trust-and-use-remains￾challenge-post-conflict-cotedivoire and https://www.transparency.org/files/content/corruptionqas/Overview_of_corruption_in_Cote_dIvoire.pdf. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – II 2a How has access to justice changed? 2b What change, if any, was noted in the administration of justice, especially criminal justice activities? 3 To what degree have JSSP training activities been integrated by targeted recipients (judiciary police, magistrates, and court staff) into their respective work roles? 4 To what extent have the attitudes of the general public and media changed toward the Ministry of Justice and the legal aid office? 4a What are the key barriers that still need to be address to effectuate a change in the perception of legitimacy? 5 How effective have JSSP tools been in raising awareness and improving public understanding of and access to legal information and the public defense fund? 5a What are the key lessons, best practices, and challenges learned from this approach? METHODOLOGY This performance evaluation used a non-experimental evaluation design using a mixed￾methods approach to collect quantitative and qualitative data to respond to the five key evaluation questions described above. Please see Annex C to review the Evaluation Design Matrix, detailing the data sources, data collection methodologies, analysis methods and perceived limitations at the inception of the evaluation process. A team comprised of Renaud Beauchard, Evaluation Team Leader, Magda Raupp, Senior Evaluation Expert, and Pierre Olivier, Local Expert conducted field research in Côte d’Ivoire for a period of three weeks. During fieldwork, they conducted in depth interviews with 58 relevant stakeholders in Bouake, Man, Yopougon, Sinfra, Agboville, and Boundiali. Please see Annex E for a full list of stakeholders interviewed by the team. In addition to qualitative research, DI conducted an exit poll survey with 1,069 court users in eight different locations to assess court users’ perceptions on access to justice, the quality of service delivery, level of information about their rights, and their opinions about the justice system in general, including perceptions on the level of corruption in the courts. The exit poll was conducted by the Centre de Recherche et de Formation sur le Developpement Integré (CREFDI ) in a total of eight pilot courts, including courts where the project intervened –Man, Bouake, Yopougon, Sinfra, Agboville, Boundiali(treatment group)—and courts where the project did not --Divo and Aboiss (control group). Participants in the exit polls were approached as they entered and departed various courts. All were 18 years of age or older. In addition to the exit poll, CREFDI also conducted focus groups with court users in the six intervention courts. JSSP incremental implementation model gave the evaluators the opportunity to make comparisons between jurisdictions that had benefitted from the program for over three COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – III years and jurisdictions that benefitted from the intervention later.2 The evaluation design also allowed the team to compare jurisdictions that benefitted from the program (intervention) and those that had not participated in the program (control). In addition to interviews conducted during field work, the evaluation team reviewed JSSP M&E data, and assessed whether or not there were differences between pilot courts that benefitted early from JSSP as opposed to those that benefitted later. This information was then triangulated with the results of the exit poll survey, which contrasted court user perceptions in intervention and control courts. The intention was to clarify the effects of the program and provide insights to inform USAID, other donors, and the Ministry of Justice policy and decision-makers. The quantitative analysis is based on an exit poll survey to determine whether there were statistically significant differences among court users’ perceptions in intervention and control areas and, to assess, if relevant, the size of these differences. EVALUATION LIMITATIONS The qualitative analysis used a mix of desk reviews, interviews of key informants, focus groups. Compiling all the information from the interview onto a spreadsheet, the evaluation team took repeated steps to generate systematic, verifiable, credible information and minimize several types of bias. For example, to reduce response or selection bias, the team designed data collection instruments to elicit open, honest, and unbiased responses and avoid leading questions. Similarly, to reduce acquiescence bias, the team triangulated information by comparing findings obtained through the use of different data collection methods. Finally, to reduce co-intervention bias, the team conducted a thorough review of the implementing environment and donor activities in the justice sector, including the European Union, the Agence Francaise de Developpement (AFD) and the Projet "Accès à la Justice" (PALAJ) implemented with the Association des Femmes Juristes de Côte D'Ivoire. The goal of this review was to understand programmatic overlap as well as potential synergies and lessons learned. 2 The JSSP targeted 10 pilot courts. In 2014 a new court was added to the list and then in the summer of 2015, the list of pilot courts changed again when 7 new localities were added, which brings the total number of pilot courts concerned by the project to 18 over its total duration. The current list of pilot courts include • The 10 original courthouses listed in the RFP: Bouafle, Dabou, Bouake, Abengourou, Gagnoa, Dimbokro, Bouna, Lakota, Sinfra and Man; • The courthouse of Yopougon which was added as a pilot court in 2014 after USAID and the EU agreed in January 2014 that ProJustice would take over the EU computerization program, which was using Yopougon as a pilot court; • The courthouses of Agboville, Bondoukou, Korhogo, Boundiali, Daloa, Guiglo and Séguéla, which were added as pilot courts in the summer of 2015. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – IV FINDINGS IMPROVEMENT IN CASE MANAGEMENT AND CHANGE IN CASE MANAGEMENT CAPACITY 1. Overall, the new tools designed for criminal proceedings and introduced by JSSP are well accepted in pilot courts. These tools include: (i) the Criminal Registry Office (Bureau d’ordre pénal “BOP”); (ii) the Criminal Justice Unit (Pôle judiciaire pénal “PJP”); (iii) the detention tables for examining judges; (iv) the procedural templates; (v) the Quality Control and Management Unit (Unité de contrôle de qualité et de gestion, UCQG). 2. The Criminal Registry Office (Bureau d’ordre pénal) is implemented well and has been accepted in the various courthouses, with the exception of Yopougon. 3. The Criminal Justice Unit (Pôle judiciaire pénal) is used as a tool of dissemination in the courthouse’s jurisdiction and is credited with a jump in quality of the work performed by judiciary police officers 4. The procedural templates are useful tools established by a pool of Ivorian judges (both prosecutors and bench) to ensure compliance with criminal procedure. 5. The preventive detention monitoring tables are a very simple feature consisting of wall￾mounted metal boards that show and manage pretrial detention times.3 6. The departure of the court coordinators has generated problems of sustainability of ProJustice tools and activities in some jurisdictions (eg Sinfra and Bouna). 7. One important challenge to the sustainability of the new tools is the lack of resources and the lack of maintenance of equipment. 8. The Quality Management Control Unit (Unité de Contrôle Qualité et Gestion (UCQG) integrates solutions of demand-side governance in the court management by using the suggestion boxes to determine the UCQG agenda. 9. A computerized case management application (SIGP or Système d’information pour la gestion des procédures judiciaires et des actes administratifs) has been developed and is being experimented in two pilot courts by a contractor. 10. It is too soon for the evaluation team to evaluate the impact of the computerized case management system (CMS) but some issues affecting sustainability are already apparent. 11. Based on interviews with the MoJHR officials, the evaluation team believes the MoJHR is not fully involved in the goals of this project . The donor community, on the other hand, is investing more in providing hardware (computers and software) than on providing technical assistance 12. There is not a statistically significant difference in reduction of case backlog between jurisdictions that have participated the longest in the program --Bouaké, Dabou, Man and Youpogon— and the two control jurisdictions —Aboisso and Divo. 3 See a picture of a detention table in ProJustice’s Quarterly Progress Report July-September 2014 and Annual Progress ReportApril 2013 – September 2014, p. 11. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – V 13. According to data collected by JSSP, the level of satisfaction of court users with services provided by the courts varies significantly from court to court and is not positively correlated with activities implemented by the program JSSP INTERACTIONS WITH NGOS AND CIVIL SOCIETY ORGANIZATIONS TO PROMOTE ACCESS TO JUSTICE FOR THE GENERAL PUBLIC AND OUTREACH TO THE OFFICE OF LEGAL AID 14. NGOs reported a “thirst for rights” ” (i.e a desire of a rule of law system) on the part of the population but it remain defiant towards justice institutions. 15. In certain localities, community’s opinion leaders demanded payments for participating in the awareness campaigns. 16. NGOs reported MoJHR’s lack of cooperation regarding events taking place in the courthouses where the project operates. CHANGES IN ACCESS TO JUSTICE 17. Overall, evidence collected in the field shows a trend towards greater use of the judicial system. CHANGE IN THE ADMINISTRATION OF JUSTICE 18. The introduction of transparency tools such as the court signage and the court fees posters has contributed to the diminution of the phenomenon of intermediaries named “margouillats.” 19. Changes in the administration of justice seem to vary from one jurisdiction to the next making it difficult to talk about an overall change as a result of JSSP activities. Improvements were noted in population survey in Agboville, Boundiali and Man, but not in Bouaké, Sinfra and Yopougon. 20. The exit poll survey of court users shows that perceptions of corruption in the court has improved in the courthouses of Agboville, Boundiali and Man but not in other courts involved in the evaluation. 21. The suggestion boxes installed by JSSP are used to determine the agenda of the Quality Management Control Unit meetings. 22. JSSP support of the general inspectorate has led to meaningful changes (all courthouses have been inspected twice, sanctions have been imposed including on judges) and the inspection is perceived as an example of good practice. INTEGRATION OF TRAINING ACTIVITIES BY TARGET RECIPIENTS INTO THEIR WORK ROLES 23. Joint training of judiciary police officers/prosecutors/examining judges have contributed to the increase in the cooperation among the various actors of criminal justice. 24. The Criminal Justice Unit has been instrumental in disseminating the joint training innovation in their jurisdictions. 25. Exchanges between actors exposed to JSSP tools and others have had a beneficial effect in their willingness to introduce JSSP tools. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – VI CHANGE OF ATTITUDES OF THE GENERAL PUBLIC AND MEDIA TOWARD THE MOJHR AND THE LEGAL AID OFFICE 26. Very few applications were made to the office of judicial assistance. In spite of awareness campaigns, only 191 applications were received by the Office of judicial assistance between June 2015 and January 2017, with 48 rejected. 27. Effective signaling and displays about fees have contributed to reduce the phenomenon of informal intermediaries, a common practice before JSSP. KEY BARRIERS THAT NEED TO BE ADDRESSED TO ATTAIN CHANGE IN THE PERCEPTION OF LEGITIMACY 28. The reform of the justice sector is not a priority sector for the Government of Côte d’Ivoire; it is mostly donor-driven. 29. The budget of the MoJHR’s Directorate of civil and criminal affairs to run the Office of judicial assistance is very low. 30. No agreement was reached with the Bar association on a scale of fees. 31. The Ivorian Bar is a key barrier to the perception of legitimacy of the judicial sector as a whole. 32. The Office of Judicial Assistance does not monitor what happens after it has granted an application. 33. Suggestion boxes highlight a sector affected by corruption, delays and a lack of attorneys. 34. Both NGOs and several key informants report an overall skepticism toward the judiciary on the part of the public. 35. Perception of legitimacy is not the same in all localities. EFFECTIVENESS OF JSSP TOOLS IN RAISING AWARENESS AND IMPROVING PUBLIC UNDERSTANDING OF AND ACCESS TO LEGAL INFORMATION AND JUDICIAL ASSISTANCE 36. Awareness of and access to legal information was made easier by tools such as the signaling and the display of various court fees in the courthouses but overall the activities concerning access to justice have limited impacts. 37. Awareness campaigns focused on the rights and benefits of judicial assistance but neglected to clarify the potential downsides (see ProJustice flyers on judicial assistance and land certificates). 38. Practical guides and compendia of criminal cases were widely distributed to jurisdiction officials and NGOs. 39. The information and orientation kiosks, which ProJustice claims to have implemented, have not been seen in operation in courthouses the team visited. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – VII CONCLUSIONS IMPROVEMENT IN CASE MANAGEMENT AND CHANGE IN CASE MANAGEMENT CAPACITY The low technology tools such as the BOP, the PJP, the procedural templates, the detention tables, and institutions like the Quality Management Control Unit (Unité de Contrôle Qualité et Gestion, created out of JSSP work) show potential improvements in case management capacity. However, several factors, including the low priority of the justice sector in the GoCI’s policy agenda, the massive turnover of judicial staff, and the chronic lack of budget for maintenance and servicing of the equipment raise issues of sustainability. The MoJHR is neither technically nor institutionally prepared for a successful implementation of the computerized case management system. JSSP INTERACTIONS WITH NGOS AND CIVIL SOCIETY ORGANIZATIONS TO PROMOTE ACCESS TO JUSTICE FOR THE GENERAL PUBLIC AND OUTREACH TO THE OFFICE OF LEGAL AID Despite meaningful contributions made by the project (e.g. directory of NGOs operating in the field of justice, open doors in the courthouses etc.) and despite a desire for a functioning judicial sector on the part of the population, the project’s interactions with NGOs was lacking in content to reconcile Ivorian justice with their judicial system. Focusing extensively on the structurally weak judicial assistance scheme, the awareness campaigns conducted in the program lacked a backbone and were too focused on communicating about fragile institutions like judicial assistance. CHANGES IN ACCESS TO JUSTICE Access to justice includes both the formal ability to appear in court and engaging the wider social context of the Ivorian court system and overcoming the systemic barriers faced by different members of the community. The Ivorian Bar, with just over 500 attorneys, has restricted conditions of entry, and a poor track record of success when it comes to representation of low- income citizens. This is a structural impediment to access to justice. With the limitation of access to reliable statistics produced by the MoJHR as well as limitation of time and resources, the evaluation team and exit poll survey could not identify a positive correlation between the project’s activities and an increase of court users. The evaluation team can, however, credit the program with outputs geared at the promotion of women in the magistracy and the legal profession. CHANGE IN THE ADMINISTRATION OF JUSTICE The evaluation team credits the JSSP for having designed certain tools intended to stimulate a demand for a better administration of justice (UCQG, transparency tools such as signage, COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – VIII courts fees posters) and observed a genuine concern on the part of interviewed judicial officials for an improvement of the court users’ experience. With its transparency tools, JSSP also appears to have had an effect, albeit not quantifiable, on the practice of informal intermediaries known as “margouillats.” The exit poll survey reveals a discrepancy in terms of citizens’ perception of court administration with Agboville, Boundiali, and, to a lesser extent, Man faring better than the other courthouses. Little change is to be expected from the Judicial Assistance initiative. With barely a couple hundred applications and operational deficiencies, the Judicial Assistance Initiative will be unlikely to drive enough litigants to stimulate a demand for a better administration of justice. INTEGRATION OF TRAINING ACTIVITIES BY TARGET RECIPIENTS INTO THEIR WORK ROLES The interviews with the prosecutors, examining judges, and a judiciary police officer reveal a discernable trend of a more cooperative management of criminal proceedings, which is reinforced by the PJP, emphasizing the overall comprehensiveness of the program. This trend is corroborated by the evaluation conducted by ProJustice of the joint training of magistrates/judiciary police officers. BARRIERS TO THE LEGITIMACY OF JUDICIAL INSTITUTIONS AND OF THE OFFICE OF JUDICIAL ASSISTANCE Barriers to legitimacy include:  The low priority status of the justice sector in the GoCI’s general policy;  The lack of operationalization of the judicial assistance scheme  An excessive focus on the part of the implementer on formal processes as opposed to actual processes (e.g. flyer on judicial assistance, Tetra Tech DPK’s response on the decentralization of the payment process of judicial assistance)  A business bar with a poor track record on legal assistance to low-income citizens. EFFECTIVENESS OF JSSP TOOLS IN RAISING AWARENESS AND IMPROVING PUBLIC UNDERSTANDING OF AND ACCESS TO LEGAL INFORMATION AND JUDICIAL ASSISTANCE The findings suggest that awareness campaigns were not the most effective tool given the structural problems affecting the judicial system. In both the design and the implementation, insufficient attention was paid to the structural issues plaguing the judicial sector. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – IX RECOMMENDATIONS Findings Conclusions Recommendations The five new tools are well accepted (findings 1 through 5 and 8) Strong potential for case management improvement but no systematic collection of statistics to measure their effect by MoJHR 1. Dedicate the remaining time left to properly assess the structural problems affecting the Judicial Assistance initiative. 2. ProJustice: Find a way to work with the Directorate of evaluation and statistics to improve the quality of information and use this information to improve decision making (particularly BOP). 3. Extension to the whole jurisdictional network of preventive detention tables) No maintenance of basic equipment by MoJHR (finding 7) MoJHR does not have institutional structure, human resources for computerized CMS MoJHR not in capacity to own the Computerized CMS 4. USAID: Mobilize the donor community to find funding for technical assistance to build technical and institutional capacity of MoJHR Depending on turnover, the tools survive or not (finding 6). The tools sustainability is grounded upon individuals, not upon institutional appropriation. 5. ProJustice: Work with the DACP to generate a list of instructions (circulaires) to ensure the dissemination of the various tools in the entire jurisdictional network. Too much focus on judicial assistance in awareness campaigns (finding 13, 37) Judicial assistance scheme not operational (findings 29, 30, 31, 32). Implementation focused too much on formal processes not enough on actual processes (particularly regarding judicial assistance). 6. Perform a diagnostic assessment on what is lacking as a prerequisite for a properly functioning judicial assistance and monitoring procedure after granting judicial assistance. 7. At minimum, an agreement with the Bar on a scale of fees is required. 8. Champions in the judiciary to work with NGOs. 9. Reorient civil society intervention on access to justice to chronic issues affecting the Ivorian justice system and conduct awareness campaigns on the issue such as the illegal use of criminal custody as a tool for debt collection and prolonged preventive detention. 10. Consider refocusing judicial assistance on a specific category of applicants (women victims of violence, handicapped, detainees in preventive detention) to conduct a pilot experiment with the OJA. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – X Signage, court fees posters have contributed to reduce the phenomenon of “margouillats”. Signage and court fees posters have been great simple tools to achieve important results. 11. Investigate further the effect on margouillats phenomenon. 12. Consider equipping other courthouses with signage. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 1 BACKGROUND Once hailed a vibrant economy that possessed an efficient judicial system with well-trained judges and lawyers, Côte d’Ivoire faces an uphill battle to recover from two decades of conflict culminating in a complete breakdown of the rule of law between 2002 and 2008. As a result of the prolonged state of crisis, the country has suffered a serious decline in its ability to ensure effective administration of and access to justice throughout its territory, particularly in a sizable portion of the Centre Nord Ouest (CNO). While courthouses in the CNO reopened after the Ouagadougou agreements in 2008, criminal justice did not resume in full until 2012 as a result of a lack of penitentiary facilities. 17 out of 37 courthouses and 22 out of 33 penitentiary facilities were damaged and looted during the post-electoral crisis in 2011. However, the rule of law challenges the country faces are not limited to transitional issues. Though there is consensus that the restoration of state authority has resulted in the resumption of formal justice, simply restoring defunct justice sector institutions in the former CNO zone will not ensure the effective administration of justice or advance national reconciliation. Côte d’Ivoire largely inherited its legal institutions from France, albeit with a very different political regime. Though judges and lawyers received the same technical training as their French counterparts until 1984, the inquisitorial basis of the French criminal justice system formed a toxic cocktail with the politics of successive regimes following the one-party period of Houphouet Boigny. As a result, the judicial system remained fraught with corruption and a lack of independence further entrenched a culture of impunity rather than combatting it. The situation worsened after judges stopped receiving pre-service training in France. Since then, judges have received limited training in an increasingly tense political climate characterized by violent conflicts over land rights, particularly in the North and West. These conflicts are exacerbated by the chronic failure of the judicial system to resolve disputes. Perceived at the outset of the crisis as corrupt, biased, inefficient, extremely punitive, and controlled by politics4 , the formal justice sector faces competition from the traditional justice sector, which varies greatly from one region to another, both in terms of legal issues at stake and community structure (e.g., dynastic structure in the North vs. elected leaders in the West). Besides the population’s general distrust of Ivorian justice, the justice sector, particularly criminal justice, faces new challenges. These include the presence of demobilized, armed, and unemployed young men; the surge of drug-related crime in the region; and the threat of terrorist groups in the Sahel-Sahara zone, which manifested itself violently in Grand Bassam on March 14, 2016, when Côte d’Ivoire experienced its first Islamist terrorist attack that left 16 persons dead. Awareness that the restoration of the rule of law in Côte d’Ivoire is key to national reconciliation prompted the donor community to pour substantial resources into the 4 USAID-funded Democracy and Governance Assessment conducted in Cote d’Ivoire in 2011 identified critical governance challenges confronting Cote d’Ivoire in the area of low levels of government accountability and consequent low levels of popular support and legitimacy of democratic institutions. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 2 country’s justice sector. Donors coordinate their actions through a sector-based strategy synthesized in the government’s strategic document— adopted by the council of ministers in June 2013 — DOPS— which constitutes a guiding framework for both Government and donors implemented under PAPS. The PAPS for the years 2017-2020, prepared with JSSP assistance, is currently under review in the general secretariat of the Government. In this context, USAID funded the Justice Sector Support Program (JSSP) under cooperative agreement No. 624-A-13-00002 with a funding amount of $18.6 million from 2013 to 2018. JSSP was designed by USAID/West Africa/Regional Peace and Governance Office as part of a larger justice reform package supported by the Office of Transition Initiatives (OTI) and other donors, particularly the United Nations Mission in Côte d’Ivoire (UNOCI). Its mandate is to coordinate with OTI to prioritize its activities in courthouses that OTI had already equipped or was renovating and refurbishing under a separate program. Donors also tasked JSSP with establishing coordination between the Ministry of Justice and UNOCI to ensure delivery of a complete reform in the regions where USAID operates. Although UNOCI has recently left this program, the French Government and the European Union continue to provide significant support to Côte d’Ivoire’s justice sector. The USAID-funded Democracy and Governance Assessment conducted in Côte d’Ivoire in 2011 concluded that weaknesses in the justice sector are a result of corruption, lack of independence, inadequate resources to maintain courtroom functioning, and lack of trained legal professionals. This leads to low levels of popular support and leaves the country fragile and more vulnerable to violence. JSSP was designed to improve the Government of Côte d’Ivoire (GoCI) delivery of justice services, especially criminal justice. JSSP is based on the following theory of change: "IF the supply of justice services in Côte d’Ivoire are improved through capacity building, case management strengthening, transparency and accountability activities at the local level and leadership development and institutional strengthening at the national level; and IF demand for more fair, professional and accessible justice is fostered through legal awareness campaigns and legal aid strengthening activities, THEN there will be significant improvement in the administration of justice and service delivery and increased usage of justice sector services.” JSSP aims to address two main objectives: (1) improve the administration of justice and (2) increase access to justice. JSSP’s expected results were:  Improved court management and speedier trials, by sufficiently trained officials and better, more uniform adjudication of cases;  Decreased time it takes for a case to be opened and concluded;  Strengthened capacity of the inspectorate and increased monitoring of judicial performance;  Greater public confidence in the justice sector;  More defendants are able to access the fund for legal defense; COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 3  Greater public awareness and an increased use of the formal judicial system. The key project implementation arrangements are: The development of tools designed for a better administration of justice and their experimentation in pilot courts with the assistance of court coordinators; The design of an IT computerization master plan; The development of a computerized case management application and its experimentation in 2 pilot courts;  Training of trainers for the national judicial training institute;  Joint trainings between judiciary actors and judicial police officers;  The development of pedagogic tools for the purpose of continuing education;  Material and technical assistance to the general inspectorate of judicial and penitentiary services;  Assistance to the drafting of a new judicial assistance decree;  Material and technical assistance to the office of judicial assistance;  Awareness campaigns on access to justice and judicial assistance. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 4 PURPOSE OF THE EVALUATION AND EVALUATION QUESTIONS The primary objectives of this mid-term performance evaluation were: (1) to assess and document the extent to which the project is achieving its objectives; and (2) to make justified recommendations to improve the efficacy of the project, thereby contributing to the ambitious but vital goal of improving justice sector effectiveness, transparency, and accountability in Côte d’Ivoire. The evaluation findings should serve as a learning tool that leads to informed decisions about programmatic adjustments necessary to achieve the overall goal of the JSSP activity. The evaluation findings can also be used to inform a new Project Appraisal Document (PAD) in the democracy sector and for future programs. The evaluation responds to the following five evaluation questions formulated by USAID: 1. To what extent did JSSP interventions in the 18 pilot courts lead to improvements in case management? a. What change, if any, was noted in the case management capacity in each court? Was it increased? 2. To what extent did JSSP interventions with non-governmental organizations and civil society organizations promote access to justice for the general public and outreach to the Office of Legal Aid? a. How has access to justice changed? b. What change, if any, was noted in the administration of justice, especially criminal justice activities? 3. To what degree have JSSP training activities been integrated by targeted recipients (judiciary police, magistrates, and court staff) into their respective work roles? 4. To what extent have the attitudes of the general public and media changed toward the Ministry of Justice and the legal aid office? a. What are the key barriers that still need to be addressed to effectuate a change in the perception of legitimacy? 5. How effective have JSSP tools been in raising awareness and improving public understanding of and access to legal information and the public defense fund? The evaluation team structured its findings, conclusions and recommendations around both the core evaluation questions and illustrative sub-questions and the set of intermediate results or components. Please see the Evaluation Matrix in Annex C. This evaluation aims to provide USAID/Côte d’Ivoire with a clear and comprehensive understanding of the effectiveness and achievements of the JSSP in order to guide future programming in democracy and local governance in Côte d’Ivoire. Please see the statement of work in Annex B. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 5 METHODOLOGY This performance evaluation used a non-experimental evaluation design using a mixed-methods approach to collect quantitative and qualitative data to respond to the five key evaluation questions formulated by USAID. Please see Annex C to review the Evaluation Design Matrix, detailing the data sources, data collection methodologies, analysis methods and perceived limitations at the inception of the evaluation process. To the extent possible, the team interviewed men and women for this evaluation. The smaller number of women surveyed in the exit poll is a reflection of the reality—fewer women than men entered or departed the courthouses during the four days when the survey was conducted. Before arriving in Côte d’Ivoire for three weeks of fieldwork, the evaluation team conducted a comprehensive review of program documents. The information gathered during the desk review helped inform the team’s understanding of the justice sector in Côte d’Ivoire and of JSSP’s programmatic approaches, opportunities, and constraints. This initial review also helped the team gather comparative data and gain a preliminary understanding of program goals, implementation plans, and performance monitoring efforts. The evaluation used quantitative and qualitative methods to systematically collect data. As detailed in greater length below, the team collected data through a desk review, in-depth interviews (IDIs) with key informants, group interviews (GIs), focus groups, and existing surveys. The team also reviewed and analyzed the indicator data collected by JSSP’s M&E unit. In addition, DI designed an exit poll survey of 1069 court users in 8 pilot courts (866 respondents in the intervention group and 203 respondents in two pilot courts serving as the control group to determine whether there were statistically significant differences among court users’ perceptions in intervention and control areas and, to assess, if relevant, the size of these differences. By utilizing multiple evaluation methods and triangulating results, the team aimed to strengthen the validity of its findings and answer the evaluation questions and illustrative sub-questions in the Scope of Work. JSSP incremental implementation model gave the evaluators the opportunity to make comparisons between jurisdictions that had benefitted from the program for over three years and jurisdictions that benefitted from the intervention later. Figure 1 below provides an overview of JSSP implementation strategy. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 6 Figure 1. JSSP phased-in implementation model The intention of this evaluation design was to clarify the effects of the program by comparing results of pilot courts in different stages of implementation and also compare these results with control courts where JSSP did not intervene. The overall goal of this evaluation is to inform USAID, other donors, and the Ministry of Justice policy and decision-makers on how to use results of this evaluation and make recommendations for further programming in the justice sector. SAMPLING METHODS Given the implementation model adopted by JSSP, the team collected data in jurisdictions and courthouses at different stages on implementation: locations where the project has been present for close to four years (Bouake, Man, Yopougon); one courthouse where the project had a permanent presence from 2013 to 2015 (Sinfra); two courthouses where the project has operated on a permanent basis since the summer of 2015 (Agboville and Boundiali); and the two control jurisdiction (Aboisso and Divo). In this manner all implementation arrangements were included. Most of the interviews with ministry officials and project partners and beneficiaries located in Abidjan were conducted during the first week of field work. This allowed time for officials in Abidjan to assure those outside the capital of the evaluation’s legitimacy and provided the team a larger picture from the ministry’s perspective before embarking on a fact-finding mission in the various courthouses. 2013-2017 BOUAKÉ DABOU MAN YOPONGOU 2016-2017 AGBOVILLE BONDOUKOU BOUNDIALI DALOA GUIGLO KORHOGO SEGUELA 2013 – 2016 ABENGUROU ALAKOTA BOUAFLE BOUNA DIMBROKO GAGNOA SINFRA Control ABOISSO DIVO COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 7 For the collection of quantitative data, the evaluation team developed an exit poll survey to administer to court users. The survey was conducted by CREDFI and selected each 4th or 6th (where the universe of court user is larger) person entering or departing the courthouse. The surveys were conducted either in French or in Dioula according to the individual’s preference. The smaller number of women surveyed is a reflection of the reality—fewer women than men entered or departed the courthouses during the four days when the survey was conducted. The survey includes questions designed to assess public attitudes toward the Ministry of Justice, perceptions of the Office of Legal Aid, awareness and understanding of the Public Defense Fund, and general knowledge of rights and how to exercise those rights, and the ease of access to justice in Côte d’Ivoire. The survey adopts an “exit polling” strategy rather than a national probability sample, which reduces the extent to which survey results can be generalized to the entire population. The questions guiding the JSSP evaluation specifically address perceptions of individuals who have accessed the courts. Global survey research shows that, in general, very few people have direct contact with courts of law outside of obtaining basic services, like issuance of identity documents. The same is true of Côte d’Ivoire. 2015 Afrobarometer data show that only 6% of the 1,199 individuals sampled in Côte d’Ivoire have direct experience with the courts. Therefore, a national probability sample would not return sufficient number of court users to be meaningful to the JSSP evaluation. DATA COLLECTION For this evaluation we used a mixed-method approach, which integrates different evaluation methods, at every stage of the evaluation process, drawing on both quantitative and qualitative data. The success of the method utilized—as is the case in any evaluation—depends greatly on the quality of data collected or already available. All quantitative or qualitative studies rely on the data being of a sufficiently high quality, and that requires regular data quality checks. The findings reported in the ‘Findings’ section below are based on three sources of data: (1) JSSP M&E data on indicators provided regularly to USAID; (2) the fieldwork (IDIs and Observations) conducted by the evaluation team during a three week period in-country; and (3) the field work conducted by CREFDI (exit poll survey and focus groups). The mixed method approach implemented by the evaluation team to collect the information needed to answer the evaluation questions is detailed and discussed below In addition, the evaluation team collected primary quantitative and qualitative data from a number of sources including USAID staff, project implementers, project beneficiaries, NGOs, and other key stakeholders. By triangulating information from these different sources of information, the team was able to respond to the key evaluation questions, substantiate their findings and conclusions, and draw informed recommendations on how to improve overall project results. By triangulating information from these different sources of information, the team was able to respond to the key evaluation questions, substantiate their findings and conclusions, and draw informed recommendations on how to improve overall project results. Desk Review. Before arriving in Côte d’Ivoire the evaluation team conducted a comprehensive review of program documentation, which included the JSSP proposal, and the reports submitted to USAID as well as documentation produced by other donors. The team reviewed documents COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 8 describing the judicial system in Côte d’Ivoire and existing government documents and plans related to the judicial system. Reports and papers written by specialists in judicial reform were also reviewed. The information gathered in this manner helped to inform the team’s understanding of the justice sector in Côte d’Ivoire and of JSSP’s programmatic approaches, opportunities, and constraints. This initial review also helped the team gather comparative data and gain a preliminary understanding of program goals, implementation plans, and performance monitoring efforts. The team also reviewed JSSP M&E reports that track the performance of project indicators to demonstrate the achievement of targets and objectives. The evaluators used the raw existing data to conduct further analyses that focused on the outcomes of the project on its beneficiaries in order to provide USAID, the Ministry of Justice, and other stakeholders with information that facilitates decision making. Interviews with key informants. The evaluation team conducted 58 in-depth interviews with key informants and with stakeholders in Abidjan, Agboville, Boundiali, Bouaké, Man, Yopougon, and Sinfra. The interviewees included USAID staff, project implementers, project beneficiaries, NGO leaders, and other key stakeholders. Annex E provides a list of the 58 stakeholders interviewed for this evaluation. These detailed interviews—the qualitative segment of the multi-mixed evaluation approach— constitute the basis of the conclusions later reached by the team. At the end of field work, the team read the interview notes, summarized and coded the information, and identified recurrent themes or ideas. Then it compared the information obtained through different interviews with data obtained from results of the focus groups, the exit poll survey and JSSP M&E reports. Exit Poll Surveys. DI designed an exit poll survey instrument to be administered to court users by its sub-contractor CREFDI. The survey included questions designed to assess public attitudes toward the Ministry of Justice, perceptions of the Office of Legal Aid, awareness and understanding of the Public Defense Fund, general knowledge of individual rights and how to exercise those rights, and the ease of access to justice in Côte d’Ivoire. The selection of the jurisdictions provides a spectrum of JSSP implementation strategy. The exit poll was conducted in courts where the project intervened—Man, Bouake, Yopougon, Sinfra, Agboville, and Boundiali—and in courts where the project was not present—Divo and Aboisso (control jurisdictions). Participants were approached as they entered and departed the courthouses. 1,069 court users took the survey, 866 in the intervention and 203 in control jurisdictions. All were 18 years of age or older. Of those who took the survey, 74% in intervention jurisdictions and 85% in the control jurisdictions were male. Table 1. Jurisdictions selected for the exit poll survey Treatment (2013—2017) Bouaké, Man, Yopougon, Treatment (2013—2015) Sinfra Treatment (June 2016—2017 Agboville, Boundiali Control Divo, Aboisso Source: CREFDI report COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 9 The exit poll survey was conducted in these eight courts on April 14, 18, and 19. Of those entering or departing the court each 6th person in some jurisdictions and 4th in the smaller jurisdictions was approached and asked to take the survey. The overall refusal rate was 29%. Please see Annex H for the results of the exit poll survey. Focus groups. In addition to administering the exit poll survey CREDFI conducted focus groups in six communities— Agboville, Bouaké, Boundiali, Man, Sinfra, and Yopougon. These locations were selected because they were the same where the evaluation team would be conducting qualitative interviews with court officials making triangulation of the data possible. EVALUATION LIMITATIONS Selection Bias In all qualitative research, there is a possibility of selection bias because respondents are selected purposively, not randomly, and therefore, evaluation findings are not generalizable across the population. People selected for interviews may have preconceived information about the project being evaluated, and their perceptions may differ from those individuals not interviewed. The evaluation team mitigated this by stressing the independence of the evaluation process, explaining the purpose of the evaluation, and ensuring confidentiality. In addition, the Team triangulated responses from respondents in different courts and contrasted these responses with results from the exit poll survey. Halo Bias There is a known tendency among respondents to under-report socially undesirable answers and to alter their responses to approximate what they perceive as the social norm. The lack of priority given to the justice sector by the GoCI and its heavy dependence on foreign donors resulted in a bias on the part of ministry officials, who do not wish to be too critical of the project. Most meetings ended with words of praise and a wish for a continuation of USAID support to the judicial sector and a JSSP follow-on. However, the court officials interviewed were more candid in their responses. As the mission progressed, criticism from the interviewees in the pilot courts of other informants such as foreign aid counterparts and/or occurrences such as the monitoring committee on computerization started to indicate trends. With these trends as evidence of possible dissatisfaction, the team was able to elicit more candid responses from ministry officials. Acquiescence Bias This is a type of cognitive bias where respondents answer questions in the way they think the interviewer wants them to rather than according to their true beliefs. Although it is impossible to determine the extent of this bias in the present evaluation, it is conceivable that respondents may have either been reluctant to criticize the program during interviews and focus groups or had an incentive overstate the effectiveness of program activities in the expectation that this would bring additional donor support. The team minimized this bias by reminding respondents of the independence of the Evaluation Team and the importance of eliciting candid and honest responses. Time and resource constraints Available resources and the short data collection period limit the scope of an evaluation and the amount of data that can be collected. As mentioned before, the evaluation team was only able to visit six pilot courts out of a total of 18 courts benefitted by the program. Generalizing results of COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 10 the evaluation is difficult, but by comparing courts at different stages of implementation, the Evaluation Team was able to minimize this bias. In addition, the team triangulated information obtained through interviews and focus group discussions with results of the exit poll survey. Co-intervention bias This type of bias occurs when program beneficiaries are simultaneously benefitting from other (unaccounted for) interventions and it is difficult to determine additional marginal impact of program activities. In Côte d’Ivoire, several donors have been working in justice support programs. To mitigate this bias, the team conducted a review of the implementing environment and donor activities in the justice sector, seeking to understand programmatic overlap as well as potential synergies. As detailed in the Annex D, the team interviewed members of the EU, the AFD, and PAJAJ who had implemented projects in the justice sectors. Comparability and reliability of data JSSP’s baseline data was only conducted for 10 pilot courts, making comparisons between baseline data and actual results of all 18 pilot courts impossible. Moreover, the exit poll was only conducted in 8 courts, so comparisons between the baseline data and the exit poll survey is similarly impossible. The team used the survey data to compare results from courts where the project had intervened (intervention courts) against courts where the project did not intervene (control courts) and triangulated information collected during interviews and focus groups with results from the exit poll survey. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 11 FINDINGS The findings presented in this section are based on the analyses of information from several sources, including desk review; raw data made available by the JSSP M&E system (please see Annex D for a complete list of JSSP indicators); 58 interviews with key informants; and court exit polls and focus groups as reported by CREFDI. To facilitate the smooth interpretation of this document, the findings are organized by the USAID-formulated evaluation questions presented in the Statement of Work. Evaluation Question # 1: To what extent did JSSP interventions in the 18 pilot courts lead to improvements in case management? Evaluation Question #1(a): What change, if any, was noted in the case management capacity in each court? Was it increased? 1. Overall, the five new tools designed for criminal proceedings and introduced by JSSP—(i) the Criminal Registry Office (Bureau d’ordre pénal “BOP”); (ii) the Criminal Justice Unit (Pôle judiciaire pénal “PJP”); (iii) the detention tables for examining judges; (iv) the procedural templates; and the new tool designed for overall court administration, (v) the Quality Control and Management Unit (Unité de contrôle de qualité et de gestion, UCQG)—are well accepted in pilot courts.5 Informants revealed that, with the notable exception of the Courthouse of Yopougon, where these tools are facing a resistance from the prosecutor’s office, the case management tools are well accepted and applied by the various users and applied. Interestingly, the potential spillover effect of these tools was also highlighted by an interview with a prosecutor in a non-pilot court interviewed (the prosecutor in Toumodi) who through continuing education and trainings had been introduced to the procedural frameworks and the BOP in February 2017. He had also just attended training on the PJP and stated that he was “amazed” by the potential of these new tools and hoped to use them in his courthouse. 2. The Criminal Registry Office (Bureau d’ordre penal) is implemented well and has been accepted in the various courthouses, with the exception of Yopougon. This tool, designed for the prosecutor’s office, is basically an excel spreadsheet centralizing all event dates in the life of a criminal case. It is updated on a daily basis to include minutes, complaints, and other legal acts. Overall, the prosecutors interviewed praised the BOP as a useful tool that gives them a clear picture of the status of all criminal matters and a “traceability of all proceedings which enter the prosecutor’s office docket”, as well as a tool to collect court statistics. However, except in Boundiali where the BOP is used by the Court coordinator to generate monthly statistics, it appears that it is more the BOP’s potentiality for statistical purposes that are praised rather than its actual statistical use. In other courthouses, statistics are collected using the BOP only for the sake of inspections by the general inspectorate, neither for the purposes of court management nor for sending statistics to the evaluation and statistics 5 Though not directly related to case management, the transparency tools developed by JSSP: solutions boxes, Court Fee posters and Court signage are also very well accepted. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 12 directorate. The production of statistics in the courthouses and at the ministry level is still unreliable and in its infancy (a “total void” according to the director). For example, the directorate of evaluation and statistics created in 2013 has only been in operation for a few months. The prosecutor in Bouaké, a strong advocate of the BOP and of other JSSP tools, described the BOP as a tool allowing him to “inspect himself,”that is, to implement good practices in time processing and quality control that are typically sanctioned by the general inspectorate. However, the BOP is rejected in Yopougon by the prosecutor’s office mainly because of the intermittent presence of the clerk entering information on the BOP. The prosecutors in Yopougon consider that the BOP slows down their work, risks generating friction with the general inspection and, because of that, the prosecutor’s office does not rely on it. It also appears that the BOP is no longer in use in Bouna, which is one of the seven courthouses where JSSP operated on a permanent basis until the summer of 2015. The president of the Boundiali courthouse, who used to be the resident prosecutor in Bouna, reported that the BOP gradually stopped being used there after the justice coordinator departed and has now been completely abandoned after his departure. Other issues discussed during the interviews included the suspension of the BOP in Man from November 2016 to January 2017 after the computer on which the BOP was installed broke down. This is an indication of the limitations of procedures that require the use of technology. 3. The Criminal Justice Unit (Pôle judiciaire pénal) is used as a tool of dissemination in the courthouse’s jurisdiction and is credited with a jump in quality of the work performed by judiciary police officers.6 The PJP interacts with the joint trainings/JPO/Magistrates and points toward an improvement in the work done by the judiciary police. The PJP is a regular meeting held among actors of the judiciary police and judicial actors involved in criminal proceedings. In courthouses where there is a court coordinator, the coordinator attends the PJP meetings. Except in Yopougon, where the PJP is not well accepted in the prosecutor’s office, the PJP is welcomed by prosecutors and examining magistrates (“EJ”).7 In particular, the EJs consider that the PJP is an essential tool to “put them on the map” with JPOs who, until the PJP institution, tended to neglect and snub them in favor of the prosecutor, their sole interlocutor in the judiciary in their eyes. Thanks to the PJP and joint training, several examining judges have confided that the judiciary police officers now understand who they need to report to (the EJ, and not the prosecutor) when acting on a rogatory commission. Except in Yopougon, the prosecutors think that, as a result of the PJP and joint trainings, the JPOs perceived the prosecutor’s office more as a director of investigation than the next step in the procedure, or someone to whom to report only when “the police does not know what to do anymore.” Certain prosecutors consider that the PJP has had a virtuous effect on the level of illegal criminal custody, although they avowed that this was difficult to measure. 6 There is no available quantitative data to support his finding, neither in JSSP M&E plans nor in the surveys conducted for this evaluation.. The information collected to support this finding derives from interviews with stakeholders. 7 According to one interviewed prosecutor, the instructions decided at the PJP are never enforced and they deplore the fact that the police commissioner delegates to one of his subordinates, showing a low priority on the part of the police there. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 13 4. The procedural templates are useful tools established by a pool of Ivorian judges (both prosecutors and bench) to ensure compliance with criminal procedural steps. For example, they standardize the form of “soit-transmis” whereby the prosecutor requests some information from an administration (e.g. a gendarmerie brigade or a prison administrator) or asks the police to initiate a preliminary investigation. Like the BOP and the PJP, the procedural templates appear to be generally accepted and used by the various actors in a criminal proceeding and the PJP, here again, is a channel through which the procedural templates are disseminated to the actors who should use them.. At the courthouse of Yopougon prosecutors are reluctant to use the templates and continue using the old forms. 5. The preventive detention monitoring tables are a very simple feature consisting of wall￾mounted metal boards that show and manage pretrial detention times.8 This provides the examining judges a simple tool to ensure that the detention warrants (mandats de dépôt) are processed on time.9 This is meant to resolve a chronic problem in Côte d’Ivoire, where a very large part of detainees are in pre-trial detention. 10 Even though in theory, Ivorian law mandates that when a pretrial detention warrant is not processed or renewed on time the detainee should be released from jail, in practice, detainees, unaware of their rights, fail to ask to be released and remain in detention until their case is processed. Judging from the discussions with all the examining judges met, including the dean of examining judges of Yopougon, this tool is very well accepted by its users and highly praised. The examining judges interviewed all displayed amazement at how a simple tool can help mitigate such a complex and serious problem. We use the word mitigate only, as even though the detention tables appear to be a huge improvement in examining judges’ chambers, several judges interviewed deplored the long delays involved when a special renewal is requested before the accusation chambers in criminal matters. 6. The departure of the court coordinators has caused problems in some jurisdictions. The role of the court coordinators was to monitor and support the new tools. Their move to new courthouses has caused some negative effects—in Sinfra, for example the tools continue to be used in spite of the departure of the court coordinator while in Bouna they no longer seem to be used. The difference between Sinfra and Bouna is traceable to the circumstance that the same prosecutor, chief clerk, and president have stayed there since the beginning of the project while Bouna has suffered from a much more important turnover (e.g. the BOP has not survived the departure of the head prosecutor to Boundiali)11 . 8 See a picture of a detention table in ProJustice’s Quarterly Progress Report July-September 2014 and Annual Progress Report April 2013 – September 2014, p. 11. 9 These are not renewable in felony cases and could be renewed only once without a special authorization from the accusation chamber in criminal cases (e.g. murders, rapes). 10 This is a chronic issue on the whole African continent, which has one of the lowest proportions of detainees of all regions, yet, where the prisons are over-crowded with detainees in pretrial detention. See the following article, in which a comment from the head of a leading Ivorian NGO reports that about 45% of Ivorian detainees are in pre-trial detention. https://www.opensocietyfoundations.org/voices/raising-profile-pretrial-detention-africa. See also the documented report from OSI, entitled Presumption of Guilt: the Global Overuse of Pretrial Detention, available at: https://www.opensocietyfoundations.org/publications/presumption-guilt-global-overuse-pretrial-detention. 11 Tetra Tech DPK rightly points out that “the reasons for changing 7 pilot courts out of 11 and moving court coordinators to the new ones at the end of 2015 was not at all a termination of donor funding, but rather a decision COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 14 7. One important challenge to the use of the new tools is the lack of resources and the lack of maintenance of equipment. Prosecutors and other actors complain about not having even functioning copiers to make copies of the procedural templates. They also complain about the deficient means of communications to transmit the filled templates from the police to the prosecutor’s office and vice-versa when the telephone landline is cut off or not in working condition. Instead of sending instructions by fax, they are given orally over cell phone, resulting in issues of traceability in the criminal investigation. 8. The UCQG integrates solutions of demand-side governance in the court management by using the suggestions boxes to determine the UCGQ agenda. Well accepted everywhere, this tool, which is a regular meeting chaired by the president during which issues regarding the courthouse management (including time processing, reception of the court users etc.) are addressed, appears to be taking place as planned in all the courthouses visited. In courthouses where the suggestion boxes are being used and processed by the court coordinators, the suggestion boxes are used to determine the meeting agenda. In courthouses where there is a court coordinator, he/she redacts the minutes of the meeting, sometimes concurrently with the chief clerk (greffier en chef). In courthouses where there is no longer a court coordinator (e.g. Sinfra), the UCQG remains in place, but its agenda is not based on the suggestion boxes, which have fallen out of use after the justice coordinator left in the summer of 2015. 9. A computerized case management application (SIGP or Système d’information pour la gestion des procédures judiciaires et des actes administratifs) has been developed and is being experimented in two pilot courts by a contractor. This component was originally designed to be funded by the EU, but it was transferred to USAID as a result of an agreement entered into in February 2014 by which PSSJ provided assistance to the MOJHR for the preparation of a master plan for the computerization of the judicial and penitentiary systems. Interviewees, including those in the donor community, consider that the plan provides a good foundation but needs to be refined (AFD expert). An external contractor led by an Ivorian State-owned enterprise, the SNDI, which had developed the integrated information system of the ministry of Finance (SIGFIP), was selected to develop software adapted to the Côte d’Ivoire judicial system. The development of the software started in April 2016. At the time of the evaluation, the application had been developed and was under deployment in the two pilot courts— Yopougon and Dabou—since January 2017. 10. It is too soon for the evaluation team to evaluate the impact of the computerized CMS but some issues concerning sustainability are already apparent. The evaluation team attended a meeting of the computerization monitoring committee, which includes stakeholders from the CMS experiment. That meeting revealed that (i) the MOJHR is not providing all the necessary equipment required for the experiment to be conducted (e.g., cabling problems); (ii) there are conflicts between the head of the IT service of the MOJHR and the consortium, the former accusing the latter of not performing well its transfer of skills, the jointly planned by USAID and GoCI dating back to the Request for Applications and also since the launch of JSSP in 2013 as a way to maximize the impact of JSSP by including more jurisdictions throughout the life of the project. Thus, over a 5-year period, JSSP was intended to cover 18 jurisdictions representing most geographical areas of Côte d’Ivoire. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 15 latter reproaching the former for not being cooperative; (iii) certain court officials (particularly the prosecutor’s office in Yopougon) consider that the computerization effort adds undue burden onto their overstretched schedules. Interviewed after the meeting, the evaluation team interviewed separately the head of the IT service and of the consortium. The head of the IT service informed the evaluation team that this service responds directly to the chief of staff and was staffed with 10 technicians with little decentralization in courthouses. The representatives from the consortium confirmed that from the beginning they have had difficulties obtaining involvement on the part of the technicians from the MOJHR. According to them, the head of the IT service himself does not lead by example by not attending trainings and that the general repeated strikes in the civil service made matters worse as no technician from the MOJHR was available at all in January during the national strike. There is concern regarding the viability of the project after the two-year guarantee (which involves maintenance of the system) expires in 2020. As reported by the representatives of the consortium, if the lack of cooperation on the part of the MOJHR is not resolved very soon, the ministry will have to use an external contractor after the expiration of the period of guarantee. The CEO of the SNDI, who is a high level civil servant, has asked for a meeting with the MOJHR’s chief of staff to discuss the situation with the ministry’s IT service. The consortium also expressed concern that as JSSP ends there could be a further reduction on the willingness of the judiciary officials to cooperate. The evaluation team visited the courthouse of Yopougon and sat for a demonstration with the only clerk currently using the SIGP. Given the size of the courthouse’s docket in Yopougon, the computerization is a phased-in implementation, procedure by procedure. We observed several technical issues already reported to the monitoring committee and were told that, as a result of these issues, the clerk performs double entry in order to anticipate eventual losses of information. Except from the technical issues, the application is simple to use and the clerk was well accustomed to it but this was a very small portion of the whole courthouse’s docket. 11. Based on interviews with the MoJHR officials, the evaluation team believes the MoJHR is not fully involved in the goals of this project. The donor community, on the other hand, is investing more in providing hardware (computers) and software than on providing technical assistance.12 During the other interviews with officials from the MOJHR (e.g., the director of civil and criminal affairs, the chief of staff, the inspector general), the evaluation team raised questions regarding the computerization experiment which elicited some lofty aspirations such as “…thanks to the computerization, the Ivorian justice is entering into the 21 st century…” or “…the computerization will solve our case management problems...” When asked whether they realized that computerization was a difficult process, that there had been many experiments of case management systems with a poor track record of success, even in OECD jurisdictions with such applications, the response invariably was: “…but there 12 Tetra Tech DPK challenged this finding about lack of political will arguing that “the MoJHR may lack enough technical, financial, human and material means to implement a thorough IT policy, but there is a real will to move forward on the computerization process of the judicial system.” COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 16 are donors involved, first and foremost the French Agency of International Development…” When asked about their plans concerning the computerization, the French Agency of International Development said that they were planning to invest in hardware and equipment (cabling, computers, and servers) but had no plan to fund any technical assistance. The AFD had an STTA conduct a diagnosis of the IT question at the MOJHR. The STTA’s recommendations were that the MOJHR engage in institutional reorganization to integrate the new IT dimension into the full jurisdictional network and not simply at central administration. Neither the AFD nor the EU seem to be much aware of how the development and experimentation of the application has been conducted and, though they are aware that the MOJHR does not have the resources to get to the next step, they were totally unaware of the poor level of cooperation between the consortium and the MOJHR. 12. There is not a statistically significant difference in reduction of case backlog between jurisdictions that have participated longest in the program— Bouaké, Dabou, Man and Youpogon—and the two control jurisdictions—Aboisso and Divo. This means that the percentage of reduction in case backlog would have been the same without the program. The evaluation team used the baseline data collected by JSSP in 2016 and included the Jan￾Mar 2017 data to conduct an analysis of the available data for indicator 1.2.1 and compare the performance of the intervention and control jurisdictions. Intervention jurisdictions started the period with 17,939 cases and showed 13,148 remaining at the end of the period. Control jurisdictions started with 4,194 cases and showed 2,921 cases remaining at the end of the same period. The difference observed between the means of jurisdictions that have participated in the program the longest—Bouaké, Dabou, Man and Youpogon—and the two control jurisdictions—Aboisso, and Divo is not statistically significant as shown in Figure 2. Figure 2. Percentage reduction in case backlog Source of data: JSSP M&E system 13. According to data collected by JSSP, the level of satisfaction of court users with services provided by the courts varies significantly from court to court and is not positively correlated with activities implemented by the program. JSSP tracked user satisfaction with services provided in the courts as an indicator to measure greater public confidence in the justice system, one of its overall objectives. In the first quarter of 2017, JSSP surveyed 2080 26.7 30.3 0 20 40 60 80 100 Intervention Control % COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 17 court users in control and intervention jurisdictions—1496 men and 326 women.13 Respondents who selected 1 (very satisfied) or 2 (satisfied) on a 1 – 5 scale for at least six items were considered to be satisfied with the services provided. Figure 1 shows the level of satisfaction in all jurisdictions in the period between January and March, 2017. The courts marked in red are those where JSSP worked since 2013. The courts in blue were added in 2016 and the ones in green are the control group. As can be seen in the Figure below, the level of satisfaction is lower in jurisdictions where the program has been implemented the longest. In control jurisdictions, the level of satisfaction with court services is high, even though these jurisdictions have not received benefits from the program. Figure 3. Percentage of users surveyed reporting satisfaction with court-provided services Source of data: JSSP M&E system Still, these results must be viewed with caution. Selection bias is a problem in surveys of service users because only those who agreed to be surveyed are represented. These individuals may differ in important aspects from those who did not agree to participate, and may differ even more so from prospective court users who did not visit the courthouse. Moreover, the reasons for going to the courthouse vary from person to person in many ways – applying for an ID card, taking care of land certificates, and facing criminal justice procedures are likely to leave different impressions on satisfaction ratings. When leaving the courthouse some individuals are likely to be more satisfied than others depending on the reason for their visit—applying for an ID document is certainly easier and less bureaucratic than getting the title to a piece of land. 13 The low percentage of female participation reflects the fact that, in all jurisdictions, most courts users are male. With the exception of the 2nd and 4th Quarter of 2016, JSSP did not provide disaggregated figures by gender on most of its indicators. On those two quarters, where disaggregated figures are available, the level of satisfaction between males and females is virtually the same. See JSSP M&E reports. 87% 77% 87% 71% 93% 89% 98% 87% 91% 93% 92% 83% 98% 0% 20% 40% 60% 80% 100% 120% Bouaké Dabou Man Yopougon Agboville Bondoukou Boundiali Daloa Guiglo Korhogo Séguéla Aboisso Divo 2013-2017 2016-2017 Control COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 18 Results from DI’s exit poll also suggest a negative conclusion about JSSP influence may be premature. There is minimal substantive difference and no statistically significant difference between satisfaction with courts as expressed by exit polled court users of JSSP courts and other, non-beneficiary courts. Instead, users who accessed justice at JSSP courts were more likely to respond that they were “treated fairly – without taking into account ethnicity, sex, social and financial status or age,” although these differences are not significant with 90 percent or more confidence. These findings are not enough to tip the balance to a positive conclusion about JSSP intervention effects on levels of satisfaction with the justice system, but they reveal that JSSP M&E data do not tell the whole story. Evaluation Question # 2: To what extent did JSSP interventions with non-governmental organizations and civil society organizations promote access to justice for the general public and outreach to the Office of Legal Aid? 14. NGOs reported a “thirst for rights” (the existence of the rule of law) on the part of the population but said that citizens remain defiant towards justice institutions. NGOs reported both during the group interview and individual interviews on-site, having received some negative feedback from opinion leaders and persons from the general public on the political will for reform on the part of justice officials. These people expressed doubts regarding GoCI’s willingness and ability to fulfill the judicial assistance promises to the full extent of the communications made during the awareness campaigns. This is confirmed by the focus group qualitative survey conducted by CREFDI where the investigation shows that “…in terms of judicial assistance, we note that the awareness of court users in this study is mitigated by the possibility to actually obtain judicial assistance…” COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 19 There appears to be a difference in perception of the judicial assistance in various parts of the territory. In Bouaké, Agboville, Boundiali and Man, citizens understand without ambiguity the principle of judicial assistance. However, in Sinfra, Man and Yopougon, “a majority of users insist on the impossibility of having recourse to judicial assistance when needed.” Furthermore, court users display “total ignorance concerning who they need to contact as well as the procedures they need to follow to benefit from this assistance.” Respondents in this investigation report that they ignore the Office of judicial assistance’s existence and its mission or the judicial protection fund of the MoJHR and its usefulness. The numbers reported by the Office of legal assistance confirms this: between June 2015 and January 2017, only 191 applications for judicial assistance were received, mostly from the Abidjan region. 15. In certain localities, community’s opinion leaders demanded payments for participating in the awareness campaigns. Among the difficulties reported by the NGOs, was the stiff competition from other donor-funded programs to attract opinion leaders. This is true particularly in the West, which, according to one NGO representative, has been “flooded with donors’ money.” As a result of this, the NGOs noted that it has become common practice for opinion leaders to demand payment for participating in meetings. 16. NGOs reported MoJHR’s lack of cooperation regarding events taking place in the courthouses where the project operates. Another operational issue voiced unanimously by the four NGOs conducting awareness campaigns is the requirement from courthouses that a letter from the MOJHR be presented before activities in their courthouses are allowed. NGOs complained about undue delays in obtaining such authorization from the directorate of civil and criminal affairs at the MOJHR.14 Evaluation Question # 2 a: How has access to justice changed? 17. Overall, as shown in Figure 5, evidence collected in the field shows a trend towards greater use of the judicial system. Please note that the figures shown below take into account all individuals who seek the courthouse regardless of the service requested: provision of ID cards, certified document as well as the resolution of criminal cases or other that require judicial procedures. Please note that August is the period of vacation of the judiciary. 14 Tetra Tech DPK contended this finding stating that “the only activities for which authorizations from the MoJHR are needed are the open house days. As this activity requires a special organization in the tribunals (suspension of hearings, presentations and conferences, magistrates and judicial personnel mobilized to guide the public and answer questions) the MoJHR needs to plan with the tribunal concerned. This has never caused any delays in the activity, as authorization is sought and granted ahead of time.” COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 20 Figure 5. Trend noticed in number of individuals using the courts15 Source of data: JSSP M&E system 18. The introduction of transparency tools such as the court signage and the court fees posters has contributed to the diminution of the phenomenon of intermediaries named “margouillats.” In some courts, these two tools implemented by JSSP limited the phenomenon as “margouillats”, or informal intermediaries who collect fees from court users to guide them in the courthouses (this sometimes involves court personnel). The interviews conducted in Agboville, Boundiali and Man indicated that the phenomenon was decreasing and this was expressly correlated with the signaling and the posters displaying the various fees. However, both the interviews and the focus group discussion groups conducted by CREFDI revealed that these measures have had much less effect in Bouaké and Yopougon where, the practice of using informal intermediaries still appears to be endemic. Concerning Sinfra, the phenomenon was reported to be rather insignificant during the interviews conducted with the court officials. Although the MoJHR does not collect statistics and therefore, measuring the reduction of these intermediaries is difficult, there is no question that decreasing the impact of informal intermediaries translates in decreasing the cost of access to justice for court users. Evaluation Question 2 b: What change, if any, was noted in the administration of justice, especially criminal justice activities? 16 19. Changes in the administration of justice seem to vary from one jurisdiction to the next making it difficult to talk about an overall change as a result of JSSP activities. Improvements were noted in the population survey in Agboville, Boundiali, and Man, but not in Bouake, Sinfra, and Yopougon. On courthouse reception and time processing requests, users tend to have an appreciation of improvements in the courthouses of Agboville and Boundiali which are both new pilot courts and in Man, where JSSP has been operating since 2013. Surprisingly, the report produced by the JSSP M&E system for the first quarter of 2016 reports that Agboville has one of the lowest rates of backlog case processing but, paradoxically, the exit poll survey found that Agboville shows one of the highest level of perception of court improvement. No improvement is observed in the qualitative or 15 Information on court users was not disaggregated by gender in JSSP’s M&E reports. JSSP consistently surpassed their target on this indicator. By Year 4, the number of court users increased 75 percent against the baseline. The target was a 10 percent increase against the baseline. See JSSP M&E reports. 16 This question is addressed on page 13 of the CREFDI qualitative report in which four main parameters have been observed (reception, time processing, corruption and fairness and impartiality of the services). 92,549 117,871 100,200 89,396 132,872 Oct-Dec 15 Jan-Mar 16 Apr-Jun 16 Jul -Sep 16 Jan-Mar 17 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 21 quantitative study conducted by CREFDI in the courthouses of Bouaké and Yopougon where the project has had a permanent presence since 2014 and in the courthouse of Sinfra where the project had a permanent presence until the summer of 2015. This finding is confirmed by our own quantitative analyses of JSSP data, which showed no significant differences between intervention and control jurisdictions (see Figure 6). When comparing across all JSSP courts and non-beneficiary courts, few changes in overall satisfaction with services can be ascertained. And striking inequalities exist between JSSP and other court users’ perceptions of fair treatment relative to their most recent other visit to the courthouse. However, increased perceptions of fairness are more prevalent among non￾beneficiary court users than among JSSP court users. This could be a function of increased awareness among JSSP court users about the quality of treatment they should be able to expect; however, with descriptive statistics alone there is no way to carefully explain this statistically significant anomaly. 20. The exit poll survey of court users shows that court users’ perception on corruption in the court has improved in the courthouses of Agboville, Boundiali and Man but not in other courts involved in the evaluation. These jurisdictions show court users see reduction of corruption practices, while corruption is perceived to continue to be a problem in Bouaké, Sinfra and Yopougon (see Figure 7). Regarding fairness and impartiality of the judicial system, the same trend is observed with Agboville, Boundiali and Man on the rise while Bouaké, Sinfra and Yopougon on the downturn. At the same time, citizens in JSSP court communes are not necessarily more likely to feel informed about their rights, and express significantly less confidence that they would be know how to report a case of judicial corruption, such as being asked to pay a bribe by a court official (see Table 2). Figure 6. Court performance perceptions over time COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 22 With respect to citizens’ knowledge on how to report judicial abuse and corruption cases, the exit poll surveys shows that a high percentage of court users do not know how to report corruption cases. See Table 2 below.17 17 ProJustice reports comparable percentage of public knowing how to report on judicial abuses and corruption cases in Bouaké and Boundiali, but significantly higher ones in Agboville, Yopougon and the control courts. Courts April 2016 Oct. 2016 January 2017 Agboville 25% 32% 24% Bouaké 25% 13% 15% Boundiali 22% 11% 15% Man 26% 34% 28% Yopougon 15% 13% 25% Aboisso 34% 39% 34% Divo 21% 11% 34% Figure 7. Exit Poll Survey on Perceptions of Corruption COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 23 Table 2: Percentage of Court Users Reporting Knowledge of Corruption Reporting Procedures If an agent at the courthouse is soliciting a bribe and you wish to report this to the authorities, would you be sure to know how to report the behavior? Place Yes No Do not know/Cannot read Total Yopougon 15.4% 81.4% 3.3% 100% Bouaké 14.5% 84.9% 0.6% 100% Man 19.1% 79.8% 1.1% 100% Divo 24.4% 72.3% 3.4% 100% Agboville 12.3% 87.7% 100% Aboisso 20.2% 75% 4.8% 100% Sinfra 7.7% 89.7% 2.6% 100% Boundiali 13.3% 84.9% 2.2% 100% Total 16.4% 81.1% 2.5% 100% Source: CREFDI, Exit Poll Survey. 21. The suggestion boxes installed by JSSP are used to determine the agenda of the UCQG meetings. JSSP has installed suggestion boxes in all pilot jurisdictions and in the central ministry to allow the population to express itself on issues of justice administration. The boxes were installed in various locations in the pilot courts as well as in other locations— town hall, legal clinics, the prefectorate, and the directorate of civil and criminal affairs in Abidjan. The intention was to provide a direct forum of expression for the population on questions as diverse as reception of the public, corruption, time it takes to process a case, among others. At least in Boundiali, Man, Agboville and Bouaké, the analysis from these boxes is used to prepare the agenda of the UCQG meetings. The suggestion boxes are not in use anymore in Sinfra. This was not mentioned and discussed concerning Yopougon (the first official who highlighted the practice was the president in Boundiali whom the team interviewed after the visit in Yopougon). 22. JSSP support of the general inspectorate has led to meaningful changes and the inspection is perceived as and example of good practice. Assistance to the general inspectorate has led to an increase in the number of inspections. In fact, it can be said that the inspection was reborn in 2012. Based on the information collected, all courthouses have been inspected twice since 2013 and in most courts visited an inspection had been completed within six months from our visit and for the first time, judges and clerks have been disciplined. Also, and this is specifically linked to JSSP, the inspectors have received trainings about inspecting offices of ministerial justice officers (bailiffs and notaries), who play an extremely important role in the enforcement of judicial decisions and land rights. Following these trainings, the general inspectorate has conducted inspections in offices of 30 notaries and 80 bailiffs. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 24 Importantly, notaries and bailiffs have also received trainings funded under JSSP. Finally, some efforts have been made in awareness campaign conducted by NGOs to explain the role of the general inspectorate. Evaluation Question # 3: To what degree have JSSP training activities been integrated by targeted recipients (judiciary police, magistrates, and court staff) into their respective work roles? 23. Joint trainings of judiciary police officers, prosecutors, and examining judges have contributed to the increase in the cooperation among the various actors of criminal justice. The level of appreciation of these trainings was particularly obvious when interviewing EJs who complained that prior to the joint training, they were ignored by the JPOs who did not report to them when they initiated a rogatory commission or another act of investigation within their purview. Both EJs and prosecutors established causation between the joint training and an increase in the quality of the JPO’s work.18 According to the prosecutors, the joint training has contributed to strengthen their roles as directors of a criminal investigation and clearly leads toward cohesion of the judiciary police and the prosecutor’s office work. The only interview conducted with JPOs (the gendarmerie brigade in Man) confirmed that this vision is shared by JPOs. Training has also been the occasion for the dissemination of material produced with the assistance of JSSP (for example, guide on violence against women, prosecutor’s guide, criminal case law treatise, etc.).19 24. The Criminal Justice Unit has been instrumental in disseminating the joint training innovation in their jurisdictions. Except for the prosecutor’s office in Yopougon, all prosecutors, EJs and JPOs interviewed highlighted that one of the benefits of the PJP has been the dissemination within the jurisdiction where they operate the new tools and practices learned during joint training. Prosecutors in Yopougon and officials at the appellate level in Bouaké were more critical. Officials, such as the president of the Court of Appeals and the president of the accusation chamber of Bouaké at the appellate level consider the training sessions and the material prepared by JSSP to be too theoretical. Prosecutors in Yopougon have expressed doubts as to whether participants do not fall back in the usual routine as soon as they return to their courthouse after the training..20 25. Exchanges between actors exposed to JSSP tools and others have had a beneficial effect in their willingness to introduce JSSP tools. The mix of personnel from pilot courts who have been introduced and learned to use the JSSP tools and of personnel from non-pilot courts appears to have had a beneficial effect with the personnel from non-pilot courts showing eagerness to experiment the tools in their own courts.21 18 All these findings result from overall trends identified in interviews with the various key informants, but no quantitative data backing these trends was collected because it was simply not available. 19 Tetra Tech DPK wanted to stress that “It is necessary to mention here the positive impact of the project’s joint trainings for investigating judges (JJs) and their clerks, in order to improve efficiency of investigating offices, reduce case backlogs and decrease procedural delays..” 20 Evidence to support this finding is based on interviews. No quantitative data exists to support this finding. 21 This was made evident in the interview with the prosecutor of Toumodi. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 25 Evaluation Question # 4: To what extent have the attitudes of the general public and media changed toward the Ministry of Justice and the legal aid office? 26. Very few applications were made to the office of judicial assistance22. In spite of the awareness campaigns conducted, only 191 applications for judicial assistance were received between June 2015 and January 2017 with 48 (28%) being rejected. The exit poll survey shows that court users surveyed still do not know much about judicial assistance. The analyses of the survey responses show that 6 out of 10 court users do not know about the existence of a judicial assistance office and 9 out of 10 do not know how to access judicial assistance. Of those who have heard of the judicial assistance office, most do not believe that it will be responsive to their needs. This is clear from the overwhelming number of respondents who could not form an opinion on whether the Office of Legal Aid or the Public Defense Fund met the needs of ordinary citizens. However, court users polled in JSSP court communes are significantly more uncertain than were respondents in other court communes. JSSP programming therefore may play some role in transforming strong negative opinions about Ministry of Justice services into more neutral uncertainty. Still, opinions about the Ministry of Justice itself are not substantively or statistically different across JSSP court communes and non-beneficiary commune courts. 22 The statistics from the DACP are not reliable (eg, the statistics from Oct-Dec. 2016 mention 26 received applications, 25 granted applications and 4 rejected applications). Figures 8, 9, and 10: Exit Poll Survey Perceptions on Whether Various Bodies Meet Citizen Needs COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 26 27. Effective signaling and displays about fees have contributed to reduce the phenomenon of informal intermediaries, a common practice before JSSP. These two tools implemented by JSSP contributed to limit the use of “margouillats”, or informal intermediaries who collect fees from court users to guide them in the courthouses (this sometimes involves court personnel). The interviews conducted in Agboville, Boundiali and Man indicated that the phenomenon was decreasing and this was expressly correlated with the signaling and the posters displaying the various fees. However, both the interviews and the focus group study conducted by CREFDI reveal that these measures have had much less effect in Bouaké and Yopougon where, the practice of using informal intermediaries still appears to be endemic. Concerning Sinfra, the phenomenon was reported to be rather insignificant during the interviews conducted with the court officials. The reduction of informal intermediaries contributes to increase the legitimacy of justice institutions. These informal intermediaries are an endemic problem everywhere in Africa and contribute to the perception that going to the courthouse is an experience that generates confusion for citizens who are taken advantage of by persons who “know their way” into the system. Evaluation Question # 4a. What are the key barriers that still need to be addressed to effectuate a change in the perception of legitimacy? 28. The reform of the justice sector is not a priority for the government; it is mostly donor￾driven. Representatives from the donor community describe a sector given a very low priority by the GoCI as illustrated by the share of the national budget allocated to the justice sector (0.96%) and the funds allocated to the MoJHR’s judicial assistance fund (XOF 170 M as of now. 23 Donors such as the European Union deplore the short-term vision of the MoJHR and took this into account by changing the nature of assistance provided to the sector from technical assistance to budgetary support. 29. The budget of the MoJHR’s Directorate of Civil and Criminal Affairs (DACP) to run the Office of Judicial assistance is very low. The budget for the office of judicial assistance is only 189 MXOF (initially 500 MXOF, then 250MXOF and now 189 MXOF). This budget is meant to facilitate and speed up payments to judicial auxiliaries taking judicial assistance cases who would not have to be process payments through the Judicial Agent of the Treasury, a process which was considered extremely long. 23 Sources: AFD, CELeX and DACP. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 27 30. No agreement was entered into with the Bar association on a scale of fees. An essential part of running a judicial assistance system rests on paying the providers of legal services. This condition has not been met. One agreement is under negotiation between the MoJHR and the bar association, but the evaluation team was told that it merely involves payment of lodging and transportation costs to lawyers who would perform services paid under the judicial assistance scheme. 31. The Ivorian Bar is a key barrier to the perception of legitimacy of the judicial sector as a whole. The Ivorian Bar is business oriented and as such restricts entry (only 12 to 15 new attorneys are admitted each year) to a Bar that numbers 515 attorneys for a population of 22 million inhabitants24. Only two attorneys have offices outside of Abidjan (one in Bouaké, one in San Pedro).25 The Bar has a poor track record in terms of assistance to the low income persons as illustrated by the failure of an experiment supported by the European Union which attempted to institutionalize free legal consultations in the TPI of Abidjan plateau and the MACA prison in Yopougon. 32. The Office of Judicial Assistance does not monitor what happens after it has granted an application. Even though very few judicial applications have been filed, there is already evidence of rationing as illustrated by the relatively high number of rejected applications (about 25%) and the process of determining eligibility of citizens to receive judicial assistance.26 The whole decree of October 2016 is in itself an indication of the intent of the GoCI to ration legal aid and is very conducive to arbitrary determinations.27 The gap between the lack of operationalization of the office of judicial assistance and the “rationing” disposition of the MoJHR on one hand and the overoptimistic communication of individuals’ rights to judicial assistance on the other hand could contribute to the public negative perception of the judicial system and increase the suspicion that the system is not fair.28 33. Suggestion boxes highlight a sector affected by corruption, delays, and a lack of attorneys. The boxes specifically target the improvement of the perception of legitimacy of the judiciary as a whole, not of the Office of judicial assistance or the Ministry of Justice. Though relatively few citizens fill the questionnaire (for example, 256 questionnaires filled in Boundiali in Oct-Dec. 2016), the sampling made by the team in the courthouses on a dozen questionnaires in Agboville and Man reveal that the perception of corruption is very high (almost invariably, the respondents check the box corruption/lack of professionalism and 24 Source: ProJustice. See also the website of the Ivorian bar http://www.ordredesavocats.ci/FR/annuaireavocat.php 25 Source: AFD 26 Tetra Tech DPK contested this finding by stating: “The draft report offers no evidence that any of the rejected applications were for budget “rationing” reasons. The DACP representative during the seminar contradicted this information. No application was ever rejected on the ground of budget scarcity.” 27 The decree establishes a thorough screening process of the personal financial situation of the applicant, including the possibility to hear the adverse party, and does not contain any clear indication of the level of income that makes one applicant eligible to receive judicial assistance. Compare, for example with France, where the judicial assistance, named aide juridictionnelle is indexed on the consumer price index. Per instructions of the French ministry of justice, the income threshold is 1,007 to be eligible to 100% gratuity, 1,510 to be eligible for partial support (25% to 55% of the legal costs). See eg. http://www.net-iris.fr/indices-taux/social/48- aide-juridictionnelle-bareme-plafond-ressources. 28 Tetra Tech DPK seems to confuse what is in legal text and the practice. Their staff contested this finding and stated: “This is not correct. Once the judicial assistance office has granted an application, it transfers the case to the Bar and follows up with the Bar. Once an attorney is found to take the case, the Bar normally informs the judicial assistance office. For obvious reasons, the judicial assistance office does not get involved in the case resolution and in the process that follows.” COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 28 most forms contain some comments specifically mentioning corruption). Respondents complain about time it takes to process requests and cases as well as about the lack of availability of attorneys. Except for the perception of corruption, these findings match the analysis conducted and reported by JSSP M&E team in the courthouse of Agboville (Analyse des Résultats de la Collecte de Questionnaires des Boîtes Relais dans la Jurisdiction de Boundiali, Oct-Dec 2016). The perception of corruption (30%) in Boundiali appears to be lower than the sampling made by the evaluation team in Agboville and Man. 34. Both NGOs and several key informants report an overall skepticism toward the judiciary on the part of the public. Many respondents mention that the figure of the judge is still equated with prison. NGOs report “a thirst for rights” on the part of the citizens, yet they also recognize a lack of trust of the population toward the judiciary. This is reflected in the low number of applications (191) to the office of judicial assistance. 35. Perception of legitimacy is not the same in all localities. The qualitative and quantitative study conducted by CREFDI reports on the three aspects of quality of services, integrity of the court personnel and fairness and impartiality concludes that the court users perceive an improvement in Agboville, Boundiali and Man, but deterioration in Bouaké, Sinfra and Yopougon. Evaluation Question # 5: How effective have JSSP tools been in raising awareness and improving public understanding of and access to legal information and the public defense fund? 36. Awareness of and access to legal information was made easier by tools such as the signaling and the display of various court fees in the courthouses but overall the activities concerning access to justice have limited impacts. Interviews with the NGOs point in the direction of increased awareness of basic legal information and access to legal information. However, the exit poll survey shows that, except in Boundiali, the awareness of legal rights is much weaker than indicated in the reports of JSSP M&E team. 37. Awareness campaigns focused on the rights and benefits of judicial assistance but neglected to clarify the potential downsides (see ProJustice flyers on judicial assistance and land certificates). The tools used by JSSP have essentially consisted of awareness campaigns on judicial assistance, on how to access the judiciary to solve legal matters, and on land certificates. The NGOs focused on informing the citizens of their rights to legal assistance and on provisions of the code of civil procedure on judicial assistance but neglected to clarify more difficult aspects of legal assistance and other rights programs which were the focus of awareness campaigns. For example, when applying for a land certificate the applicant must bear the cost of a survey by a geometer, which is in the range of several hundred thousand XOF and was not highlighted in the campaigns. In addition, many awareness campaigns on judicial assistance were conducted before the decree organizing the judicial assistance scheme was adopted (Oct. 2016) which contains much more restrictions than the provisions in the civil procedure code on eligibility - and before the program was fully operational. 38. Practical guides and compendia of criminal case law were widely distributed to judicial officials and NGOs. JSSP has published several guides (guide on violence against women, the COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 29 prosecutor’s guide), and a compendia of criminal case law which have been widely distributed to judicial officials. 39. The information and orientation kiosks, which ProJustice claims to have implemented, have not been seen in operation in courthouses the team visited. Because of a lack of human resources affected to that purpose, none of the courthouses visited had an information kiosk in operation. The answers to the remaining Evaluation Question # 5a (What are the key lessons, best practices, and challenges learned from this approach?) will be addressed in the next section of the report. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 30 CONCLUSIONS The conclusions presented and discussed here are based on the findings reported in the previous section and integrate the findings obtained from the various data sources, including qualitative IDIs, observations conducted by the evaluation team, quantitative analyses, information made available by the JSSP M&E system, and the exit poll surveys and focus groups conducted by CREFDI. They are grouped by theme to facilitate understanding of the issues. IMPROVEMENT IN CASE MANAGEMENT AND CHANGE IN CASE MANAGEMENT CAPACITY The tools implemented by ProJustice show a strong potential for case management improvement but it is difficult to measure their real impact because of an unavailability of systematic collection of statistics (see especially finding No. 2). The various fortunes of 2013-2015 pilot courts in terms of tool appropriation casts doubts as to whether extending the program to 18 courthouses instead of 10 as originally planned was a good decision (see finding 6). Turnover of judicial officials at the onset of the program limited the capacity of the pilot courts. 10 courts might have led to better results in terms of sustainability of the tools than expanding to 18 courthouses. The lack of resource and absence of a proper servicing of the equipment observed in the various courthouses leads to the conclusion that the conclusion of an agreement with the MoJHR and the EU about transferring over to ProJustice the experimentation of a computerized CMS was entered into without sufficient due diligence. If the MoJHR cannot ensure the maintenance of simple equipment such as computers, copiers, and telecommunication facilities, it was doubtful that it was in a position to maintain a computerized CMS. A pre-condition for the successful implementation of a computerization initiative in the justice sector is that the final user has a governance structure to appropriate and extend the CMS to the entire jurisdictional network. MoJHR did not have the capacity to pilot a computerized jurisdictional network. It is either too soon to measure the effect of JSSP tools or that they fell short of achieving that objective (see finding 12). JSSP INTERACTIONS WITH NGOS AND CIVIL SOCIETY ORGANIZATIONS TO PROMOTE ACCESS TO JUSTICE FOR THE GENERAL PUBLIC AND OUTREACH TO THE OFFICE OF LEGAL AID The awareness campaigns focused too much and too early on a judicial assistance initiative. Campaigns tended to communicate only positives without explaining hidden costs. Because of the serious challenges and constraints faced by the Ivorian justice system, NGOs were facing an uphill battle to reconcile the Ivorian citizens with their justice system. This has no doubt limited the range of possibilities of activities conducted with NGOs, particularly after the legal clinic component was removed from the project in early 2014. Judicial assistance was too structurally weak for the government to meet demand. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 31 There is an interest to hear about access to justice, not that the rallies, the radio programs and the broadcast actually furthered access to justice. The sole reliable indicator in this respect is the number of judicial assistance applications which, between April 2015 and January 2017, did not exceed 200 applicants. This number is small compared to the audiences reached by the program and the proportion of the population a-priori eligible. CHANGES IN ACCESS TO JUSTICE Evaluating access to justice should not only consider the formal ability to appear in court but in a broader sense, i.e. engaging the wider social context of the Ivorian court system, and overcoming the systemic barriers faced by different members of the community. Regarding the formal ability to appear in court, we can only report, based on the JSSP M&E raw data, the trends noticed in the number of individuals using the courts which points toward an overall increase of court users (see finding 16) between October 2015 and March 2017. But the period is too short and the evolution too hieratic to discern a string trend toward an increase. It is also impossible to extrapolate that this inconclusive trend would be a consequence of JSSP activities. At most, we can establish a correlation, but this evaluation did not have the resources to conduct an impact evaluation. The findings from the exit poll survey are also inconclusive as to warrant a conclusion that the program has made significant improvement in access to justice. Although in terms of services related to the establishment of administrative documents such as extract of criminal records and criminal proceedings, the survey notes an improvement in all six pilot courts, in these courthouses court users have little knowledge on judicial assistance, which paradoxically, is inversely proportional to the focus of awareness campaigns conducted under JSSP activities. With regard to certain categories of justice users which may be affected by systemic barriers, particularly women, the evaluation team can only credit the program for outputs, but it is way too early to measure the impacts of these activities. Finally, the Ivorian Bar, with just over 500 attorneys, has restricted conditions of entry and a poor track record of success when it comes to representation of low- income citizens. This is a structural impediment to access to justice. CHANGE IN THE ADMINISTRATION OF JUSTICE The evaluation team can credit ProJustice for having designed certain tools with a view to stimulate a demand for a better administration of justice The transparency toolsmay be correlated with a trend, observed from interviews with court officials that revealed a concern for reducing delays and a genuine concern and desire to improve the experience of court users. Regarding measurable changes on citizens’ perception of court administration, the exit poll survey shows a disparity among courthouses with Agboville, Boundiali (2 courthouses on the program recently) and Man (on the program since 2013) showing an improvement both on the perception of the quality of judicial services and corruption-free environment while Bouaké, Sinfra and Yopougon even reveal an increase of the perception of corruption and integrity (see finding 18). These findings lead to the conclusion that the program had mixed fortunes depending on the courthouses and that the most recent courthouses tend to perform better than courthouses that COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 32 have been benefited by the program since 2013 and courthouses that have been on the program for four years (in all fairness, at least concerning Bouaké and Yopougon, these are much larger courthouses than Agboville and Boundiali). Finally, little change is to be expected from the Judicial Assistance initiative. With barely a couple hundred applications and operational deficiencies, the Judicial Assistance Initiative will be unlikely to drive enough litigants to stimulate a demand for a better administration of justice. INTEGRATION OF TRAINING ACTIVITIES BY TARGET RECIPIENTS INTO THEIR WORK ROLES Based solely on interviews with prosecutors, EJs and one judiciary police officer, reveal that there a discernable trend of a better perception on their part of a more cooperative management of criminal proceedings which is reinforced by the PJP, emphasizing here again the comprehensive aspect of the project. BARRIERS TO THE LEGITIMACY OF JUDICIAL INSTITUTIONS AND OF THE OFFICE OF JUDICIAL ASSISTANCE The very fact that justice is a low priority sector awarded a mere 0.96% of the government’s budget is a barrier to the legitimacy of judicial institutions. Despite apparent good will from the individual actors interviewed for this study and their interest in improving the relationship between Ivorian citizens and justice institutions, there is a sense that the sector is abandoned by government and commitment to reform is purely donor-driven. This lack of commitment on the part of the government risks causing fatigue on both sides. The main conclusion is that the office of judicial assistance is a scheme that can be described as an empty shell. The judicial assistance office is crippled by a very low budget and an absence of an agreement with the providers of legal services paid under the judicial assistance scheme. The judicial assistance aspect of the program reveals an excessive focus on formal processes (as mandated in the legal text) and an insufficient focus on actual practices. This is illustrated by the contents of the flyer distributed on awareness campaigns on judicial assistance, which is for the most part a mere copy and paste of the provisions of the code of civil, commercial and administrative procedure which can lead a citizen to believe that the only requirement to receive legal assistance is to apply and provide a certificate of non-taxation or justify income under a certain threshold (which, in passing is not regulated in the October 2016 decree). Therefore, the conclusion on judicial assistance is that, four years into the implementation, JSSP has achieved the following: i) JSSP has provided support to the office of judicial assistance in training, IT equipment, technical assistance to develop statistical tools; ii) assistance toward passing a decree in October 2016 that provides, inter alia, the decentralization of the judicial assistance screening process and iii) awareness campaigns promoting judicial assistance and informing that the office of judicial assistance now directly administers its own budget (starting in 2017); iv) less than 200 applications filed between April 2015 and January 2017 with 48 rejections. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 33 EFFECTIVENESS OF JSSP TOOLS IN RAISING AWARENESS AND IMPROVING PUBLIC UNDERSTANDING OF AND ACCESS TO LEGAL INFORMATION AND JUDICIAL ASSISTANCE The findings related to the access to justice component of the project suggests that awareness campaigns were not the most effective tool. Both exit poll survey conducted by CREFDI and the very low number of applications compared to the number of persons and opinion leaders reached by awareness campaigns suggest that the awareness campaigns conducted were not effective in raising awareness and improving public understanding of and access to legal information and the public defense fund. The awareness campaigns were simply premature given the low level of operationalization of the judicial assistance initiative. The ineffectiveness of the campaigns is illustrated by the low number of applications filed before the office of judicial assistance. The program’s access to justice activities was the judicial assistance component which itself was crippled by the low priority given to the justice sector by the GoCI. Other structural limitations include a very small bar vis-à-vis the size of the population, exclusive concentration of attorneys in Abidjan, and little interest of the attorneys in assisting low-income population. Much like computerization of case management, the judicial assistance component of the project illustrates that development programs must adjust to the low or high priority given the sector by the government. The main lesson learned from the project’s difficulties with the computerization and with the judicial assistance initiative is that donors should think twice before investing in sectors that the government considers low priority. Donors must consider the level of political will needed on the part of the government for projects to succeed and project design should be adjusted and recalibrated to reflect the government’s high or low levels of priorities. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 34 RECOMMENDATIONS With the above findings and conclusions in mind, USAID/Côte d’Ivoire should consider the following general recommendations for ProJustice and future justice sector strengthening programs. The following table summarizes findings, conclusions and recommendations of this evaluation. Findings Conclusions Recommendations The five new tools are well accepted (findings 1 through 5 and 8). Strong potential for case management improvement but no systematic collection of statistics to measure their effect by MoJHR 1. Dedicate the remaining time left to properly assess the structural problems affecting the Judicial Assistance initiative. 2. ProJustice: Find a way to work with the Directorate of evaluation and statistics to improve the quality of information and use this information to improve decision making (particularly BOP). 3. Extension to the whole jurisdictional network of preventive detention tables) No maintenance of basic equipment by MoJHR (finding 7) MoJHR does not have institutional structure, human resources for computerized CMS. MoJHR not in capacity to own the Computerized CMS 4. USAID: Mobilize the donor community to find funding for technical assistance to build technical and institutional capacity of MoJHR Depending on turnover, the tools survive or not (finding 6). The tools sustainability is grounded upon individuals, not upon institutional appropriation. 5. ProJustice: Work with the DACP to generate a list of instructions (circulaires) to ensure the dissemination of the various tools in the entire jurisdictional network. Too much focus on judicial assistance in awareness campaigns (finding 13, 37) Judicial assistance scheme not operational (findings 29, 30, 31, 32). Implementation focused too much on formal processes not enough on actual processes (particularly regarding judicial assistance). 6. Perform a diagnostic assessment on what is lacking as a prerequisite for a properly functioning judicial assistance and monitoring procedure after granting judicial assistance. 7. At minimum, an agreement with the Bar on a scale of fees is required. 8. Champions in the judiciary to work with NGOs. 9. Reorient civil society intervention on COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – 35 access to justice to chronic issues affecting the Ivorian justice system and conduct awareness campaigns on the issue such as the illegal use of criminal custody as a tool for debt collection and prolonged preventive detention. 10. Consider refocusing judicial assistance on a specific category of applicants (women victims of violence, handicapped, detainees in preventive detention) to conduct a pilot experiment with the OJA. Signage, court fees posters have contributed to reduce the phenomenon of “margouillats”. Signage and court fees posters have been great simple tools to achieve important results. 11. Investigate further the effect on margouillats phenomenon. 12. Consider equipping other courthouses with signage. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – A-1 ANNEX A: REFERENCES Agence Française de développement, Fiche du projet de renforcement du système judiciaire et pénitentiaire et de la protection des droits de l’homme en Côte d’Ivoire, 2016. Akindes, F., Charlick, R. & Daddieh, C., USAID’s Côte d’Ivoire Democracy and Governance Assessment, 2011. Analyse des Résultats de la Collecte de Questionnaires des Boîtes Relais dans la Jurisdiction de Boundiali, Oct-Dec 2016. Dakolias, M. (2014) Court Performance around the World: A comparative Perspective. Yale Human Rights and Development Journal, Issue 1, Vol 2, Article 2. European Union Commission, Fiche du projet d’appui à la réforme et modernisation du système judiciaire et pénitentiaire en Côte d’Ivoire, 2011 Gorée Institute, Rapport d’évaluation à mi-parcours du projet d’appui à l’amélioration de l’accès à la justice et aux droits en Côte d’Ivoire, Nov. 2014 MoJHR, Document d’orientation de la politique sectorielle du ministère de la justice en Côte d’Ivoire, 2012-2015 ONUCI Rule of Law, UNDP, UNICEF, Fiche du projet amélioration de l’accès à la justice dans l’ex zone Centre Nord Ouest, Programmation 07/2012 – 07/2013 ONUCI Rule of Law, l’organisation et le fonctionnement du système judiciaire en Côte d’Ivoire, 2007 ONUCI Rule of Law, Etat des lieux du système judiciaire ivoirien, 2010 Open Society Foundation, Presumption of Guilt, The Global Overuse of Pretrial Detention, 2014. ProJustice, Rapport de mission d’évaluation à froid des formations conjointes, Collaboration Magistrats d’instruction et greffiers d’instruction dans les 11 juridictions pilotes de ProJustice, Etat des lieux et recommandations, March 2017 ProJustice, Rapport de mission d’évaluation à froid des formations conjointes Collaboration magistrats-officiers de police judiciaire dans les 7 nouvelles juridictions pilotes de ProJustice, Etat des lieux et recommandations, March 2017 ProJustice, Rapport de la qualité des données dans les 11 juridictions pilotes de ProJustice, Dec. 2016 ProJustice, Rapport de rétro information dans le cadre du suivi de la performance des juridictions pilotes de ProJustice en Côte d’Ivoire, Mai 2016 ProJustice, Rapport de rétro information dans le cadre du suivi de la performance des juridictions pilotes de ProJustice en Côte d’Ivoire, Mai 2016 ProJustice, Rapport de rétro information dans le cadre du suivi de la performance des juridictions pilotes de ProJustice en Côte d’Ivoire, Oct. 2016 ProJustice, Rapport de rétro information dans le cadre du suivi de la performance des juridictions pilotes de ProJustice en Côte d’Ivoire, Commentaires des justiciables, Oct. 2016 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – A-2 ProJustice (Cote d’Ivoire) Performance Monitoring Plan-Indicator Table Q1-Q4 ProJustice (Cote d’Ivoire) Performance Monitoring Plan Baseline FY 2016 ProJustice, Performance Monitoring Plan Indicator table, year 4 ProJustice, Reference data PMP, FY2016 ProJustice, Training Database, FY 2016 ProJustice, Quarterly Progress Report, Oct.. – Dec.. 2013 ProJustice, Quarterly Progress Report, Jan. – Mar. 2014 ProJustice, Quarterly Progress Report, Apr. – Jun. 2014 ProJustice, Quarterly Progress Report, Jul. – Sept.. 2014 and Annual Progress Report Oct. 2013 – Sept. 2014. ProJustice, Quarterly Progress Report, Oct.. – Dec.. 2014. ProJustice, Quarterly Progress Report, Jan. – Mar. 2015 ProJustice, Quarterly Progress Report, Apr. – Jun. 2015 ProJustice, Quarterly Progress Report, July – Sept. 2015 and Annual Progress Report Oct. 2014- Sept. 2015. ProJustice, Quarterly Progress Report, Oct.. – Dec.. 2015 ProJustice, Quarterly Progress Report, Apr. – Jun. 2016 ProJustice, Quarterly Progress Report, July – Sept. 2016 and Annual Progress Report Oct. 2015- Sept. 2016 ProJustice, Quarterly Progress Report, Oct.. – Dec.. 2016 ProJustice, Schéma directeur informatique du Ministère de la Justice, des Droits de l’Homme et des Libertés Publiques, des juridictions et des maisons d’arrêt, 2015 ProJustice, Rapport de mission d’Etat des lieux de l’utilisation des outils développés par ProJustice dans cinq juridictions pilotes, March 2017 ProJustice, Performance Monitoring and Evaluation Plan, FY 2013-2014 ProJustice, Year Three (FY16) Workplan. UNDP, Etude d’évaluation du système judiciaire ivoirien, 2008 White, H. & Saborwal, S. Quasi Experimental Design and Methods. Methodological Briefs: Impact Evaluations. No. 8, UNICEF Office of Research-Innocenti. www.unicef-irc.org COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –B-1 ANNEX B: STATEMENT OF WORK AND EVALUATION ACTIVITIES COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –B-2 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –B-3 AID-624-T0-17-00001 Mid-term evaluations of USAID/CI's DG Program 2. Develop the evaluation methodology 3. Test and verify the evaluation methodology 4. Deploy a field team 5. Collect the relevant data to inform the evaluation 6. Conduct oral debrief meetings with USAID on the preliminary findings of the evaluation 7. Host a stakeholder's workshop to present the draft evaluation findings for their validation and inputs 8. Draft Final Report 9. Submission of COR approved final report to the DEC C.4. Results : Deliverables and Outputs The Contractor must furnish a set of deliverables and reports- (see Section F.4) C.5. Team Composition The Contractor must propose a diverse evaluation team characterized by expertise in different key components of the JSSP activity. As a means of building local capacity to undertake evaluations, the team composition should at least have one local expert as a key member of the evaluation team. C.6. Place of Performance The place of performance is C6te d'lvoire covering 11 pilot jurisdictions areas. The extent of travel will be determined by the evaluation design and data needs as agreed upon between the Contractor and USAID. The TOCOR will serve as the primary Point of Contact for the Performance Evaluation. C.7. Logistics The Contractor will be responsible for all logistics including coordinating all travel throughout the 11 pilot jurisdictions areas, lodging, printing, office space, equipment and car rentals, financing from the contract award and managing dissemination of results. The USAID/Cote d' lvoire Democracy and Governance Officer in coordination with the TOCOR will provide support to set up initial meetings with key government officials and other stakeholders. C.8. Existing Sources of Information The Mission will share the following documents with the successful evaluation team to facilitate the desk review. The applicable documents are listed below: 1. Quarterly progress reports (FY2014- FY2015) 2. JSP baseline report 3. JSP Performance Monitoring and Evaluation Plan (PMEP) 4. Annual work plans (FY2014- FY2016) 5. List of JSSP Pilot Courts [END OF SOW OF JSSP] Page 7 of 35 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-1 ANNEX C: EVALUATION MATRIX Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations 1. To what extent did JSSP interventions in the 11 pilot courts lead to improvements in case management JSSP has instituted 3 specific mechanisms to improve the case management systems, which are the Criminal Registry Office (Bureau d’ordre penal “BOP”), the Criminal Justice Unit (Pôle judiciaire penal “PJP”) and the quality Management Control Unit (Unité de contrôle qualité gestion “UCQG”). A large part of the evaluation will therefore concern the status of these mechanisms. The analysis will also address whether improvement of case management corresponds with the introduction of the various tools developed by JSSP (procedural templates, preventive detention monitoring tables, signage, judicial fee table, solutions boxes). This question is linked with precise program indicators: (3: percentage reduction in case backlog, 4: number of cases purged; 5: average time taken to process cases). The baseline study has done a good job developing categories which could form the basis of sub-indicators for the purposes of the present evaluation (reduction of preventive detentions, monitoring of the decisions not to prosecute certain offences, differentiation of the criminal proceedings with or without examination, implying a minor or not etc.). How do beneficiaries rate the program’s effectiveness at improving case management? What specific aspects or elements of the program were most effective and/or useful and which were least effective and/or useful? To what extent is the court system affected by corruption problems? Did the JSSP introduce any measure to address the problem of corruption? Review of project documents Key Informant Interviews with court personnel, legal professionals, project staff Focus groups with key stakeholders (court administrators, judges, prosecutors) Descriptive analysis to illustrate different project activities Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups JSSP introduced a new component to design and implement a computerized case management system in 2 pilot courts. The baseline survey does not incorporate results for this element of the project and therefore, information to assess results of this aspect of the program will require interviews with court staff of the 2 pilot courts in Yopougon and Dabou. Data from informants will depend on who is asked and how informed each person interviewed and/or surveyed is. Baseline data is only available for 10 pilot courts COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-2 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations 1a. What change, if any, was noted in the case management capacity in each court? Was it increased? This question relates to the effects of the JSSP activity at the micro-level of each court. DI will emphasize the effectiveness of the court coordinators established by the program in each pilot court through a case study in each courthouse. How long does it take on average to process a case? Did court users perceive any change in the time it takes to process their case(s)? Are users more or less satisfied with the court service delivery? Key Informant Interviews with court personnel, project staff Focus groups with key stakeholders (court administrators, judges, prosecutors) Site visits Descriptive analysis to illustrate different project activities Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups. Baseline data for courthouses not included in the baseline report is unavailable, and therefore, there is no element to compare improvements achieved from project’s inception. The computerization dimension will be an added element to consider in the cases of the pilot courts of Yopougon and Dabou. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-3 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations 2. To what extent did JSSP interventions with non￾governmental organizations and civil society organizations promote access to justice for the general public and outreach to the Office of Legal Aid? This question concerns two very different activities. The first is the improved access to the ministry of justice’s Public Defense Fund and increase capacity and outreach of the Office of Legal Aid. The second is support for networking, public information campaigns, and awareness-raising activities to improve the public’s understanding and access to legal information, which includes the community legal advisors part and the legal outreach one. The PDF and OLA elements tie to indicators 2.1.1, 2.1.2, 2.1.3, 2.1.3, 2.1.4, 2.1.5. The legal outreach activity relates to indicators 2.2.1 while the community legal advisors activity relates to indicator 2.2.2. To what extent did the project enhance the capacity of the Office of Legal Aid? Are the project’s indicators calibrated to assess capacity enhancement? How did citizens gain greater access to the Public Defense Fund? Do citizens perceive any change in their ability to access services? How effective were outreach activities in improving public’s understanding of the justice system and awareness of legal aid available to them? Exit poll survey Key informant interviews with leaders of NGOs, journalists, court personnel and project staff Focus group discussion with leaders of NGOs, community leaders in the zones of intervention, legal advisors, and legal outreach, especially panels of women. Site visits Descriptive statistics for exit poll data Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups Although the question only mentions the Office of Legal Aid (OLA), we must assume that the Ministry of Justice’s public defense fund (PDF) was an involuntary omission and should be covered in the investigation. Data from informants will depend on who is asked and how informed each person interviewed and/or surveyed is. Baseline data is only available for 10 pilot courts 2a. How has access to justice changed? This question relates to indicators 1, 8 and 10; 2.1.3;2.2.2.. The team will compare and contrast actual number of citizens using the courts with users’ perceptions of access to the justice system and overall assessment of service delivery. Access to justice involves not only the number of citizens using the court system, but also citizens’ belief in the legitimacy of that system, perception that the system works on their behalf and does not discriminate What are notable and specific changes citizens perceive with respect to access to justice? How satisfied are citizens with the ability of the courts to resolve their cases? Review of project documents Exit poll of court users Key informant interviews with NGOs, journalists, legal professional and project staff Descriptive statistics for exit poll data Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to Citizens interviewed for the exit poll may not understand the difference between access to the courts and access to justice. Exit polling data may produce “non￾attitudes” – responses that do not reflect true beliefs because the COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-4 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations based on wealth, status, gender, ethnicity, religion etc., and the ability of the system to resolve cases in a timely manner. Focus group discussion with leaders of NGOs, community leaders in the zones of intervention, legal advisors, and legal outreach, especially panels of women Site visits assess differences between baseline data and evaluation team findings, and between survey data and data collected through interviews and focus groups issues at hand are arcane and complex and not topics the ordinary citizen holds true opinions on. The exit poll survey is designed to cross￾validate survey responses to allow us to control for this in the analysis. Baseline data available only for 10 pilot courts 2b. What change, if any, was noted in the administration of justice, especially criminal justice activities? The question examines changes in the administration of justice from a perspective of the demand for services— whether legal aid and community legal advisers have generated demand for a better administration of justice. It relates to EQ2 as well as to EQ1. Are there any specific areas where change is notable in the case of criminal justice? What is the most important problem still remaining for citizens when they need to resolve a criminal case? Desk review of project documents Key informant interviews with NGO, journalists, legal professionals and project staff Focus group discussion with NGO leaders, community leaders in intervention zones, legal advisors, and legal outreach, especially panels of women Site visits Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups Baseline data available only for 10 pilot courts COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-5 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations 3. To what degree have JSSP training activities been integrated by target recipients (judiciary police, magistrates and court staff) into their respective roles? The approach will center on interviews with trainees as well as the institutions supplying these trainings. The question is the only one where the program components concerning the IGSPJ can fit. It is in this part of the evaluation that tools like the Solutions boxes could be evaluated. The program indicators for this question include indicator 2 and sub-indicators 1.1.1, 1.1.2 and 1.1.3 and 1.3.2. What are some examples of how new skills have been integrated into beneficiaries’ roles? What has been most useful and least useful in these trainings? What have been the greatest challenges in the implementation of new skills into beneficiaries’ work? Desk review Key interviews with court officials, officers of judiciary police, judges, IGSPJ inspectors and project staff Focus group discussion with local leaders, community leaders, especially panels of women Site visits Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and evaluation findings, and between survey data and data collected through interviews and focus groups Other donors are also involved in the provision of assistance in judicial training and capacity building of the General inspectorate. Attribution of any change to JSSP will be difficult to isolate. Informants may respond positively to ensure more assistance will be provided 4. To what extent have the attitudes of the general public and media changed toward the ministry of Justice and the office of Legal Aid? This question is linked primarily to indicator 1, 10 and 13 and sub indicators 1.3.1, 1.3.3, 1.3.4, 2.1.2 and 2.2.1. Attitudes about the ministry of justice and the office of legal aid are related to perceptions of citizens in their interaction with the courts and news reports and communication material disseminated by the Ministry of Justice. To what extent citizens understand the functions of the office of Legal Aid? How easy is it to obtain financial aid in case citizens cannot afford the costs of legal assistance? What are the means the Ministry of Justice has used to disseminate information about its services, particularly the office of Desk review of project’s documents Exit poll of court users Key informant interviews with OLA, court officials, and project staff Site visits Descriptive statistics for exit poll data Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and Citizens may not be aware of the Legal Aid Office No available baseline data for a number of pilot courts COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-6 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations The evaluation will be concerned with assessing how well informed citizens are about the functions and services of the OLA and the PDF. On the other hand, the evaluation will also assess whether the OLA and the PDF contribute to offering real judicial remedies to litigants and citizens who cannot afford its normal costs. Finally, the evaluation will assess how well aware are citizens of the importance of these services and to what extent they have used them. Legal Aid? information collected by evaluation team, and between survey data and data collected through interviews and focus groups 4a. What are the key barriers, if any, that still need to be addressed to effectuate a change in perception of legitimacy? What are the challenges the Ministry of Justice faces to improve its services and increase access to justice? To what extent is citizen satisfaction with justice services improve the legitimacy of the Ministry of Justice? What is the single most important problem citizens’ face in gaining access to justice and obtaining assistance from the office of Legal Aid? Desk review Exit poll of court users Key informant interviews with OLA, court officials, and project staff Site visits Descriptive statistics for exit poll data; correlation between variables Trend analysis to find patterns, similarities across courts Content analysis of interview and Focus group data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups Citizens interviewed may not have had access to the Office of Legal Aid. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-7 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations 5. How effective have JSSP tools been in raising awareness and improving public understanding of and access to legal information and the public defense fund? This question relates primarily to indicators 1, 10 and 13 and sub indicators 1.3.1, 1.3.3, 1.3.4, 2.1.2 and 2.2.1. Like Q4, this is a question related to citizens’ perceptions. But this question refers mostly to JSSP efforts to raise awareness, independent of any other communication and dissemination efforts not supported by the project What are the most effective means of raising awareness about available legal aid and the public defense fund? How easy is it for citizens to understand legal information? What innovations, if any, has JSSP introduced to increase outreach and messaging effectiveness? Desk review of project’s documents Exit poll of court users Key informant interviews with OLA, court officials, and project staff Focus group discussion with leaders of NGOs, community leaders in the zones of intervention, legal advisors, and legal outreach, especially panels of women Site visits Descriptive statistics for exit poll data; correlation between variables Trend analysis to find patterns, similarities across courts Content analysis of interview data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through interviews and focus groups No available baseline data for a number of pilot courts 5a. What are the key lessons, best practices and challenges learned from this approach? What do JSSP staff consider to be the most important lessons drawn from the implementation of this project? What specifically they could have done differently? What are the remaining challenges? Desk review of project documents, identification of specific indicators that have not reached their expected targets Interviews with project staff and court officials Trend analysis to find patterns and similarities across informants Content analysis of interview data Comparative analysis to assess differences between baseline data and information collected by evaluation team, and between survey data and data collected through COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – C-8 Evaluation Question (Interpretation and Approach) Sub-Questions Data Sources, Data Collection Tools Analysis methods Limitations interviews and focus groups COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –D-1 ANNEX D: LIST OF PROJECT AND PERFORMANCE INDICATORS The JSSP has 13 key performance indicators. Data on the 13 indicators were collected during the baseline study conducted in 2013. However, those data are limited to the 10 original courthouses listed as pilot courts in the RFA. Project Outcome Performance Indicators Indicator # JSSP Indicator Title 1 Average level of court user satisfaction with service provided in the courts 2 Collaboration between judiciary police, magistrates and court staff (score) 3 Percentage reduction in case backlog 4 Number of pending cases purged 5 Average time taken to process cases 6 Percentage of public knowing how to report judicial abuse and corruption of cases 7 Number of judicial abuses and corruption cases reported by the public 8 Number of citizens accessing justice system 9 Percentage of annual increase in the budget distributed through the Public Defense Fund 10 Percentage of population that knows how to access the Public Defense Fund COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –D-2 Indicator # JSSP Indicator Title 11 Number of individuals who received legal aid counseling, 12 Number of courts where the PDF is accessible through decentralization 13 Percentage of public indicating awareness of their legal rights and how to exercise those rights (Surveys) Project Performance Indicators at Output Level Indicator # Indicator Title 1.1.1 Number of judges and judicial personnel trained with ProJustice assistance 1.1.2 Collaboration between judiciary police, magistrates, and court staff is improved 1.1.3 Number of new procedures/policies developed for the INFJ with ProJustice assistance 1.3.2 Number of inspectors trained with ProJustice assistance 1.3.4 Number of judges and/or court personnel sanctioned for improper behavior 2.1.3 Number of individuals receiving counseling from community legal advisors 2.1.5 Percentage of legal aids lawyers showing improved knowledge of how to represent indigents 2.2.2 Number of individuals receiving counseling from community legal advisors COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –E-1 ANNEX E: LIST OF PEOPLE INTERVIEWED NAME INSTITUTIONAL AFFILIATION/TITLE 1. KONE AISSATA President Agboville Courthouse (Tribunal de Première Instance “TPI”) 2. DJE NOEL Prosecutor TPI Agboville (Substitut résident “SR”) 3. TAPE Igor Chief Clerk (Greffier en chef) 4. LOUKOU ALLOU ST GUILLAUME Clerk 5. JOHNSON KOUASSI FRANCK Clerk 6. ABOAD DENIS MICHEL Clerk 7. DAGNOGO Klofanhan N’Golo Director of Civil and Criminal Affairs, MOJHR 8. GOUIN GUY-ROGER Deputy Director in charge of commercial affairs and intellectual property, MOJHR 9. LOROUGNON ARTHUR PHILIPPE Deputy Director of civil affairs and the seals (director of the office of legal aid), MOJHR 10. GNEBLE Labe Coordinator, ONEF 11. MONEY STEPHANE MIDH 12. ADOU HONOREE Coordinator, AFJCI 13. KROU ADJE MICHEL Coordinator, Transparency Justice 14. OUATTARA Soualilho Director of statistics, MOJHR 15. KOUASSI Bernard Deputy Chief of Staff, MOJHR 16. LEBRY Marie Léonard Judge at the Supreme Court, Fmr. Head at the Court of Appeals of Bouake 17. Jean Luc Bolé Head of the IT service at the MoJHR 18. Kambité Tchoukité COO, INova 19. Kouakou Kouadio Abraham Donga Project Director, INova 20. BISSOUMA Guy Eric Quality Control Expert, Consortium 21. DIABATE Safiatou SNDI COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –E-2 22. Me KOUMAN ADAMA Clerk for ex parte requests to the President of the TPI of Yopougon. First clerk to work with the computerized CMS. 23. Me SIEI ATHANASE Head Clerk (Greffier en Chef) TPI of Yopougon 24. Mme YAVO Jocelyne Deputy Head Prosecutor 25. KONAN KAN Prosecutor (Substitut du Procureur) 26. KAPLY DESIRÉ Président, TPI Yopougon 27. FODJO KADJO ABO General Inspector of the Judiciary and Penitentiary Services 28. KOUAKOU BROU Executive Director, National Judges Training Institute (INFJ) 29. CISSE MAKOUENI DELPHINE Chair, Judges School (Ecole Nationale de la Magistrature) and President of the Association of Female Judges 30. COULIBALY MAHOMED Vabé Chief of Staff, MOJHR 31. Me KASSY Head Clerk, TPI Boundiali 32. MAMBE SYLVAIN President TPI of Boundiali since January 2017,.. Formerly President of TPI of Bouna 33. GOGOUA Examining judge, TPI of Boundiali 34. (Name not available) Chief Clerk, TPI of Boundiali 35. OUATTARA SANOGO Local coordinator MIDH in Korhogo 36. François LE BUR French judge, former head of the technical assistance for the EU 37. SORO NALOURGO Head clerk, TPI of Man 38. DAGNOHA DRISSA President, TPI of Man 39. YAO JONAS KOFFI EJ, TPI of Man 40. VEH BLONDE Coordonator, ONG ONEF Man 41. TEBAHI HUGUES ANICET Prosecutor (SP), TPI Man 42. Eric ASSEMIAN Deputy Head of gendarmerie brigade 43. ATTA KOUAME EFORI PAUL President, TPI Sinfra 44. N’GUESSAN OKOUN KOUASSI MARC Prosecutor (Substitut Résident), TPI of Sinfra COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –E-3 45. Me OUATTARA Maïmouna Chief Clerk, TPI of Sinfra 46. TIA TIA SSOU ANDRE Prosecutor (substitute resident), TPI of Toumodi 47. TAHIROU DEMBELE First Pdt, Court of Appeals of Bouaké 48. SERI BALLET General Prosecutor Appeal Court of Bouake 49. Judge KAMIN Président de la Chambre d’Accusation (since 2017) of Appeal Court of Bouake, former President of TPI of Bouaflé and formerly at the Directorate of Civil and Criminal Affairs 50. SORI NAYL President, TPI Bouaké 51. KONE BRAMAN Prosecutor (Procureur de la République “PR”)) at the Bouake TPI 52. KAKOU TANOH Vice Pdt, TPI of Bouaké, Dean of Examining Judges 53. Gaëlle ANGOT Chargée de mission, French Agency of International Development (AFD) 54. TERRA DE JONG Country Director, Property Rights and Artisanal Diamond Development (PRADD II) 55. David Pascual BADUENES Governance attaché, EU Delegation 56. Liliane KONAN Chair CELEX (donor coordination) Chairwoman of the COMAJ (ProJustice steering committee) 57. KOSSONOU YEBOUE Dean of examining judges (“EJ”) at the TPI of Yopougon 58. Elodie RICHE Governance Attachée, French Embassy COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-1 ANNEX F: QUESTIONNAIRE FOR STAKEHOLDER INTERVIEWS EQ Codes Question Interviewee Category EQ/EO Links INTRO Lead in discussions. What is your role in [organization]? How long have had this role? Which JSSP approaches/activities have you been involved with? How has the project changed since start up? BACKGROUND QUESTIONS AS REQUIRED #1 By which part of the program where you personally concerned in terms of administration of Justice? Criminal Registry Office (BOP) Criminal Justice Unit (PJP) Quality Management Control Unit (UCQG) Procedural Templates Computerization Judges, Prosecutors Court Clerks Other Court personnel Attorneys EQ1 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-2 #2 To what extent did the new mechanisms implemented by ProJustice improved the following dimensions of the judicial system? Please use a scale of -2 to 2, where -2 means Worsen to a great extent and 2 Improve to a great extent. PLEASE INDICATE ONE ANSWER FOR EACH DIMENSION Dimensions Worsen to a great extent (-2) Worsen to some extent (-1) It will not bring any change (0) Improve to some extent (1) Improve to a great extent (2)  Criminal Registry Office (BOP)  Reduction of case backlog  Criminal Justice Unit (PJP)  Quality Management Control Unit (UCQG)  Number of pending cases purged  Criminal Registry Office (BOP)  Criminal Justice Unit (PJP)  Quality Management Control Unit (UCQG)  Average time to process cases  Criminal Registry Office (BOP)  Criminal Justice Unit (PJP)  Quality Management Control Unit (UCQG)  Quality of decisions rendered  Criminal Registry Office (BOP)  Criminal Justice Unit (PJP)  Quality Management Control Unit (UCQG)  Diminution of duration of preventive detention  Criminal Registry Office (BOP)  Criminal Justice Unit (PJP)  Quality Management Control Unit (UCQG)  Preventive Detention Tables Judges, Prosecutors Court Clerks Other Court personnel, Attorneys, Community Leaders, Citizens, CSOs EQ1a, EQ2(b) #3 On a scale of 1 to 5, how would you rate the presence of court coordinators in each pilot court1: most helpful; 5 least helpful. Explain. Judges, Prosecutors Court Clerks Other Court personnel EQ1 #4 Regarding the joint training of magistrates and judicial police officers, do you think that the training actually contributed to a change in practice concerning the cooperation between the judiciary police and the magistrates, or do you think that the old routine reinstated after the participants went back to their courthouse or police station? Judges, prosecutors, JPOs EQ3 #5 Do you believe that the trainers selected by ProJustice to conduct the joint training between magistrates and JPOs make a meaningful difference concerning the cooperation between the judiciary police and magistrates? Yes. No. Explain? Judges, prosecutors, JPOs EQ3 #6 Have you noticed any change in the way the public is received in courthouses as a result of ProJustice activities? Yes. No. Explain. Judges, Prosecutors, court clerks, court staff. EQ2(a) EQ (4). #7 Regarding the computerization, do you estimate that the introduction of computerized case management system is motivated by a real concern about improving the justice delivery for court users or do you think the computerization is more of a communications operation? Judges, prosecutors, court clerks in pilot courts of Yopougon and Dabou EQ1 EQ1(a) #8 Have you been consulted about your needs by the developers of the computerized case management system? Yes. No. Judges, prosecutors, court clerks in pilot courts of Yopougon and Dabou EQ1, EQ1(a) COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-3 #9 Please evaluate to which extent you agree with the following statements: Fully disagree Mostly disagree Mostly agree Fully agree Don’t know The judicial system is faster The judicial system commands a greater trust in the institutions The judicial system is more accessible The judicial system is fairer, more impartial and less corrupt Judges, Prosecutors Attorneys EQ1, EQ1(a), EQ2 (a) and (b) #10 Please evaluate to which extent you agree with the following statements: Fully disagree Mostly disagree Mostly agree Fully agree Don’t know The trainings were oriented toward the betterment of rendering justice on a day-to￾day basis. The training were too reminiscent of law school teaching The training provided for an opportunity to increase cooperation among services (as applicable, judiciary police and prosecutors, among various actors of the judicial system: prosecutors and examining judges, judicial staff and private practice). The dispensers of continuing legal education listen to our needs The persons trained could disseminate what was learned to their peers. Judges, prosecutors, gendarmes and judiciary police officers who received training at the IFJ and the National Police School. INFJ and ENP staff? EQ3 #11 Have you heard of the following tools implemented by ProJustice? If so, have you seen the benefice of any of them? 1) Solutions boxes 2) Criminal Jurisprudence digest 3) Procedural templates 4) Practical Guide for prosecutor’s office 5) Code of judicial conduct 6) Handbook to fight sexual and gender-based violence Judges, Prosecutors, court staff, IGSPJ EQ 1 EQ3 #12 Do you estimate that the Inspection General of Judiciary Services performs is more visible now than prior to 2014? Have your courthouse been inspected by the IGSPJ in the last 12 to 24 months? Has it resulted in any specific outcome? Judges, Prosecutors #13 How would you rate the following tools about providing access to justice? 0 Unheard of; 1 Heard of, but did not notice any practical effect 2) Small measurable practical effect 3) significant practical effect 1) Access to Justice Guide? 2) Information and guidance kiosks 3) Posting the judicial fee scale in Pilot courts 4) Signage in pilot courts Judges, prosecutors, court staff, attorneys EQ2(a), EQ5 Do you estimate that the JSSP activities (e.g. seminars on professional conduct for magistrates, capacity building of the Inspection of judicial services in their role of counseling for the judiciary) have contributed to modify the perspective of your colleagues and yourself on ethical aspects? #14 Have you observed an increased awareness of judicial issues after the outreach campaigns took effect? If so, how would you rate it on a scale of 0 to 2: 0: negligible 1) noticeable 2) substantial Judges, prosecutors, court staff, attorneys EQ2(a) EQ5 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-4 #15 Have you noticed a change of perception in the past 3 years about judicial affairs? If so, would you say that the change is toward more legitimacy? Less? The same? Judges, Prosecutors, court staff, attorneys, MoJ staff EQ4 #16 What influence did NGOs have on the integrity of the judicial system as a mechanism of external control? Please indicate only one answer. 1. Negative 2. Neutral Positive Judges, Prosecutors, attorneys, MoJ Staff EQ2(b) #17 Do you support the reforms and mechanisms instituted under the JSSP in general? Fully To an extent No. (Explain why you don’t support the reforms and mechanisms) Judges, Prosecutors, CSOs, inspectors EQ1, EQ2(b) EQ3 #18 Why do you support the reform? Judges, Prosecutors, CSOs, inspectors EQ1, EQ2(b) EQ3 #19 Why not? Judges, Prosecutors, CSOs, inspectors EQ1, EQ2(b) EQ3 #20 What is your general impression of the quality of work of the judiciary in the past few years? 1. Very low 2. Low 3. Moderate 4. High 5. Very high Do you believe that the programs targeted at access to Justice have contributed to change your judgment on the judicial system? Yes. No. CSOs. Local radio EQ2, EQ 2(a) #21 Do you believe that the legal aid assistance dispensed through the Ministry of Justice and the Office of Legal Aid makes a difference? Yes. No. If you answered Yes, do you believe the aid dispensed goes to those in need of it or is it dispensed through arbitrary standards? Yes. No. Yes. No. Explain. CSOs, local radio EQ 2 EQ 2(a) EQ4 #22 Do you think the JSSP component promoting access to Justice have triggered a positive response from the judiciary? Yes. No. Please explain. CSOs, local radio EQ2(b) COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-5 #23 Do you believe that the JSSP activities have had a measurable effect on following factors undermining the trust of citizens in the judicial system? Please give your assessment on a scale of 1 to 4, where 1 represents “Not at all” and 4 represents “To a great extent”. Which of these factors remains the one which undermines trust in the judicial system the most? Not at all Mostly not To some extent To a great extent Chief factor Corruption in the judicial system Political/politicians’ influence on the court Poor, non-transparent personnel policy – how staff is recruited and promoted, appointed to senior posts Inadequate penalties for corruption Length of proceedings Content of court decisions Sensationalist / exaggerated media reports Lack of fairness Selective initiation of cases by the prosecution Other CSOs, local radio EQ2 EQ4 EQ5 #24 Do you believe that the awareness campaigns on the rural land tenure are useful contributions to alleviate the tensions concerning land? Yes. No. If not, please explain what is lacking in your perspective? CSOs community leaders #25 Do you estimate that the JSSP activities have contributed to render the media rendition of legal matters more articulate and better structured to inform the citizens? Yes. No. Explain Local radios CSOs EQ5 #26 Do you believe that the JSSP access to justice activities are sustainable, i.e. their benefice will survive once the program retreats? Yes. No. Explain. CSOs EQ5 #27 Did you ever seek information on your legal rights? If so, was it prior to JSSP activities started? Or since then? If the latter, was your access to legal right information a consequence of the program outreach? Community leaders, Citizens EO2, EO2(a) #28 How easy or difficult was it for you to access such information? 1. Very difficult 2. Mostly difficult 3. Mostly easy 4. Very easy Community leaders, Citizens EQ2, EQ2(a), EQ5 # 29 Do you believe that the solutions boxes installed in pilot courthouses are really meant to improve the way justice is delivered or do you think that making a suggestion is like sending a message in a bottle on the ocean? Community leaders, Citizens, court users EQ2(b) EQ4 #30 What sources of information on legal rights and the work of the judiciary are available to citizens? [Interviewer] Accept multiple responses. Show card. Community leaders, citizens EQ5 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-6 1. Internet 2. Television 3. Radio 4. Dailies and magazines 5. Court bulletin boards 6. Brochures, leaflets 7. Information service (via the telephone) 8. Information counter 9. Registry desk 10. Archive 11. Court staff 12. Lawyers 13. Friends, relatives, colleagues 14. Other For each of these sources of information, please indicate whether they were available before the JSSP activities and, if so, whether the JSSP has contributed to make them more accessible. #31 How accessible is the judicial system to you personally now compared to before the JSSP activities Still very inaccessible No change Still mostly inaccessible No or very little change Mostly accessible Small to moderate improvement Very accessible Substantial improvement Don’t know 1. In terms of finances – given the court-related costs court taxes, trial costs, travel costs)? 1 2 3 4 9 2. In terms of finances – given the attorney-related expenses? 1 2 3 4 9 3. In terms of geography – given the distance of the courthouse? 1 2 3 4 9 4. In terms of layout – how easy was it to find your way and move around the courthouse? 1 2 3 4 9 5. In terms of access to information 1 2 3 4 9 Community leaders, citizens EQ2, EQ2(a) EQ4 #32 I will now read out a number of statements on the judicial system. Please rate your agreement with each statement on a scale of 1 to 4 where 1 represents ‘fully disagree’ and 4 represents ‘fully agree’: Fully disagree Mostly disagree Mostly agree Fully agree Don’t know JSSP activities have contributed to better laws on the book 1 2 3 4 9 Community Leaders, Citizens EQ4 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-7 JSSP has contributed to a better enforcement of the laws in practice 1 2 3 4 9 JSSP has contributed to strengthen equality before the law 1 2 3 4 9 #33 Do you believe that the JSSP activities have had an effect on the following factors undermining trust in the judicial system? Please give your assessment on a scale of 1 to 4, where 1 represents “Not at all” and 4 represents “To a great extent”. Please rank by order the factors on which you think JSSP has had the most effect, if any. If you estimate that it had no effect, please enter 0. Not at all Mostly not To some extent To a great extent Chief factor Corruption in the judicial system Political/politicians’ influence on the court Poor, non-transparent personnel policy – how staff is recruited and promoted, appointed to senior posts Inadequate penalties for corruption Length of proceedings Content of court decisions Sensationalist / exaggerated media reports Lack of fairness Selective initiation of cases by the prosecution 10. Quality of the treatment of the public Other Community leaders, Citizens EQ2(b), EQ4 #34 Have you ever heard of the register to requests for legal aid? Have you observed that the delivery of registers to record requests for legal aid has contributed to an increase of citizens who receive legal aid? Community Leaders, Citizens in courthouses where registers for legal aid have been delivered. EQ2 EQ5 #35 According to information you have, were any efforts made to suppress corruption in the judiciary in the period 2013 to 2017? 1. No, no efforts were invested Community Leaders, Citizens, CSOs, attorneys EQ2(b), EQ4 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –F-8 2. Yes, minor efforts were invested 3. Yes, major efforts were invested DK (Don’t know – do not read out) If you answered 2 or 3, do you refer to JSSP efforts or other programs? #36 How efficient were those efforts? 1. They were inefficient 2. They were efficient, but not to a sufficient extent 3. They were very efficient DK (Don’t know – do not read out) Community Leaders, citizens, CSOs, attorneys Eq2(b), EQ4 #37 Have you heard about JSSP activities about reforming the administration of Justice? Yes. No. If you answered Yes, can you specify anything that has been done within the framework of this judicial reform? Community Leaders, citizens, CSOs, attorneys EQ1, EQ 2(a) and (b) #38 How well informed are you about the justice administration reforms launched under JSSP? Please reply on a scale of 1 to 5, where 1 represents ‘not informed at all’ and 5 represents ‘very well informed’. 1. Not informed at all 2. Mainly not informed 3. Fairly informed 4. Mainly informed 5. Very well informed Community Leaders, citizens, CSOs, attorneys EQ5 #39 When you think about the quality of judiciary work before and after the JSSP pilot courts, did you notice any difference? Was the judiciary work after the beginning of 2010 of higher quality, lower quality, or you did not notice any difference with regards to quality of judiciary work? Attorneys, Citizens who have appeared in court before and during JSSP EQ4 #40 When you think about access to legal information, did you notice any changes after the beginning of JSSP? 1. Access to information became easier 2. Access to information became more difficult 3. I did not noticed any changes Citizens, Citizens who have appeared before a court during JSSP, Community leaders EQ2(a) and (b), EQ4 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-1 ANNEX G: EXIT POLL SURVEY QUESTIONNAIRE Justice Sector Strengthening Program Mid-term evaluation survey Questionnaire Questionnaire du sondage pour l’évaluation à mi-parcours du programme de renforcement du Secteur de la Justice INFORMED CONSENT – CREFDI to use its standard intro Consentement éclairé – Hello. My name is ____________. I am from [CREFDI], an independent research organization. I do not represent the government of Côte d’Ivoire or any political party. We would like to learn about your experience at the court today and in the recent past. Bonjour. Je me nomme .Je suis du CREFDI, un centre de recherche indépendant. Je ne représente ni le gouvernement de CÔTE D’IVOIRE ni un parti politique. Nous aimerions apprendre de vos expériences avec le tribunal aujourd’hui et dans un passé récent. Your answers will be confidential, so please feel free to tell us what you think. This interview will take about 15 minutes. Vos réponses seront confidentielles, alors s’il vous plait sentez-vous libre de nous dire ce que vous pensez. Cette interview vous prendra environ 15 minutes. If you are happy to proceed, then let’s begin. Si vous êtes d’accord, commençons! NOTE: DO NOT BEGIN THE INTERVIEW UNLESS THE RESPONDENT GIVES HIS/HER CONSENT. IF THE RESPONDENT REFUSES, LEAVE THE HOUSE AND RECORD THE REFUSAL. Remarque: ne débuter pas l’interview à moins que le répondant donne son consentement. Si le répondant refuse, quitter le domicile et enregistrer le refus. If consent is granted: Si le consentement est obtenu: Thank you. As we go through the questions, please keep in mind that you may say ‘I don’t know’ to any question when you do not know the answer, and you may say ‘pass’ to any question when you do not feel comfortable answering. You may also terminate the interview at any time. Do you understand? Merci. Lors du parcours des questions, s’il vous plaît gardez à l’esprit que vous pouvez dire « je ne sais pas » à toute question lorsque vous ne savez pas quoi répondre, et vous pouvez dire « passer » à toute question lorsque vous ne vous sentez à l’aise pour répondre. Vous pouvez aussi arrêter l’interview à tout moment. Comprenez vous? [Proceed with interview only if answer is positive]. Démarrer l’interview seulement si la réponse est positive. Let’s begin. Commençons! COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-2 METADATA (to be filled out by enumerator) P1. Time interview started: Heure de début de l’interview: P5. Day of Interview Jour de l’interview Single code Monday Lundi 1 Tuesday Mardi 2 Wednesday Mercredi 3 Thursday Jeudi 4 Friday Vendredi 5 Saturday Samedi 6 Sunday Dimanche 7 P2. Interviewer name: Nom de l’interviewé: P3. Interviewer gender: F M Genre de l’interviewé: P4. Interview Date: Date de l’interview: D1. Court Tribunal NR 99 D2. Town Ville DK 88 NR 99 SECTION 1. Exit Interview Début de l’interview COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-3 1. Think of scale from 0 to 10, where 0 is the lowest score and 10 is the best. About how satisfied are you with the services provided in the courts on the scale from 0 to 10? Sur une échelle de 0 à 10, où 0 est le score le plus bas et 10 est le plus élevé. A quel point êtes-vous satisfait des services fournis par les tribunaux sur une échelle de 0 à 10 ? Score: DK 88 NR 99 2. Do you agree or disagree that you were treated fairly – without taking into account ethnicity, sex, social and financial status or age? Êtes-vous d’accord ou en désaccord que vous avez été traité équitablement- sans tenir compte de votre ethnie, sexe, statut social et financier ou âge ? Strongly agree Fortement d’accord 1 Somewhat agree Quelque peu d’accord 2 Somewhat disagree Quelque peu en désaccord 3 Strongly disagree Fortement en désaccord 4 DK 88 NR 99 3. Other than today, have you visited a court sometimes during the past 12 months? Hormis aujourd’hui, vous est il arrivé de vous rendre dans un tribunal durant les 12 derniers mois? Yes Oui 1 No Non 2 DK 88 NR 99 4. Compared to the last time you went to a court of justice, do you think the services provided in the courts today are a lot better, somewhat better, somewhat worse, a lot worse, or were your experiences about the same? COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-4 Comparez à la dernière fois où vous étiez au tribunal, pensez-vous que les services fournis dans les tribunaux aujourd’hui sont beaucoup mieux, quelque peu mieux, quelque peu pire, beaucoup pire ou vos expériences étaient les mêmes. A lot worse Beaucoup pire 1 Somewhat worse Quelque peu pire 2 About the same Les mêmes 3 Somewhat better Quelque peu mieux 4 A lot better Beaucoup mieux 5 DK 88 NR 99 5. Compared to the last time you went to a court of justice, do you think you were treated more fairly or less fairly, or were your experiences about the same? Comparer à la dernière fois où vous vous êtes rendu au tribunal, pensez-vous que vous avez été traité plus équitablement ou moins équitablement, ou de la meme manière. Less fairly Moins équitablement 1 About the same Les mêmes 2 More fairly Plus équitablement 3 DK 88 NR 99 Section 2: Office of Legal Aid Bureau d’aide juridique 6. Have you ever heard of the Ministry of Justice Office of Legal Aid? Avez-vous entendu parler du bureau d’aide juridique du Ministère de la Justice? COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-5 Yes Oui 1 No Non 2 DK 88 NR 99 7. If you or someone in your family needed free legal assistance from the Ministry of Justice, would you know how to access it? Si vous ou quelqu’un de votre famille avez besoin d’assistance juridique gratuite du Ministère de la Justice, savez-vous comment avoir accès à cela ? Yes Oui 1 No Non 2 DK 88 NR 99 8. How well do you think the Office of Legal Aid meets the needs of people like you, or don’t you know enough to say? A quel point pensez-vous que le bureau d’aide juridique reponds aux besoins des gens comme vous, ou n’en avez vous pas suffisamment entendu parler pour vous prononcer Completely meets peoples’ needs reponds complètement aux besoins des gens 1 Somewhat meets peoples’ needs reponds quelque peu aux besoins des gens 2 Somewhat fails to meet peoples’ needs Reponds peu les besoins des gens 3 Completely fails to meet peoples’ needs Ne reponds pas du tout aux besoins des gens 4 DK 88 NR 99 Section 3: Public Defense Fund Fonds public de protection COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-6 9. Were you a party involved in a case today, either as plaintiff, litigant, witness or victim? Étiez-vous impliqué dans une affaire aujourd’hui, soit comme plaignant, défendeur, témoin ou victime ? Yes Oui 1 Proceed to question 11 Continuer à la question 11 No Non 2 Proceed to question 10 Continuer à la question 10 DK 88 NR 99 10. Have you ever been involved in a case either as plaintiff, litigant, witness or victim? Avez-vous été impliqué dans une affaire soit comme plaignant, défendeur, témoin ou victime? Yes Oui 1 Proceed to question 11 Continuer à la question 11 No Non 2 Proceed to question 13 Continuer à la question 13 DK 88 NR 99 11. Did you receive financial assistance to ensure your right to a defense in court? Avez-vous reçu une assistance financière pour garantir votre droit de défense au tribunal? Yes Oui 1 No Non 2 DK 88 NR 99 12. Did you receive help from a public defender or did you hire your own lawyer? Avez-vous reçu une aide d’un avocat commis d’office ou avez-vous engagé votre propre avocat ? Public defender Avocat commis d’office 1 Hired lawyer Avocat engagé 2 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-7 Not applicable Non applicable 3 DK 88 NR 99 13. Has anyone you know ever received help from a public defender or the Ministry of Justice Public Defense Fund? Ya t’il quelqu’un que, vous connaissez, a-t-il reçu une aide d’un avocat commis d’office ou du fonds de protection du Ministère de la Justice ? Yes Oui 1 No Non 2 DK 88 NR 99 14. How would you rate the quality of the service received from the public defender? Comment noteriez-vous la qualité du service reçu de l’avocat commis d’office? Poor Faible 1 Fair moyen 2 Good Bien 3 Excellent excellent 4 DK 88 NR 99 15. How well do you think the Public Defense Fund meets the needs of people like you, or don’t you know enough to say? Comment pensez-vous que le fonds public de protection repond aux besoins des gens comme vous, ou n’en avez vous as suffisamment entendu parlé pour vous prononcer? Completely meets peoples’ needs 1 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-8 Repond completement aux besoins des gens Somewhat meets peoples’ needs Repond quelque peu Repond completement aux besoins des gens 2 Somewhat fails to meet peoples’ needs repond peu aux besoins des gens 3 Completely fails to meet peoples’ needs Ne repond completement pas du tout aux besoins des gens 4 DK 88 NR 99 16. In general, how well do you think the Ministry of Justice meets the needs of people like you? En général, a quel point pensez-vous que le Ministère de la justice repond aux besoins des gens comme vous? Completely meets peoples’ needs Repond completement aux besoins des gens 1 Somewhat meets peoples’ needs Repond quelque peu Repond completement aux besoins des gens 2 Somewhat fails to meet peoples’ needs repond peu aux besoins des gens 3 Completely fails to meet peoples’ needs Ne repond completement pas du tout aux besoins des gens 4 DK 88 NR 99 Section 4: Rights & Procedural knowledge Droits & connaissance procédurale 17. In general, do you think you are well informed about your rights? En général, pensez-vous que vous êtes bien informé sur vos droits? Yes Oui 1 No Non 2 DK 88 NR 99 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-9 18. Suppose you were asked to pay a bribe by a court official and wanted to report this case of judicial corruption. Are you confident that you would know how to report it? A supposer que qu’un agent du tribunal vous demande de payer un « pot de vin » et vous voulez rapporter ce cas de corruption. Êtes-vous sur que vous saurez comment le rapporter ? Yes Oui 1 No Non 2 DK 88 NR 99 Section 5: Demographics DEMOGRAPHIE 19. What is the highest level of school that you have completed? Quel est votre plus haut niveau d’instruction? Illiterate/none Illetré/aucun 1 Primary incomplete Primaire inachevé 2 Primary complete Primaire achevé 3 Secondary incomplete Secondaire inachevé 4 Secondary complete Secondaire achevé 5 High school Grande école 6 University undergraduate incomplete Universitaire inachevé 7 University undergraduate complete Universitaire achevé 8 Post university incomplete 9 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-10 Post universitaire inachevé Post university complete Post universitaire achevé 10 DK 88 NR 99 20. How old are you? Quel âge avez vous? NR 999 21. Sex of respondent (by observation) Sexe du répondant (par observation) Male Mâle 1 Female Femelle 2 22. Please tell me about the work you are now doing. Which of the following best describes your present status? S’il vous plait parlez-moi de l’emploi que vous faites maintenant. Laquelle des affirmations suivantes décrit le mieux votre statut actuel ? Employed full time (32+ hrs/week) Employé à plein temps (+32hrs/semaine) 1 Employed part time (15-32 hrs/week) Employé à mi-temps (entre 15 et 32 hrs/semaine) 2 Employed less than 15 hrs/wk Employé à moins de 15hrs/semaine 3 Unemployed Chômeur 4 DK 88 NR 99 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT –G-11 23. What is your current marital status? Quel est votre statut matrimonial actuel? Married or living together as married Marié(e) ou en concubinage 1 Widowed Veuf(ve) 2 Divorced Divorcé(e) 3 Married but separated/not living together Marié(e) mais séparé/ ne vivent pas ensemble 4 Single, never married Célibataire, jamais marié 5 DK 88 NR 99 THANK YOU FOR TAKING THE TIME TO ANSWER THESE QUESTIONS. MERCI D’AVOIR PRIS LE TEMPS DE REPONDRE à CES QUESTIONS. P6. TIME THE INTERVIEW FINISHED: TEMPS DE FIN DE L’INTERVIEW NR 999 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-1 ANNEX H: EXIT POLL SURVEY RESULTS (CREFDI) QUANTITATIVE SURVEY FOR JSSP PERFORMANCE EVALUATION 2017 IN CÔTE D’IVOIRE Including additional statistics from control group April 2017 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-2 METHODOLOGY SELECTION OF TARGET COURTS Côte d’Ivoire administratively and geographically is an association of regions (31) and districs (Abidjan and Yamoussokro). The representant of government in the region is the Prefect of Region. The regions are divided into departments and the representant of Government in the department is the Prefect of Department. The department is also divided into sous-prefectures and the representant of Government in a sous￾prefecture is the sous-prefect. We can also conclude that Côte d’Ivoire is the association of all sous-prefectures (considering the smallest level). The commune is a part of a sous-prefecture and is ruled by an elected mayor. All villages situated in a distance of 7 km from the commune are part of the commune. Beyond the 7 km the villages are considered as part of the sous-prefecture but no more part of the commune. Actually there is no more than one target court in a commune or sous-prefecture. (1) The final target courts that Mr. Renaud Beauchard and CREFDI agree to visit are: 1. Sinfra 2. Agboville 3. Bouake 4. Boundiali 5. Man 6. Yopougon (2) The communes where the baseline survey was conducted 1. Abengourou 2. Bouaflé 3. Bouaké 4. Bouna 5. Dabou 6. Dimbokro 7. Gagnoa 8. Lakota COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-3 9. Man 10. Sinfra CRITERIAS OF SELECTION 1/ Select 8 target courts considering those where evaluation team will go 2/ Select 8 target courts considering those where the baseline where conducted 3/ Select 8 target courts considering the period of implementation of JSSP. N° TREATMENT/CONTROL TARGET COURT PERIOD OF IMPLEMENTATION 1 TREATMENT MAN 2 SINCE 2013 BOUAKE 3 YOPOUGON 4 SINFRA 2013 - 2015 5 AGBOVILLE SINCE JUNE 2016 6 BOUNDIALI 7 CONTROL DIVO 8 ABOISSO 4/ Select 8 target courts considering the different categories of courts where the JSSP is implemented. COURT COURT OF FIRST INSTANCE COURT OF APPEAL SUPREME COURT AGBOVILLE MAN ABIDJAN ABIDJAN ABOISSO GAGNOA ABENGOUROU LAKOTA KORHOGO BOUAKE DABOU DIVO DALOA BONDOUKOU DIMBOKRO MAN BOUNA DALOA KORHOGO BOUNDIALI BOUAFLE SINFRA ABENGOUROU GUIGLO YOPOUGON SEGUELA PLATEAU BOUAKE COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-4 SELECTION OF RESPONDENTS Quantitative survey Sample size: 1069 respondents Treatment group: 866 respondents Control group: 203 respondents Sample universe: people visiting target courts, age 18 and older (both ivorians and non ivorians); at the end of each day a list of people visiting target courts is produced. Sample design: PPS considering people visiting target courts and selected with the same nth in all courts Respondent selection: exit interviews; selection of every nth individual; no more gender parity. Survey Languages: french, dioula (Approximated level of confidence of 95%) (Approximated margin of error between 3% and 5%) COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-5 Qualitative survey (qualitative interviews and focus groups) Target areas and groups: - 6 focus groups with population (Boundiali, Sinfra, Yopougon, Bouaké, Man, Agboville) considering that Renaud may work in the same target courts with officials. People interviewed in the quantitative survey were involved in the focus groups. Planning Overview TASK DURATION COMPLETION DAY TRAINING 4 days April 8th - 9th & 11th - 12th PRE TEST 1 day April 10th FIELDWORK 7 days April 14th - 20th DATA CLEANING 3 days April 21th - 22th DRAFT REPORT 1 day April 23th NOTE ON THE QUALITATIVE SURVEY REPORT The report of the qualitative survey should be read in conjunction with this report on the quantitative survey in order to better understand the results and thereby to evaluate performance. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-6 THE SAMPLE YOPOUGON BOUAKE DIVO MAN AGBOVILLE ABOISSO BOUNDIALI SINFRA Friday, April the 14th REFUSALS 87 6 10 2 0 3 2 10 UNIVERSE OF COURT USERS 2.093 618 181 138 172 126 40 74 RESPONDENTS 133 36 25 24 16 22 8 6 EACH Nth INDIVIDUAL = 4 Tuesday, April the 18th REFUSALS 76 15 18 0 1 5 2 2 UNIVERSE OF COURT USERS 1.901 924 504 362 225 189 169 22 RESPONDENTS 177 68 53 34 28 28 34 2 EACH Nth INDIVIDUAL = 6 Wednesday, April the 19th REFUSALS 52 24 11 0 0 4 3 3 UNIVERSE OF COURT USERS 1.373 1.252 385 252 275 342 126 161 RESPONDENTS 149 68 41 28 21 34 15 19 EACH Nth INDIVIDUAL = 6 Thursday, April the 20th REFUSALS 1 UNIVERSE OF COURT USERS 95 RESPONDENTS 12 EACH Nth INDIVIDUAL = 6 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-7 UNIVERSE OF STUDY 11.647 352 RESPONDENTS 459 172 119 86 65 84 57 39 UNIVERSE PER COURT 5.367 2.794 1.070 752 672 657 335 352 REFUSALS RATES 32% 21% 25% 2% 2% 13% 11% 29% RESPONSE RATES 68% 79% 75% 98% 98% 88% 89% 71% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-8 RESULTS TREATMENT GROUP Age years Maximum 85 18-35 ans 64% 36-55 ans 28% 56-87 ans 09% Sexe % Male 74 Female 26 Education % No formal education 05 Primary 12 Secondary 42 Post-secondary 41 Religion % Christian 67 Muslim 27 Other 06 Only one third of court users are muslims while there are 34% of christians and 43% of muslims in Côte d’Ivoire. CONTROL GROUP Age years Maximum 76 18-35 ans 66% 36-55 ans 28% 56-87 ans 06% Sexe % Male 85 Female 15 Education % No formal education 03 Primary 08 Secondary 54 Post-secondary 35 Religion % Christian 77 Muslim 18 Other 05 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-9 1. To what extent did JSSP interventions in the 11 pilot courts lead to improvements in case management? a. What change, if any, noted in increased the case management capacity in each court? There is a general enhancement of quality of service in the courts where JSSP is implemented. Even if we observe the same thing in Divo’s court where JSSP is not implemented. Tableau croisé VILLE/COMMUNE * 4. Comparé à la dernière fois ou vous entiez au tribunal, pensez-vous que les services fournis dans les tribunaux aujourd’hui sont beaucoup mieux, quelque peu mieux, pire, bien pire ou vos expériences étaient les mêmes ? % dans VILLE/COMMUNE 4. Comparé à la dernière fois ou vous étiez au tribunal, pensez-vous que les services fournis dans les tribunaux aujourd’hui sont beaucoup mieux, quelque peu mieux, pire, bien pire ou vos expériences étaient les mêmes ? Bien pire Pire Les mêmes Quelque peu mieux Beaucoup mieux Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 2,1% 10,7% 34,7% 33,7% 14,2% 4,7% 100,0% BOUAKE 1,4% 8,8% 41,9% 31,1% 14,2% 2,7% 100,0% MAN 1,2% 6,1% 30,5% 32,9% 25,6% 3,7% 100,0% DIVO 12,2% 18,3% 35,7% 31,3% 2,6% 100,0% AGBOVILLE 4,7% 21,9% 37,5% 34,4% 1,6% 100,0% ABOISSO 2,4% 3,6% 54,2% 14,5% 19,3% 6,0% 100,0% SINFRA 2,7% 10,8% 32,4% 21,6% 27,0% 5,4% 100,0% BOUNDIALI 2,2% 17,8% 44,4% 33,3% 2,2% 100,0% Total 1,5% 8,9% 33,5% 32,2% 20,1% 3,9% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-10 Tableau croisé VILLE/COMMUNE * 1. Considerant une echelle de 0 à 10, ou 0 est le score le plus bas et 10 est le plus eleve. A quel point etes-vous satisfait des services fournis par les tribunaux sur une echelle de 0 a 10 ? % dans VILLE/COMMUNE 1. Considerant une echelle de 0 a 10, ou 0 est le score le plus bas et 10 est le plus eleve. A quel point etes-vous satisfait des services fournis par les tribunaux sur une echelle de 0 a 10 ? Total 0 1 2 3 4 5 6 7 8 9 10 Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 2,4% 0,4% 1,5% 4,4% 11,8% 19,1% 16,4% 16,7% 17,5% 5,0% 4,6% 100,0% BOUAKE 2,3% 1,2% 4,7% 12,8% 16,3% 12,2% 13,4% 26,7% 5,2% 4,7% 0,6% 100,0% MAN 1,1% 1,1% 1,1% 3,4% 15,7% 5,6% 13,5% 28,1% 12,4% 17,9% 100,0% DIVO 1,7% 2,5% 2,5% 5,9% 21,8% 4,2% 21,8% 21,0% 3,4% 15,2% 100,0% AGBOVILLE 1,5% 1,5% 4,6% 7,7% 13,8% 9,2% 26,2% 26,2% 4,6% 4,6% 100,0% ABOISSO 1,2% 1,2% 1,2% 9,5% 11,9% 22,6% 19,0% 23,8% 2,4% 4,8% 2,4% 100,0% SINFRA 5,1% 7,7% 2,6% 5,1% 5,1% 17,9% 35,9% 15,4% 5,1% 100,0% BOUNDIALI 6,7% 4,4% 15,6% 2,2% 11,1% 26,7% 17,8% 15,6% 100,0% Total 1,9% 0,7% 1,4% 3,6% 9,5% 17,1% 12,5% 17,0% 22,4% 6,2% 7,4% 0,3% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-11 In fact, in the case below, Divo’s court where JSSP is not implemented gets the best improvement rate. We may assert that the general improvement is greatly due to the dynamism of the courts personnel. Tableau croisé VILLE/COMMUNE * 5. Comparé a la dernière fois ou vous vous êtes rendu au tribunal, pensez-vous que vous avez été traite plus équitablement ou moins équitablement, ou de la même manière ? % dans VILLE/COMMUNE 5. Comparé a la dernière fois ou vous vous êtes rendu au tribunal, pensez-vous que vous avez été traite plus équitablement ou moins équitablement, ou de la même manière ? Total Moins equitablement De la même mani?re Plus equitablement Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 16,8% 44,9% 30,8% 7,5% 100,0% BOUAKE 11,4% 49,0% 36,2% 3,4% 100,0% MAN 14,6% 35,4% 46,3% 3,7% 100,0% DIVO 15,7% 25,2% 57,4% 1,7% 100,0% AGBOVILLE 15,6% 32,8% 48,4% 3,1% 100,0% ABOISSO 7,3% 47,6% 41,5% 3,7% 100,0% SINFRA 13,5% 35,1% 45,9% 5,4% 100,0% BOUNDIALI 4,4% 44,4% 51,1% 100,0% Total 14,2% 41,5% 39,4% 4,9% 100,0% In terms of fair treatment, Sinfra, where the JSSP was implemented and then stopped, gets the worst rate. Tableau croisé VILLE/COMMUNE * 2. Etes-vous d’accord ou en désaccord avec l’affirmation selon laquelle vous avez ete traite équitablement- sans tenir compte de votre ethnie, sexe, statut social et financier ou age ? % dans VILLE/COMMUNE COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-12 2. Etes-vous d’accord ou en désaccord avec l’affirmation selon laquelle vous avez ete traite équitablement￾sans tenir compte de votre ethnie, sexe, statut social et financier ou age ? Total Fortement d’accord Quelque peu d’accord Quelque peu en désaccord Fortement en désaccord Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 49,1% 23,0% 13,6% 13,4% 0,9% 100,0% BOUAKE 72,7% 18,6% 5,8% 2,3% 0,6% 100,0% MAN 77,5% 10,1% 7,9% 3,4% 1,1% 100,0% DIVO 79,0% 10,1% 7,6% 3,4% 100,0% AGBOVILLE 63,1% 20,0% 9,2% 7,7% 100,0% ABOISSO 33,3% 44,0% 13,1% 4,8% 4,8% 100,0% SINFRA 46,2% 12,8% 7,7% 30,8% 2,6% 100,0% BOUNDIALI 88,9% 4,4% 2,2% 4,4% 100,0% Total 59,8% 20,1% 10,2% 8,9% 1,0% 100,0% 2. To what extent did JSSP interventions with non-governmental organizations and civil society organizations promote access to justice for the general public and outreach to the Office of Legal Aid? a. How has access to justice changed? At Boundiali where the JSSP is implemented, 6 court users out of 10 didn’t go to the court in the 12 last months. And Divo and Aboisso where the JSSP is not implemented got the best rates. Tableau croisé VILLE/COMMUNE * 3. Sans tenir compte d' aujourd'hui, vous est-il deja arrive de vous rendre dans un tribunal durant les 12 derniers mois? % dans VILLE/COMMUNE COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-13 3. Sans tenir compte d' aujourd'hui, vous est-il deja arrive de vous rendre dans un tribunal durant les 12 derniers mois? Oui Non Total VILLE/COMMUNE YOPOUGON 67,5% 32,5% 100,0% BOUAKE 71,5% 28,5% 100,0% MAN 76,4% 23,6% 100,0% DIVO 74,8% 25,2% 100,0% AGBOVILLE 72,3% 27,7% 100,0% ABOISSO 88,1% 11,9% 100,0% SINFRA 76,9% 23,1% 100,0% BOUNDIALI 44,4% 55,6% 100,0% Total 71,0% 29,0% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-14 Tableau croisé VILLE/COMMUNE * 17. En général, pensez-vous que vous êtes bien informe sur vos droits ? % dans VILLE/COMMUNE 17. En général, pensez-vous que vous êtes bien informe sur vos droits ? Oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 27,9% 71,5% 0,7% 100,0% BOUAKE 20,9% 77,9% 1,2% 100,0% MAN 37,1% 62,9% 100,0% DIVO 35,3% 64,7% 100,0% AGBOVILLE 36,9% 63,1% 100,0% ABOISSO 20,2% 77,4% 2,4% 100,0% SINFRA 23,1% 76,9% 100,0% BOUNDIALI 33,3% 66,7% 100,0% Total 28,3% 71,0% 0,7% 100,0% b. What change, if any, was noted in the administration of justice, especially criminal justice activities? 3. To what degree have JSSP training activities been integrated by targeted recipients (judiciary police, magistrates and court staff) into their respective work roles? COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-15 Even if Divo where JSSP is not implemented gets the best rate, in all of the courts the court users are not confident that they know how to report bribery. Tableau croisé VILLE/COMMUNE * 18. A supposer qu'un agent du tribunal vous demande de payer un pot de vin et vous voulez rapporter ce cas de corruption. Êtes-vous sûr que vous saurez comment rapporter ce fait ? % dans VILLE/COMMUNE 18. A supposer qu'un agent du tribunal vous demande de payer un pot de vin et vous voulez rapporter ce cas de corruption. Êtes-vous sûr que vous saurez comment rapporter ce fait ? Oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 15,4% 81,4% 3,3% 100,0% BOUAKE 14,5% 84,9% 0,6% 100,0% MAN 19,1% 79,8% 1,1% 100,0% DIVO 24,4% 72,3% 3,4% 100,0% AGBOVILLE 12,3% 87,7% 100,0% ABOISSO 20,2% 75,0% 4,8% 100,0% SINFRA 7,7% 89,7% 2,6% 100,0% BOUNDIALI 13,3% 84,4% 2,2% 100,0% Total 16,4% 81,1% 2,5% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-16 4. To what extent have the attitudes of the general public and media changed toward the Ministry of justice and the legal aid office? What are the key barriers that still need to be addressed to effectuate a change in perception of legitimacy? At least 6 court users out of 10 in all courts don’t know legal aid office and 9 out of 10 don’t know how to access to legal assistance. Tableau croisé VILLE/COMMUNE * 6. Avez-vous déjà entendu parle du bureau d'assistance juridique du Ministère de la Justice? % dans VILLE/COMMUNE 6. Avez-vous déjà entendu parle du bureau d'assistance juridique du Ministère de la Justice? Oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 28,7% 71,1% 0,2% 100,0% BOUAKE 30,8% 69,2% 100,0% MAN 36,0% 64,0% 100,0% DIVO 22,7% 76,5% 0,8% 100,0% AGBOVILLE 33,8% 66,2% 100,0% ABOISSO 26,2% 73,8% 100,0% SINFRA 25,6% 74,4% 100,0% BOUNDIALI 28,9% 71,1% 100,0% Total 29,0% 70,8% 0,2% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-17 Tableau croisé VILLE/COMMUNE * 7. Si vous ou quelqu'un de votre famille avez besoin d'assistance juridique gratuite du Ministère de la Justice, savez-vous comment avoir accès a cela ? % dans VILLE/COMMUNE 7. Si vous ou quelqu'un de votre famille avez besoin d'assistance juridique gratuite du Ministère de la Justice, savez-vous comment avoir accès à cela ? Oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 5,5% 91,2% 3,3% 100,0% BOUAKE 2,9% 93,0% 4,1% 100,0% MAN 5,6% 94,4% 100,0% DIVO 4,2% 85,7% 10,1% 100,0% AGBOVILLE 6,2% 93,8% 100,0% ABOISSO 9,5% 90,5% 100,0% SINFRA 7,7% 92,3% 100,0% BOUNDIALI 4,4% 95,6% 100,0% Total 5,3% 91,5% 3,2% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-18 Tableau croisé VILLE/COMMUNE * 8. A quel point pensez-vous que le bureau d'assistance juridique réponds aux besoins des gens comme vous, ou n'en avez-vous pas suffisamment entendu parler pour vous prononcer ? % dans VILLE/COMMUNE 8. A quel point pensez-vous que le bureau d'assistance juridique réponds aux besoins des gens comme vous, ou n'en avez-vous pas suffisamment entendu parler pour vous prononcer ? Total Réponds complètement aux besoins des gens Réponds assez aux besoins des gens Réponds peu aux besoins des gens Ne réponds pas du tout aux besoins des gens Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 3,6% 5,6% 13,2% 9,2% 68,4% 100,0% BOUAKE 1,7% 14,5% 2,3% 81,4% 100,0% MAN 2,3% 4,5% 5,7% 2,3% 85,2% 100,0% DIVO 5,0% 16,0% 27,7% 8,4% 42,9% 100,0% AGBOVILLE 6,2% 23,1% 20,0% 9,2% 41,5% 100,0% ABOISSO 7,1% 6,0% 15,5% 41,7% 29,8% 100,0% SINFRA 7,7% 2,6% 89,7% 100,0% BOUNDIALI 4,4% 11,1% 11,1% 6,7% 66,7% 100,0% Total 3,4% 7,2% 14,7% 9,6% 65,0% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-19 5. How effective have JSSP tools been in raising awareness and improving public understanding of and access to legal information and the public defense fund? What are the key lessons, best practices and challenges learned from this approach? Aboisso where JSSP is not implemented gets the best rate in terms of financial assistance received. Sinfra gets the best rates of positive perception of Public Defense Fund with only 10%. Tableau croisé VILLE/COMMUNE * 11. Avez-vous reçu une assistance financière pour garantir votre droit a être défendu au tribunal ? % dans VILLE/COMMUNE 11. Avez-vous reçu une assistance financière pour garantir votre droit a être défendu au tribunal? Oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 5,0% 93,8% 1,2% 100,0% BOUAKE 10,0% 90,0% 100,0% MAN 100,0% 100,0% DIVO 3,8% 96,2% 100,0% AGBOVILLE 12,0% 88,0% 100,0% ABOISSO 18,2% 81,8% 100,0% SINFRA 12,5% 87,5% 100,0% BOUNDIALI 100,0% 100,0% Total 6,1% 93,2% 0,7% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-20 Tableau croisé VILLE/COMMUNE * 12. Avez-vous reçu une aide d'un avocat commis d'office ou avez-vous engage votre propre avocat ? % dans VILLE/COMMUNE 12. Avez-vous reçu une aide d'un avocat commis d'office ou avez-vous engage votre propre avocat ? Total Avocat commis d’office Avocat que vous avez engagé? Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 12,5% 66,7% 20,8% 100,0% BOUAKE 44,4% 44,4% 11,1% 100,0% MAN 100,0% 100,0% DIVO 6,7% 40,0% 53,3% 100,0% AGBOVILLE 10,0% 90,0% 100,0% ABOISSO 83,3% 16,7% 100,0% SINFRA 100,0% 100,0% BOUNDIALI 100,0% 100,0% Total 12,0% 65,0% 23,0% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-21 Tableau croisé VILLE/COMMUNE * 14. Comment noteriez-vous la qualité du service reçu de l'avocat commis d’office ? % dans VILLE/COMMUNE 14. Comment noteriez-vous la qualité du service reçu de l'avocat commis d’office ? Faible Moyenne Bonne Excellente Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 16,7% 16,7% 50,0% 16,7% 100,0% BOUAKE 25,0% 50,0% 25,0% 100,0% DIVO 50,0% 50,0% 100,0% AGBOVILLE 100,0% 100,0% ABOISSO 100,0% 100,0% Total 14,3% 28,6% 28,6% 14,3% 14,3% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-22 Tableau croisé VILLE/COMMUNE * 13. Ya t il quelqu'un que, vous connaissez, qui a reçu une aide d'un avocat commis d'office ou du fonds de protection du Ministere de la Justice ? % dans VILLE/COMMUNE 13. Ya t il quelqu'un que, vous connaissez, qui a reçu une aide d'un avocat commis d'office ou du fonds de protection du Ministere de la Justice ? oui Non Total Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 12,3% 86,8% 0,9% 100,0% BOUAKE 9,3% 89,5% 1,2% 100,0% MAN 6,7% 93,3% 100,0% DIVO 10,9% 88,2% 0,8% 100,0% AGBOVILLE 12,3% 87,7% 100,0% ABOISSO 13,1% 85,7% 1,2% 100,0% SINFRA 20,5% 79,5% 100,0% BOUNDIALI 4,4% 95,6% 100,0% Total 11,2% 88,0% 0,7% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-23 Tableau croisé VILLE/COMMUNE * 15. A quel point pensez-vous que le fonds public de protection répond aux besoins des gens comme vous, ou n'en avez-vous pas suffisamment entendu parle pour vous prononcer? % dans VILLE/COMMUNE 15. A quel point pensez-vous que le fonds public de protection répond aux besoins des gens comme vous, ou n'en avez-vous pas suffisamment entendu parle pour vous prononcer? Total Répond complètement aux besoins des gens Répond assez aux besoins des gens Répond peu aux besoins des gens Ne répond pas du tout aux besoins des gens Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 3,1% 6,0% 13,4% 12,1% 65,3% 100,0% BOUAKE 0,6% 1,8% 7,0% 2,3% 88,3% 100,0% MAN 1,1% 3,4% 10,1% 85,4% 100,0% DIVO 5,9% 12,6% 42,0% 10,1% 29,4% 100,0% AGBOVILLE 7,8% 10,9% 17,2% 25,0% 39,1% 100,0% ABOISSO 6,0% 4,8% 22,6% 42,9% 23,8% 100,0% SINFRA 10,5% 2,6% 5,3% 81,6% 100,0% BOUNDIALI 2,2% 6,7% 4,4% 13,3% 73,3% 100,0% Total 3,1% 6,1% 14,9% 13,2% 62,7% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – H-24 Tableau croisé VILLE/COMMUNE * 16. En général, à quel point pensez-vous que le Ministère de la justice répond aux besoins des gens comme vous ? % dans VILLE/COMMUNE 16. En général, à quel point pensez-vous que le Ministère de la justice répond aux besoins des gens comme vous ? Total Répond complétement aux besoins des gens Répond assez aux besoins des gens Répond peu aux besoins des gens Ne répond pas du tout aux besoins des gens Ne sait pas [Ne pas lire] VILLE/COMMUNE YOPOUGON 8,6% 22,4% 49,9% 11,4% 7,7% 100,0% BOUAKE 6,4% 27,3% 55,8% 6,4% 4,1% 100,0% MAN 27,0% 32,6% 23,6% 12,4% 4,5% 100,0% DIVO 14,3% 30,3% 47,1% 5,0% 3,4% 100,0% AGBOVILLE 24,6% 27,7% 35,4% 10,8% 1,5% 100,0% ABOISSO 19,0% 13,1% 28,6% 39,3% 100,0% SINFRA 12,8% 51,3% 23,1% 7,7% 5,1% 100,0% BOUNDIALI 15,6% 11,1% 31,1% 8,9% 33,3% 100,0% Total 12,6% 25,1% 44,0% 11,9% 6,4% 100,0% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-1 ANNEX I: TETRATECH STATEMENT OF DIFFERENCES COTE D’IVOIRE JUSTICE SECTOR STRENGTHENING PROGRAM MID-TERM PERFORMANCE EVALUATION DRAFT REPORT STATEMENT OF DIFFERENCES PREPARED BY TETRA TECH DPK (in response to draft report submission) The present statement of differences is intended to contribute to improving the draft report of the mid-term evaluation of the Justice Sector Strengthening Program (JSSP). The statement of differences is a list of comments by Tetra Tech and JSSP management. The document below is divided in two parts, general comments and specific comments. A. General Comments 1. The report does not mention anywhere that despite being titled a “mid-term” performance evaluation, in fact the evaluation is coming after more than four years of activity and with only about 9 months of technical implementation remaining. While the report may need to keep that title, it is our view that the short implementation period remaining argues for reducing the number of recommendations and ensuring that they are focused on any future justice sector activities. This is because with only a few months before project close-out, JSSP will be unable to implement most of the recommendations. 2. The report clearly sets out the five questions that USAID wanted answered through the evaluation, but then most of the report is not spent answering those questions. The questions are included as headings but the material under each heading does not answer the question. As just one example, on pages 16-17 the section is supposed to be about whether JSSP interventions improved case management, but there are long paragraphs about the departure of court coordinators and about the supposed lack of involvement of the MoJHR and the donor community’s focus on hardware. These do not address the question. Another example is on page 20, where the question is whether JSSP interventions promoted access to justice, but one long paragraph talks about opinion leaders wanting payment to participate in awareness campaigns, and COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-2 another paragraph talks about MoJHR lack of cooperation. Again, these do not answer the question. 3. The background section of the draft report discusses at length the serious challenges and constraints faced by the Ivoirian justice system, however, these are not discussed adequately in the other sections of the report. The report also lacks an objective discussion on how these historical issues and administrative and political challenges may have significantly constrained the JSSP interventions. In the light of the state of the Ivoirian justice sector, which is perceived to be “corrupt, biased, inefficient, extremely punitive, and controlled by politics”, immediate, linear, and consistent results throughout the intervention areas may not be expected. 4. The draft report often mixes assessments of the JSSP program, activities, and performance with assessments of the institutional and political environment without mentioning the systemic approach and actions taken by the project to reinforce the MoJHR institutional capacity. This is the case for the computerization program: the draft report only mentions in passing the fact that prior to developing the software, JSSP prepared a Strategic Master Plan for the computerization of the judicial sector to provide the Government of Côte d’Ivoire (GoCI) with a planning and policy development tool. The same remark can be made regarding judicial assistance: the draft report does not mention the support (training, IT equipment, technical assistance to develop statistics tools) provided by JSSP to the central Office of Judicial Assistance (OJA) which has improved OJA’s capacity since 2013. The criticisms of the way OJA processes the individual requests for judicial assistance does not mention the improvements made such as the progressive decentralization of the screening process and the fact that OJA now directly administers its own budget whereas before 2017, all payments connected to a judicial assistance case had to be processed by the Judicial Agency of Treasury (Agence Judiciaire du Trésor). To mention these activities would help develop a more global view of JSSP. 5. The draft report does not cover many of the activities implemented by JSSP. For example, the assessment of JSSP Objective 2 (improve and increase access to justice) concentrates solely on the judicial assistance subcomponent but does not mention any of the other subcomponents, such as support to increase gender parity in the magistracy, publication of practical guides and compendia widely distributed to judicial officials and NGOs, information and orientation kiosks in the pilot courts, etc. It also does not define access to justice, but seems to interpret it as solely meaning the numbers of people accessing the courts. 6. The draft report often appears to take opposite views from those expressed by informants interviewed. It also sometimes relies too much on one informant’s COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-3 views, ignoring remarks made by other informants. The harsh criticisms on the computerization and judicial assistance programs are based on only one informant’s view (former team leader of the European Union (EU) funded project PARMSJP) whilst at least 4 informants for the computerization program and 3 for judicial assistance expressed much more favorable opinions but are not mentioned in the draft report itself; they only appear in annex 3. It is our view that significant reliance on this one particularly negative informant has affected the tone of the whole report. 7. There is an inordinate amount of time and attention paid to the computerization program, and at the same time there is an extreme negativity about everything connected to the computerization activity. It appears that most of the negative comments about the computerization program have come from an interview with the former head of the EU-funded PARMSJP project. It bears noting that that project failed to implement a computerization program within budget (1 million euro) and within its timeline, and thus handed over the activity to USAID/JSSP, which managed to implement it in less than one year and for less than $1 million. For that reason, this person may now have reasons to criticize JSSP (and USAID) for implementing a computerization program within budget and within a very tight timeline. It is unfortunate that the negative comments from one person have permeated what should have been an unbiased evaluation. We would also note that the computerization program is only a pilot program, and was only launched in February 2017, so it would be unwise to make sweeping statements about the effectiveness or the sustainability or even the implementation of the program which has only just been launched and is still being adjusted. We would also point out that the computerization activity is just one activity (representing about 3% of the funding) of the project and yet it has assumed an outsized role in this evaluation report. 8. Some conclusions and recommendations seem in contradiction with the assessment as outlined in the draft report and with the objectives of the program. “Consider disinvesting from the computerization experiment” (recommendation 3) at this stage, after such an important investment from JSSP, would seem counterproductive and a major loss of past investments. It would leave the software as presently is, and leave the requests for improvements from some end users unattended. In light of the extreme negativity about the computerization program throughout the report, it is also a dubious recommendation. B. Specific Comments COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-4 1. Page 2, Finding 1: In addition to the case management tools mentioned here, the transparency tools developed by JSSP are also well accepted in pilot courts: Solution Boxes, Court Fee Posters, and Court Signage. 2. Page 2, Finding 7: “One important threat to the use of the new tools is the lack of resources and the lack of maintenance of equipment”. This comment seems to refer to additional IT equipment required for the case management information system (CMIS). No particular equipment is needed for the court administration tools developed by JSSP, or if so it has already been provided by the project. 3. Page 3, Finding 11: “The MoJHR seems to be uninvolved [on the CMIS]”. This statement is not factual. The MoJHR may lack enough technical, financial, human, and material means to implement a thorough IT policy, but there is a real will to move forward on the computerization process of the judicial system. Page 3, Finding 17: “Perception of corruption of the court has improved in the courthouses of Agboville, Boundiali and Man but not in other courts involved in the evaluation”. This assertion is contradicted by JSSP’s data regarding the two PMP indicators on corruption: “Percentage of public knowing how to report on judicial abuse and corruption cases”; and “percentage of public reporting corruption attempts”. a. Percentage of public knowing how to report on judicial abuse and corruption cases For this indicator, there is significant improvement in six pilot jurisdictions, instead of just three as reported by the evaluation team (see table below): Courts April 2016 October 2016 January 17 Agboville 25% 32% 24% Bondoukou 30% 9% 37% Bouaké 25% 13% 15% Boundiali 22% 11% 15% Dabou 24% 30% 23% Daloa 13% 22% 28% Guiglo 22% 23% 18% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-5 Korhogo 16% 16% 20% Man 26% 34% 28% Séguéla 10% 7% 13% Yopougon 15% 13% 25% Aboisso 34% 39% 34% Divo 21% 11% 34% b. Percentage of public reporting corruption attempts The following table shows that all pilot courts have experienced a significant improvement. Similar changes are observed in the control courts due to the fact that JSSP also shares the collected data with them so that they can take decisions based on their strengths and weaknesses. This should also be taken into account in the interpretation of these indicators. Courts April 2016 October 2016 January 17 Agboville 19% 3% 4% Bondoukou 7% 3% 2% Bouaké 18% 4% 4% Boundiali 4% 1% 1% Dabou 16% 6% 4% Daloa 32% 4% 1% Guiglo 5% 5% 3% Korhogo 19% 3% 2% Man 5% 7% 3% Séguéla 7% 5% 4% Yopougon 37% 26% 10% Aboisso (c) 24% 6% 2% Divo (c) 15% 5% 2% COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-6 4. Page 3, Finding 20: “In terms of reduction in case backlog the differences observed between the means of jurisdictions that have participated in the program the longest— Bouaké, Dabou, Man and Yopougon—and the two control jurisdictions—Aboisso, and Divo is not statistically significant”. The comparison of control courts with these pilot courts is not relevant for this indicator. Indeed, the courts of Bouaké, Man, and Yopougon are much bigger than Aboisso and Divo, so the reduction in case backlog is less significant than in other pilot courts given the higher numbers of cases processed. To be relevant, the comparison with control courts for this indicator should be done with pilot courts of comparable size. 5. Page 3, Finding 21: “Joint training of JPOs/Prosecutors/IJs have contributed to the increase in the cooperation among the various actors of criminal justice”. It is necessary to mention here the positive impact of the project’s joint trainings for investigating judges (IJs) and their clerks, in order to improve efficiency of investigating offices, reduce case backlogs, and decrease procedural delays. 6. Page 4, Finding 25: “There is no statistically significant difference between court users in intervention and control jurisdictions in terms of the satisfaction with the services provided by the courts”. The latest court users’ satisfaction survey conducted by the project in March 2017 showed that the average level of court users’ satisfaction in many of the pilot courts has been higher (91% - 98%) than the two control courts (90%). As such a sweeping conclusion such as, “the program does not appear to be instrumental in increasing court users’ satisfaction with the services provided or influence their perception related to the effectiveness of the courts”, is not accurate. The fact that JSSP interventions have not yielded consistent results across all the pilot courts is also not something to critique the project for; rather, it is to be expected given the different dynamics involved in each of the pilot courts. Namely, all pilot courts are different in size and were at different starting points when JSSP began work in the court. A good impact study should not simply compare the results at any given time without taking into account the initial level of the different groups. Trend analysis and historical data should build on this impact study. This is valid for all PMP indicators. In addition, JSSP informed the evaluation team that control courts are small in comparison to most pilot courts. There are no other large jurisdictions of the size of Yopougon, Bouaké, Korhogo, or Man that could have been included in the group of control courts. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-7 Taking into account all of the above, apart from Dabou, the pilot courts of same or comparable size (Agboville, Bondoukou, Boundiali, Guiglo, and Séguéla) had statistically better scores than the control group. This is illustrated in the following table: Courts April 2016 October 2016 January 17 Agboville 94% 91% 93% Bondoukou 77% 98% 89% Boundiali 96% 97% 98% Guiglo 98% 98% 91% Séguéla 90% 81% 92% Pilot Courts 91% 93% 93% Aboisso (c) 84% 86% 83% Divo (c) 71% 81% 98% Control Courts 73% 83% 90% 7. Page 4, Finding 28: “Suggestion boxes highlight a sector affected by corruption, delays and a lack of attorneys”. The reasoning is reversed here. It is because there are issues regarding corruption, procedural delays, and lack of attorneys in the Ivoirian justice sector that JSSP had put in place solution boxes in all pilot courts so that (i) court users can express their opinion on the delivery of justice services, and (ii) justice sector decision-makers can accordingly take corrective actions at both the local and national levels. 8. Page 4, Finding 30: “Perception of legitimacy is not the same in all localities”. This statement is not clear. 9. Page 4, Conclusion 2: “The BOP has facilitated the production of information on a case but in terms of statistics production, its potential has not yet been realized”. This statement is contradicted by most of the informants mentioning the BOP (Criminal Registration Office) in Annex 3, as well as by feedback received from pilot courts which systematically use the BOP to issue the statistics regularly requested by the MoJHR. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-8 10. Page 4, Conclusion 3: “Without reliable statistics, it is difficult to assess the changes in case management in each courthouse”. It is not clear whether this is a general statement about the Ivoirian justice system or about JSSP’s pilot courts. The MoJHR has tackled this issue recently by creating the Direction of Planning and Statistics (DPS). JSSP believes that this is a key move to improve monitoring and performance of the Ivoirian justice sector, and therefore has decided to provide technical and material assistance to the DPS in order to strengthen its capacity to collect and assess judicial data. 11. Page 4, Conclusion 4: “Case management reforms are threatened by the low priority status given to the justice sector by the GoCI”. The reasoning is reversed in this sentence. It is because the justice sector shows serious weaknesses in Côte d’Ivoire that USAID decided to finance JSSP in order to develop sustainable case management reforms. It is not necessarily an issue of lack of will, but rather of lack of financial, human, and material means. 12. Page 4, Conclusion 5: “The serious risks involved in implementing a computerization project in a sector such as Justice that is given low priority the GoCI may have been underestimated”. This statement is not based on fact. The MoJHR may lack the financial, human, and material means to implement a thorough IT policy, but JSSP over the four years of its operation has observed real political will to move forward on the computerization process of the judicial system. 13. Page 4, Conclusion 6: “A pre-condition for the successful implementation of a computerization initiative in the justice sector is that the final user has a governance structure to appropriate and extend the CMS to the entire jurisdictional network. In the current context, that precondition is lacking”. This statement ignores the systemic approach and actions taken by JSSP to reinforce the chances of success of the CMIS program. It should be recalled that prior to developing the CMIS software, JSSP prepared a strategic Master Plan for the computerization of the judicial sector providing GoCI with an IT planning and policy development tool. As noted above in the General Comments section, the volume of negative comments about the computerization program is disproportionate (due to the dollar amount of the activity), and they are also questionable because the main negative interlocutor is from the EU project, which has a direct incentive to undermine USAID’s work in this area which succeeded where the EU project did not. 14. Page 5, Conclusion 16: “Prosecutors and IJs establish a direct causation link between the joint trainings and the improvement of JPOs work”. It is also necessary to mention here the positive impact of the project’s joint trainings on COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-9 IJs and their clerks, which have improved efficiency of investigating offices, reduced case backlogs, and decreased procedural delays. 15. Page 6, Conclusion 19: “The main barrier is the low priority accorded by the GoCI to the Justice sector”. As noted above, the reasoning is reversed in this sentence. It is because the justice sector shows serious weaknesses in Côte d’Ivoire that USAID decided to finance JSSP in order to develop sustainable case management reforms. It is not an issue of lack of political will, but rather of lack of financial, technical, human, and material means. This is the very purpose of JSSP and generally speaking, of justice reform programs in developing countries. We also do not understand where this conclusion comes from: which interlocutors from the GoCI have stated that the justice sector is a low priority? On the contrary, the highest levels of government have supported reform of the justice sector over the four years of JSSP implementation, and the MoJHR has supported and participated extensively in numerous activities both in Abidjan and in the regions. 16. Page 6, Recommendation 1: “Have a dedicated donor coordination meeting on the computerized CMS to expose and understand the reasons behind the short￾comings of the experimentation in Yopougon and Dabou”. This statement is not based on fact. The test phase of the CMIS in Dabou has been very positive so far; unfortunately, the head of this pilot court was not interviewed by the evaluation team. Regarding Yopougon, we have concerns that one person’s reactions, the Chief Prosecutor of Yopougon – someone who has only been in that position for two months and has not really been trained on the CMIS program – is being used as the basis for a sweeping critique. 17. Page 6, Recommendation 3: “Without signs of appropriation of the computerized CMS system (…), consider disinvesting from the computerization experiment”. See general comments above, #8. 18. Page 6, Recommendation 4: “Assist the MoJHR to provide the most useful and low technology tools (such as [preventive detention] tables) to non-pilot courts. Other court administration tools developed by JSSP should be mentioned here: BOP, PJP (Criminal Judicial Unit), procedural templates, and UCQG (Quality Control and Management Committee). 19. Page 6, Recommendation 5: “Provide assistance to the courthouses and the MoJHR’s directorate of evaluation and statistics to realize the full potential of the BOP for statistical purposes”. This is already an ongoing activity of JSSP; see also comments above on support to DPS. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-10 20. Page 6, Recommendation 6: “Work with the Directorate of civil and criminal affairs and the Inspectorate on instructions to be signed by the Minister to direct the use of JSSP tools (BOP, PJP, procedural templates, preventive detention tables)”. JSSP is already working with the MoJHR on this matter, and in particular with the DACP and the General Inspectorate. In accordance with its sustainability plan, JSSP is in process of:  Advocating with the MoJHR for the institutionalization, through administrative instructions, of the use of the management tools developed with the assistance of JSSP: BOP, PJP, UCQG, Pretrial Detention Table, procedural templates, Solution Boxes, and other best practices from the pilot courts;  Identifying champions in pilot courts who will help the MoJHR to extend the tools nationwide;  Transferring know-how to the MoJHR on the implementation of these tools, particularly through the INFJ and the General Inspectorate. 21. Page 6, Recommendation 7: “Investigate the reasons why the jurisdiction of Yopougon has such a negative attitude related to JSSP tools”. This seems an extrapolation from one person’s reactions, the Chief Prosecutor of Yopougon – someone who took office only two months ago and has not really been trained on the computerization program. In addition, our understanding from our conversations with the Chief Prosecutor of Yopougon is that her concerns are not related to all JSSP tools, but more precisely to the BOP and CMIS – and only in relation to the Prosecutor’s Office. This assertion cannot be extended beyond the Prosecutor’s Office and should be amended accordingly. See also the comments of the president of the Yopougon court in Annex 3 of the draft report. 22. Page 7, Recommendation 12: “Convince the MoJHR that the PJP is an essential tool to ensure that training activities are integrated in the long term”. We do not understand this comment as the MoJHR is already convinced of the importance of the PJP thanks to the work of JSSP both in the field and at the central level. JSSP is also working with the DACP to ensure the sustainability of the PJP in the long term (i.e. through administrative instructions). 23. Page 7, Recommendation 13: “Assist the MoJHR with nomination movement to ensure that the JSSP tools spread though the jurisdictional network and are not threatened in pilot courts”. We do not understand the language of this comment. We note that JSSP is working with INFJ so that dissemination of the JSSP tools is included in INFJ’s initial and continuing education programs. 24. Page 7, Recommendation 20: “Recalibrate future programming according to the lessons learned from the experience of working with sector considered a low COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-11 priority by the GoCI. High tech investments should not be made in this context”. It is because the justice sector shows serious weaknesses in Côte d’Ivoire that USAID decided to finance JSSP in order to develop sustainable reforms. It is not an issue of lack of political will, but rather of lack of financial, technical, human, and material means. This is the very purpose of JSSP and generally speaking, of justice reform programs in developing countries. In addition, after the end of JSSP, USAID and/or other donors should keep providing technical and material assistance to the General Inspectorate to improve efficiency and transparency of court operations, and to increase trust of citizens in the Ivoirian justice system. Please also see our general comments about the extreme negativity about the computerization activity – and in fact on most all JSSP interventions – throughout the evaluation report. 25. Page 7, Recommendation 21: “Efficiency of the courts should become the number one indicator for JSSP since congestion, costs, and delay are the problems most often pointed out by court users in most countries. Efficiency of the courts can be quantitatively measured and provides a sound basis for comparison”. It is not clear what is meant here by “efficiency” of the courts. JSSP has at least 3 indicators that measure improved case management: 1.2.1 Percentage reduction in case backlog 1.2.3 Mean case disposition time in courts assisted by USG in the area of case management 1.2.5 Ratio of all disposition to new case filings in courts assisted by USG in the area of case management 26. Page 7, Recommendation 22: “There are validity and reliability problems associated with self-reporting since those who agree to be surveyed may differ in significant ways from those who prefer not to. JSSP has several important indicators based on self-reporting (indicators 1, 1.3.6, 3.1, 2.1.2, 2.2.1, and 3.3) and should explore new ways to collect the data needed to track the performance of these indicators”. We understand and agree that the question of validity is an inherent weakness of perception-based surveys that rely on self￾reporting. However, the issue of selection bias or self-selection as indicated above may not be a factor in our context as the respondents are selected randomly to answer the survey questions. Moreover, the assertion that respondents who agreed to be surveyed may have different characteristics from those who chose not to respond is not based on any evidence. It should also be noted that this is exactly the same method used by CREFDI for its quantitative evaluation (although with perhaps a more neutral name): 'Exit poll Survey' or 'purposive sampling'. We find it odd that the evaluation report decries COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-12 a method that it uses itself. Moreover, we note that the sample size per jurisdiction was half of JSSP’s. Last, we do not understand how court users’ perception about the justice sector and knowledge of their rights could be captured otherwise than through self￾reporting by those same court users. No doubt, the six indicators mentioned above are important in gauging the project performance but in total, JSSP tracks 23 indicators relying on different data sources. 27. Page 8, last paragraph: The real strategic document governing the reform process is the “Document Opérationnel de Politique Sectorielle” (DOPS) adopted by the Council of ministers in June 2013. The DOPS has no time limits. The document that is referred to in the draft report is the “Plan d’Action de Politique Sectorielle” (PAPS). A first PAPS was adopted in 2013 to cover years 2013-16. A second PAPS was prepared in December 2016 (with JSSP assistance, not mentioned in the draft report), and is still under review in the General Secretariat of the Government. 28. Page 11, Figure 1. JSSP phased-in implementation model: There are errors in the list of pilot courts in two boxes. The boxes are reversed. 2016-2017: Agboville, Bondoukou, Boundiali, Daloa, Guiglo, Korhogo, Séguéla. 2013-2016: Abengourou, Bouaflé, Bouna, Dimbokro, Gagnoa, Lakota, Sinfra. 29. Page 13, Potential Bias: “The lack of priority given to the justice sector by the GoCI and its resultant heavy dependence on foreign donors resulted in a bias on the part of ministry officials, who do not wish to be too critical of the project. Most meetings ended with words of praise and a wish for a continuation of USAID support to the judicial sector and a JSSP follow-on”. The draft report makes sweeping assumptions about why interlocutors at the Ministry would criticize or praise the JSSP. Why is this a bias? MoJHR did not hesitate to criticize JSSP when it considered it necessary, e.g. misunderstanding in January 2016 on the solution boxes. On the contrary, the fact that most senior officials praise JSSP’s outcomes of the project should be considered as an indicator of its success. 30. Page 13, Potential Bias: “As the mission progressed, criticism from the interviewees in the pilot courts of other informants such as foreign aid counterparts and/or occurrences such as the monitoring committee on computerization started to indicate trends”. We don’t understand this remark. Most comments from the interviewees in pilot courts as they appear in Annex 3 COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-13 are extremely positive. In addition, donors’ responses may be motivated by other considerations than the efficiency of JSSP (e.g., potential bias coming from conflicting interests between donors). 31. Page 13, Potential Bias: “With these trends as evidence of possible dissatisfaction, the team was able to elicit more candid responses from ministry officials”. This method is very questionable. If evaluators were looking for “bad” answers, it is always very easy to frame accordingly the questions asked of the interviewees. This confirms the sense in reading the entire report that it is disproportionately negative. (For example, see footnote 10 which assumes – with no evidence whatsoever – that those not answering questionnaires are more likely to have even more negative responses than those answering questionnaires!) We reluctantly come to the conclusion that the interviewers set out to portray JSSP and USAID in a negative light, and so all evidence is interpreted to be the most negative possible. 32. Page 17, Finding 6: “Their departure upon the termination of donor funding has caused some negative effects”. The reasons for changing 7 pilot courts out of 11 and moving court coordinators to the new ones at the end of 2015 was not at all a termination of donor funding, but rather a decision jointly planned by USAID and GoCI dating back to the Request for Applications and also since the launch of JSSP in 2013 as a way to maximize the impact of JSSP by including more jurisdictions throughout the life of the project. Thus, over a 5-year period, JSSP was intended to cover 18 jurisdictions representing most geographical areas of Côte d’Ivoire. 33. Page 18, Finding 9, final part: “Work started in January 2016…” In fact the development of the software started in April 2016. 34. Page 19, Finding 11: We do not understand the reference to “vague answers” but we note that the computerization program is being implemented in two pilot courts and was only launched in February 2017 so this may explain why officials in the central MoJHR provided “vague answers.” 35. Page 20, Finding 12, and page 24, finding 23: “Between June 2015 and January 2017, only 173 applications for judicial assistance were received, mostly from the Abidjan region”. Based on statistics from the DACP, the number of applications received during the period mentioned is not 173 but rather 191, as per the following breakdown: Quarter Number of COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-14 Applications Oct-Dec 2016 26 Jul-Sep 2016 14 Apr-Jun 2016 39 Jan-Mar 2016 28 Oct-Dec 2015 26 Jul-Sep 2015 19 Apr-Jun 2015 39 Total 191 36. Page 20, finding 14: This is not correct. The only activities for which authorizations from the MoJHR are needed are the open house days. As this activity requires a special organization in the tribunals (suspension of hearings, presentations and conferences, magistrates and judicial personnel mobilized to guide the public and answer questions) the MoJHR needs to plan with the tribunal concerned. This has never caused any delays in the activity, as authorization is sought and granted ahead of time. 37. Page 21, Finding 16: “….. appreciation of improvements in the courthouse of Agboville, which has been a pilot court since the summer of 2015, and in Boundiali, which is a new pilot court…” Both Agboville and Boundiali are new pilot courts where JSSP started operating approximately at the same time. 38. Page 27, Finding 31, last line: “…the awareness of legal rights is much weaker than indicated in the reports of JSSP M&E team”. The draft report should include the source of information and display the figures comparatively. Given that JSSP used the same sampling method as CREFDI, the problem must come from the sample size. JSSP’s sample was 415 people per jurisdiction, what was the size of CREFDI’s sample? Moreover, in the absence of NGO activities during 2016, JSSP carried out two surveys on access to justice within 6 months; their results were practically identical. This shows that JSSP’s sampling methods are reliable, because they produced the same results. 39. Page 30, Conclusion 5: “The serious risks involved in implementing a computerization project in a sector such as Justice that is given low priority the GoCO may have been underestimated”. This is not correct and this is precisely why JSSP started the computerization program by preparing and distributing to the MoJHR a Master Plan for the computerization of the judicial sector. It also is another example of a sweeping negative statement or critique of JSSP that permeates the report. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-15 40. Page 30, Conclusion 7: Should this paragraph not be under the next section? 41. Page 31, Conclusion 8: It would be possible to measure the impact of the JSSP awareness campaigns by measuring the audiences. Thousands of people attending the rallies and listening to the radio programs, broadcast both nationwide and locally. 42. Page 31, Conclusion 9: “The campaigns were conducted before the decree of October 2016 was formally approved and before any evidence that the Office of Judicial Assistance was viable”. The campaigns were conducted before and after the decree of October 2016, and were adapted after the decree to reflect the new organization. 43. Page 31, Conclusion 10: “The judicial assistance office does not monitor what happens after it has granted an application”. This is not correct. Once the judicial assistance office has granted an application, it transfers the case to the Bar and follows up with the Bar. Once an attorney is found to take the case, the Bar normally informs the judicial assistance office. For obvious reasons, the judicial assistance office does not get involved in the case resolution and in the process that follows. “Even though very few judicial applications have been filed, there is already evidence of rationing as illustrated by the relatively high number of rejected applications…” The draft report offers no evidence that any of the rejected applications were for budget “rationing” reasons. The DACP representative during the learning seminar contradicted this information. No application was ever rejected on the ground of budget scarcity. 44. Pages 31 & 32, Conclusion 11: The whole paragraph reflects an erroneous interpretation of a positive development in the management of the judicial assistance budget. There are several factual errors in the paragraph. i. The exact budget allocated to the DACP for 2017 is XOF 189 million, not 170 million; ii. This amount does not represent the overall budget allocated by the State to judicial assistance. It only represents a portion that is given for direct administration by the DACP to facilitate and speed up payments to judicial auxiliaries (attorneys mostly) taking judicial assistance cases. Until 2017 all payment requests by auxiliaries had to be processed through the Judicial Agency of Treasury. That caused major delays in the process for which the attorneys were always resentful and became reluctant to take judicial assistance cases. The decentralization of the payment process under the direct responsibility of the DACP is an experiment made to facilitate payments and improve the relationship with the Bar. Once the XOF189 million budget allocated this year to the DACP is entirely spent, other requests for judicial assistance will be funded through the ordinary State budget and payments will be processed routinely by the Judicial Agency of Treasury. JSSP has worked intensively with the DACP to develop this COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-16 experiment, as well as on the decentralization of the screening of judicial assistance requests in each tribunal. 45. Page 32, Conclusion 12. “This has not been considered or dealt with by JSSP”. This was carefully considered by JSSP. This is the reason why we organized training sessions for attorneys on judicial assistance and on the payment mechanisms of fees related to judicial assistance cases. 46. Pages 34 and 35, Conclusion 23: There seems to be a confusion between legal and judicial assistance. Initially, in the technical proposal, JSSP was supposed to invest in both mechanisms: legal assistance, through the recruitment of additional community legal advisers for legal clinics to support the PALAJ project (in which JSSP is involved as a member of the PALAJ steering committee) funded mainly by the EU and UNICEF; and judicial assistance, through a multi-dimensional program composed of institutional support, training, and awareness campaign on access to justice and judicial assistance. Based on a mutually agreed division of responsibilities between JSSP and the EU-funded PARMSJP project, JSSP decided to focus on judicial assistance and to implement the planned program. To say that JSSP “never recovered from that initial choice” is totally out of context and is again an example of the automatic negative interpretation of everything to do with JSSP that permeates the entire evaluation report. “…and had difficulties in finding a distinctive voice in the heavily populated donor landscape of civil society driven activities”. JSSP is to the best of our knowledge the only project supporting judicial assistance, and we did it in coordination with the PARMSJP (with whom we co-organized one seminar) and with other donors active in the judicial sector. 47. Page 36, Recommendation 10: “Develop better synergies with the legal clinics experiment”. JSSP has a direct relationship with the legal clinics and with the PALAJ project. Since the PARMSJP project ended in 2015, the legal clinics have been funded by the French C2D (Contrat désendettement-développement), which will stop its support in July 2017. Thereafter the MoJHR is supposed to take over the funding of legal clinics but this remains to be confirmed. If this does not happen, it will probably mean the end of the legal clinic experiment. Thus we do not understand the above recommendation to develop better synergies with legal clinics whose future looks uncertain. We have in the past included legal advisers of legal clinics in some of JSSP training activities to develop common objectives but we are not planning further synergies as legal and judicial assistance have their own specific mechanisms. JSSP also believes that, thanks to JSSP’s complementary approach (reinforcing the MoJHR’s central and local institutions managing judicial assistance, and boosting demand by a better information to the public), it is judicial assistance that actually has a much better chance of long-term sustainability. COTE D’IVOIRE JSSP MIDTERM PERFORMANCE EVALUATION REPORT – I-17 48. Page C-17, Interview of Kouakou Brou, Director of the National Judicial Training Institute (INFJ): “Better coordination with the INFJ when ProJustice organizes trainings”. JSSP always coordinates with INFJ before launching any training initiatives. All the project’s training modules are designed and developed together with the INFJ team, and are implemented in accordance with INFJ’s needs and priorities. JSSP supported the INFJ itself with its strategic plan, and has worked closely to plan and make changes to the initial and continuing training for magistrates. 49. Page C-24, Interview of François Lebur, former head of the technical assistance for the EU. “The selected consortium [for the development of the CMIS] was not the preferred candidate of the MoJHR’s head of IT services”. This statement is false. Mr. Bole was a full member of the selection committee for the RFP on the CMIS, and he attended all meetings where he participated actively. The selection committee unanimously – including Mr. Bole – chose the selected consortium. 50. Page C-43, Interview of Elodie RICHE, Governance Attachée, French Embassy: “ProJustice not always clear in exchanges with donors. Insularity”… “Other donors are not consulted enough on the elaboration of the new justice sector action plan conducted by ProJustice”… “ProJustice does not coordinate its trainings enough with the INFJ”… “ProJustice coordination is too institutional. ProJustice should participate more in informal gatherings among donors”. This is the first time we hear such complaints, either from the French Embassy or from any other donor. On the contrary, the MoJHR and most donors active in the justice sector praise JSSP’s openness, sharing of information and willingness to cooperate (e.g. GIZ for Germany, JICA for Japan). In addition, JSSP has always invited the French Embassy and AFD to coordinate, whether bilaterally or together with other donors, in several key areas such as training of magistrates and clerks, CMIS, and judicial assistance. We fear that – as indicated openly in the report – the evaluators actively solicited negative feedback and have then included only negative feedback in this evaluation report.