1 September 27, 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the USAID Dominican Republic Monitoring, Evaluation, and Learning Platform. .. .... Final Report: Whole-of-Project Mid-Term Performance Evaluation of the Criminal Justice System Strengthened Project FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 2 Photo caption: Meeting of members of the Local Coalition for Security and Crime Prevention supported by USAID/DR through the PASJ activity, in Puerto Plata, Dominican Republic. Photo credit: Participación Ciudadana. This publication was produced for review by the United States Agency for International Development. It was prepared by Panagora Group for the USAID/Dominican Republic Monitoring, Evaluation, and Learning Platform Task Order, IDIQ No. AID-OAA-I-15-00025, Task Order No. AID￾517-TO-16-00003. Panagora Group’s Evaluation Team: Charles Jakosa, Melanie Reimer and William J. Cartier. Recommended citation: USAID Dominican Republic Monitoring, Evaluation, and Learning Platform Task Order. 2018. Final Report. Whole-of-Project Mid-Term Performance Evaluation of the Criminal Justice System Strengthened Project. Silver Spring, Maryland: Panagora Group. Panagora Group Contact: William J. Cartier Chief of Party Panagora Group 8601 Georgia Avenue, Suite 905 Silver Spring, MD 20910 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 3 FINAL REPORT WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION OF CRIMINAL JUSTICE SYSTEM STRENGTHENED PROJECT September 27, 2018 USAID Dominican Republic Monitoring, Evaluation, and Learning Platform Contract No. AID-OAA-I-15-00025, TO No. AID-517-TO-16-00003 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or of the United States government. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 4 TABLE OF CONTENTS Acronyms 5 I. Executive Summary 7 Evaluation Background and Purpose 7 Evaluation Questions, Findings, and Conclusions 7 Effectiveness of the CJSSP Activity 7 Effectiveness of the ICITAP Activity 9 Effectiveness of the PASJ Activity 10 Evaluation Recommendations 16 I1. Evaluation Purpose, Questions and Methodology 20 Justice System Strengthed Project Background 20 Evaluation Purpose and Questions 21 Evaluation Approach and Methodology 23 Constraints and Limitations 26 III. Findings and Conclusions 28 CJSSP Results and Challenges 28 ICITAP Results and Challenges 34 PASJ Results and Challenges 37 Overall Project Results 42 Links to Overall Project Goal 47 Project Coordination 51 Challenges and Course Corrections 54 Project Engagement with GODR 56 Sustainability of Project Results 58 Project Results in GBV 61 IV. Recommendations 64 Annex A. Summary of Responses to Survey of Trainees 68 Annex B. Comparison of PAD to Activity Mechanisms 74 Annex C. Written Resources Reviewed 85 Annex D. Key Informant Interviews 89 Annex E. Data Collection Instruments 95 Annex F. Evaluation Scope of Work 112 Annex G. Conflict of Interest Declarations 118 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 5 ACRONYMS ADR Alternative Dispute Resolution AECID Spanish Agency for International Cooperation for Development AGO Attorney General’s Office AIEH Asociación de Empresas Industriales de Herrera y Provincia Santo Domingo AMCHAM American Chamber of Commerce AMELP Activity Monitoring, Evaluation, and Learning Plan AMUPREV Municipal Alliances for Violence Prevention ANJE National Association of Young Entrepreneurs AOR Agreement Officer Representative CA Cooperative agreement CBO Community based organization CBSI Caribbean Basin Security Initiative CDCS County Development and Cooperation Strategy CJH Community Justice House CJSSP Criminal Justice Systems Strengthened Project COIN Center for Research and Integrated Treatment CLIMA Urban Resilience to Climate Change Project CNM National Council of the Magistracy (Consejo Nacional de la Magistratura) CONADIS National Council on Disability CONANI National Council for Youth and Adolescence CONEP National Business Council COR Contract Officer Representative CSO Civil society organization DN National District, Santo Domingo DNP Dominican National Police DO Development Objective DQA Data Quality Assessment DR Dominican Republic ENJ National Judicial School ENMP National School of the Public Ministry FEDOMU Dominican Federation of Municipalities FGD Focus Group Discussion FINJUS Foundation for Institutionality and Justice FY Fiscal Year GBV Gender Based Violence GBVVU Gender Based Violence Victims´ Unit GCC Global Climate Change GODR Government of the Dominican Republic IADB Inter-American Development Bank IAWJ International Association of Women Judges ICITAP International Criminal Investigative Training Assistance Program FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 6 INACIF National Forensics Institute INL Bureau of International Narcotics and Law Enforcement Affairs IP Implementing Partner IR Intermediate Result JIG Judiciary Inspector General KII Key Informant Interview LGBT Lesbian, Gay, Bisexual, and Trans-gender MEL Monitoring, Evaluation, and Learning MINERD Ministry of Education of the Dominican Republic MIP Ministry of Interior and Police MP Public Ministry MW Ministry of Women PACAM Foundation for Abused Women PAD Project appraisal document PAPA Participating agency program agreement PASJ Civil Society Action for Accountable Justice and Security PC Citizen Participation (CSO Participación Ciudadana) PLD Dominican Liberation Party (Partido de la Liberación Dominicana) PMP Performance Monitoring Plan PWD People with Disabilities QPR Quarterly Progress Report RP Results Package SDW Santo Domingo West SOW Scope of Work TA Technical Assistance USAID United States Agency for International Development USAID/DR United States Agency for International Development/Dominican Republic USG United States Government WB World Bank FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 7 I. EXECUTIVE SUMMARY EVALUATION BACKGROUND AND PURPOSE This mid-term performance evaluation of the USAID/DR Criminal Justice System Strengthened Project uses a whole-of-project lens. The purpose of the Justice Project is to “strengthen the Government of the Dominican Republic’s (GODR) criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”. It responds to the USAID/DR Country Development Cooperation Strategy (CDCS) Intermediate Result (IR) 1.3: Criminal Justice Institutions Strengthened, which contributes to the CDCS Development Objective (DO) 1: Crime prevention strengthened. The theory of change is that the Project “will advance the DO1 crime prevention objective by improving the deterrent value of prosecutions as a government response to crime and increasing the overall transparency and credibility with which the Dominican criminal justice system operates”. The Project Approval Document (PAD) identifies two sub-purposes: 1) increase GODR output and effectiveness in prosecuting crime; and 2) increase public engagement and oversight in Dominican law enforcement and crime prevention efforts. This evaluation looks at the performance of three mechanisms or Activities under the PAD:1 • Criminal Justice System Strengthened (CJSSP). Contract with Chemonics International; $21,536,654, starting on June 8, 2015 and ending on June 7, 2020. The objective is to strengthen the criminal justice system in the Dominican Republic by improving the quality and effectiveness of prosecution. • Institutional Capacity and Transparency Strengthening for Police Reform (ICITAP). Participating Agency Program Agreement (PAPA) with the United States Department of Justice; $2,800,000, starting on July 20, 2015 and ending on July 19, 2019. The objective is to strengthen institutional capacity and transparency within the Dominican National Police (DNP). • Civil Society Action for Accountable Justice and Security (CSAAJS/PASJ). Cooperative Agreement (CA) with Participación Ciudadana (PC); $6,400,000, starting on June 22, 2015 and ending June 21, 2018. The objective is to increase public awareness on criminal justice and public safety, especially in the institutional roles and responsibilities of the Public Ministry and the DNP in crime persecution and prevention. EVALUATION QUESTIONS, FINDINGS, AND CONCLUSIONS USAID/DR identified ten evaluation questions (EQs) about the effectiveness of the three Activities under the Justice Project, including some sub-questions for clarification. The EQs and respective sub￾questions are listed in the body of this report. Effectiveness of the CJSSP Activity EQ1. Have the three Activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ2. What have been the major achievements and challenges of the CJSSP activities conducted to increase prosecutor effectiveness and access to justice for vulnerable groups? 1 In this report “Activity” or “Activities” refer to the implementation of the mechanisms funded by USAID, whereas lower case “activity” or “activities” are the actions undertaken by the implementing partners. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 8 Main Findings: • With CJSSP, USAID aimed to strengthen management of high volume fiscalías (prosecutors’ offices) by expanding and adapting management models established under a previous USAID activity. However, progress on adapting/improving these management models was set back in 2016 by technical difficulties in finalizing an assessment conducted by CJSSP, and by the division of Santo Domingo Province fiscalía (the primary target for this intervention) into three separate fiscalías. • In the first two years of the Activity CJSSP dedicated significant effort to assessments, consultations and planning processes related to a diverse range of areas of intervention, designed to update and deepen the understanding of issues, and inform strategy and program design. The implementer team’s attention was thus spread across multiple potential subject areas and target institutions, which delayed the design and implementation of interventions. • Because CJSSP was unable in the first year to make progress towards implementing a new management model, when leadership of the Attorney General’s Office (AGO) changed, there were no significant technical assistance activities under way in the targeted fiscalías, and it took most of Year 2 to engage AGO senior management again. AGO counterparts in the fiscalías speak highly of the CJSSP team’s efforts to engage with the PM and the AGO but say that little was achieved. • In Year 2, CJSSP provided support to the anti-corruption unit of the AGO, which has been well received. And in the beginning of Year 3 it initiated technical assistance activities in the Santo Domingo West (SDW) fiscalía. • The planned expansion of the Community Justice Houses (CJH) from 8 to 15 did not happen, as the GODR was not open to assuming operational costs, in part owing to the close association of the CJHs with sub-grantee PC, whose social activism generates resistance in the GODR. An assessment by CJSSP showed that CJHs sustainability is more challenging than originally thought, despite efforts to create their own legal identity. • CJSSP supported a large-scale training effort reaching almost 3,000 participants, primarily through the Public Ministry’s National School (ENMP). Many participants received multiple trainings. However, CJSSP monitoring data does not identify the number of unique individuals, which could be in the range of 1,000, as participants report taking multiple courses. This would represent almost all of the estimated 1,000 to 1,200 prosecutors in the country. • The majority of respondents to the evaluation survey on the training programs, mostly recent entries to the AGO, said that they had applied most of what they learned in their work, and that training had helped them to speed up case management. Training participants in FGDs said that training was of high quality and relevance, and some learning had been directly applied in their subsequent work. However, little information has been captured by CJSSP on the effects of its training efforts. Training via the ENMP was not focused on the targeted localities in the Duarte corridor or on high volume fiscalías as set out in the PAD. • Despite efforts by CJSSP to engage with the AGO on oversight and integrity of the prosecution service, this core institution has shown little interest. CJSSP extended the activity to the Judiciary Inspectorate General (JIG), which has shown positive effects but contributed to dispersion in activities, as it is outside of the PAD purpose. Work on DNP oversight was removed from scope by USAID, along with other activities related to the police. Conclusions: • CJSSP results are mostly outputs spread over a wide array of results areas with diverse beneficiaries and counterparts, and with no clear arrangements for geographic focus or co￾location with other Justice Project Activities. The large-scale training effort funded by CJSSP is FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 9 appreciated by justice operators interviewed, and especially among the Junior Prosecutors (fiscalizadores) who responded to the survey. One weakness was the inability to tie training to the work on new management models. • Overall, beyond the training, CJSSP has made little measurable progress towards expected Activity results over the first two and a half years of implementation. This, despite broad recognition among justice sector operators of CJSSP’s knowhow and consistent efforts at engaging AGO leadership, and the quality of the training funded by USAID through the ENMP and ENJ. • Insofar as improving prosecutor effectiveness, the services for vulnerable groups and training of justice operators have reached large numbers at significant cost without any clear evidence of impact or change, while most other areas of the Activity remain in the initial stages, especially in terms of enhanced oversight of the AGO. Dramatic changes in scope of the Activity by USAID and other challenges, primarily external, have significantly affected the ability of CJSSP to move beyond the foundation-laying phase of the interventions. Effectiveness of the ICITAP Activity EQ1. Have the three Activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ 3. What have been the major achievements and challenges of the ICITAP activities conducted to strengthen police management and transparency? Main Findings: • The ICITAP work plans have dropped two of three intended results relating to strengthening police responsiveness to vulnerable populations, external oversight, internal ethics, and police￾community relations. Activities were instead restricted to strengthening management processes and focusing on Directorates in Police Headquarters in Santo Domingo, though those efforts did have an impact nationwide, e.g., budgeting, salary increases throughout the DNP, etc. This de￾scoping forestalled the achievement of PAD results in the targeted localities in the Duarte corridor. • In strengthening management processes, ICITAP has been instrumental in operationalizing the capacity-building objectives of the DNP’s 2016-2020 Strategic Plan, creating 28 annual operational plans covering all DNP Directorates and requiring activity-based budgeting. Almost unanimously, senior DNP management stated that ICITAP support had helped to create a shift in the culture of the DNP away from “management by personality”. • The DNP’s share of the overall GODR budget for current expenditure increased from 1.60% in FY15 to 1.72% in FY18. While this cannot be fully attributed to ICITAP, informants stated that activity-based budgeting allowed the DNP to better defend its budget request. Conclusions: • ICITAP has been able to engage with DNP senior leadership in Santo Domingo, which has allowed it to generate significant, sustainable improvements in DNP management that in the long term should improve policing. However, USAID/ICITAP eliminated those activities which would have a direct impact on the PAD results of increased police responsiveness, accountability, transparency and relations with the community within the short time frame of the CDCS. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 10 Effectiveness of the PASJ Activity EQ1. Have the three Activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ4. How effective have the civil society activities been in creating demand for justice and monitoring law enforcement and crime prevention efforts, both at the national and local levels? Main Findings: • Many informants see the objective of PASJ to train leaders of civil society organizations (CSOs) on human rights and the duties of justice sector operators and the police, and to take direct action on crime prevention, rather than monitoring law enforcement, crime prevention or prosecution. There have been few activities directly relating to prosecution effectiveness or oversight, other than a series of community dialogues to identify priority issues for local coalitions. • Some actions to raise awareness of journalists, communicators, and CSOs working on rights of vulnerable populations have been small scale, of short duration and lacking follow-up. Some specialized NGOs working in these areas criticize PC for not working effectively with existing networks. • PASJ has supported a national coalition of organizations to lead policy research and advocacy on a range of policy reforms. The coalition has been dominated by organizations closely linked to the business sector, as opposed to activist or human rights organizations. Divisions in the coalition have emerged on high-profile policy issues in Year 2. • The PASJ IPs have been effective in convening civil society organizations (CSOs) to support justice strengthening. The national coalition has had an important role in approval of legislation directly linked to the PAD purpose, such as the Police Law and Law on Firearms, while other issues addressed by the coalition, such as laws on political parties, elections and regulation of notary publics, are outside the PAD framework. • At the local level, PASJ has been effective in creating “five and a half” coalitions and mobilizing broad based local coalitions to lead crime prevention activities.. They have not engaged with the PM, AGO or local fiscalías on issues of prosecutor effectiveness. The local coalitions have worked on prevention and response to gender-based violence (GBV); police corruption and abuses of vulnerable populations; and mediation of school violence. However, they lack a broad membership base and practical “know how”, and have gained little traction with municipalities, GODR agencies, fiscalías or DNP. Conclusions: • Overall, PASJ has made progress towards most of the expected results of the CA in the first two years, but few of the results in the CA are consistent with the focus of the PAD on prosecutor effectiveness. Moreover, PASJ activities at the national level have been widely dispersed thematically, leading to the “miniaturization” of many actions aimed at awareness raising about justice sector issues. • The PASJ coalition made important contributions to the approval of two legislative reforms at the national level and has had effective engagement with justice sector operators and sector experts, but its efforts have been dispersed in advocacy for reforms unrelated to the PAD results. Differences on how to address controversial issues undermined coalition activities. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 11 • The local coalitions have made progress in mobilizing local CSOs and in identifying issues, but they have a low capacity for initiating prevention activities, which undermines their ability to engage with other local stakeholders. • While PASJ has addressed some issues of prosecutor effectiveness and oversight at the national level, this has not been taken up by the local coalitions. They have focused on crime prevention and deterrence. Overall Results of the Justice Project EQ5. Taken as a whole, have the three Activities produced the results set out in the CDCS, “Criminal justice institutions strengthened”, and in the Justice PAD, “to strengthen the GODR criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”? EQ6. Taken as a whole, have the Activities in support of justice sector reform supported the expected links between the overall goal of crime prevention and the intermediate results of improvement in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities? Main Findings: • The CJSSP Year 1 and 2 Work Plans specified many expected results and hundreds of diverse actions. In CJSSP, many tasks have not been implemented, such as the management model in the SD fiscalía; strengthened prosecutor/ police coordination; and expansion of CJHs to decongest the formal justice system. Also, USAID instructed the implementer not to work with the DNP, 911 responders or forensics institute, which curtailed their ability to implement activities aimed at key PAD and Activity results. • The PASJ mechanism and plans also specified many results, Sub-IRs and hundreds of dispersed, small-scale actions. Significant resources were allocated to FINJUS and spread across a “legis￾lative agenda” comprising 10 different laws, of which four have no direct links to prosecutor effectiveness or crime prevention. • PASJ does have activities with potential for impacts within the CDCS time frame through direct engagement of local coalitions with justice sector operators and police to push for changes in crime prevention/investigation/ prosecution in targeted localities. The pilot project on mediation of conflicts in two schools has shown positive results, which could have an immediate impact on preventing crime, but it has not yet been expanded to schools in other localities though local coalitions supported by PASJ. The local coalitions have not engaged on issues of prosecution. • In all three Activities, most interventions related to prosecutor/police coordination, police response to crimes against vulnerable populations, professional conduct of prosecutors and police, and external oversight/engagement with the DNP have been eliminated. This weakened their contribution to achievement of the PAD Results Framework. In ICITAP, the activities eliminated from work plans were precisely those that could have had a direct impact on police effectiveness and responsiveness within the CDCS time frame. • The design of the mechanisms and work plans did not fully incorporate PAD guidance on co￾location, and neither USAID nor the IPs coordinated on prioritizing and sequencing interventions in targeted localities in the Duarte corridor. Nor was there an effort at joint, project-wide coordination with local counterparts; the PASJ local coalitions had no links to CJSSP or ICITAP. • The PAD provided some guidance on monitoring and evaluation, specifying many output and outcome indicators. In designing the three mechanisms, however, USAID/DR did not develop a coordinated approach to performance measurement to ensure that there were indicator data for linking Activity outcomes to the higher-level PAD results, e.g., guidance on how to attribute FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 12 results in targeted localities in the Duarte corridor; on the need for joint data collection strategies; and on how to conduct baselines and data collection in the six localities. Conclusions: • USAID’s design of CJSSP, ICITAP and PASJ Activities reflects the PAD results framework, which has defined many results, with their respective inputs, outputs, and outcomes. The thematic dispersion in the PAD was accentuated in USAID’s design of the CJSSP RFP and the PASJ RFA, shown by the large number of tasks/Sub IRs, which were carried over into hundreds of actions in the annual work plans. This led to a “miniaturization” of many actions, resulting in a lessening of impact due to a dispersion of budget resources and reduction in the ability to build synergies among actions. • Some of the tasks/Sub IRs that could produce results within the time frame of the CDCS in the targeted localities in the Duarte corridor were not implemented. In the case of CJSSP Results Package 1, with perhaps the most direct link to the PAD purpose, there has been little discernible or measurable progress beyond relationship-building and strengthening of the knowledge base. In ICITAP, the activities that could generate short term impacts were eliminated. In PASJ, the dispersion of activities meant that citizen engagement with local police or justice sector operators lacked depth and continuity, a problem worsened by the absence of ICITAP support. • USAID/DR did not have a whole-of-project approach to monitoring and evaluation, which should have cascaded into the three Activity monitoring, evaluation, and learning plans (AMELPs). This was particularly acute with respect to the correct specification of output and outcome indicators to link to PAD results; issues of attribution of results in the targeted localities; and the design and implementation of data collection. • With respect to the overall results of the three Justice Project Activities, the evaluation concludes that they have not yet produced significant progress toward the overall results set out in the CDCS and Justice PAD referenced above, owing in part to thematic dispersion and lack of depth, scale, and/or continuity of many actions, weak/delayed implementation, lack of political buy-in from key GODR partners, or elimination by USAID of some critical results and activities in the first half of the Project period. Justice Project Management EQ7. Has there been adequate coordination among the different justice sector Activities? EQ8. Has the Mission been opportune in identifying emerging challenges and making course corrections in the different Activities? Main Findings: • There has been a major coordination issue between USAID and INL in relation to activities with the DNP. This issue was made more difficult by two factors. First, the ICITAP Activity was initially managed from outside the Dominican Republic. Once USAID saw that this made implementation more difficult, it corrected the staffing structure, though delays in implementing that change had Activity place-holder management rather than a permanent program manager in-country for approximately one year. This delay meant that changes that permanent management to the Activity would have had the ability to design and effect had not yet been performed at the time of the evaluation. Second, staffing changes in the INL Office in the Dominican Republic after the design of the ICITAP Activity, which had been done in concert FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 13 with INL, resulted in a disconnect exacerbated by the absence of long-term Activity management. • Responding to INL’s opposition to the ICITAP Activity’s involvement in operational areas of the DNP, USAID made the decision to restrict ICITAP to strengthening of DNP HQ management processes, which has been largely successful. However, this decision has had a negative impact on the Justice Project overall, not just the Activity. It has not been possible to engage with the DNP on CJSSP activities to strengthen responsiveness to GBV and case management. Nor has PASJ been able to gain support of DNP for crime prevention partnerships with the CSO coalitions in the six localities. • Coordination between USAID and the IPs has been limited to information sharing and some operational coordination. USAID has not demonstrated a whole-of-project approach in its approach to managing the three Activities, particularly with respect to co-location, and the implementing arrangements to reinforce impacts in the six targeted localities. • With respect to emerging challenges, a major challenge of CJSSP was the uncertain scope of work with GODR entities due to INL objections. This issue has been resolved by dropping the relevant components, which undermined CJSSP’s contribution to higher level PAD results. The change in AGO early in Year 2 also created a severe challenge. • Other challenges undermined CJSSP progress, notably the unexpected restructuring of the Santo Domingo Province fiscalía into three entities, only one of which has become operational. Also, CJSSP’s assessment findings about the unexpectedly high cost of CJH operations borne by USAID led to an opportune decision to halt expansion. However, half way into the project implementation period, the possible sustainability of the CJHs has not been resolved. • Insofar as ICITAP’s principal challenge, the USG Mission has not been effective in resolving the coordination problem with INL, which has severely limited ICITAP’s original expected results, and has forestalled expected synergies with PASJ activities in support of local coalitions and police-community partnerships. ICITAP contributed to the problem through weak activity management structure, but which was resolved recently with new activity leadership. Conclusions: • The Justice Project PAD dedicates considerable attention to the external constraints to police reform, but a major constraint to implementation of the ICITAP Activity has been internal to the USG Mission in the DR. The agreement reached between USAID and INL on how to coordinate actions with the DNP has not upheld. ICITAP’s initially weak internal management also contributed to delaying a resolution; however, new ICITAP leadership has begun to address these issues. • Several salient implementation issues have emerged in the first two years of the Justice Project, which have not been resolved opportunely. There has not been a timely response to the issue of the very limited progress to date on CJSSP’s planned interventions with the AGO and fiscalías. Consequently, there were few substantive program results to create buy-in on the part of the AGO. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 14 GODR Engagement and Sustainability EQ9. Have USAID/DR and the IPs been effective in engaging the GODR in the justice sector Activities, and ensuring buy-in and support for proposed reforms? EQ10. Which of the justice sector Activities (and individual components) supported by USAID/DR are producing sustainable results? Main Findings: • CJSSP has engaged with many GODR entities in pursuit of its diverse lines of action. There is widespread respect for the CJSSP team’s knowhow and understanding of sector issues, and appreciation for USAID’s ongoing investment in the justice sector. But interviewees suggested that USAID (and the Embassy) had not been effective in efforts to cultivate relations with relevant GODR stakeholders. • As for sustainability of CJSSP results, the results in Years 1 and 2 are still too incipient to judge. The critical work on the management model and associated actions are just now beginning to be implemented in a pilot project in SDW. • Engagement between ICITAP and the DNP is strong. There are continuous, formal interactions between ICITAP and the DNP leadership, including a DNP-convened weekly meeting between ICITAP and DNP’s executive management. Interviews indicated constant presence of DNP officials in the ICITAP offices for both formal and unstructured meetings. • ICITAP results in strengthening core management processes in the DNP look to be sustainable, in that they are based in the Police Law and buttressed always with formal changes in policies and SOWs. For example, although changes in Human Resource policies face resistance from entrenched interests, they are difficult to reverse once legal-administrative obligations are created. GODR support for the ongoing DNP reforms remains strong. • During Years 1 and 2 of the PASJ Activity, FINJUS has had effective engagement with justice sector operators at all levels; with individual legislators and legislative committees; and during Year 1 with political party leaders and campaign teams. The panelists and participation lists of the round tables and conferences read like a “who’s who” of the leadership of the justice sector. • The engagement of PASJ local coalitions with municipalities and GODR agencies is difficult and sporadic, owing in part to GODR antipathy towards PC. Similarly, engagement with the DNP is uneven, owing to constant rotation of DNP officers, and unwillingness to discuss the sensitive issues of police effectiveness, transparency and probity. The absence of support from DNP national leadership undermines efforts to establish police-community crime prevention partnerships. • PASJ national coalition results in legislative reforms (Police Law and Law on Firearms) are sustainable by virtue of their legal foundations, although this is always conditioned by the implementing arrangements and continued GODR support. However, the results of the local coalitions are at best incipient. The coalitions have just begun their activities and require several years of support to solidify their membership base and to become valid interlocutors with local GODR and CSO counterparts. Conclusions: • The engagement of IPs with GODR agencies has been continuous and generally quite successful, with the notable exception of CJSSP’s challenging relations with the new AG since Year 2. CJSSP has built strong relationships at lower levels of the AGO and the fiscalías. And the relationship with AGO leadership has recently improved. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 15 • Sustainability in this area of CJSSP will depend on its ability to produce critical results soon, and to effectively leverage its growing engagement with the new AGO team on a strategy for expanding the elements of the new management model (once thoroughly developed and tested in SD West) to the other high volume fiscalías. • The ICITAP results look to be sustainable. The next cycle of leadership change in the DNP is entirely predictable, thus ICITAP and USAID are able to develop a strategy for managing that change. • The principal challenge with respect to PASJ sustainability is to consolidate the local coalitions, reinforcing their leadership capacity and membership; provide them with more resources to support their priorities in crime prevention plans; and support their greater engagement with GODR and local counterparts. GBV Response in the Justice Project There are several GBV related sub-questions. This evaluation report also presents the consolidated findings on how GBV has been addressed by the three Activities; and in the conclusions it refers to opportunities for program activities. Main Findings: • CJSSP supports direct legal and psychological services to GBV victims (and their children) via the CJHs and PACAM, and via training on handling of GBV cases for justice sector operators. However, the effect of those activities on prosecution effectiveness and specifically on rates of case collapse are not captured in CJSSP monitoring. • With ICITAP, GBV-related activities were curtailed. The plan had been to have a robust effort that was to establish and strengthen DNP standards and procedures for family and gender violence (F/GBV) service provision and strengthen DNP first responder capacity for F/GBV emergency calls under the 911 system. The Team understands that USAID is planning to engage in this area with DNP. • PASJ project documents and reports show that GBV has been a recurring topic in the activities with local coalitions. In the initial participatory diagnosis with community leaders in the targeted localities in the Duarte corridor, GBV was identified as one of the major issues in crime and citizen security. Subsequently, in the action plans developed with local coalitions, prevention and institutional response to GBV has been one of the priority actions. This finding from the reports is supported by the KIIs. • The weakest elements in the institutional response to GBV are the lack of access to counselling and mediation services to prevent GBV, the lack of awareness of rights and support mechanisms for victims, and the deficient training of DNP agents in attending to cases of GBV. Conclusions: • The institutional structure of GBV response in the Dominican Republic is complex, involving multiple actors, making it difficult at times for the victim to navigate. Any new programming activities funded by the Justice Project need to be viewed in light of the risk of further complicating efforts by involving new operators such as the Public Defender’s Office (ODP) or opening new units within the AGO, or with NGOs whose services may not ultimately be sustainable due to funding or other constraints. • Any proposal to increase OPD and AGO involvement also runs the risk of creating a parallel structure in the places where the MW provides this service. The public defender FGD participants and KIIs with the AGO and members of civil society all stated that the services FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 16 provided by the MW to GBV victims is adequate; is being coordinated with other institutions; and requires more resources at the local level to cover demand. As mentioned, one of the major problems to address is coordination with AGO and DNP. EVALUATION RECOMMENDATIONS The evaluation generated fourteen discrete recommendations. A common thread in the recommendations is the need to reinforce the whole-of-project management of the different Activities. 1. Following on the evaluation findings and conclusions, it recommends that USAID/DR assess the feasibility of returning to the Justice Project PAD’s principals of co-location and measurement of impacts on the increase of GODR output and effectiveness in prosecuting crime and increasing public engagement and oversight of crime prevention, deterrence and prosecution efforts in the targeted localities of the Duarte corridor. The recommendation is extensive to all three Activities (CJSSP, ICITAP, and PASJ). Feasibility should be assessed in terms of the time remaining, the possibility of extending the existing mechanisms, and the re-scoping of the Activities to focus activities on those results closely linked to the PAD purpose and objectives. The rescoping should be done on a whole-of project basis, rather than piecemeal changes to activities. 2. One of the conclusions of the evaluation is that the Justice Project has not achieved many of its expected results in its first two years, owing to a mix of external and internal factors. In the context of USAID/DR’s current CDCS stocktaking, the evaluation team recommends that the Mission consider extending the PAD implementation time frame through to the end of FY2019 (September 2020). Without such an extension, even with the recommendations put forward here, it will be challenging to make substantive and sustainable progress towards project purpose and objectives in the time remaining. 3. The evaluation recommends that USAID move quickly to find a solution to the problem of INL coordination. USAID should seek Embassy support to create and actively coordinate a justice sector working group of the Activity CORs, INL, the RSO, and IPs to deconflict efforts and seek synergies among the Justice Project activities and USG agencies. This group should be chaired by the Deputy Chief of Mission. A top priority is to define the precise planned areas of intervention and counterparts for the remaining period of the Justice Project, to enable each agency and IP to operate freely within that agreed scope, and to provide clarity to GODR and other external stakeholders. While impacting directly ICITAP, this recommendation is crucial for the achievement of synergy between all the IPs in the co-located activities. 4. While the evaluation team supports the DO1 decision to give support to co-location of the Justice Project Activities in SDW, it recommends that it be expanded to other localities in the Duarte corridor. With respect to CJSSP, a tightly focused SOW and budget should be defined, eliminating the work on oversight (RP2) and reducing work with vulnerable groups, maintaining only GBV. USAID should take a more proactive stance to ensure that rescoping allocates budget resources to work with the fiscalías in Santiago and National District to incorporate the SDW pilot activity approach. Depending on budget resources, it may not be possible to maintain activities in supporting a new management model in the fiscalías and also work on ADR. USAID should decide on its priority and select one or the other for expansion in multiple localities. CJSSP support to training should be focused on specific positions in the AGO and targeted fiscalias, delivering skills and knowledge that directly contribute to the narrowed SOW, designed FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 17 to support the institutional capacity development objectives (relating to the prosecutor management model, use of alternative case disposition and related actions). 5. Assuming that Recommendation 3 is successfully implemented, USAID should direct ICITAP to develop a focused work plan modification, to support two interventions in addition to the existing management strengthening activities: improved DNP response to GBV; and improved police-community relations through engagement with the local coalitions supported by PASJ and participation in activities promoted by the coalitions in the selected municipalities in the Duarte corridor. ICITAP should engage with the DNP’s Direccion Especializada de Atencion a la Mujer y Violencia Intrafamiliar on this strategy and link it to the broader support for GBV prevention and response in the design of the Justice Project activities, and in particular with the PASJ local coalitions. 6. The changes to the ICITAP Activity should be strictly aligned with the PAPA language, developed through a formal modification of the ICITAP work plan. The changes should also be formally approved by the DNP national leadership, and led by specific Directorates within the DNP, who would support ICITAP and PASJ activities in the targeted localities (as in the example in Recommendation 5 on the location of the GBV activities in the DNP). The design of the modification should be done as a joint exercise between ICITAP and PASJ to develop a strategy for engaging with the DNP at the local level. If possible, USAID should consider extending the PAPA through to 2020 to coincide with CJSSP. 7. With respect to PASJ, USAID should direct PC to focus activities on local coalitions in the targeted localities, and work towards an extension through FY2019, again in the framework of the PAD extension. The rescoping should eliminate most of the advocacy activities done through FINJUS and the coalition (except possibly advocacy on PM/AGO, DNP budget resources and ADR). It should also eliminate the training and events for raising public awareness about human rights and the justice system and instead focus on activities on supporting coalitions to engage with local stakeholders: municipalities, DNP, fiscalías (especially GBV Units), GODR agencies (especially MW), schools, business associations, and other CSOs. The action plans for the coalitions should incorporate GBV and school violence prevention (in the latter extending the pilot project in school mediation), and actions with the DNP to strengthen police effectiveness and responsiveness. 8. USAID should reassess plans for victim support activities to be housed within the Public Defender’s Office and explore alternative opportunities within existing systems, e.g., MW and its victim legal assistance services, or the incipient institutional networks of GBV services at the local level. At the same time, USAID should reassess the current support to PACAM counselling services to victims of GBV, considering carefully the potential for sustainability, and explore alternative means of supporting similar services via the MV. The design of this component should be done at the Justice Project level to support complementary interventions, co-located in the targeted localities. For example, support to the MW, along with promotion of the institutional networks to improve overall GBV response, and prevention activities with the PASJ coalitions. 9. The evaluation recommends that the proposed changes in the three Activities be designed to respond to CDCS and Justice Project guidance, to identify synergies and mechanisms for joint action in the targeted localities and with local stakeholders to advocate for and support GODR, municipal and civil society efforts. The evaluation recommends that the DO1 team hold a facilitated workshop to finalize the Justice Project tasks/sub IRs, and the guidance for co-location FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 18 and coordination between IPs and with the local coalitions. This could be followed by a workshop with the IPs to improve the proposed design. The design of implementation arrangements in the SOWs and work plans should give priority to supporting the local coalitions and their role as a coordinating mechanism for crime prevention. For example, even if CJSSP were to implement GBV activities, it should be implemented in close cooperation with the local coalitions, through the common coordinating mechanism with fiscalías, DNP, GODR agencies and other stakeholders in each targeted locality. Similarly, with any CJSSP activities on expanding support for ADR, which should also be co-located. 10. USAID, CJSSP and PASJ should accelerate support for use of Alternative Dispute Resolution (ADR) methods as a means of conflict management and justice system decongestion, including mediation and conciliation, in a manner that ensures fairness and respect for rights, quality of service and coordinated approaches among institutions of government and civil society. If this recommendation is taken up, and if stakeholder consultation and pending analysis by FINJUS (under contract to CJSSP) indicate that it will add value, support should be provided to PASJ (through FINJUS) to develop legislation to regulate and promote the use of mediation. 11. New GBV-focused programming through the Justice Project should seek to avoid further complicating an already complex institutional structure. The emphasis should be on improving and reinforcing existing services provided by the fiscalías through the GBV Victims Units in the targeted localities in the Duarte corridor; the service provided by the MW in these same localities; the DNP’s own specialized Directorate; the local PASJ coalitions that work to support community awareness raising and victims’ understanding of GBV services; and the existing networks of GODR, health service providers, NGOs and community organizations in the target localities. This can be done within the existing mechanisms, without involving UNFPA. 12. SAID and IPs should ensure that robust and comprehensive M&E systems are in place to measure, at a minimum, progress against CDCS and PAD targets and goals. In rescoping the three Activities, the Mission should update and improve the PAD log frame and specify the output and outcome measures to be incorporated into the Activity design and AMELPs. The limited time remaining in the Justice Project, budget restrictions and attribution challenges may make it impossible to use some of the original PAD outcome indicators. However, the Mission should identify at least one robust outcome indicator for each Activity linked to the PAD purpose and sub-purposes (and the CDCS PMP). The indicators (both outcome and output) should be sufficient to provide essential information about achievements and to highlight issues for management attention. 13. USAID should give guidance to IPs to urgently devise and execute methods to measure the impact of training (past and upcoming) on knowledge, attitudes, and practices of participants and functioning of institutions. Ideally, this would include measuring the effect of CJSSP training on actual case disposition and the individual parties to cases as indirect beneficiaries, e.g. assessment of satisfaction of vulnerable persons after a case has been handled by the fiscalía or CJH or courts, to observe trends before and after personnel are trained or sensitized. 14. USAID and implementing partners should continue to seek out and collaborate with champions and allies occupying key positions in the justice institutions, making every effort to move forward rapidly with reforms while those individuals or teams are in place while also pursuing strategies to safeguard gains made and protect against sudden loss of those allies. The strategy should include but not be limited to securing written agreements at the highest levels that outline the authorized types of intervention or areas of cooperation with other institutions and FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 19 pressing for the formal adoption and socialization of new or revised processes and rules that contribute to improvements in the institution. This may be achieved by a formal MOU, however, given the time remaining in the Justice Project implementation period, it may be more practical to seek other mechanisms, so as not to slow the work of the activities. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 20 II. EVALUATION PURPOSE, QUESTIONS, AND METHODOLOGY CRIMINAL JUSTICE SYSTEM STRENGTHENED PROJECT BACKGROUND This mid-term performance evaluation of the USAID/DR Criminal Justice System Strengthened Project uses a whole-of-project lens consistent with Automated Directives System (ADS) 201.3.5.13, looking at the constituent activities of the project and examining their progress toward achievement of the overarching project purpose.2 The purpose of the project as set out in the PAD is to “strengthen the Government of the Dominican Republic’s (GODR) criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”. It thus corresponds to the CDCS IR 1.3: Criminal Justice Institutions Strengthened, which contributes to the CDCS Development Objective 1: Crime prevention strengthened. This whole-of-project evaluation pays attention to USAID program design guidance, which requires that IRs in the CDCS be defined at a level of ambition that is “manageable and attainable given the Mission’s resources, staff, and influence”. As set out in ADS 201.3.3.9 (1), a PAD includes “a theory of change, describing how the process of change is expected to take place and how USAID intends to directly and/or indirectly influence these changes”. The theory of change in the USAID/DR Justice Project PAD says that the project “will advance the DO1 crime prevention objective by improving the deterrent value of prosecutions as a government response to crime and increasing the overall transparency and credibility with which the Dominican criminal justice system operates”. The PAD identifies two sub￾purposes: 1) increase GODR output and effectiveness in prosecuting crime;3 and 2) increase public engagement and oversight in Dominican law enforcement and crime prevention efforts. In developing the project purpose and theory of change, the PAD identifies four distinct systemic factors to be addressed in the design of the implementing mechanisms: • Limited Prosecutor Absorption Capacity. There is especially limited prosecutor absorption capacity in Santo Domingo Province and the National District, and in the case of crimes against vulnerable populations (in particular, GBV). The problem of capacity is felt in all fiscalías, resulting in case backlog and low rates of prosecution. • Low Access to Justice. There is especially low access to justice in marginalized communities, which do not have access to alternative dispute resolution and have a low level of trust in the police, which increases case backlog and system congestion by increasing the number of minor offenses processed within the formal justice system. • Weak Police Management & Transparency. Interagency conflict and poor coordination between police and prosecutors critically weakens justice system ability to prosecute crime, while weak operational management undermines DNP performance. This is exacerbated by police abuse and corruption. 2 See USAID. 2016. ADS 201 Additional Help: Whole-of-Project Evaluation (WOPE). Washington D.C.: USAID Learning Lab. 3 A critical element in the PAD theory of change is that the activities will “reduce case backlog and system congestion by reducing the number of minor crimes and disputes processed within the system”. USAID. 2014. Project Appraisal Document: Criminal Justice System Strengthened Project, p.10. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 21 • Inadequate Public Engagement & Oversight. CSO advocacy in crime and citizen security (e.g. police reform) has failed to gain traction under the weight of other justice sector reforms. Public advocacy in the justice sector has not addressed community level issues of crime and security where public demand and frustration are highest. To effectively address these factors, the PAD guidance on the technical approach and implementation proposes a geographic focus on the Duarte corridor, and within that corridor, it highlights specific localities, including the Province of Santo Domingo, National District, Santiago, La Vega and Puerto Plata. For the proposed activities on prosecutor effectiveness, the PAD focuses on the “highest volume crime districts” of National District, Santo Domingo Province and Santiago”. Following the PAD implementation plan, the Justice Project has been implemented under three separate mechanisms: • Criminal Justice System Strengthened Project (CJSSP). Contract with Chemonics International; $21,536,654, starting on June 8, 2015 and ending on June 7, 2020. The objective is to strengthen the criminal justice system in the Dominican Republic by improving the quality and effectiveness of prosecution. The Activity will strengthen the Public Ministry’s management and technical capacities to process critical cases by expanding and adapting new management models; and increase the transparency and accountability of Dominican law enforcement and crime prevention efforts by strengthening internal controls and external oversight for both the Public Ministry and National Police. • Institutional Capacity and Transparency Strengthening for Police Reform (ICITAP). Participating Agency Program Agreement (PAPA) with the United States Department of Justice; $2,800,000, starting on July 20, 2015 and ending on July 19, 2019. The objective is to strengthen institutional capacity and transparency within the DNP. The Activity will improve police – prosecutor coordination throughout the criminal investigations process; strengthen DNP human resources planning and management; strengthen DNP community relations capacity by creating police-community partnerships; and strengthen external oversight mechanisms for the DNP. • Civil Society Action for Accountable Justice and Security (CSAAJS in English, or PASJ in Spanish). Cooperative Agreement (CA) with Participación Ciudadana; $6,400,000, starting on June 22, 2015 and ending June 21, 2018. The objective is to increase public awareness on criminal justice and public safety, especially in the institutional roles and responsibilities of the Public Ministry and the National Police in crime persecution and prevention. The Activity will improve public awareness of crime and criminal justice issues; increase access and availability of information and support resources for victims and witnesses; increase use of the Dominican Freedom of Information Act (FOIA) and related legislation to exert social control actions and monitor criminal justice system’s actors, including the police; and support civil society organizations in pressing for reforms in the criminal justice and public safety systems. EVALUATION PURPOSE AND QUESTIONS The evaluation team met numerous times with the Mission Program Office to confirm USAID/DR’s goals and objectives, to clarify the evaluation questions, and during the evaluation field work also conducted follow up interviews with DO1 staff to validate the evaluation questions and the type of information and insights that will be most useful to USAID’s decision making. The Mission expressed concerns about the alignment of activities with the Justice Project PAD, the effectiveness of specific FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 22 results areas (Sub IRs) within each of the constituent activities, and their effectiveness in achieving the project purpose. In addition to formulating ten evaluation questions (EQs) the Mission provided sub￾questions to clarify the intent. In this report, the evaluation questions formulated by the Mission have been re-ordered and grouped to facilitate the analysis and presentation of findings and conclusions (the original order and numbering can be seen in the evaluation SOW in Annex F). Effectiveness of the Justice Project Activities In the presentation of findings and conclusions on the effectiveness of the constituent activities of the Justice Project, the report groups EQ1 together with each of the Activity-specific questions, i.e., EQ1 and EQ2 for CJSSP; EQ1 and EQ3 for ICITAP; and EQ1 and EQ4 for PASJ. EQ1. Have the three activities achieved the expected results of their contracts or agreements for the first half of the implementation period? If not, why not and what are the recommendations for course-correction for the second half of the award period? If the expected results have been achieved, are there implementation challenges we should be aware of for the remainder of the award? EQ2. What have been the major achievements and challenges of the activities conducted to increase prosecutor effectiveness and access to justice for vulnerable groups? Has the Activity met its targets? Have interventions been designed and implemented such that they specify, generate and document changes in management practices that can provide models for reform? If not, what changes need to be made to the activities to achieve this? Have the activities to improve access of vulnerable groups (and especially for victims of GBV) been effective? What else needs to be done? EQ3. What have been the major achievements and challenges of the activities conducted to strengthen police management and transparency? Are the activities set out in the approved Work Plans being implemented as agreed? Which of the activities are not being implemented? Why? Have the activities to improve access of vulnerable groups (and especially for victims of GBV) been effective? Give specific examples. What changes to the activities are needed to ensure that they meet all the objectives of this Activity? EQ4. How effective have the civil society activities been in creating demand for justice and monitoring law enforcement and crime prevention efforts, both at the national and local levels? Beyond the local implementing partners (IPs) themselves, are other civil society organizations or leaders involved and invested in these activities? Is there a coalition of CSOs working to improve performance and accountability (e.g. transparency, GBV, judicial corruption)? How has the private sector participated in these coalitions? What are the obstacles to bringing more CSOs/private sector organizations into these activities? Do the grantees and sub-grantees have the capacity to achieve results? Overall Results The second set of evaluation questions have to do with the progress made during the first two years of implementation towards the Justice Project’s overall purpose, “to strengthen the Government of the Dominican Republic’s (GODR) criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”. EQ5. Taken as a whole, have the three activities produced the results set out in the CDCS, “Criminal justice institutions strengthened”, and in the Justice PAD, “to strengthen the GODR criminal justice system by improving the timeliness and effectiveness of criminal FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 23 prosecutions”? Has the design of the mechanisms (contracts and agreements) responded fully to the CDCS and PAD results framework? EQ6. Taken as a whole, have the activities in support of justice sector reform supported the expected links between the overall goal of crime prevention and the intermediate results of improvement in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities? What data do we have? What data do we need/how can we better measure results and linkages going forward? Address specifically the activities to promote justice for vulnerable groups and especially victims of GBV. Are there specific evaluation exercises that might be conducted to demonstrate these links? Project Management The Justice Project theory of change supposes a high degree of complementarity and synergy between the constituent activities, which requires effective steering and guidance from USAID. The evaluation questions consider issues of Project and Activity-level coordination and adaptation. EQ7. Has there been adequate coordination among the different justice sector activities? How have USAID/DR and IPs ensured coordination? Give specific examples of effective coordination (or challenges to coordination) between implementing partners and/or between GODR counterparts. Has there been effective coordination among USAID, INL and the IPs in the justice sector project? EQ8. Has the Mission been opportune in identifying emerging challenges and making course corrections in the different activities? Are there quality performance data for decision-making by USAID? Do the Mission DQAs show that IPs generate quality performance data for ensuring effective management and monitoring? Sustainability In that this is a mid-term evaluation, the questions include issues of engagement with counterparts and sustainability, with the intention of identifying tactics to improve PAD implementation going forward. EQ9. Has USAID/DR and the IPs been effective in engaging the GODR in the justice sector activities, and ensuring buy-in and support for proposed reforms? Give specific examples of GODR buy-in (or not) from each of the activities and their respective components including subgrantees activities. Have the activities to improve access of vulnerable groups (and especially for victims of GBV) been taken up by GODR partners? EQ10. Which of the justice sector activities (and individual components) supported by USAID/DR are producing sustainable results? Give examples of the main program interventions that will continue (or not) producing results in the long-term. Address specifically the activities to promote justice for vulnerable groups and especially victims of GBV. Where will the project get traction in GODR? What measures need to be taken to ensure greater sustainability of results in each of the activities? EVALUATION APPROACH AND METHODOLOGY Following ADS 201.3.3.9 (3), the PAD is understood to be an “organizing framework that should be adapted as new evidence emerges, circumstances change, and tactics require adjustments”. Thus, while the evaluation data collection focuses on each of the three constituent activities of the project, the evaluation has a utilization-focus which in its data collection, analysis, findings, conclusions and recommendations gives emphasis to how USAID activities have aligned with the PAD theory of change, and “how they have contributed to the project purpose and what tactics within the PAD may need adjustment”. The tactics under evaluation are the results frameworks and activities within each of the FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 24 three implementing mechanisms. As will be discussed below, the three activities within the Justice Project have many intermediate results (IRs) and Sub-IRs or tasks, and hundreds of individual actions. Given limitations of space, the findings and conclusions presented in this evaluation report give emphasis to the analysis of those IRs and Sub-IRs deemed to be most critical for achieving the overall project purpose. Gender-Sensitive Perspective Consistent with USAID’s evaluation policy and recognizing that effects of integration and the success of the Activity might vary across gender, the evaluation team applied a gender perspective to the entire evaluation process. The team included both female and male researchers, and a gender balance among respondents was sought during fieldwork. During data collection, the team endeavored to limit the potential for one gender to dominate the conversation at the expense of another. Substantively, the data collection and presentation of findings and conclusions has paid attention to one critical gender issue in USAID/DR programming, gender-based violence. GBV is referenced throughout the findings and discussed in a separate section. Socially-Sensitive Perspective Transparent ethical considerations have been applied throughout the evaluation in data collection, analysis, and reporting. During the data collection process, the evaluation team took steps to 1) protect the anonymity of participants, 2) ask for permission prior to posing any questions, 3) avoid questions that can be unsettling to the participants, and 4) avoid actions and postures that could distort responses. The evaluation team applied these considerations to respect cultural, socio-economic, and physical limitation differences throughout the evaluation process, and particularly in data collection in marginal communities and with vulnerable populations, including individuals identifying as lesbian, gay, bisexual or transsexual (LGBT) and persons with disability (PWD). Data Collection The evaluation included a review of relevant project-related documents; analysis of Activity monitoring, evaluation, and learning (MEL) data collected by the IPs; the conduct of KIIs with IPs, government counterparts, beneficiaries and other stakeholders; and focus group discussions (FGDs). In addition, the team conducted project implementation observations in the target municipalities; and applied a short web-based survey of selected participants of CJSSP-funded training. Document Review. During the initial phase of the evaluation and prior to arrival of the expatriate team members in the DR, the team conducted a thorough review of background documents related to the project and its context, which provided a deeper understanding of the operations to date. This also allowed the team to finalize the evaluation design and data collection tools. Many additional documents not available prior to the initiation of in-country evaluation activities were provided later by the IPs. The documents reviewed for this evaluation are listed in Annex C. Review of Activity Progress Reports and Performance Data. During the implementation process, the IPs have collected a wealth of data and produced reports and special studies to document their progress and orient their planning. In the case of CJSSP and to a much lesser extent ICITAP and PASJ, these data are robust and representative of project outputs, but as discussed below, not indicative of project outcomes. The evaluation team has examined existing Activity monitoring and evaluation data to determine project activities and outputs over the implementation period. That data included: FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 25 • IP activity monitoring, evaluation and learning plans (AMELPs) • Baseline surveys and reported quantitative data generated by activities • Quarterly and annual progress reports • Complementary contextual data provided by MEL staff of each of the IPs • Special studies, assessments or activity designs conducted by the IPs Key Informant Interviews (KIIs). The evaluation team proposed to conduct as many as 140 KIIs with USAID/DR staff, key GODR officials interacting with IPs, local and international NGOs and donors and the implementing partner organizations. While some planned KIIs were not conducted owing to problems of availability or willingness of informants, overall the data collection exceeded the proposed target, conducting a total of 143 KIIs. An important limitation was the inability to obtain an interview with senior AGO leadership. The evaluation team made a conscientious effort to ensure that KIIs reflected the entire universe of the Justice Project stakeholders equitably, avoiding bias toward any gender, racial or socio-economic group, etc., and included PWD, LGBT people, and women impacted by the activities. USAID and IP recommendations helped orient informant selection, but some KIIs were selected among stakeholders or beneficiaries not prioritized by them to ensure a balanced universe of informants that includes stakeholders not favored by project interventions. See the KII summary table below. Annex D lists the persons interviewed and their institutional affiliations. The report narrative does not identify respondents in the interest of protecting their anonymity. During data collection, four informants, all senior lawyers who have attended CJSSP or PASJ activities, requested not to be included in the KII list or referenced by institutional affiliation; this request was respected. They are included in the summary KII table, but not listed in Annex D. Table 1: Summary of KIIs Conducted by Activity, Type and Gender ACTIVITY/TYPE TOTAL USG-USAID 4 CJSSP staff, consultants 16 CJSSP counterparts, subgrantees or beneficiaries 34 ICITAP staff 11 ICITAP counterparts, subgrantees or beneficiaries 18 PASJ and FINJUS staff 14 PASJ counterparts, subgrantees or beneficiaries 40 Others 2 KIIs requesting not to be named 4 Total 143 Data collection protocols were used to guide KIIs. They were semi-structured, with probing questions to pursue information that would respond to the key evaluation questions. The KIIs were sufficiently flexible to allow for a natural flow of the conversation, as the evaluation team identified points of particular interest to address the research questions. The team conducted most KIIs in Santo Domingo (including Santo Domingo West), as well as National District, Santiago and Puerto Plata. Notes were taken discretely and a summary report of each KII prepared as soon afterwards as was logistically possible. The KIIs were not taped in order to avoid intimidation of the respondents. Some KIIs with stakeholders located outside the DR were conducted by telephone and Skype. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 26 Focus Group Discussions. The evaluation team used FGDs with groups of 53 Justice Project counterparts or beneficiaries to determine the sense of ownership of and satisfaction on the part of counterparts and beneficiaries of the project, as well as to promote idea sharing and collaborative group assessments. The seven FGDs conducted were as follows: Table 2: Summary of FGDs Conducted by Activity, Type and Gender ACTIVITY/TYPE TOTAL Prosecutors in Santiago engaged by CJSSP 7 Prosecutors in Santo Domingo West engaged by CJSSP 6 Public Defenders 4 Members of PASJ local coalition in Santiago 6 Members of PASJ local coalition in Santo Domingo West 6 Members of PASJ local coalitions in Puerto Plata 8 Members of PASJ local coalition in National District1 16 Total 53 The MEL Platform Chief of Party and staff coordinated with IPs to convene participants and assist in scheduling at appropriate times. When convening FGD participants, the evaluation team clarified that participants would not receive financial compensation or reimbursement. The FGDs were facilitated using a structured discussion guide adapted to each group. All FGDs were approximately 90 minutes each. In each session one team member took notes discretely. Survey of Justice Sector Operators. The evaluation team conducted a web-based survey of justice sector operators from the Public Ministry (PM) and fiscalías trained via CJSSP through the Public Ministry’s National School (ENMP in Spanish). According to CJSSP reports, 1,449 criminal justice personnel were trained with their technical or financial support in Year 1 and 1,519 in Year 2. The purpose of the survey was to measure the perceptions and experience of justice sector operators in relation to utility of the training and their perception of reform efforts in the justice sector in topics related to CJSSP interventions. Some results of the survey are presented in the findings and conclusions in response to EQ1 and EQ2 on the effectiveness of CJSSP. A summary of responses is presented in Annex A. The limited number of participants for whom contact information was available (310 individuals) combined with the low response rate mean that the results probably do not reflect the entire population of training participants. Thus, the results of the survey are merely suggestive and are biased towards new entrants. CONSTRAINTS AND LIMITATIONS Generalizability Due to time and budget constraints, data collection was limited to a subset of the Activities’ IP staff, GODR and non-governmental counterparts, and beneficiaries. Therefore, the evaluation team has taken care in generalizing findings beyond this sampling of respondents. However, at the same time, the evaluation team contends that the document review, KIIs, FGDs, and to a limited extent the survey, are all broadly representative of the information available on the three activities. As mentioned above, one limitation was the inability to obtain interviews with senior AGO leadership. With this caveat, the evaluation team considers that the relatively small sample size does not weaken overall evaluation findings, in that triangulating the different data sources gave greater consistency in findings and conclusions. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 27 Potential Bias Recall bias was a problem, as some informants relevant to the PASJ Activity blended their responses with perceptions about other non-USAID activities conducted by PC. In the KIIs and FGDs, upon realizing the issues, the evaluation team used probing questions to overcome this potential bias. There may also have been some instances of response bias, as project counterparts, particularly among staff in the fiscalías, training schools, CJHs and members of the PASJ national and local coalitions, evidently sought to avoid negative perception of project interventions, possibly with understanding that negative evaluation results could result in reduced funding and support. Again, to counter this potential bias, the evaluation team endeavored to use probing questions about concrete experiences and results. There may also have been some degree of selection bias, particularly with respect to the PASJ local coalitions, where the PASJ community promotors convened the participants in the FGDs. Nevertheless, in practice, the evaluation team observed that the FGD discussions were sufficiently large and diverse to allow for a mix of perceptions, both negative and positive, about project results and challenges. With respect to the survey of training participants, ENMP was only able to furnish emails for 310 unique individuals who had participated in the training. Of these, only 130 participants completed one or more survey questions, a response rate of 41 percent. For individual questions, the number of responses mostly ranged between 65 and 70 participants, which is 22 percent at best, below the average of 33 percent reported in the literature on web-based surveys. It should be noted that 67 percent of respondents were junior prosecutors (fiscalizadores), many of them recent entries into the AGO. This self-selection thus biases the survey towards more junior staff in the AGO, with unknown effects on the survey responses. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 28 III. FINDINGS AND CONCLUSIONS This section presents the findings and conclusions for each of the three activities (CJSSP, ICITAP and PASJ). The report combines discussion of EQ1 with each of the Activity-specific questions, due to the shared focus on results and challenges, i.e., EQ1 and EQ2 for CJSSP; EQ1 and EQ3 for ICITAP; and EQ1 and EQ4 for PASJ. CJSSP RESULTS AND CHALLENGES EQ1. Have these three activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ2. What have been the major achievements and challenges of the activities conducted to increase prosecutor effectiveness and access to justice for vulnerable groups? FINDINGS The CJSSP contract sets out a diverse range of anticipated results or outcomes as well as a list of defined tasks directed towards its overall objective of a strengthened criminal justice system. The purpose is described as the achievement of Results Package 1: Prosecutor Effectiveness Improved (RP1), and Results Package 2: Prosecutor and Police Oversight Strengthened (RP2). Under those two RPs, seven results areas are described along with associated sub-results, which have guided implementation and reporting under the contract, and thus have been used as the basis for evaluation of results and challenges of CJSSP. The contract also describes 13 “Expected Outcomes” ranging from rather general to highly specific (two have numerical targets). These outcomes have not been mentioned until recently in CJSSP reporting, and USAID and Chemonics indicated they had not been a point of reference for managing the contract, primarily because most did not align well with the Results Packages and/or depended on working with the national police—a counterpart that in March 2016 USAID instructed Chemonics to avoid, so as not to overlap with the ICITAP award and other U.S. agencies. In mid-2017, USAID advised against work with the forensics institute (INACIF), which Chemonics viewed as further constraining the potential to achieve the expected outcomes. These 13 outcomes were reportedly being renegotiated at the time of data collection. 4 CJSSP Results Results Package 1: Prosecutor Effectiveness Improved As outlined in the contract, USAID aimed via RP1 to “strengthen Public Ministry technical and management capacity to process critical cases by expanding and adapting management models established under the 2008- 2013 rule of law portfolio”. In fact, the four defined results areas extend well beyond that scope, requiring engagement of multiple justice sector operators via highly diverse interventions, as shown in the following analysis. 4 Chemonics staff highlighted that some of these outcomes appeared to be inconsistent with international standards for protection of rights within a democratic justice system (example: “increased convictions”), and therefore they have been interpreted and applied by the implementer to ensure alignment with those standards and with the other objectives of the contract. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 29 Maximize use of criminal case decongestion mechanisms5 One strategy to support decongestion was the continued operation and expansion of the network of Community Justice Houses from eight to 15, via a subgrant. Ultimately, only one new CJH was opened and another was closed, before the expansion plan was halted due to growing concerns over sustainability and CJSSP research that invalidated PAD assumptions regarding the CJH contribution to decongestion. Significant technical assistance (TA) and training to the CJHs (and PC as parent organization) were said by staff as have improved information capture and management by the CJHs and PC, allowing them to track and report on their services much more quickly and accurately. Assisted by CJSSP in-depth analysis of operational costs and sustainability options, in 2017 PC decided to transition ownership of the CJHs to an independent association, which was formally launched in March 2018. Fundraising efforts (also supported by USAID and Chemonics) have so far netted one modest grant to the association from the GODR, and there are hopes this may evolve into a yearly subsidy. IP progress reports show that the CJHs have collectively served 87,189 people in vulnerable communities since CJSSP began, including more than 27,000 Alternative Dispute Resolution (ADR) processes (including mediation and conciliation); this means the Activity is on track to meet original targets in the contract. Although the 2016 evaluation of the CJHs showed a high rate of case resolution, the effect on criminal justice decongestion is difficult to establish as the majority of ADR interventions are civil disputes. While various informants noted that resolving these cases was having a positive effect by defusing conflicts that might have escalated to become criminal in nature, there is insufficient data to draw conclusions about the ensuing effects on the criminal justice system. Beyond the CJHs, CJSSP is taking a multi-actor approach to promoting ADR, including continued training and TA on conciliation and other abbreviated case disposition methods for prosecutors in fiscalías, and planned support to defining a national ADR strategy. As of the time of this evaluation, CJSSP was funding FINJUS to conduct a comprehensive assessment of all legal dispositions, practices and organizations related to ADR in the country, as the basis for an anticipated strategy proposal for expanding use of ADR mechanisms. As well, plans are well advanced to collaborate with the Ministry of Interior by training community leaders in mediation, and to support the judiciary’s new ADR coordination unit. Improved case management and monitoring A core output in this results area was a comprehensive assessment by CJSSP of the three largest fiscalías, begun in early 2016 to lay the foundation for development of a management model for those hubs of prosecution activity. However, Chemonics said that defective analysis by the consultant necessitated extensive verification (including consultations with GODR counterparts) and revision before finalization of the report about 10 months later. Initially used as an internal resource by CJSSP, the “Comparative Assessment of Prosecutor Office Management Models” was shared in late 2017 with the AGO, which indicated the study was valuable and would be used alongside the existing model (developed with support of a previous USAID project) to guide design of an adapted new model. CJSSP has begun work with the AGO in that process, providing TA and training to develop a pilot for SD West fiscalía, and aiming for replication by the GODR in other large fiscalías. To establish baseline data, reports indicate that more than 20,000 case files have been registered and digitized by CJSSP￾funded data entry specialists in SD West. As well, five draft protocols related to distinct types of cases or prosecutorial functions have been developed and presented to the AGO by CJSSP consultants, with a view to enhancing case processing and management in multiple fiscalías (once approved). The main CJSSP result in terms of inter-institutional coordination is establishment of an Information Technology (IT) Committee comprising 12 government and judiciary agencies, which has undergone a 5 This results area was called “Expand alternative dispute mediation” in the contract. However, Chemonics opted to broaden the scope by adjusting the wording in their reports and workplans, and USAID raised no objection. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 30 year-long planning process. Progress reports and committee members indicate that CJSSP has assisted the Committee to define parameters for a pilot for data integration in the justice sector. Chemonics and two committee members said that launch of this data sharing platform is imminent, although other committee members were less optimistic, noting that not all institutions had made commitments to share information on the platform. Strengthened legal protection of vulnerable groups CJSSP has focused considerable energy on improving awareness and skills among justice operators and enhancing systems and procedures for handling of vulnerable persons. CJSSP emphasized vulnerable groups including victims of GBV in its subgrants, while incorporating this theme in other training and TA activities for prosecutors, CJHs, public defenders and others. The subgrant to the Dominican non￾governmental organization Patronato de Casos y Mujeres Maltratadas (PACAM) aimed primarily to improve management of GBV cases and provide psychological assistance to victims, mostly users of the CJHs. Reports show that PACAM counselled 888 women/children over two years, trained 170 prosecutors, and provided information to 1,173 people in communities. CJH staff said that PACAM’s work had added value to their services, by responding to a pressing need among their users. One objective was to reduce the number of victims who discontinue the cases against their aggressors, but at time of writing the evaluation team had not been provided with monitoring data on the effects on so-called “case collapse” among beneficiaries. In fact, most of those who have received counselling do not have active cases; although the plan was to target those women who were pursuing legal action, that proved to be unworkable (primarily due to high demand from other clients), and the decision was made to offer the service to other GBV victims as well. The Center for Research and Integrated Treatment (Centro de Investigación y Orientación Integral or COIN), another Dominican non-governmental organization, received a subgrant for sensitization and training of justice system actors to improve how they handle cases involving members of the LGBT community. COIN reported reaching 261 people, and participants spoke favorably about the impact of the innovative training methods. One CJH director indicated that the awareness sessions had made a noticeable difference in the attitudes of its personnel. This one-year subgrant was terminated shortly before completion due to grant administration and reporting problems, although manuals developed with CJSSP funding were later published with other funding. With unanticipated funding from the U.S. State Department, CJSSP managed a grant to the U.S.-based International Association of Women Judges (IAWJ) for awareness-raising and training of judges on GBV cases. In cooperation with the National Judicial School (ENJ in Spanish), IAWJ held in-depth consultations with more than 350 diverse stakeholders to analyze the issues and inform design of a new training curriculum for judges, which was subsequently adopted by ENJ. They trained 21 judges as trainers, and two training events were subsequently delivered via the ENJ for other judges. While the stated goal was to reduce rates of GBV case discontinuation, that result is not being monitored. Strengthened human resources CJSSP reports indicate 1,449 criminal justice personnel were trained with their technical or financial support in Year 1 (target was 820), and 1,519 in Year 2 (more than five times the target of 300); those trained included prosecutors, public defenders, judges, forensics experts, psychologists, community justice house staff and other personnel. 6Chemonics said that training was scaled up as sector needs were identified (including through assessments and consultations), and as progress was impeded in other areas of activity. They report investing more than $1.8 million in training so far (of a total expenditure of 6 It is not possible to establish the number of unique individuals trained. From the KIIs and the survey, it is evident that most of the almost 3,000 participants had received multiple training courses. Thus, the number of justice operators receiving training is a subset of the almost 3,000 mentioned in the CJSSP reports. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 31 approximately $11 million as of December 31, 2017), largely on subjects of specific relevance to the contract results areas. Analysis of the CJSSP contract and reports shows that the public defenders, judiciary, forensics institute and CJH were targeted more than originally anticipated; Chemonics indicated that training needs across the board were greater than expected. Of the key topics in the contract, case management (including triage of cases) and ADR methods were hardly touched by training to date (defined as courses of eight hours or more in duration), while substantial training has been delivered on topics not specifically mentioned in the contract, notably communications, criminal investigation, oral litigation, and human rights—though the latter is related to handling of vulnerable populations, which was highlighted in the contract. Training for prosecutors in case management and ADR was logically tied to development of management models, where CJSSP has only recently started to make inroads with the AGO. Looking beyond formal training events captured by those records, CJSSP reports having provided substantial technical assistance to a range of justice sector counterparts with a significant component of ADR and subjects related to case management. Nearly all training was delivered in conjunction with the ENJ and ENMP, which both indicated that CJSSP financing and training expertise allowed for delivery of more courses of a high standard. Although some were new courses, they did not point to any resulting changes in their regular curricula or training offerings, nor to any Training of Trainers initiatives (specifically mentioned in the contract), except for the IAWJ-designed course to prepare judges as trainers. Interviews with Chemonics and the schools suggested that the core functioning of the schools was not much affected by CJSSP support, which was seen by the schools more as a “bonus” that allowed them to expand their course offerings in the short term. Training participants in FGDs said that training was of high quality and relevance, and some learning had been directly applied in their subsequent work. Chemonics reports that knowledge testing was done by trainers before and after each workshop, but that data has not been analyzed across trainings to look for trends in knowledge change among participants, and monitoring of knowledge retention or change in performance of participants has not been done.7 The online survey among ENMP training participants, although it did not focus specifically on those trained with CJSSP support, provides a useful snapshot of their views on training received in the last two years. The responses are weighted towards Junior Prosecutors (fiscalizadores). For example, courses focused on investigations, evidence and litigation were highlighted as among the most useful; those three subjects were extensively supported by CJSSP. Just over 50 percent of respondents said they could dispose of cases much more quickly than before they received training via the ENMP. Half of respondents emphasized continued training within the AGO as a high priority to improve how cases are handled by prosecution service. The survey findings are described in detail in Annex A. Results Package 2: Prosecutor and Police Oversight Strengthened According to the contract, RP2 intended to “increase the transparency and accountability of Dominican law enforcement and crime prevention efforts by strengthening internal controls and external oversight for the Public Ministry and National Police…” As work with the national police was removed from CJSSP, the scope was significantly narrowed. RP2 activities to date have had a strong focus on the judiciary, notably after the 2016 Judicial Summit—financially and technically supported by CJSSP based on guidance from USAID. 7 Chemonics staff indicated that this gap in data had been recognized, although specific plans to address it were not yet clearly defined. They see the long-term solution as involving the incorporation of training impact measurement in the performance evaluation systems of the GODR entities employing the participants. This was also mentioned by Spanish International Development Cooperation Agency (AECID) and the ENMP. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 32 Oversight management and professional conduct of prosecutors (and police) Despite ongoing efforts by CJSSP to engage with the AGO on oversight and integrity of the prosecution service, that core institution has shown little interest. CJSSP was able to deliver one combined training of 52 inspectors from the AGO and judiciary, but many planned activities were not feasible, such as updating the AGO code of ethics and improving personnel evaluation systems. Although not squarely within RP2, CJSSP has taken advantage of the AGO’s authorization to work with their anti-corruption unit (PEPCA), delivering training and a detailed manual to guide the investigation and preparation of these complex cases; PEPCA said the manual is still an internal document but is in daily use by its team. PEPCA staff indicated that their effectiveness in prosecuting corruption cases has notably improved since receiving the manual and related training. Following the Judicial Summit, the Judiciary Inspectorate General (JIG) Office has been revitalized with additional resources and a greatly increased rate of investigations and sanctions. The number of inspections and related investigations has climbed from a retrospective baseline of 175 inspections/ investigations in 2015 and 261 in 2016, to 459 in 2017.8 CJSSP has provided training to the JIG team, and TA to develop a new regulation for disciplinary procedures in the judiciary, which the JIG Office considered important to strengthen its oversight function.9 As well, training on the code of ethical conduct was delivered to 600 public defenders in Year 1. Although the judiciary and public defense were not the priority targets anticipated for RP2, USAID staff commented that advances in oversight and integrity systems in other national justice institutions could encourage the AGO to be more proactive in its own integrity systems. The evaluation team is unable to assess the veracity of this claim. Citizen and civil society oversight of justice system operators CJSSP reports and staff indicated that this results area was considered the domain of the PASJ implementers and was included in the CJSSP contract to be used only as needed; thus, virtually no activity has been undertaken directly by CJSSP. However, it is worth noting that Chemonics and FINJUS (as subcontractor to PC under PASJ) have consulted and coordinated on a number of relevant initiatives, including organization and follow-up of the Judicial Summit in 2016, inclusion of the JIG in a public forum where it engaged actively with civil society actors, and other less formal exchanges of ideas and information. CJSSP Challenges A significant cross-cutting challenge identified by Chemonics was the uncertain scope of work with some GODR counterparts; over a period of months, USAID ordered suspension of planned activities involving the DNP, 911 (emergency call) responders, and INACIF. Due to the shifting parameters, CJSSP lost time and energy in the analysis and preparation of interventions in several areas that were later removed from their mandate. Chemonics and USAID said that USAID budget uncertainty had affected planning and implementation, noting also that Dominican state budgets for institutional partners to implement reforms are extremely limited. CJSSP also highlighted the change in AGO leadership following 2016 elections, which severely limited work with the prosecution service for six to nine months until relations could be re-built, as discussed in EQ9. CJSSP’s considerable efforts at building a relationship with senior AGO leadership were thwarted. The Justice Project PAD explicitly stated as an assumption that “the Public Ministry (AGO) will continue to accept USAID sponsored management models for case management and the adjudication of justice…”, but in fact the CJSSP and USAID have faced considerable challenges in 8 Figures are drawn from latest statistical reports provided to the evaluation team by the IG, via Chemonics. They differ slightly from figures previously reported by CJSSP. 9 That regulation was under consideration by the Superior Council of the Judiciary as of the time of writing, with formal approval expected in the near future. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 33 securing that acceptance since the AGO change. So far, approval to proceed is limited to SD West fiscalía, and key informants indicate low levels of interest at present in changing models within the other two high-volume fiscalías (National District and Santiago).10 This might reflect the low level of interaction between CJSSP and these fiscalías to date; however, the evident good will and respect for the CJSSP team shown by the leadership of these fiscalías may create a foundation for cooperation in subsequent years. Implementation of a full-scale national Integrated Case Management System (the original aim of the contract) proved too costly for the GODR after feasibility studies were undertaken with partial CJSSP support, and progress on management models for prosecutors was set back by technical difficulties in the assessment by CJSSP (as described on page 27) and division of Santo Domingo Province fiscalía into three separate fiscalías. Informants and document review indicated that significant time and resources has been devoted by CJSSP to in-depth research, consultations and assessments. The evaluation team identified at least 26 major assessments and studies done by consultants starting in January 2016, of which 15 took place in 2016, and 11 in 2017-18. The first study produced in relation to management models or functioning of the prosecution was dated October 2016. Some USAID and Chemonics staff suggested that this focus on research was necessary due to the three￾year gap in USAID programming in the justice sector, which had resulted in major gaps in knowledge within the mission; the studies were largely designed to fill those gaps and guide project strategy. While the evaluation team is not in a position to judge the extent to which all of these analyses were in fact necessary or useful, it became clear that the implementer team’s attention has been spread across a highly diverse set of topics and target institutions, not all of which were contemplated in the PAD or contract (ex. a study on environmental crimes, and several assessments of communications capacities). Challenges to ADR and other decongestion methods have included the unexpectedly high cost of CJH operations and the large proportion borne by USAID; the CDCS 2014-2018 estimate that 75% of costs were from non-USAID sources was found to be inaccurate, instead it was 36%. Another key assumption underlying this Activity design was that the prosecution function was a primary source of congestion in the criminal justice system; a soon-to-be-released study by FINJUS, funded by CJSSP, has found that the judiciary may in fact be a more serious bottleneck in the system. Sustaining the CJH at a full scale of operations will continue to be a challenge going forward, as will the existence of major congestion factors in the judiciary and other components of the criminal justice system. CONCLUSIONS CJSSP has established itself as a legitimate interlocutor in the DR justice sector, building numerous relationships over the first two and half years of implementation. The critical exception was the AGO senior leadership for a good part of Year 2. Despite success in building institutional and personal relationships, CJSSP has produced, in the words of its own management team, a “smattering of results” thus far; mostly outputs, these are spread over a wide array of thematic areas with diverse beneficiaries and counterparts, with no strong geographic focus. The challenges presented by this high dispersion were exacerbated by the lack of recent engagement by USAID in the justice sector, and the shifting focus of its mandate from USAID in the first year. Overall, CJSSP has made little demonstrable progress towards its contractually defined expected results over the first two and a half years of implementation. With respect to improving prosecutor effectiveness (RP1), services for vulnerable groups and training of justice operators have reached large numbers at significant cost with inconclusive results, owing in part to monitoring shortfalls. Most other areas of activity remain in the initial stages, especially in terms of enhanced oversight (RP2). The contract has a broad scope and yet is highly prescriptive, essentially requiring a highly dispersed approach by the implementer. When advances on some anticipated results were stymied, usually by 10 Page 11 of Justice Project PAD. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 34 USAID guidance to avoid overlap or lack of GODR cooperation, the scope was stretched to include targets of opportunity and emerging priorities. In spite of this flux and early evidence that contractual expectations were unclear to both parties, decisions on formal reorientation of the Activity have not yet been taken. As detailed in EQ8 below, the informal and delayed nature of communications by USAID on these issues have undermined the implementer’s ability to pursue a clear set of priorities within the reduced scope. As a consequence of dispersion, lack of clarity and other setbacks, progress described in interviews and the latest reports often sounds like the first six months of a new Activity; strategies have just been finalized, management models are being drafted, a data sharing platform is about to be launched, etc. Extensive research and stakeholder consultations have helped to fill knowledge gaps and guide development of activities and have laid a better-informed foundation for the rest of the contract period, but application of that knowledge-base towards production of concrete and measurable outcomes is still lagging. Work on justice personnel capacity has been extensive, including assorted topics and target groups not anticipated in the contract, while reforms with potential to complement training are still in the nascent stages in various results areas. Overall, CJSSP has probably trained almost all of the estimated 1,000 to 1,200 prosecutors in the country. Despite this impressive figure, a dearth of in-depth monitoring data (and associated issues with reliability) 11 makes it difficult for the evaluation team to comment on the results of RP1 investments in training on justice personnel capacity and performance. Nor can we say much about the actual effect of enhanced services to GBV victims and other vulnerable persons, and the impact of community justice services and ADR on decongestion of the system. Evidence of change is almost entirely anecdotal at this stage, thus, better monitoring strategies on all of these fronts are urgently needed. The CJSSP MEL Plan is almost entirely made up of output indicators. As for RP2, scant activities have had positive effects in unexpected areas (the AGO anti-corruption unit’s capacity and judiciary’s disciplinary systems), while progress towards expected results has been minimal considering that the contract is halfway to completion. ICITAP RESULTS AND CHALLENGES EQ1. Have these three activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ3. What have been the major achievements and challenges of the activities conducted to strengthen police management and transparency? FINDINGS The PAPA mechanism describes the ICITAP-led Activity as contributing to “USAID Result # 2: Police Transparency and Management Strengthened”, while the purposes are: (1) to improve police – prosecutor coordination throughout the criminal investigations process, (2) to strengthen DNP human resources planning and management, (3) to strengthen DNP community relations capacity and (4) strengthen external oversight mechanisms for the DNP (ex., public information, FOIA compliance). The ICITAP results framework specifies three sub IRs, as follows. 12 • Result 2.1: Police – Improved Delivery of Services to Vulnerable Groups • Result 2.2: DNP Management & Accountability Strengthened • Result 2.3: DNP External Oversight and Community Relations Improved 11 Monitoring system weaknesses are described under EQ6. A Data Quality Assessment in 2017 identified issues with two of seven CJSSP indicators, notably the one on rates of case disposition. 12 See PAPA at 16-18. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 35 ICITAP Results The original three results areas above were cut back soon after start-up.13 The activities under Result 2.1 to increase DNP timeliness and responsiveness in preventing family and gender violence, protecting PWD, members of the LGBT community, and other vulnerable populations were eliminated. So, too, were actions under Result 2.3 to strengthen external oversight of DNP crime prevention and community engagement activities, and improved enforcement of ethics and accountability policies and procedures. The remaining Result 2.2 seeks to increase DNP transparency and accountability in workforce planning and oversight and strengthen institutional management capacity. According to informants in the Planning and in the Human Resources Directorates, the ICITAP activity has focused on operationalizing the capacity-building parts of the DNP’s 2016-2020 Strategic Plan. Further, the Activity has been instrumental in the creation of 28 annual operational plans that align with the DNP’s Strategic Plan and that cover all sub-divisions within the DNP. Through these plans, these sub-divisions are now budgeted using activity-based budgeting, and each budget is approved by the Planning Directorate. In the past, the budgets were created by the DNP Director of Administration and Finance with no institutional process for any other input. Starting in 2018, the DNP is initiating a quarterly review of implementation of all 28 operational plans to understand if gaps in implementation exist and to address such gaps. Almost unanimously, informants in the four Directorates engaged by ICITAP plus past and present senior DNP management stated that this planning effort was instrumental in shifting the culture of the DNP away from “management by personality”, thus creating greater stability, continuity, transparency and accountability in internal decision-making. These informants conveyed a belief that the use of protocols and standard operating procedures (SOPs) developed with ICITAP assistance will have a lasting effect on DNP management. This was further reflected by interviews with senior DNP management; the Team was informed that this change to a rules-based management culture was one of the senior management hopes for the Activity when it was initiated. Last in the theme of the Activity’s “planning” results, is the ability of the DNP to garner increased funding due to the activity-based budgeting that their new operational plans enable it to present to the Ministerio de Hacienda for the annual budget allocation process and be able to demonstrate to Hacienda budget execution from the prior year’s plans. The DNP’s proportional share of the overall GODR budget for current expenditure increased from 1.60% in FY15 to 1.72% in FY18. While causality cannot be directly attributed to the ICITAP Activity, informants stated that the activity-based budgeting allowed the DNP to better defend its last budget request. ICITAP support to strengthening the DNP Human Resources policies and SOPs has also born substantial fruit. Job descriptions and human resources oversight are in place for two of the 15 Directorates with two others near completion. The Human Resources efforts further have led to the creation of a DNP-wide salary scale for staff. This scale means that positions salaries are now determined objectively, and, as many informants stated, raises are not done at the arbitrary whim of a supervisor. This has had the widest impact on the DNP as it has affected the approximate 37,500 DNP staff throughout the country. ICITAP has plans to expand to four additional Directorates per the FY18 draft workplan and draft amended PAPA. Expanding to other Directorates, though, will require additional coordination with INL as ICITAP’s plans encompass the Investigation, Prevention, and Education Directorates, all of which INL stated are foci for its assistance. 13 See Question 4, supra, discussing how these initiatives largely align with the results in the PAD Sub-Purpose 2 (1.3.2): “Police Transparency and Management Improved.” FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 36 ICITAP Challenges The ICITAP Activity has withdrawn from initiatives to improve the police response to the problems that the public expects them to handle. ICITAP has eliminated support for better relations with vulnerable populations, to bolster transparency activities, to enhance police/prosecutor interactions, and to establish police-community partnerships for crime prevention, leaving almost exclusively initiatives to support the internal public management support described in the PAPA. Despite these significant changes there have been no formalized amendments to the PAPA memorializing the reduction in scope of the Activity. As discussed more fully below in EQ6, lack of interagency coordination affected efforts by ICITAP to work on support to vulnerable populations and transparency activities. The Justice PAD notes that INL was consulted in the design of the PAD results framework, and that USAID and INL reviewed the INL 2014-2018 strategy, concluding that “consultations with State INL confirm that planned USAID assistance for the DNP in the areas of institutional coordination, including investigations, human resources and operations management will not be included as components of the INL bilateral assistance portfolio (and thus not be duplicative)”. However, USAID later informed ICITAP that INL would be providing community policing assistance to the DNP and to the DNP Inspector General Directorate. In KIIs with ICITAP, staff communicated that they did not see a way to engage in assistance to vulnerable populations except through the Prevention Directorate, but ICITAP had been informed that this directorate was off limits as INL would be exclusively working with them. In addition, USAID informed ICITAP to withdraw from activities to strengthen police/prosecutor coordination because USAID was informed that neither the DNP nor MP desired this assistance, since neither felt that this was an area in which support was needed. As well, ICITAP believed that the PASJ activity would be working on external oversight of the police, and that ICITAP should therefore not engage in that area. For its part, as discussed below, the PASJ activity has limited traction with the DNP at the local level and expected ICITAP support to engage DNP leadership on police-community relations. ICITAP project management arrangements were not conducive to addressing issues proactively. For the first 21 months, ICITAP managed the activity remotely with a frequent, but not constant, management presence in the Dominican Republic. The only in-country presence was technical staff who reportedly had no knowledge of the PAPA’s requirements. In early 2017, a short-term manager was sited in Santo Domingo. Initially, he was to be in the Dominican Republic for two months until another permanent manager could be mobilized. When that permanent manager withdrew, the temporary manager was left in place, ultimately, for almost an entire year. During that period, the Year 2 Workplan expired at the end of September 2017, a new draft workplan and draft Amended PAPA were drawn up, but neither had been approved at the time of fieldwork. CONCLUSIONS The ICITAP PAPA was initially aligned with the police-oriented activities in the PAD, but the narrowing of activities has undermined the Activity’s contribution to PAD results. ICITAP has achieved strong results in a relatively narrow area of implementation, by assisting the DNP in implementing their strategic plan, and supporting operational planning and budgeting. By creating a stable, transparent policy framework and SOPs in human resources, ICITAP has had DNP-wide impacts. Those successes minimize opportunities for corruption, e.g., with more transparent budget-setting and less arbitrary assignments of salaries and raises. Moreover, ICITAP has fostered solid relationships with senior DNP management, and these relationships could be parlayed in the future to expanded engagement. However, ICITAP lacked a management structure that was sufficiently motivated and empowered to put its imprimatur on the activity and to address the reductions in scope by identifying alternatives or work￾arounds. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 37 ICITAP is positioned to engage DNP senior leadership in Santo Domingo to support activities in Results 2.1 and 2.3—including improving police response to vulnerable populations, improved community relations and enhanced transparency. However, to achieve the full range of PAD-envisioned results, which have to do with the “end products” of policing (how police respond to crime in a given community) as discussed below in EQ7, USAID must help resolve inter-agency conflicts to make more efficient and effective use of this Activity. Additionally, guidance should be given formally in activity￾management documents that clearly show what ICITAP is expected to achieve. PASJ RESULTS AND CHALLENGES EQ1. Have these three activities achieved the expected results of their contracts or agreements for the first half of the implementation period? EQ4. How effective have the civil society activities been in creating demand for justice and monitoring law enforcement and crime prevention efforts, both at the national and local levels? FINDINGS The discussion of findings in response to EQs 1 and 4 is organized into results and challenges, which refer to the three results areas of the PASJ mechanism: • Result 1: Public Oversight of Crime Prevention Improved • Result 2: Public Input in Criminal Justice and Citizen Security Policy Reform Increased • Result 3: Community Engagement in Crime Prevention Increased PASJ Results The cooperative agreement (CA) “Civil Society Action for Accountable Justice and Security” (termed PASJ in Spanish) was awarded to PC with FINJUS as the major, named sub-grantee. The design of the activity and the allocation of roles—PC with a more activist role working with CSOs in awareness raising, and FINJUS in a more technical research and policy advocacy role—repeats implementing arrangements under previous USAID/DR activities with PC and FINJUS over 25 years for democracy and governance and rule of law activities, beginning with the Democratic Initiatives Program from 1992 through 2002.14 Result 1: Public Oversight of Crime Prevention Improved. Of the 26 key informants from national level institutions interviewed about the effectiveness of the PASJ interventions, 19 considered that the activity had been effective in raising public awareness of human rights and informing the public and vulnerable populations of rights with respect to the justice sector and the police. Many informants cited the PASJ activities to train CSO leaders on human rights and the duties of the police in relation to the community. PC’s own reports mention 58 national and regional organizations participating in this training. Others highlighted the broad reach of training for journalists and communicators in the Duarte corridor on justice and security. 14 For the early history of USAID/DR’s support to PC and FINJUS see: R. A. Boglio Martínez. 2007. “U.S. civil society promotion and the construction democratic governance in the Dominican Republic”. VI Conferencia Regional de ISTR para América Latina y el Caribe. November 8-11, 2007, Salvador de Bahía, Brazil; and Checchi & Company Consulting, Inc., and DevTech Systems, Inc. 2002. Assessment of USAID Civil Society Programs in the Dominican Republic. Task Order No. AEP-I-803-00-00022-00. Santo Domingo: USAID. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 38 PASJ has dedicated significant resources to promoting public debate on legislative and policy reforms in justice and security in the framework of the 2016 election campaigns (President, Deputies, Senators and Mayors). This included moderated debates with all the Presidential candidates. All informants agreed that this raised the profile of justice reform issues in the election campaigns. Interviews with organizations that participated in access to justice training supported the initiative to include women, LGBT, PWD and people of Haitian descent. They pointed to frequent abuse or indifference of police and prosecutors to crimes against vulnerable populations and welcomed the initiatives. The evaluation team looked at PASJ’s activities to establish networks of civil society leaders in Puerto Plata, Santiago, San Francisco de Macorís, La Vega, Bonao and National District in Santo Domingo. In Year 1, PASJ conducted FGDs with GODR representatives and CSOs to try to establish a mechanism for government-civil society dialogue on justice and security; a participatory security diagnosis and then a plan of action for each local network. The networks in five cities (Bonao was dropped) became a major focus of PASJ: Of 3,389 people trained in Year 2, 2,199 were from these CSO networks. The KIIs and FGDs with members of local coalitions in Puerto Plata, Santiago and National District indicated that they understand and identify with the issues of security and crime prevention prioritized in the community diagnoses conducted by PASJ in 2016 and in their action plans. They have given emphasis to preventing and responding to GBV, working with existing networks of GODR agencies such as the MW and the National Council for Youth and Adolescence (CONANI in Spanish), fiscalías (including specialized GBV victims units), public hospitals and CSOs. They have also worked effectively with LGBT communities to promote their rights and address the frequent reports of police abuse and extortion (of money or sexual favors). While the members agree that the coalitions should continue to work on crime prevention, they consider that deterrence is critical. They believe that the DNP is failing to deter crime because of its poor relations with the community and the involvement of DNP personnel in crime. In Puerto Plata and Santiago, FGD participants highlighted how police involvement in crime or payoffs by drug traffickers to police led to a “climate of lawlessness” that causes impunity. Added to these problems are the constant police abuses, extorting people on paydays (macuteo); extorting motorcyclists to return their motorcycles when they are impounded for driving without papers; and subjecting local youth to arbitrary detainment in the frequent, large scale “round ups” (redadas), which create distrust of the police and undermine deterrence of all types of crime. Result 2: Public Input in Criminal Justice and Citizen Security Policy Reform Increased Result 2 is essentially a policy research and advocacy activity, led by FINJUS and supported by a coalition of national civil society and business organizations, with membership between 12 and 17 organizations (including PC and FINJUS).15 There has been strong buy-in from the business sector through the National Council of Private Enterprise (CONEP); Association of Young Entrepreneurs (ANJE); Herrera Industrial Association and the American Chamber of Commerce (AMCHAM). At the end of 2016, CONEP dropped out of the coalition. FINJUS itself is the creation of a business group, and several other organizations, although not representing business, have been created and led by business people. In contrast, there are no activist CSOs represented in the coalition, other than PC itself and Poder Ciudadano. None of the important Dominican human rights organizations have been involved. 15 The PASJ Annual Progress Report 2015-2016 lists 17 members of the Coalition, while the PASJ Annual Progress Report for 2016-2017 lists 14, and a list provided to the evaluation team by PC in March 2018 names only 12. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 39 The evaluation team interviewed ten of the coalition member organizations, along with four people listed in the PASJ reports as participants in advocacy activities but not members, who requested not be named in the report. All but three coalition members believe that the coalition has been effective in mobilizing members to promote legislative reforms through research, engagement with experts and justice sector operators (judges, prosecutors and lawyers), meetings with legislators, and specialized forums. All interviewees agreed that FINJUS produces high quality, relevant research on reform issues and contributes to the quality of legislative drafting. PASJ reports that it has mobilized other civil society and business organizations; in Year 2, for example, 65 organizations participated in the advocacy actions through round tables, forums, and conferences. These coalition events, while usually small in size, are covered extensively by print and electronic media. KIIs with coalition members highlighted the use of television programs (22 in total) on specialized topics of justice and crime prevention. The centerpiece of the PASJ Result 2 activities has been a “legislative agenda” made up of ten reform proposals.16 Seven PASJ coalition members highlight the success of the FINJUS approach to promoting these reforms by conducting policy research and conducting structured debates with experts, researchers, and justice sector operators. By the end of 2017, eight of ten draft laws developed with input from FINJUS had been discussed in Congressional committees, and two reforms had passed (Organic Law of the National Police, and Law for the Control and Regulation of Firearms). Through Year 2 and the beginning of Year 3, there was intense media coverage and debate on three draft laws (on Political Parties, Elections, and Penal Code) in which FINJUS figured prominently. Result 3: Community Engagement in Crime Prevention Increased In this result area, PASJ has focused on creating local crime prevention initiatives In Santo Domingo West (SDW) and the Cienfuegos neighborhood in Santiago, supported by CSOs, local authorities, business groups, DNP, regional offices of GODR agencies and individual schools. The SDW initiative is a continuation of PC’s activities with a local network of CSOs working on youth issues under a subgrant from another USAID initiative, At Risk Youth (ARY), while the Cienfuegos initiative is new. Both are supported by the CJHs in their neighborhoods, mainly by lending meeting space, but also through their legitimacy and contacts in the community. PASJ has supported a pilot program of conflict mediation in two schools in SDW and Cienfuegos. In Year 1, PASJ trained students as school mediators and “conflict managers”. Based on KIIs and FGDs with members of the two initiatives, the approach has been effective in reducing intra-school violence. In Year 2, PASJ is taking it further to develop a “protocol on violence-free schools” to address all forms of violence, including harassment and bullying. Also, PASJ began a training program to extend the pilot to the other local coalitions in Puerto Plata, La Vega, San Francisco de Macoris and National District. In Year 2, parallel to the school mediation pilot, PASJ supported the design and implementation of local crime prevention plans, with a series of workshops in Cienfuegos and SDW. While incorporating school mediation, the plans included other topics such as youth leadership, mediation and conflict resolution, adolescent pregnancy, family violence, and gender violence. The participants in Cienfuegos (including the PASJ coordinator) were unable to explain how/ why their initiatives have a narrower focus 16 Draft law of the Dominican Penal Code, draft Law on Extinction of Dominion for the Civil Confiscation of Illicit Property, Draft National Organic Law of the National Police, Draft Law for the Control and Regulation of Firearms, Ammunition, Explosives, and Other Related Materials, Bill of Notary Law, Draft Law on Political Parties and Groups, Draft Law on Reform of Electoral Law No. 275-97, Draft Law on Responsibility and Fiscal Transparency, draft Law on Protection of Victims, Witnesses and Other Procedural Subjects and proposal of Organic Law of the Ministry of Interior and Police. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 40 than the PASJ-supported coalitions in the other localities. The participants in SDW said that their activities were youth-focused and strictly limited to crime prevention. PASJ Challenges The informants most critical of PASJ emphasized the wide thematic dispersion of the program activities, also pointing to a confusion about awareness raising on human rights versus carrying out oversight of justice and police. They said that PASJ’s efforts in oversight had not been effective because its activities were too thinly spread out, with training and events on multiple topics. Also, several stakeholders were critical of the decision to incorporate elections observation on access of PWD, which they saw as outside of their scope and something imposed by USAID. “With PASJ, everything is a priority, so nothing is”, said one informant. The thematic dispersion is also illustrated in PASJ progress reports. In Year 1, there were 12 different Sub-IRs just in Result 1; this was extended to 14 in the second year. With respect to PASJ’s activities with vulnerable populations, most interviewees said that activities had been under-funded, sporadic and lacked follow up. Three considered that PASJ duplicated efforts of organizations specialized in vulnerable populations, such as PWD or LGBT communities. While PASJ produced a directory of organizations assisting vulnerable populations, informants pointed out that other specialized organizations had already produced such a directory. Some were especially critical of PC’s claim to have established a network of organizations. “A network is not a phone book”, said one informant, pointing to the lack of follow-up by PASJ. With respect to the local coalitions, the CSO leaders say they are committed to the coalition work plan but require more PASJ support to implement it. They point to the low level of resources from PASJ to support concrete crime prevention initiatives in the localities. In the FGDs in Puerto Plata, Santiago and National District, a recurring theme was the unwillingness of the DNP and the Ministry of Interior and Police (MIP) to dialogue with community groups on security and crime prevention. The FGDs and KIIs all emphasized the need for USAID support to bring the DNP to engage systematically with the coalitions, along with more budget resources. In an FGD with PASJ community promotors, the participants said that they did not understand why USAID had not instructed PC to extend the successful school mediation approach to all targeted localities. Three PASJ coalition members and four informants not in the coalition agreed that the approach of the PASJ coalition working through dialogue with small numbers of people was effective in cases like the laws on police or firearms, however, all believe that the coalition has no traction on issues that affect political interests of the governing party (i.e., appointments of judges, or the draft laws on political parties, elections). They also said that FINJUS has not taken a consistent position on what one informant termed “the elephant in the room”: the control of the judiciary by ruling party (Partido de la Liberación Dominicana or PLD in Spanish) operators in the National Council of the Magistracy (CNM in Spanish), who have consistently favored PLD nominees in appointments to the high courts. “The big problem continues to be the CNM, and it cannot be addressed through a USAID project”, said one interviewee, adding that “it is just too political and too sensitive”. All ten coalition members and the four non-members said that coalition activities in support of judicial reform had waned in Year 2. Most agreed that the coalition lost its unity of purpose and as a result maintained a “low profile” in the aftermath of the Judicial Summit, and particularly during appointments to the high courts by the CNM in 2017. They said that the low level of activity reflected in part a lack of consensus on how to approach the “thorny issue” of the PLD’s political control of the appointment process managed by the CNM. All agreed that as elections approach in 2020, it will be difficult to maintain PASJ coalition activities on policy reform. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 41 Several informants mentioned that the PASJ national coalition had lost an opportunity in not effectively addressing the issue of budget resources allocated to the AGO and the DNP, which are directly related to the Activity objectives. Rather, the national coalition dispersed its advocacy activities in multiple reform issues unrelated to the Justice Project objective of increasing effectiveness of criminal prosecution. Several coalition members and all four non-members interviewed thought that USAID involvement in the areas relating to judicial independence, appointment of judges, political party reform, campaign financing, and elections was increasingly “unproductive” and resented by the ruling party. Several pointed out that the situation is quite different from the post-Balaguer transition, when it was easier to build a consensus on reform. Today, despite worrisome governance problems, business groups have found an accommodation with the ruling party and are unwilling to support constant, public pressure on the PLD. Seven interviewees said that the PLD was pressuring business leaders to back off public criticism of the government and the party on sensitive topics of judicial independence and political party reforms.17 CONCLUSIONS With respect to effectiveness in creating demand for justice and monitoring law enforcement and crime prevention efforts, PASJ has made contributions to the approval of two legislative reforms at the national level. The PASJ national coalition has been dominated by business interests to the detriment of activist CSOs and has been weakened by differences on how to address issues of justice sector independence and political party and electoral reforms, which led to a reduction in coalition activities and public profile. A previous assessment of USAID support to a PC and FINJUS partnership, conducted in 2002, noted that they have different approaches, the former supported by activist professionals with a focus on social mobilization, the latter supported by business elites with a focus on policy research and advocacy. It concluded that “they still lack coordination to devise strategies and form united fronts to press for greater impact in legal reform.18 Under the PASJ activity, this tension continues and limits the coalition’s effectiveness in promoting reform. At the local level, PASJ has been effective in creating five and a half coalitions and mobilizing broad based local coalitions to lead crime prevention activities. The “half coalition” is in SDW, where the group formed under the ARY activity is not as broad based as in the other five localities, as it lacks the participation of community-based organization (CBO) leaders at neighborhood level, such as Juntas de Vecinos (neighborhood councils). PASJ has adopted an inductive approach, giving wide scope to each community in defining its strategies, however it recognizes the need for developing more structured interventions. While local coalitions welcome the support for awareness raising on human rights, they want support and budget resources for more concrete initiatives to strengthen crime prevention, deterrence, investigation and prosecution at the local level. One area of particular importance that has been overlooked by USAID until now are the local networks to support community prevention and institutional response to GBV. Also, the coalitions want to scale up the school mediation methodology in their communities piloted in Years 1 and 2. Local coalitions point out the need to work on crime deterrence, too. However, they require more support from PASJ in building partnerships with local authorities, GODR agencies and the DNP and in conducting activities. In the case of the police, given 17 Several KIIs also noted that many businesses were vulnerable to investigation and prosecution under recent money laundering legislations and see real possibilities of government reprisal against those business leaders who take an overly critical stance. They say that this has had a chilling effect in the second year of the Justice Project. 18 Checchi & Company Consulting, Inc., and DevTech Systems, Inc. 2002, cited above. Page 184. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 42 often conflictive relations with the regional and local DNP officers, there is need for agreement with DNP leadership in Santo Domingo to create sustainable crime prevention partnerships. Overall, PASJ has made progress towards some of its expected results in the first two years. However, the CA contains many areas of interventions and PASJ’s activities are spread thinly, raising questions of the depth, scale and sustainability of some results (this will be taken up again in relation to overall results in EQ 5 below). The national coalition has taken on high-profile topics that do not directly relate to the Justice Project results framework, such as appointments of judges and political party and electoral reform. While these issues are intrinsically important to democratic governance in the DR, they do not contribute to the CDCS and PAD results within the time frame of the CDCS, and coalition interventions may well undermine possibilities of cooperating with GODR in achieving intermediate results directly related to the overarching PAD results of crime reduction (prevention/deterrence/ investigation/prosecution). OVERALL PROJECT RESULTS EQ5. Taken as a whole, have the three activities produced the results set out in the CDCS, “Criminal justice institutions strengthened”, and in the Justice PAD, “to strengthen the GODR criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”? FINDINGS As to whether the Justice Project activities have produced the results set out in the CDCS and PAD, the evaluation team finds it challenging to answer, due to the lack of data about project outcomes to date. This challenge will be addressed in more detail in response to EQ6. This section presents findings relating to the concordance between the PAD and the design of the three mechanisms; and the degree of dispersion of the expected results/tasks/actions in the mechanisms. Concordance between PAD and Mechanisms CJSSP With respect to CJSSP, the outputs and targets in the PAD correspond closely to the 13 overarching expected outcomes in Section C of the contract. However, those expected outcomes do not match up with the “results packages” and tasks in the contract, which has caused confusion as to priorities of the activity and the actual deliverables of the contract. Over the first two years of implementation USAID instructed the CJSSP implementer not to work with the DNP, 911 responders or forensics institute, which in practice curtailed their ability to fully implement certain Outcomes/Results that would have naturally included those entities. These were: • Expected Outcomes: Public Ministry / National Police Working Relationships and Coordination Improved, Public Ministry Capacity to Manage Criminal Investigations Improved, etc. • Result 1.3: Strengthened legal protection of vulnerable groups. • Results Package 2: Prosecutor and Police Oversight Strengthened, notably Result 2.1: Strengthened oversight management; Sub IR 2.1.1 Improved responsiveness and handling of citizen complaints; and Result 2.2: Improved professional conduct of prosecutor and national police through revitalized ethics and accountability standards. ICITAP Comparing the ICITAP PAPA with the PAD, the activity results statements map closely to the PAD outcome and expected results. However, in the first two years, USAID instructed ICITAP to withdraw FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 43 from most proposed areas of activity, focusing efforts on Result 2.2: DNP Management & Accountability Strengthened. The results areas eliminated were: • Result 2.1 areas of activity: Establish and strengthen DNP standards and procedures for family and gender violence (F/GBV) service provision; Strengthen DNP professional knowledge of, and service delivery to members of the LGBT community, persons with disability, and other vulnerable groups. • Result 2.2 areas of activity: Strengthen DNP compliance with existing GODR transparency and public oversight laws (ex., FOIA, budget, procurement); Strengthen ethics oversight and enforcement within the National Police. • Result 2.3 areas of activity: Establish and assign a cadre of vetted DNP community liaison officers. This included as an indicator “community/police partnerships established”. Overall, the activities that were left focused on reform of DNP administrative functions, such as planning, budgeting, and human resource management, which are expected to have long-term impacts on DNP management, but not necessarily on governance practices related to ethics, external accountability, and public oversight, or on policing practices in localities targeted by the CDCS. Comparing the results and Sub IRs dropped from the CJSSP mechanism with what was dropped from ICITAP, the evaluation team finds that three results relating to DNP governance—strengthening DNP’s GBV/vulnerable groups protocols, internal integrity systems, and external transparency/accountability—were eliminated from both CJSSP and ICITAP. PASJ The PASJ CA reflects all the expected results set out in the PAD, and expands them considerably, adding Sub IRs and “illustrative activities”. Nevertheless, some proposed results and activities have not been implemented by PASJ in the first two years. These include: • Result 1.2 areas of activity: Assessment of the state of affairs of judicial career, Public Ministry, Public Defense and the National Police, as well as the consequences regime; Citizen Monitoring of the Public Ministry and the National Police career system and the consequences regime. • Result 1.4 areas of activity: Information campaign and public outreach to inform society of the Institutional Integrity System of the Public Ministry, the Public Defense National Office and the judiciary, as well as mechanisms for citizen’s action. Thus, two of the three DNP governance outcomes eliminated from CJSSP and ICITAP—relating to DNP internal integrity systems and external accountability—were not implemented by PASJ either. PASJ has not worked with the DNP on these issues, other than inviting police officers to attend forums and workshops on the new Police Law. Similarly, PASJ has not had a clear focus on strengthening prosecution. A related finding is concerned with the PAD assumptions about synergies between ICITAP and PASJ. The PAD states, “USAID assistance to strengthen DNP collaboration with civil society, and improve community-police relations, will support the accomplishment of Sub-Purpose 3.1: Public Oversight of Crime Prevention Improved”. The PAD assumed that the activities to strengthen community-police relations in the DNP (ICITAP) and to enhance DNP (and Public Ministry) integrity and accountability systems (CJSSP) would provide the “supply” of institutional will to respond positively to activities in support of a “demand” from civil society for dialogue with DNP (PASJ). The KIIs with PASJ stakeholders and the review of PASJ progress reports show that this did not happen. MIP and DNP leadership have resisted PASJ efforts to establish a structured, continuous dialogue on police effectiveness. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 44 Dispersion of Tasks and Actions The design of two of the mechanisms incorporates a large number of expected outcomes, results, and specific tasks (CJSSP) or Sub IRs and actions (PASJ). See the summaries in Table 3 and Table 4 below. The CJSSP Year 1 Work Plan is organized into 14 Tasks and 157 individual actions, whether in training, research, assessments, designs, workshops, meetings and forums or others such as production of publications or materials. In the Year 2 Work Plan there was an increase in the number of tasks to 17, but a reduction in the individual actions, to 70. It should be noted that this classification underestimates the CJSSP activity dedicated to training, as the multi-session courses or diplomas are counted as a single action. The Year 1 and Year 2 Work Plans are resumed in the above table. The category of “other” includes relatively minor actions such as publications, communications materials, etc. Table 3: CJSSP Results Areas, Tasks and Actions CJSSP WORK PLANS YEAR 1 YEAR 2 Results Areas 7 7 Tasks 14 17 Individual Actions Workshops, meetings, forums 22 0 Training events 32 15 Consultancies (activity designs, research, etc.) 30 15 Sub Grants 5 3 Others 68 37 Total Actions 157 70 The apparent reduction in dispersion of the CJSSP activity in Year 2 in fact reflects lack of traction on various results areas. The number of Tasks actually increased, as CJSSP looked for other activities outside of the original focus on the AGO and the fiscalías. As discussed above in the response to EQ1 and EQ4, activities aimed at improving prosecutor effectiveness (RP1) were essentially stalled during that period. Table 4: PASJ Results Areas, Sub IRs and Activities PASJ WORK PLANS YEAR 1 YEAR 2 Results Areas 11 11 Tasks (Sub IRs) 59 57 Individual Actions Workshops, meetings, forums 204 157 Training events 44 37 Consultancies (activity designs, research, etc) 34 33 Others 38 25 Total Actions 320 252 The PASJ mechanism contains more areas of intervention than CJSSP. The CA set out about 60 tasks (in this case called Sub-IRs), and to achieve these results PASJ proposed to carry out 320 individual program FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 45 actions in Year 1 and 252 actions in Year 2. While the Year 3 Work Plan had not been approved at time of writing, the draft version does not reduce considerably the Sub IRs, nor the large number of actions. Immediate versus Mediate Results The evaluation team also analyzed the design and implementation of the three Justice Project mechanisms in terms of the time frames for results of the major tasks and actions conducted, especially in the context of a CDCS that ends in 2018 CJSSP • With respect to CJSSP, the evaluation finds that there was little demonstrable progress after two and a half years on three tasks in Results Package 1 that were expected to produce immediate impacts within the time frame of the CDCS: the new management model in high￾volume fiscalías, strengthened inter-institutional coordination and case management, and expansion of CJHs to decongest the formal justice system. • As mentioned above, CJSSP’s training effort in the first two years probably reached almost all of the estimated 1,000 to 1,200 prosecutors in the country. In the internet survey conducted by the evaluation team, the ENMP training was positively evaluated by the junior prosecutors (fiscalizadores), but it is not possible to measure the immediate impact on expected CJSSP results, as the training was not integrated with other CJSSP interventions. For example, as mentioned in other parts of this report, the training was not designed to support the specific CJSSP interventions to establish a new management model for the fiscalías. • In Results Package 2, there was minimal progress made on tasks relating to AGO oversight and integrity, which could have produced a short-term result in prosecutor effectiveness. Instead, while working to overcome AGO resistance, CJSSP has focused efforts on judicial oversight, which has contributed to positive change but has less potential for direct impact on the activity’s expected result of improving the quality and effectiveness of prosecutions. ICITAP • The ICITAP activity was de-scoped considerably. The remaining tasks under Sub-IR 2 are aimed at internal management reforms, such as planning, budgeting and human resources, whose impacts on police effectiveness will be seen in the long term, not within the CDCS time frame. • The tasks that were eliminated under Sub-IRs 1 and 3 were designed to generate immediate impacts on police effectiveness, through changes in policing practice in specific kinds of cases (GBV or crimes against LGBT) and in the specific localities targeted by USAID, which could have been observed within the time frame of the USAID/DR CDCS. PASJ • Result 1 of the PASJ focuses on “awareness raising” about judicial reform and human rights, aimed at leaders of CSOs, journalists, and vulnerable groups. While they aim to involve DNP in workshops on rights of vulnerable populations, there is no specific intervention with the DNP. • Only one of the Result 1 Sub IRs, the coalitions of CSOs to support crime prevention initiatives and partnerships, is focused on direct oversight of justice sector operators and proposals for changes in crime prevention/investigation/prosecution in the targeted localities. • In Result 2, of ten prioritized legal reforms, four have nothing to do with the objective of crime reduction or prosecutor effectiveness (Bill of Notary Law, Draft Law on Political Parties and Groups, Draft Law on Reform of Electoral Law No. 275-97, Draft Law on Responsibility and Fiscal Transparency). FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 46 • PASJ Result 3 emphasizes pilot projects on crime prevention, focusing on mediation of conflicts in schools. While this could have a direct, and immediate impact on reducing violent crime in targeted communities, it is a small-scale, pilot initiative, and has not been integrated into the local coalitions supported by PASJ. CONCLUSIONS The first conclusion about overall results is that the design of CJSSP, ICITAP and PASJ reflects the expected outcomes and outputs of the PAD, even while the PAD itself is poorly structured, without a clear results framework, and with many inputs, outputs, outcomes and indicators, presented in a disorganized fashion. A second, related conclusion is that the thematic dispersion in the PAD was reflected and accentuated in the design of CJSSP and PASJ, shown by the large number of tasks/Sub IRs in the CJSSP contract and the PASJ CA, which are carried over into the hundreds of actions set out in the annual work plans. This led to a “miniaturization” of some actions, with little depth, scale and/or continuity. While the design of the ICITAP PAPA was more focused, outside of the core management reforms in the DNP, the other reforms—strengthening coordination with the AGO on investigations, improving response to crimes against vulnerable populations, strengthening of DNP integrity systems, DNP external accountability, and community relations—were eliminated from ICITAP activities. They were also eliminated from CJSSP and not effectively addressed by PASJ interventions, owing to the inability to engage DNP leadership. Thus, the evaluation concludes that the police reform component of the Justice Project missed the opportunity to contribute to the overall PAD result of crime prevention/reduction within the CDCS time frame. The management reforms may have important impacts in the longer term. The evaluation also concludes that many of the tasks/Sub IRs that could produce results within the time frame of the CDCS in the targeted localities in the Duarte corridor were not implemented effectively. In the case of CJSSP RP 1, the implementing partner has provided no measurable evidence of progress in prosecutor effectiveness. Indeed, CJSSP was unable to develop indicator data to measure case disposition in the large fiscalías. As mentioned, in ICITAP, the activities that could generate short term impacts in DNP crime prevention and investigation were eliminated. In PASJ, the dispersion of resources in multiple IRs meant that citizen engagement with justice sector operators in targeted localities lacked depth and continuity. The absence of ICITAP support for strengthening police-community relations undermined PASJ effectiveness, which was unable to gain traction with local police without support of national DNP leadership. Thus, the evaluation team concludes that the Justice Project activities have not yet produced significant, attributable results except in the few areas highlighted in the responses to EQs 1, 2, 3 and 4 above. The lack of depth, scale, and/or continuity of many actions undermined efforts in many areas. In the case of CJSSP, the dispersion of efforts may partially explain why the activity has achieved so little tangible change in the first two and a half years. ICITAP results, too, may not be seen in the life of the PAD in that they have limited the interventions to reforms of core DNP management functions, excluding the planned interventions aimed at improving quality and effectiveness of the police response to crime and violence in specific communities. The PASJ advocacy interventions were thematically dispersed across at least ten different legislative reforms, some with no direct link to crime prevention/reduction. In contrast, the PASJ activities in support of local coalitions have produced concrete results in terms of community oversight and partnerships with other GODR actors in crime prevention. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 47 LINKS TO OVERALL PROJECT GOAL EQ6. Taken as a whole, have the activities in support of justice sector reform supported the expected links between the overall goal of crime prevention and the intermediate results of improvement in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities? FINDINGS To respond to this question, the evaluation team reviewed the Justice PAD results framework and proposed indicators, and the CJSSP, ICITAP and PASJ Activity Monitoring, Evaluation and Learning Plans (AMELPs) and progress reports. The document review was supplemented with KIIs with IPs and USAID technical officers. Implementation of the PAD’s Geographic Focus The PAD refers to “increasing access to justice in marginalized communities; reducing case backlog and pretrial detention in the…highest volume judicial districts; increasing the transparency and accountability of GODR law enforcement and crime prevention efforts; and decreasing crime and conflict in targeted areas”. In principal, it requires a geographic locus for activities in marginalized communities/judicial districts/targeted areas, which is developed further in the references to activities “geographically focused on the Duarte Corridor” and to “increase access to justice and establish community crime prevention partnerships… focused on Santo Domingo and Santiago provinces”. As shown in Table 5 above, the PAD guidance on geographic focus is maintained in the CJSSP, ICITAP and PASJ mechanisms. The CJSSP work to support the new prosecutor management model was focused on Santo Domingo province but included National District (DN) and Santiago in some assessment activities and for monitoring of case disposition. Also, CJSSP intervened on vulnerable populations in those three localities (and beyond) through sub-grants. In the ICITAP PAPA, the proposed activities to strengthen DNP community relations capacity through police-community crime prevention partnerships were limited to Santo Domingo, Ward 3 of the National District (DN) and Santiago.19 However, as discussed above, these activities were subsequently discontinued. PASJ worked with all six localities in activities with vulnerable populations and crime prevention coalitions, adding a seventh locality, Bonao (Monseñor Nouel – MN) but that was dropped. Table 5: Proposed Geographic Location of Justice Project Activities INTERVENTION SD DN STGO. PP LV SFM MN CJSSP Prosecutor management model X CJSSP Community justice houses X X X X X X CJSSP Vulnerable populations (GBV, LGBT) X X X ICITAP Crime prevention partnerships X X X PASJ Vulnerable populations X X X X X X PASJ Crime prevention coalitions X X X X X X PASJ ADR/mediation X X 19 See ICITAP PAPA pg. 15. “Training and TA to strengthen DNP community relations capacity will be coordinated with other USAID-funded assistance to establish and strengthen community crime prevention partnerships …in targeted areas of Santo Domingo province and Santiago. This will include, but not be limited to: a) Ward 3 of the Santo Domingo National District… and b) Santiago communities of Cienfuegos, La Joya and La Ciénaga.” FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 48 While there was an assumption in the PAD about geographic focus and co-location of activities, the latter expressed sometimes obliquely in Section C of the individual mechanisms, the KIIs with the three IPs and document review show no effective coordination of activities in specific localities to achieve synergies between multiple interventions. For example, while the CJSSP activity was enjoined to prioritize Santo Domingo province in its case management model to strengthen the fiscalias, this was not reflected in the ICITAP or PASJ mechanisms or work plans, to prioritize this same locality and sequence activities in coordination with CJSSP interventions. When CJSSP made a sub-grant to PACAM on GBV, there was no effort at coordinating the interventions (outreach, campaigns in targeted communities, work with CBOs, etc.) with PASJ local coalitions in the National District. Similarly, in the subgrant to COIN, there was no apparent coordination with PC, which was also working on vulnerable populations; nor was a connection made to the PASJ local coalition working on this same topic. Nor were the three work plans designed to include the possibility of joint coordination by IPs with counterpart institutions in the field. The KIIs show that the IPs did not coordinate with fiscalias, DNP, MP, MIP or municipalities to support broader crime prevention initiatives in the targeted localities, as suggested by the PAD in its reference to the USAID regional activity, Municipal Alliances for Violence Prevention (AMUPREV in Spanish) implemented in the DR by the Federation of Dominican Municipalities (FEDOMU).20 The KIIs and FGDs with the coalition members introduced a note of caution about the desire of the MIP to include only “politically tame” CSOs in the Boards, and for that reason the MIP has invited associations of Juntas de Vecinos whose leaders are linked to the PLD. However, in the face of the evident politicization of the CSO component of MIP’s local Boards, neither USAID nor the IPs developed a strategy to establish alternative mechanisms for joint coordination with GODR in targeted localities. PAD Monitoring, Evaluation and Learning Guidance Despite the PAD emphasis on geographic focus, the section on Monitoring, Evaluation and Learning (MEL) is silent on whether it considers this focus to be more than the sum of its parts, i.e. whether co￾location will produce any sort of “critical mass” of crime prevention or crime prosecution effectiveness. Nor does it discuss how to measure performance in targeted localities. In the PAD narrative, there is a proposal for high-level impact indicators, “Citizen confidence in Justice” and “Citizen Confidence in Police”, and a reference to the “LAPOP survey”, i.e., the national level Latin American Public Opinion Project survey funded by USAID/DR in previous years. However, there is no discussion of how the LAPOP survey could measure and attribute impacts of USAID interventions in specific (sub-national) localities. The Justice PAD’s Monitoring and Evaluation Plan and Learning Approach and Logical Framework mention five different impact indicators: “Percentage change in robbery/assault victimization (people over 18 yrs. – Targeted Areas”; “Number of Cases Accepted for Prosecution”; “Percentage Increase in Cases Prosecuted”; “Percentage Decrease in Minor Crimes”; and “Rate of reports of Intra family/family/ domestic violence per every 100,000 inhabitants”. Under “data sources”, there is a brief mention of MIP and/or DNP statistics, with no discussion of the availability of disaggregated data in specific localities (e.g., Ward 3 National District, Santiago, Cienfuegos neighborhood). Similarly, there is no mention of baseline measures, or the frequency of follow-on data collection for performance monitoring. 20 Since 2013, FEDOMU had begun to work with the MIP through the legally mandated mechanism of the Public Board for Security, Citizenship and Gender (Mesa de Seguridad, Ciudadanía y Género). The PAD proposed that the approach be incorporated into the activity designs, but it is not found in any of the mechanisms. KIIs with PASJ community promotors indicate no follow up with FEDOMU to explore the potential of the Boards. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 49 Performance Measurement in the Activity MEL Plans All three of the AMELPs have undergone revisions and modifications to reflect changes in the tasks/Sub IRs and actions. Reviewing the AMELPs for Years 1 and 2, of 24 performance measures mentioned in the PAD Logical Framework, only six are reflected in the AMELPs, of which three could be termed outcome (or impact) indicators. They are: • Number of cases received/processed according to the procedures adopted (disaggregated by type location, etc.), reflected in CJSSP Indicator 1. • Number of internal management norms, policies and procedures developed and implemented, reflected in ICITAP Indicators 3, 4 and 5. • Number of CSO/community crime prevention actions implemented, reflected in PASJ Indicator 1.3.2. Overall, as summarized below in Table 6, the AMELPs have a relatively small number of impact indicators. CJSSP has one outcome indicator, while PASJ has three. Most indicators measure inputs and outputs, such as people trained, or beneficiaries of actions. In principal some output indicators could be interpreted as “outcomes”. For example, if the strategy to increase GODR support to the CJHs had resulted in a large increase in budget support and in cases mediated by the CJHs, this could be interpreted as a sustainable outcome in creating capacity to reduce congestion in criminal cases entering the fiscalías in the targeted locations, assuming that the CJHs are correct in saying that their ADR work “de-escalates” violence in the community and prevents conflicts from becoming criminal cases. 21 However, this didn’t happen, and the mediation targets in the CJSSP AMELP reflect a “steady state” of existing CJH financial, logistical and human resource capacities. Likewise, CJSSP might have monitored the critical variable of “case collapse” in cases of GBV handled by PACAM or partner GBV units in the fiscalías, measuring how the training creates long term changes in institutional responses to GBV. Table 6: Summary of Output/Outcome Indicators in Justice Project AMELPs ACTIVITY OUTCOME OUTPUT COMMENTS CJSSP 1 6 CJSSP Indicator 1 measures the use of alternative case disposition procedures by the Fiscalias. However, in Years 1 and 2 the low quality of data collected meant that it was unusable for measurement purposes. ICITAP 3 5 Some indicators termed “outputs” in the ICITAP AMELP are in fact outcome indicators, but they measure outcomes relating to internal reforms, rather than effectiveness of operations, policing, processing of complaints, etc. PASJ 2 7 PASJ outcome indicators measure effectiveness of FOIA requests, and impact of advocacy on adoption of legislative reforms. Both have issues of attribution as other factors affect these outcomes. Total 6 18 One-third of the indicators measure outcomes. While the PAD allows IPs to include other relevant indicators to illustrate outcomes, this did not happen. 21 This argument came up constantly in the interviews with CJHs. It is however, a counterfactual argument and difficult to demonstrate in the absence of rigorous baseline impact studies on crime in the CJH catchment areas. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 50 Furthermore, the AMELPs did not propose location-specific measures of any potential outcome indicators in the PAD that would have allowed USAID to measure and attribute Justice Project impacts in the targeted localities. USAID/DR has continued to support the LAPOP survey on governance and democracy issues, which measures citizen confidence in institutions (AGO, DNP, Courts, etc.) and rates of crime victimization. However, LAPOP uses a national sample, hence any measured changes in these or other indicators cannot be attributed to the Justice Project interventions in the targeted localities. Using the LAPOP survey to measure project outcomes would have required a sample in each of the localities. Nor have the Justice activities supported interventions with GODR institutions (AGO, MP, MIP, DNP) to generate disaggregated statistics on these localities, to be able to measure the PAD indicators such as “Percentage Increase in Cases Prosecuted”; or “Percentage Decrease in Minor Crimes”. In the CJSSP AMELP, the only indicator that measures a result directly related to the PAD purpose is “Percentage of new cases filed that reach case disposition in targeted District Attorney General´s Office”. In principal, this would allow for a direct measure of the impact of the new management model and other actions on the fiscalías, hence making the link to the PAD sub-purpose of “increasing GODR output and effectiveness in prosecuting crime”. However, during the first two years of CJSSP there were no reliable data collected from the three fiscalías in National District, Santo Domingo or Santiago. A Data Quality Assessment (DQA) on this indicator showed that the data were not of sufficient quality and consistency to be useful for measuring outcomes. In the case of ICITAP, while the AMELP specifies four outcome indicators, the outcomes measured are internal to the DNP, and as such, there is no reference to outcomes in the targeted localities in the Duarte corridor. More importantly, there are no measures of external outcomes. Thus, ICITAP cannot measure the effectiveness of its interventions in relation again to the PAD sub-purpose of “increasing GODR output and effectiveness in prosecuting crime”. In the PASJ AMELP, again, there is a single outcome indicator, “Number of actions as result of local/community diagnosis of the situation of violence conducted”. In principal, this indicator is directly related to the PAD sub-purpose of “increasing public engagement and oversight in Dominican law enforcement and crime prevention efforts”. The evaluation team reviewed the PASJ indicator data sources and found that the data measure only the number of workshops and training events. Other actions taken by the local coalitions—leafletting campaigns in the area of GBV, or mediation activities conducted by the student mediators, successful efforts to focus police patrolling on specific zones, or to eradicate indiscriminate group arrests (redadas) in poor communities—are not reflected in this indicator. CONCLUSIONS The Justice Project PAD contains a poorly specified results framework and log frame, made worse by a confusing narrative and lack of clarity about expected results, targets, and outcomes. Also, USAID has never developed a version of the PAD log frame to use in planning with the IPs, which also contributes to the weak links between the PAD criteria, and the Activity work plan and AMELP designs. Despite these problems, the PAD does provide guidance on geographic focus and co-locations, but the three mechanisms approved by USAID fail to provide guidance to IPs to co-locate, design, coordinate and plan activities in the targeted localities to create synergies and a common “experience curve” to learn how to work with GODR and CSO counterparts with the limited resources available. There is no discussion in any of the mechanisms or work plans about the details of co-location in the targeted localities, synergy with the other activities or the contributions of each activity to the PAD proposal for partnerships at the local and community levels. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 51 The guidance provided to IPs on the design of the AMELPs did not reflect the PAD intention to establish a set of robust outcome measures to make the link between the overall result of crime prevention/ reduction and the IRs. There was no mention of the need for generating disaggregated crime statistics and planning and coordinating M&E efforts to measure the higher-level PAD outcome indicators, baselines, etc. As a result, the AMELPs contain none of the higher-level outcome indicators. The CJSSP’s only outcome indicator directly linked to prosecutor effectiveness had serious quality issues that went unresolved during the first two years of implementation. 22 ICITAP indicators are focused on internal process change, rather than on police effectiveness. The only applicable outcome indicator during Year 1 and Year 2 in the PASJ activity leaves out important actions of the local coalitions. Thus, the lack of attention to MEL design and implementation in the Justice Project activities means that it is not possible to answer this important “so what?” question about the overall impact of the Justice Project on crime prevention and prosecution in the targeted localities in the Duarte corridor. PROJECT COORDINATION EQ7. Has there been adequate coordination among the different justice sector activities? FINDINGS Coordination of USAID and IPs with INL The PAD states that activities with the DNP will be “closely coordinated” with INL and goes on to say that implementation of activities under the Justice Project will be coordinated through the US Embassy Law Enforcement Working Group. According to the PAD, USAID Justice Project assistance would “complement the INL program in the DR by strengthening DNP management capacity, transparency and community relations. Targeted USAID TA to improve DNP responsiveness and service provision on matters of family and gender violence (F/GBV) will also complement launch and implementation of the 911 system as a shared US/Dominican priority”. The PAD notes, too, that in the design phase there was detailed consultations with State INL on areas of coordination.23 Interviews with USAID staff indicate that this agreement with INL to coordinate interventions is no longer in force. INL has staked a claim to working with various Directorates in the DNP, INACIF and the 911 service, and ICITAP understands that means those Directorates are off-limits to it, essentially creating “walled off areas” in which INL will not allow USAID IPs to work. Thus, while INL and USAID staff at the Embassy level reported sharing information, KIIs with the IPs and INL indicate that communication and coordination have not permeated to the working level. Though INL and ICITAP have offices in the same DNP building separated only by a wall, no communication occurs. This was illustrated during evaluation fieldwork; INL was supporting a training workshop in planning, an area that ICITAP is deeply engaged in with the DNP, yet ICITAP only learned of this training when DNP participants informed them. ICITAP implementation-level staff stated that synergies could exist if INL were willing to coordinate actions in the activities with the different Directorates. The KIIs uniformly indicated that the proposed coordination through the Law Enforcement Working Group is not happening. This reinforces a recent assessment of the Caribbean Basin Security Initiative (CBSI) which prominently flagged this issue and identified the need for greater coordination among USG 22 Following an external data quality assessment, CJSSP moved to correct the problem, but the improvement in the indicator came too late to be of use in monitoring the first half of program implementation in that it did not collect baseline data on case disposition prior to activity inception. 23 Justice Project PAD, p. 7. “Consultations with State INL confirm that planned USAID assistance for the DNP in the areas of institutional coordination (including investigations), human resources and operations management will not be included as components of the INL bilateral assistance portfolio (and thus not be duplicative)”. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 52 efforts in the DR.24 There is no regular meeting to deconflict initiatives and build synergies, and coordination is made more difficult by INL not having a workplan to which USAID (or IPs) could refer to inform planning and minimize overlaps. Although a meeting of IPs and INL was held in December 2017 with the aim of improved coordination and all agreed that such meetings should continue, no subsequent encounter occurred until April 2018 (just after the evaluation data collection). USAID Coordination with IPs Staff of IPs and USAID indicate that communication is generally robust between USAID, CJSSP, ICITAP, and PASJ. However, unfilled staff positions in USAID have placed a large burden on the COR/AOR responsible for all three Justice Project activities in Years 1 and 2, which limited the Mission’s ability to closely monitor the work of the IPs. The frequency of Activity monitoring by USAID has increased in the recent past with weekly USAID-IP meetings since the end of 2017. USAID has also modified the reporting requirements to receive more concise and frequent information. USAID staff said that the ramp-up and streamlining of Activity reporting were intended to allow USAID to respond to Washington more effectively and better represent the Activities in real time, though the IPs were unaware that this was the reason for the changes in interaction. Coordination Among IPs As noted in EQ6 with respect to geographic focus, the evaluation found no sign of coordination between USAID and IPs on co-location of activities or joint efforts by the IPs to promote joint actions with GODR and civil society counterparts on crime prevention/ deterrence/prosecution in the targeted localities in the Duarte corridor. That general issue aside, the findings on effectiveness of inter-IP coordination are mixed: • CJSSP and PASJ reported that there is sufficient interaction among them, and FINJUS and Chemonics reported particularly strong coordination, notably including support for the preparations of the Judicial Summit in 2016. • Interviews indicated that ICITAP and CJSSP generally understand each other’s activities, and notable synergy was achieved as the work of CJSSP on analyzing DNP communications was essentially handed over to ICITAP for follow-up when the scope of CJSSP was changed. • ICITAP and PASJ do not have a good comprehension of each other’s scope of work, owing in part to the elimination of activities in the ICITAP work plan. KIIs show that there have not been effective communications between ICITAP and PASJ even in those PASJ activities of direct relevance to ICITAP: for example, the study undertaken by PASJ on working conditions in the DNP in Year 1, or the study now underway on DNP staffing requirements for policing. • USAID convenes quarterly meetings of IPs, but all IPs expressed interest in USAID taking a more proactive coordinating role to clearly delineate roles in areas of mutual interest and ensure regular information-sharing among them (and other USG agencies). • PASJ noted that CJSSP has funded significant research on different aspects of justice sector reform and management, of which some results have not been disseminated, even in oral briefings. They state that CJSSP justifies the weak dissemination by reference to the sensitive nature of the information. This is a missed opportunity or learning and improved practice among the IPs. 24 This was a finding and recommendation in the August 2016 Final Report “Assessment of Caribbean Basin Security Initiative”, which identified the need for greater coordination among USG efforts. Further, “[b]oth INL and USAID staff reported that efforts to obtain and share information between the two agencies had not been fruitful, and some suggested that the Embassy leadership might have to play a stronger role in coordinating efforts between INL and USAID.” Id. at 49. The full report can be found here: https://dec.usaid.gov/dec/GetDoc.axd?ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2NDBmY2Uy&rI D=MjIyNDA2&pID=NTYw&attchmnt=True&uSesDM=False&rIdx=MjU3NDk0&rCFU= FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 53 • The PASJ coalitions have been working with networks of CSOs and GODR agencies in different localities (including MW, fiscalía GBV units, local hospitals and CSO service providers) to coordinate institutional responses to GBV. This experience has not been analyzed or incorporated into USAID and IP plans for improving GBV programming. Coordination with the GODR Some GODR stakeholders reported strong communication between themselves and USAID, and also with the respective IPs, e.g., the DNP with ICITAP. However, DNP senior management stated a desire for a coordinated USG voice, stating that they were sometimes unsure of how to work with INL and ICITAP. The evaluation team was unable to obtain interviews with the AG to explore this issue. IPs and USAID staff indicated that coordination with the AGO in Year 2 had been challenging owing in part to leadership changes, but recently it has improved. The MIP officials responsible for the municipal crime prevention partnerships report no interactions with USAID or the Justice Project IPs (although MIP staff working on community mediation say that they have been in discussions with CJSSP). For their part PASJ reports that it has not been possible to achieve effective coordination with MIP, but coordination with national DNP leadership has improved recently in the framework of research on DNP staffing requirements for policing. Interaction with GODR stakeholders in areas identified for increased USAID support, notably in the area of GBV, has been weak. For instance, while the Ministry of Women (MW) has had some contact with USAID and CJSSP, there is no rationale for why USAID has not been supporting MW programming parallel to efforts supported and contemplated by the Justice Project, e.g., attorney services for GBV victims, counseling for victims. Representatives of MW in Santiago and Puerto Plata are unaware of CJSSP activities. In SDW the fiscalía’s GBV unit reported some interaction with CJSSP, which was characterized by CJSSP as “regular interactions”, but this was not the case in Puerto Plata and Santiago. In contrast, all of the PASJ supported coalitions in five localities (except SDW), which focuses on youth issues) report ongoing coordination of GBV activities with the MW and Fiscalías’ GBV Units, with the local offices of the DNP and some municipal governments. The DNP’s Specialized Directorate for Women and Family Violence (Direccción Especializada de Atención a la Mujer y Violencia Familiar) reports having no interaction with USAID, ICITAP or the other IPs. CONCLUSIONS The Justice Project PAD dedicates considerable space to discussing external constraints to police reform. However, this evaluation concludes that a major constraint to the implementation of the activities under the ICITAP PAPA has been internal to the USG Mission in the DR. The agreement reached with INL on how to coordinate actions with the DNP has not held. Rather, in practice there has been a decision to exclude ICITAP and CJSSP interventions in functional areas of the DNP, including the Directorate of Prevention, which has forestalled planned ICITAP work on strengthening police response to crimes against vulnerable populations or establishing community-police crime prevention partnerships. Thus, ICITAP and CJSSP have lost the opportunity to contribute to PAD overall results of strengthened crime prevention/reduction/prosecution within the time frame of the CDCS. Among the IPs, there is an expressed need for enhanced coordination of USG efforts both to maximize their efficacy and to ensure that GODR stakeholders understand how the agencies and implementers work together. This lack of formal coordination could have a deleterious impact on planned efforts to address GBV and promote police transparency.25 With respect to GBV response, for example, USAID and the other IPs are unaware of PASJ local coalitions’ work with the CSO and GODR partnerships to promote better institutional coordination and management of GBV cases. In this sense, lack of attention 25 See Questions 1 and 2 discussing how CJSSP and ICITAP both understood that police transparency efforts were being conducted by PASJ. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 54 by USAID to ensuring close coordination of IPs’ activities has accentuated the existing challenges of co￾location in the six localities in the Duarte corridor, discussed in the section on EQ6. CHALLENGES AND COURSE CORRECTIONS EQ8. Has the Mission been opportune in identifying emerging challenges and making course corrections in the different activities? FINDINGS CJSSP With respect to CJSSP, the reaction by Chemonics and USAID to the most significant challenges that emerged after design of the activity (as detailed in EQ1), can be described as follows: • Uncertain scope of work with GODR entities, especially the DNP: This problem emerged due to INL objections to USAID intervening in a range of areas with key stakeholders and was apparent from the outset of CJSSP. However, as detailed above, INL coordination issues have still not been resolved by USAID. • New AG one year after CJSSP inception: This changeover came at a time when Chemonics was still devoting considerable effort to research and training, and thus had not yet made significant inroads with the AGO in key components related to prosecutor effectiveness, such as management models and integrity systems. The response to the closed doors at the AGO was, according to Chemonics, to persist in searching for openings where the new AG and team might be willing to cooperate, which ultimately did open doors. Meanwhile, Chemonics and USAID sought and seized opportunities with other justice institutions on a seemingly ad hoc basis, including a significant expansion of training. • Restructuring of the Santo Domingo Province fiscalia into three entities: This AGO decision undermined work already begun by CJSSP and plans for continuation, especially since establishment of the new entities has been severely delayed. Although external factors were at play, including the AGO turnover, the decision by Chemonics and USAID to focus efforts on SDW was taken only after long deliberation. If the intent of the CJSSP intervention to create a management model was to establish a pilot, confronted with the delays and challenges of working in SDW it is unclear why the location was not shifted to another high volume fiscalía. • Unexpectedly high cost of CJH operations borne by USAID: Once this was uncovered via CJSSP studies in the first year, the decision to halt expansion was taken relatively quickly. However, measures to limit CJH dependence on USAID funding have met with some resistance from USAID; interviews indicated that the CJH model and services are seen by the Mission as an important success story, despite the issues of sustainability already recognized in the CJSSP contract. • Low priority of AMELP: According to CJSSP reports, difficulties in obtaining reliable data for the lone outcome indicator of CJSSP surfaced in the second quarter of FY2016, when baselines were being set. Although Chemonics attempted to work with fiscalías to improve this data, serious problems continued by the end of FY2016.26 Since there was no assistance provided to the fiscalías in Santiago or National District, it appears there were few incentives to work with CJSSP on improving indicator data. The issue is mentioned in CJSSP reporting, but there was no 26 Annex 2 to CJSSP Annual Report for FY2016, AMEP Report, indicates at page 4 that: “… despite the efforts, the data from the Santo Domingo Province DA Office was not received correctly or timely. The data for the National District DA Office corresponding to September has not yet been reported and, in the case of the DA Office of Santiago, the Project was unable to obtain information reports from any of the departments or units.” FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 55 decisive action to improve the indicator until an external DQA highlighted the problems, two years into the activity. ICITAP • Uncertain scope of work with DNP: The main challenge in the ICITAP activity has been the inability to implement the PAD-based agreement with INL on the areas of USAID support for the DNP. The issue emerged during ICITAP start-up and has not been resolved during more than two years of implementation. While USAID has made attempts at the operational level with INL, when this failed, the problem was not effectively addressed through the US Embassy Law Enforcement Working Group. • Low priority of AMELP: A second challenge with ICITAP has been the inability to produce an activity MEL plan, with relevant performance indicators. The ICITAP team appeared to resist the requirement to use indicators to track progress (despite its thematic emphasis on strategic planning in the DNP). The AMELP was not approved until mid-Year 2 of implementation. • Patchy reporting: ICITAP has not paid attention to activity reporting, and reports are missing for some quarters. This was only resolved towards the end of Year 2. PASJ • Weak engagement with DNP: One of the principal challenges has been the inability of PASJ to engage the DNP at the national level to support its work with local coalitions and to ensure continued engagement with the regional and local DNP leadership in function of “community￾police partnerships”. This has been discussed with USAID but not well documented in PASJ progress reports. USAID has not taken effective action, owing in part to the ongoing problem with INL cooperation. • Low priority of AMELP: Like ICITAP, PASJ has encountered difficulties in designing and implementing a USAID-compliant MEL plan with relevant performance indicators. A compliant AMELP was submitted only at the end of Year 2. Performance Data and DQAs The Mission has conducted one DQA on the Justice Project, focusing on two CJSSP indicators. As mentioned above, the DQA report showed serious problems with the CJSSP Indicator 1 on the use of alternative case disposition by the fiscalías. In the course of this evaluation, it was found that CJSSP has recently initiated work to collect data on a new version of this indicator. The same DQA also identified problems of consistency of data on the training done through the ENMP. The data collection conducted for this evaluation observed similar inconsistencies in ENMP management of data on trainees. While referring to almost 3000 trainees in Years 1 and 2, ENP was only able to provide email contact information for about 300 unique individuals for an online survey (see discussion above about CJSSP training). ICITAP and PASJ have not been subjected to DQAs, owing to the lack of an approved, compliant AMELP during most of the implementation period. CONCLUSIONS Several salient implementation issues have emerged in the first two years of the Justice Project, which have not been resolved opportunely. With respect to CJSSP, there was not a timely response by the USAID/DR mission to very limited progress so far on the planned interventions with the AGO and fiscalías. The lack of early traction with those key actors (at least partly due to delays in CJSSP assessment activities as described above) may have made the activity more vulnerable to AGO leadership and other contextual shifts. CJSSP opted to move forward with other actions and counterparts, including an expanded training effort, but significant parts of RP1 and most of RP2 were stalled for nearly two years, and the decision to pursue other actions resulted in increased dispersion and decreased coherence. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 56 The most critical, cross-cutting challenge has been the inability to clarify actions with the DNP, owing to a lack of agreement with INL. As discussed above, the absence of activities with DNP in the targeted localities of the Duarte corridor has undermined the Justice Project results considerably during the first two years of activity implementation. Another cross-cutting challenge has been the weaknesses of the AMELPs, which were not addressed in a timely manner, hence one of the few outcome indicators directly linked to the PAD purpose was rendered unusable for monitoring and evaluation of CJSSP impacts, while another does not capture the full impact of the interventions. Finally, concern remains about the quality of ENMP training data and how training impacts are measured in the fiscalías and beyond. PROJECT ENGAGEMENT WITH GODR EQ9. Has USAID/DR and the IPs been effective in engaging the GODR in the justice sector activities, and ensuring buy-in and support for proposed reforms? FINDINGS CJSSP Engagement Informants uniformly indicated that key personnel of both USAID/DR and Chemonics have extensive contacts throughout the GODR justice sector institutions, which is important given the interconnected nature of the justice system. A strong connection existed with the Attorney General in office when CJSSP was launched, but elections in 2016 resulted in his replacement; this was a major setback, according to all those interviewed, which required that both USAID and Chemonics rebuild relations with a new AGO leadership team. After about six to nine months of perseverance, new entry points were found, and collaboration begun based on solid traction with the anti-corruption unit and the new fiscalía in Santo Domingo West. Although a Memorandum of Understanding is under negotiation for certain activities in relation to prosecution effectiveness and GBV responses, AGO commitment to work on accountability and ethics is still not in evidence. CJSSP was noted to have robust relations with the ENJ and ENMP, although informants observed that real buy-in could be in doubt where new training topics are not aligned with existing annual plans. The judiciary has been especially keen to engage with CJSSP on reforms, notably since the 2016 Judicial Summit; the resulting commitments led to creation of a new ADR unit and greatly increased activity level by the Inspectorate General, where levels of engagement are reported to be very high. Interviews with various stakeholders demonstrated widespread GODR interest in strengthening GBV prevention and prosecution, where CJSSP is examining opportunities for further investment. Members of the above￾described interinstitutional IT Committee have shown considerable dedication to jointly developing a data sharing platform (soon to be launched), but informants raised questions about the high-level commitment in certain institutions; the coming months will be crucial for this initiative. CJSSP has engaged with various other GODR entities in pursuit of its diverse lines of action. While space does not permit deeper analysis, documentation and interviews indicate generalized respect for the CJSSP team and appreciation for USAID’s ongoing investment in the justice sector. Many interviewees within and outside the project suggested that USAID (and the U.S. Embassy) had not been sufficiently proactive in recent years in cultivating relations with key GODR officials and institutions. ICITAP Engagement Engagement between ICITAP and the focused areas of the DNP with which it is working is strong. There is continuous and formalized interaction between ICITAP and the DNP, including, as related by the former Director General, a DNP-convened weekly meeting between ICITAP and the DNP’s executive management. The Evaluation Team also noted great familiarity between the ICITAP staff and FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 57 DNP staff, with constant presence of DNP staff in the ICITAP offices for meetings both formal and unstructured. Significantly, in KIIs with DNP senior management, i.e., Generals and Colonels, these officers took pride in their belief that they had designed the ICITAP project, though the same informants were unaware of the broader scope that ICITAP was supposed to have according to the PAPA, only being familiar with the public administration elements of the Activity. For instance, the General in charge of the Directorate of Women was unaware that the PAPA specifically calls for ICITAP engagement in that directorate. While ICITAP has robust engagement with the aspects of the DNP with which it is involved, it is not known outside of those areas. For instance, the DNP Director in Puerto Plata had no knowledge of the ICITAP interventions, although he was aware of the DNP’s efforts at strengthening planning, budgeting and human resources. PASJ Engagement In the work of the national coalition, during Year 1 and Year 2 PASJ (and particularly FINJUS) has had effective engagement with justice sector operators at all levels; with individual legislators and legislative committees; and during Year 1 with political party leaders and campaign teams. The panelists and participation lists of the round tables and conferences on different policy issues read like a “who’s who” of the leadership of the justice sector in the Dominican Republic. In the activities to prepare for the Justice Summit in 2016, FINJUS demonstrated the effectiveness of its links to the justice sector, supporting CJSSP’s efforts in this direction. PASJ was less effective in engaging with the MIP, particularly with the Security, Citizenry and Gender Boards. KIIs with MIP, PC, and PASJ coalition members show that PC’s recent high profile in the Marcha Verde27 has generated some antipathy in GODR agencies at national and local levels. The MIP has effectively excluded PC and the local coalitions from activities in the Boards established at the local level. Provincial Governor’s offices and PLD Municipalities refuse to give PASJ information about the membership or activities of these Boards; KIIs and FGDs with local coalitions found that while they have resorted to FOIA requests to MIP, Governors and Mayors, these have been routinely ignored. For its part, the MIP states that each local Board chooses which CSO representatives it wishes to invite. Interviews with MW, CONANI and the GBV Victims Units of the fiscalías in Santiago and Puerto Plata show openness to the activities of the local coalitions in strengthening GBV prevention and response. Likewise, in SDW and Santiago, the KIIs and FGDs with local groups show that there has been sustained engagement with school directors and teachers in the activities on school mediation. Despite attempts by the evaluation team to meet with the Directorate of the Preventative Police at national and local levels, this was not successful. The DNP Regional Director in Puerto Plata said that he was always willing to heed invitations from CSOs and community leaders to discuss issues of citizen security and crime, but that meetings were often tense, with people making accusations of police abuse, rather than proposing prevention measures. He “lamented the negative emphasis” of these encounters and wanted to talk about solutions. The KIIs and FGDs with coalition members in Santiago, National District and Puerto Plata indicate that engagement with the DNP is sporadic and uneven, owing to constant rotation of DNP officers, and unwillingness to discuss the sensitive issues of police effectiveness, transparency and probity. 27 This social movement, known as the “Green March” because of the color of the t shirts worn by its members, was created in late 2016/early 2017 as a response to several high/profile corruption scandals, including the bribery scandal linked to the Brazilian engineering firm, Odebrecht. PC leadership was closely involved in the creation and promotion of the movement, which in 2017 mobilized several hundreds of thousands of marchers to pressure the GODR for a robust response to the scandals. See the movement’s dedicated website: https://marchaverde.com/ FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 58 CONCLUSIONS Engagement of the GODR by CJSSP has been moderately effective overall, with several notable bright spots. Chemonics and USAID have capitalized on long-standing ties with the judiciary to positive effect, though investment there is limited by the defined targets of the activity, and needs are still significant. Relations with the AGO, the essential counterpart for CJSSP, are gradually recovering from a hiatus in top-level cooperation that undermined much of Year 2. Buy-in there on RP1 activities is evolving via various points of contact but virtually nil so far in RP2. Hindsight suggests that CJSSP did not take sufficient advantage of Year 1 to get a solid foothold with the AGO and targeted fiscalías, for example through significant technical assistance interventions with the three high-volume fiscalías, or an MoU that might have had some lasting effect on relations. USAID’s engagement during this period has been less than optimal, which appears to have hampered efforts to secure sustainable buy-in to reforms and limited high-level awareness of the contributions made by the U.S Government. The ICITAP activity with DNP has been successful in engaging senior leadership in supporting the management reforms. However, owing to the elimination of two of the three results areas set out in the PAD, ICITAP has no contact with DNP outside of the national headquarters (Palacio de Policía) With respect to PASJ, engagement with GODR has been mixed. The work of the national coalition is limited to policy debate, which has been largely successful—except as discussed above in the response to EQs 1 and 4, in sensitive areas such as judicial independence, and political party and electoral reforms. GODR engagement at the local level has been limited, owing to resistance from Governors and Mayors, and to a lack of structured interactions with DNP. Engagement with MW, CONANI and fiscalías on GBV issues has been successful, as has been the work with targeted schools on conflict resolution. USAID support for improved PASJ engagement with the GODR and DNP at the local level has been limited to date, in part due to USAID/DR human resource constraints which limit field monitoring. SUSTAINABILITY OF PROJECT RESULTS EQ10.Which of the justice sector activities (and individual components) supported by USAID/DR are producing sustainable results? FINDINGS Sustainability of CJSSP Results Given the relatively early stage of intervention in most areas of CJSSP activity, the evaluation focused on the potential for sustainability of the results outlined in EQs 1 and 2. It is not possible to say anything yet about the sustainability of the results pertaining to the prosecutor management model and related actions. KIIs with AGO officials indicated that an updated management model for fiscalías was in the works and will be informed by CJSSP assessments, ongoing consultations between Chemonics staff and consultants and the AGO, and piloting in SDW. Beyond that, given the scarcity of tangible results in the first two and a half years, it is difficult to forecast its chances of success and future contributions to the PAD purpose of improving the timeliness and effectiveness of criminal prosecutions in the targeted localities in the Duarte corridor. Informants from USAID, Chemonics and the judiciary considered the enhanced capacity of the JIG as likely sustainable. However, the judiciary was not a priority target institution for this contract, for the PAD or the CDCS. Although it is argued by CJSSP that improving judicial integrity could have a positive longer-term effect on prosecutor effectiveness, it is not clear what would be the mechanism. The AGO’s anti-corruption unit, which is more closely aligned with the contractual expected results, was seen by its leadership and by CJSSP and USAID to have made sustainable strides in its development. In both these cases, informants said that closely linking training of personnel with new guidelines for their FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 59 work was critical to sustainable changes in institutional capacity (CJSSP supported drafting of a new regulation for the JIG, and a new manual for prosecution of public corruption cases). One expert trainer suggested that by itself training would have little effect if not accompanied by that kind of systemic change. The CJHs are in transition as management by PC is taken over by a new association; informants agree this is a milestone arising from major joint efforts by Chemonics and PC to identify the best way forward for the CJHs. 28 However, in the absence of a major GODR commitment to sustaining the CJHs it is not clear how they will survive. Some funding has been secured to complement the provision of personnel for CJH operations, but several GODR informants said that the CJHs would still be linked with PC in the eyes of most government and political actors, which would inhibit major GODR financing. CJSSP funding of the CJHs beyond 2018 is uncertain according to USAID, PC and Chemonics staff, who expressed concern that some CJHs would close if new major funding was not identified soon. Interviewees said that sustainability of PACAM psychological services to GBV victims will be affected by the fate of the CJHs (where most counselling funded under CJSSP is delivered). The current subgrant with PACAM ends in July 2018, with no plan in place yet for continued service provision, although CJSSP and PACAM are seeking financial support from the GODR. Sub-grant reports and interviews with ENJ and IAWJ indicated that the new GBV curriculum had been adopted and would likely be delivered on an ongoing basis by the trained judges. Other than a new diploma course in human rights at ENMP, there was little evidence of other curriculum changes at that school, linked to CJSSP. Sustainability of ICITAP Results With respect to ICITAP, the interventions are complying with recent GODR legislation (Police Law). The SOPs that it is helping to establish in the DNP are being approved by the Ministry of Administration and Planning (MAP). To date, the SOPs from two of the directorates with which ICITAP has engaged have been approved by MAP and SOPs from the two others should be approved soon according to ICITAP staff. If ICITAP receives approval to move forward with the additional four directorates that it has proposed supporting, the planned result would be MAP-approved SOPs covering eight of the 15 directorates in the DNP. The result is that these SOPs guide not only present actions by the DNP, but they should guide future DNP activity as well. However, as DNP informants all articulated, a problem within the DNP is the lack of predictability of a staff’s duration in any position, with the Director General in place for two years. While these SOPs should in theory be followed unless officially changed, DNP management would still need to ensure that the SOPs are implemented consistently. Sustainability of PASJ Results The sustainability of the PASJ advocacy initiatives at the national level is in function of the legislative reforms achieved, particularly in the Police Law and the Law on Firearms. The sustainability of the police reform is undoubtedly a critical issue for USAID/DR and is addressed above in the section on ICITAP. The success of advocacy on other legislative reforms such as the laws on political parties, elections, notary public, and fiscal transparency remains to be seen. In any case results in these issues are not directly tied to the Justice Project results framework and their sustainability of less concern. The PASJ initiative to train and create a “network” of organizations working with PWD is nascent at best and has lacked follow on. While the directory of organizations will endure, according to FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 60 participating CSOs the initiative looks likely to be unsustainable. Similarly, the efforts at training journalists and communicators on security and justice issues, while involving a significant number of participants is not a “network” according to PC staff. The sustainability of the PASJ training in both cases is more in function of the awareness and knowledge imparted in training. The local coalitions are seen by the members and PC staff as having potential for sustainability but are still incipient. In practical terms they have existed for just over a year and are beginning to establish institutional relations and build partnerships in their communities. KIIs indicate that to be sustainable the coalitions need a constant, structured interaction with the municipalities, DNP, and GODR agencies such as MW. While the MIP´s local Boards for Security, Citizenry and Gender work in the same space, the PASJ coalitions have been effectively locked out of MIP activities, as described in EQ9. This could change in the future as the coalitions establish themselves and take on concrete activities. The KIIs and FGDs with local coalitions, PC community promotors and GODR counterparts see strong interest in continuing to work on GBV issues through a more networked approach. All emphasize the effectiveness of the work on awareness raising in marginal communities and support for greater institutional coordination to improve responsiveness. The sustainability issue needs to be addressed, and several informants suggested bringing in the municipalities and/or the MW, which although neither have operational funding, can offer staff for coordination and infrastructure for activities. Working with the MW can also “disrupt” some of the municipalities’ political sensitivities to working with PC. The pilot projects on school mediation for crime prevention in Santo Domingo West and Santiago are in their early stages; however, local coalitions indicate high interest in the approach, and the PC community promotors believe that they could easily be scaled up and incorporated into PASJ’s attempt to create more formal guidelines for the local coalitions, soon to be discussed with members (in June 2018). Some interviewees believe that having successful pilots in place allows PASJ to jump a few steps by promoting learning between school directors and MINERD officials in the pilot areas (Santiago and Santo Domingo) and the other localities in the Duarte corridor. They say that success with the school mediation would mean success and a higher profile for the coalitions, too. Finally, several informants mentioned the need for better branding of the PASJ local coalitions, with a distinctive name, perhaps an association, even with formal membership, but separate from the PC brand.29 This theme was also raised in two FGDs. The local coalition members, although overwhelmingly positive about PC’s approach and implementation activities, emphasize the need to find a strategy for institutional and political sustainability, particularly as the 2020 elections approach. The PC community promotors and the FGD participants also warn of the need to find an alternative strategy if the CJHs disappear, which they characterize as a “disaster”. CONCLUSIONS Clearly, buy-in by GODR counterparts will be a crucial element in sustainability of CJSSP results. As noted in EQ9, GODR commitment has been variable but appears to be growing as the CJSSP team builds trust relationships. Given current relations with the AGO, longer term impact of some kind on management models and prosecutions of corruption cases could occur, depending on the results of the pilot in SDW. In the judiciary, improvements in oversight and other priorities identified at the Summit are promising, but significant limitations on judicial independence could undermine those gains; again, these particular results are much less of a priority in the PAD results framework and should not absorb significant effort. 29 There were numerous references to a “membership” dimension to the PASJ coalition activity, and participants in two FGDs (Puerto Plata and Santiago) indicated their preference for formal membership, with membership cards. Others warned of the dangers of excessive hierarchy. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 61 Cultivating individual reform champions is important, but needs to run in parallel with consolidating policies, systems and procedures at institutional level—which can also shore up the investment in training of personnel. Securing formal agreements with the leadership in targeted GODR entities and in the targeted localities can bolster sustainability, as can the entrenchment of new or adapted courses in training curricula of the ENMP. There are significant questions about the long-term feasibility of the CJHs, partly due to political factors, but CJSSP has made a useful contribution towards addressing dependency on USAID funding. More work is required on branding and building local relationships to support the CJHs’ work. The principal sustainability challenge with respect to the PASJ activity is to develop a strategy to make the coalitions “the only game in town” with respect to civil society initiatives to promote effective crime prevention, deterrence and prosecution in these targeted localities in the Duarte corridor. This would require stronger support for the coalitions to make them attractive partners for municipalities, DNP and GODR agencies. There is a potential for making the coalitions more effective and relevant in prevention of GBV and school-based violence prevention through ADR, and also by building the relationship with the DNP, assuming that USAID or ICITAP can assist in gaining the support of the DNP leadership. There is potential for the expanded knowledge base emerging from CJSSP and PASJ research and analysis to have sustained benefits for the justice sector, but so far very little information has been shared outside of Chemonics, PASJ and USAID. Thought should be given to further dissemination of important findings of those studies, including abbreviated or redacted versions as needed to respect confidentiality and/or other sensitivities. This could be through dedicated in-service events with justice sector operators, rather than academic conferences or public “round tables” for experts. Finally, the impact of USAID support to criminal justice reform is jeopardized by persistent underfunding of judiciary, Public Ministry and DNP. Returning again to the PAD purpose, it could be argued that a critical objective of any policy research and advocacy activities in the PASJ activity had to be the question of the adequacy of core justice institution budget allocations in relation to their mission, functions and services. This, as opposed to those advocacy initiatives by PASJ to promote a new law on notary publics, political parties or elections, which while commendable, do not address the project purpose. PROJECT RESULTS IN GBV While USAID/DR did not formulate a specific question on the response to GBV by the Justice Project, there are several GBV related sub-questions which were discussed in the previous sections of the report. This section presents the consolidated findings on how GBV has been addressed by the three Activities; and in the conclusions it refers to opportunities for program activities in the remaining implementation time frame. FINDINGS CJSSP supports direct legal and psychological services to GBV victims (and their children) via the CJHs and PACAM. However, the effect of those services on prosecution effectiveness and specifically on rates of case collapse are not captured in CJSSP monitoring. Project data is also lacking on the efficacy of capacity-building of justice operators that aims to change negative attitudes as well as build knowledge and skills. However, at least 60% of trainees surveyed by the evaluation team indicated they had applied all or nearly all of what they learned during training on GBV, domestic violence and sex crimes. Lastly, work on systems and procedures is underway but not far advanced; CJSSP is aiming to operationalize a 2012 management model for GBV victim attention units of the AGO. With ICITAP, all GBV-related activities were curtailed from implementation of the Activity. While the plan had been to have a robust effort that was to establish and strengthen DNP standards and FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 62 procedures for family and gender violence (F/GBV) service provision and strengthen DNP first responder capacity for F/GBV emergency calls under the 911 system, none of this happened. The ICITAP PAPA Result 2.1 specifies that some of its activities to assist vulnerable populations would be performed through support to the “Directorate of Women” (Direccion Especializada de Atencion a la Mujer y Violencia Intrafamiliar). However, ICITAP was told by USAID that the Prevention Directorate was off-limits. The review of PASJ project documents and reports shows that GBV has been a recurring topic in the activities with local coalitions. In the initial participatory diagnosis with community leaders in the targeted localities in the Duarte corridor, GBV was identified as one of the major issues in crime and citizen security.30 Subsequently, in the action plans developed with local coalitions, prevention and institutional response to GBV has been one of the priority actions. The KIIs and FDGs in Santiago, Puerto Plata and National District found, too, that the PASJ local coalitions were active in prevention of GBV. In response to GBV, the PASJ coalitions have reached out to existing networks and provided support in outreach to vulnerable communities. In four localities in the Duarte corridor (National District, Santiago, Puerto Plata and La Vega) there were existing initiatives to establish networks of GODR agencies (MW, MIP and CONANI), DNP, fiscalías and GBV Victims Units, public hospitals and local NGOs to improve prevention and response to GBV. These four networks use a common mechanism for referring cases, focusing on how to provide an opportune, effective response and to avoid revictimization. In Puerto Plata, the regional office of the MW and one of the NGO members of the network agreed that when the PASJ local coalition conducted “leafletting” in marginal neighborhoods, the number of women seeking attention increased significantly, thus making the network more effective. The KIIs with PASJ local coalitions, PC’s own community promotors and GODR agencies found that the weakest elements in the institutional response to GBV are the lack of access to counselling and mediation services to prevent GBV, the lack of awareness of rights and support mechanisms for victims, and the deficient training of DNP agents in attending to cases of GBV. While the PASJ coalitions said that there are sufficient training materials and volunteers to provide sensitization and training for DNP agents on how to improve response to GBV cases, the DNP officers at the local level are often reticent to engage without an express mandate from DNP leadership in Santo Domingo. For its part, the KIIs with MW found that a major challenge was improving coordination with the DNP and fiscalías. The evaluation team also found that some informants were concerned about using ADR with GBV cases (and other violent crimes). These informants referred to Code and Law 24-97 that appear to allow ADR for “interfamilial violence”, but CJH do not allow it. This could provoke rifts among those working in this area, in addition to the risks to victims of injudicious use of ADR. It is a reason to support utilization of the mesas as means to build a common understanding of the use of ADR in GBV cases. Last, the Team saw itself and multiple KIIs reported that the physical plant for GBV-focused offices is severely lacking with GBV units lacking waiting rooms, places for children to play, or sometimes the ability to separate victims from defendants. Nor is there sufficient privacy for victims to communicate very personal details about crimes without a fear of being overheard. CONCLUSIONS The institutional structure of GBV response in the Dominican Republic is complex involving multiple actors, making it difficult at times for the victim to navigate. Any new programming activities funded by the Justice Project need to be viewed in light of the risk of further complicating efforts by involving new operators such as the Public Defender’s Office (ODP) or opening new units (within the AGO), or with 30 See Tahira Vargas. 2016. “Informe Final Consultoría Mesa de diálogo Estado-Sociedad sobre Seguridad Ciudadana en seis provincias del corredor Duarte”. Santo Domingo: Participación Ciudadana, pp. 15-16. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 63 NGOs whose services may not ultimately be sustainable due to funding or other constraints. These efforts can also further blur the lines about who does what in the justice system, e.g., the possible conflict of the AGO representing both the GODR and victims when desired results do not align; the possible conflict of interest of ODP representing the victim and the defendant in a case (this was brought up by public defenders in the FGD); and the challenge of creating practice areas in the OPD. Expansion of services by NGOs such as PACAM (presently provided through a sub-grant from CJSSP) is problematic in two ways. First, they are not likely sustainable in financial terms. Second, they not closely linked to justice project outcomes since the original plan to focus on victims with active cases in system was abandoned, and now only about 20% of those receiving counselling meet that definition. Further, as discussed above, the MW provides counselling services similar to those provided by PACAM. Any proposal to increase OPD and AGO involvement also runs the risk of creating a parallel structure in the places where the MW provides this service. The public defender FGD participants, and KIIs with the AGO and members of civil society all stated that the services provided by the MW to GBV victims is adequate; is being coordinated with other institutions; and requires more resources at the local level to cover demand. As mentioned, one of the major problems to address is coordination with AGO and DNP. UNFPA collaboration on a coordinated response to GBV was pursued by CJSSP and USAID but never finalized, primarily due to funding constraints. This is not the only option, though it could be an opportunity to leverage the long-term work of a reputable international actor with strong ties to the interinstitutional working group (mesa) on a coordinated GBV response. The mesa includes the Ministry of Public Health, MW, AGO, Judiciary, and the DNP. As of mid-2017, the Ministry of Education was also preparing to join the technical workgroup. However, in practical terms, funding UNFPA may add complexity and costs, when the main resource gap is at the local level, to extend community awareness and victims’ understanding of the GBV institutional services; to reinforce MW service provision; to improve interinstitutional coordination; and reinforce existing initiatives to improve GBV response and which involve significant volunteer contributions. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 64 IV. RECOMMENDATIONS On the basis of the findings and conclusions derived from the data collection, the evaluation team has formulated fourteen recommendations for improving Justice Project design, implementation and achievement of the results set out in the CDCS and the PAD. A common thread in the recommendations is the need for a whole￾of-project approach to Activity management, and especially to any proposed changes to the Activities during what remains of the implementation time frame. 1. Following on the evaluation findings and conclusions, it recommends that USAID/DR assess the feasibility of returning to the Justice Project PAD’s principals of co-location and measurement of impacts on the increase of GODR output and effectiveness in prosecuting crime and increasing public engagement and oversight of crime prevention, deterrence and prosecution efforts in the targeted localities of the Duarte corridor. The recommendation is extensive to all three Activities (CJSSP, ICITAP, and PASJ). Feasibility should be assessed in terms of the time remaining, the possibility of extending the existing mechanisms, and the re-scoping of the Activities to focus activities on those results closely linked to the PAD purpose and objectives. The rescoping should be done on a whole-of project basis, rather than piecemeal changes to activities. 2. One of the conclusions of the evaluation is that the Justice Project has not achieved many of its expected results in its first two years, owing to a mix of external and internal factors. In the context of USAID/DR’s current CDCS stocktaking, the evaluation team recommends that the Mission consider extending the PAD implementation time frame through to the end of FY2019 (September 2020). Without such an extension, even with the recommendations put forward here, it will be challenging to make substantive and sustainable progress towards project purpose and objectives in the time remaining. 3. The evaluation recommends that USAID move quickly to find a solution to the problem of INL coordination. USAID should seek Embassy support to create and actively coordinate a justice sector working group of the Activity CORs, INL, the RSO, and IPs to deconflict efforts and seek synergies among the Justice Project activities and USG agencies. This group should be chaired by the Deputy Chief of Mission. A top priority is to define the precise planned areas of intervention and counterparts for the remaining period of the Justice Project, to enable each agency and IP to operate freely within that agreed scope, and to provide clarity to GODR and other external stakeholders. While impacting directly ICITAP, this recommendation is crucial for the achievement of synergy between all the IPs in the co-located activities. 4. While the evaluation team supports the DO1 decision to give support to co-location of the Justice Project Activities in SDW, it recommends that it be expanded to other localities in the Duarte corridor. With respect to CJSSP, a tightly focused SOW and budget should be defined, eliminating the work on oversight (RP2) and reducing work with vulnerable groups, maintaining only GBV. USAID should take a more proactive stance to ensure that rescoping allocates budget resources to work with the fiscalías in Santiago and National District to incorporate the SDW pilot activity approach. Depending on budget resources, it may not be possible to maintain activities in supporting a new management model in the fiscalías and also working on ADR. USAID should decide on its priority and select one or the other for expansion in multiple localities. CJSSP support to training should be focused on specific positions in the AGO and targeted fiscalias, delivering skills and knowledge that directly contribute to the narrowed SOW, FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 65 designed to support the institutional capacity development objectives (relating to the prosecutor management model, use of alternative case disposition and related actions). 5. Assuming that Recommendation 3 is successfully implemented, USAID should direct ICITAP to develop a focused work plan modification, to support two interventions in addition to the existing public management strengthening activities: improved DNP response to GBV; and improved police-community relations through engagement with the local coalitions supported by PASJ and participation in activities promoted by the coalitions in the selected municipalities in the Duarte corridor. ICITAP should engage with the DNP’s Direccion Especializada de Atencion a la Mujer y Violencia Intrafamiliar on this strategy and link it to the broader support for GBV prevention and response in the design of the Justice Project activities, and in particular with the PASJ local coalitions. 6. The changes to the ICITAP activity should be strictly aligned with the PAPA language, developed through a formal modification of the ICITAP work plan. The changes should also be formally approved by the DNP national leadership, and led by specific Directorates within the DNP, who would support ICITAP and PASJ activities in the targeted localities (as in the example in Recommendation 5 on the location of the GBV activities in the DNP). The design of the modification should be done as a joint exercise between ICITAP and PASJ to develop a strategy for engaging with the DNP at the local level. If possible, USAID should consider extending the PAPA through to 2020 to coincide with CJSSP. 7. With respect to PASJ, USAID should direct PC to focus activities on local coalitions in the targeted localities, and work towards an extension through FY2019, again in the framework of the PAD extension. The rescoping should eliminate most of the advocacy activities done through FINJUS and the coalition (except possibly advocacy on PM/AGO, DNP budget resources and ADR). It should also eliminate the training and events for raising public awareness about human rights and the justice system and instead focus on activities on supporting coalitions to engage with local stakeholders: municipalities, DNP, fiscalías (especially GBV Units), GODR agencies (especially MW), schools, business associations, and other CSOs. The action plans for the coalitions should incorporate GBV and school violence prevention (in the latter extending the pilot project in school mediation), and actions with the DNP to strengthen police effectiveness and responsiveness. 8. USAID should reassess plans for victim support activities to be housed within the Public Defender’s Office and explore alternative opportunities within existing systems, e.g., MW and its victim legal assistance services, or the incipient institutional networks of GBV services at the local level. At the same time, USAID should reassess the current support to PACAM counselling services to victims of GBV, considering carefully the potential for sustainability, and explore alternative means of supporting similar services via the MV. The design of this component should be done at the Justice Project level to support complementary interventions, co-located in the targeted localities. For example, support to the MW, along with promotion of the institutional networks to improve overall GBV response, and prevention activities with the PASJ coalitions. 9. The evaluation recommends that the proposed changes in the three Activities be designed to respond to CDCS and Justice Project guidance, to identify synergies and mechanisms for joint action in the targeted localities and with local stakeholders to advocate for and support GODR, municipal and civil society efforts. The evaluation recommends that the DO1 team hold a facilitated workshop to finalize the Justice Project tasks/sub IRs, and the guidance for co-location FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 66 and coordination between IPs and with the local coalitions. This could be followed by a workshop with the IPs to improve the proposed design. The design of implementation arrangements in the SOWs and work plans should give priority to supporting the local coalitions and their role as a coordinating mechanism for crime prevention. For example, even if CJSSP were to implement GBV activities, it should be implemented in close cooperation with the local coalitions, through the common coordinating mechanism with fiscalías, DNP, GODR agencies and other stakeholders in each targeted locality. Similarly, with any CJSSP activities on expanding support for ADR, which should also be co-located. 10. USAID, CJSSP and PASJ should accelerate support for use of Alternative Dispute Resolution (ADR) methods as a means of conflict management and justice system decongestion, including mediation and conciliation, in a manner that ensures fairness and respect for rights, quality of service and coordinated approaches among institutions of government and civil society. If this recommendation is taken up, and if stakeholder consultation and pending analysis by FINJUS (under contract to CJSSP) indicate that it will add value, support should be provided to PASJ (through FINJUS) to develop legislation to regulate and promote the use of mediation. 11. New GBV-focused programming through the Justice Project should seek to avoid further complicating an already complex institutional structure. The emphasis should be on improving and reinforcing existing services provided by the fiscalías through the GBV Victims Units in the targeted localities in the Duarte corridor; the service provided by the MW in these same localities; the DNP’s own specialized Directorate; the local PASJ coalitions that work to support community awareness raising and victims’ understanding of GBV services; and the existing networks of GODR, health service providers, NGOs and community organizations in the target localities. This can be done within the existing mechanisms, without involving UNFPA. 12. USAID and IPs should ensure that robust and comprehensive M&E systems are in place to measure, at a minimum, progress against CDCS and PAD targets and goals. In rescoping the three Activities, the Mission should update and improve the PAD log frame and specify the output and outcome measures to be incorporated into the activity design and AMELPs. The limited time remaining in the Justice Project, budget restrictions and attribution challenges may make it impossible to use some of the original PAD outcome indicators. However, the Mission should identify at least one robust outcome indicator for each Activity linked to the PAD purpose and sub-purposes (and the CDCS PMP). The indicators (both outcome and output) should be sufficient to provide essential information about achievements and to highlight issues for management attention. 13. USAID should give guidance to IPs to urgently devise and execute methods to measure the impact of training (past and upcoming) on knowledge, attitudes, and practices of participants and functioning of institutions. Ideally, this would include measuring the effect of CJSSP training on actual case disposition and the individual parties to cases as indirect beneficiaries, e.g. assessment of satisfaction of vulnerable persons after a case has been handled by the fiscalía or CJH or courts, to observe trends before and after personnel are trained or sensitized. 14. USAID and implementing partners should continue to seek out and collaborate with champions and allies occupying key positions in the justice institutions, making every effort to move forward rapidly with reforms while those individuals or teams are in place while also pursuing strategies to safeguard gains made and protect against sudden loss of those allies. The strategy should include but not be limited to securing written agreements at the highest levels that outline the authorized types of intervention or areas of cooperation with other institutions and FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 67 pressing for the formal adoption and socialization of new or revised processes and rules that contribute to improvements in the institution. This may be achieved by a formal MOU, however, given the time remaining in the Justice Project implementation period, it may be more practical to seek other mechanisms, so as not to slow the work of the activities. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 68 ANNEX A. SUMMARY OF RESPONSES TO SURVEY OF TRAINEES During the data collection phase of the evaluation, a brief online survey was carried out among individuals who had attended training courses at the National School of the Public Ministry, which provided a list of 310 email contacts from their records of Public Ministry staff trained in the last two years. The most relevant data to emerge from this 16-question survey is described below. Although certain data has been used to inform the evaluation report, it should be noted that the surveyed population was not limited to those trained with the support of the project, nor were the questions phrased to elicit specific feedback on the project, which would have required a much more complex survey instrument as well as participant data that was not available from CJSSP. The body of the report refers briefly to the survey responses, as the evaluation team has doubts about the sample, which evidences a strong self-selection bias towards recent-entry, junior prosecutors in the fiscalías. General Information Of the 130 individuals who responded to the survey in whole or in part, 88 (67%) were “fiscalizadores” (Junior Prosecutors), while the other respondents were spread across other categories including administration staff and senior prosecutors, as shown in Figure 1 below.31 Of the respondents, 56% were female, and 44% male. Figure 1. Current position. 31 After completing the first question, many respondents chose to answer only certain questions. Therefore, most questions had between 65 and 73 responses. 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Responses FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 69 Training When asked to what degree they had been able to apply learning and tools acquired via training in their professional work, respondents were largely positive as shown in Figure 2. The average of those who chose (1) all or almost all of learning was applied was 59% of respondents across all training subjects. The highest rates of utilization were shown for training on Oral Litigation and Ethical Code of Conduct, while Sexual Crimes and Conciliation/Mediation were also rated highly. The subject given the lowest rating was Prosecution of Corruption Cases; 63% said they had made little or no use of that training— but only 8 respondents were in that group, which is an extremely small sample. That exception is probably related to the fact that those cases are generally handled by a specialized unit of prosecutors. If those who chose either Option 1 OR 2 are combined (all, nearly all or some aspects of training being applied), at least 60% of those who studied each subject say they have applied what they learned at least to some extent (except for Prosecution of Corruption). Figure 2. Application of Training32 Legend: 1=all or almost all applied, 2=some aspects applied, 3=nothing or almost nothing applied 32 Subjects rated by 5 or fewer respondents have been omitted from the data. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 3 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 70 When asked which courses or subjects had most helped them to improve their performance, the following were the most popular responses: • Investigation methods – 33 mentions (50% of total respondents to this question) • Litigation methods and strategies – 22 mentions • Evidence, including use of experts – 18 mentions • Gender-based violence and sex crimes – 13 mentions • Money laundering and economic crimes – 11 mentions Respondents were also asked to what extent training had affected the time it took to dispose of cases. As seen in Figure 3, just over half said they could dispose of cases much more quickly than before, and another 13% said they could handle cases slightly faster. Figure 3. Effect of training on case disposition Alternative disposition of cases Considering the current widespread interest in the GODR in expansion of alternative methods of dispute resolution, the survey sought the views of respondents on two questions. As to whether the government should create specialized centers for mediation of conflicts (before they become judicial cases), the response was a resounding yes (96% of respondents). When asked which government entity should administer this type of center, a combined 94% of respondents indicated either the Public Ministry of the Attorney General’s Office. Logro concluir (disponer) mis casos en mucho menos tiempo que antes Logro concluir (disponer) mis casos en solo un poco menos tiempo que antes No hay cambio en cuanto los tiempos de conclusión o disposición de mis casos Logro concluir (disponer) mis casos mis casos que mayor tiempo que antes No sé/no se aplica 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Responses FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 71 The survey went on to ask whether policies, procedures, protocols and tools were used in their fiscalías or units to support the use of alternative methods for case disposition. As seen in Figure 4, half of respondents said that these were used moderately, while 35% said they were used a lot. Only 13% indicated a low level of use of these methods. Figure 4. Use of alternative methods and tools The respondents indicated strong support for more use of alternative case disposition: 89% said they would support this, either because these measures provide a better solution to conflict (46%) or because they contribute to decongestion of the criminal justice system (43%). Only 8% did not favor increased use, because of the risk of impunity or denial of justice. Gender-based Violence Cases Considering the challenges of handling GBV cases in the criminal justice system, two questions were posed by the survey. Asked whether their fiscalía had implemented policies, procedures, protocols or tools to facilitate the handling of GBV cases in the last two years, 78% said yes, 8% said no, and 14% did not know. As shown in Figure 5, respondents had a reasonable level of confidence that those new measures had been effective: 40% said they were moderately effective, and 25% said they were very effective. Another 15% of respondents said the results of these measures had still to be seen. 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Responses FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 72 Figure 5. Effectiveness of measures to handle GBV cases The respondents were then asked what steps could be taken to improve the handling of GBV cases by the prosecution service (with multiple choices allowed). Improvements of procedures and tools were selected by the highest proportion of respondents (66%), while 52% mentioned specialized training of prosecutors, 46% chose sensitization of prosecutors, and 28% indicated training of other fiscalía staff. Management procedures Due to the past and current support of USAID for the development and use of management models by the AGO to improve prosecution effectiveness, the survey enquired with respondents to what extent such management models and related procedures had been effective in improving case handling in their fiscalía or unit. 40% of them said that new models had been very effective, and 32% said they had been moderately effective. Only 3% indicated they were not effective. Figure 6 depicts these responses. Las nuevas medidas son muy efectivas Las nuevas medidas son medianamente efectivas Las nuevas medidas no son efectivas Están en proceso de implementarse y está por ver su efectividad No sé 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% Responses FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 73 Figure 6. Efficacy of management models Other measures Finally, respondents were asked what more could be done to improve management systems and increase the quality and effectiveness of case handling by the Public Ministry and fiscalías. A wide range of responses were received to this open-ended question, to which each person could select up to three responses. They can be categorized as follows: • 34 (of 60) highlighted the need for more and better physical resources, ranging from buildings to transport to computers, with an emphasis on information technology tools • 30 indicated the importance of more training, for both prosecutors and other staff • 29 pointed to the need for better management of processes and information, including the use of management models and other tools for improved allocation of workloads and roles • 23 put a high priority on more personnel, including prosecutors, investigators and others, underlining the need for such personnel to be well qualified • 16 mentioned the need for better personnel management and salaries Las nuevas medidas son muy efectivas Las nuevas medidas son medianamente efectivas Las nuevas medidas no son efectivas, Aún están en proceso de implementarse y está por ver su efectividad No sé 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% Responses FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 74 ANNEX B. COMPARISON OF PAD TO ACTIVITY MECHANISMS The tables below (one per mechanism) list the outputs and targets set out in the Justice Project PAD in the first column and the most closely related content in each mechanism in the second column. As needed for sake of completeness, major activities have also been mentioned in these columns. The third column contains the observations of the evaluation team on the level of similarity between the PAD and mechanism, highlighting any important differences or gaps. CJSSP Outputs and Targets in PAD (and expected outcomes from PAD annex33) Related Outcomes or Results in Contract documents Comments Sub-purpose 1 (1.3.1) Prosecutor effectiveness increased Seven new community justice houses established (output) - Six new USAID community justice houses opened (target) Expected outcome: Seven New USAID Community Justice Houses Established and Operational in Santo Domingo and Santiago (increasing the total to 15) Sub-IR 1.1.1 Expand and systematize CJH Same as output but targeted locations specified in contract Improved coordination among justice sector institutions (expansion of coordination tables) (output) None at level of expected outcomes Sub-IR 1.2.3 Interinstitutional coordination improved Institutional coordination is mentioned as a “component” in the contract, but appears in workplans as a sub-IR Public Ministry information management and case tracking system strengthened (output) - Case tracking and detainee registry system is fully operational and is used to manage criminal caseload. (expected outcome) Expected outcome: National Case tracking and detainee registry system is fully operational and is used to manage criminal caseload Sub-IR 1.2.2 Improved use of Integrated Case Tracking Systems Contract adds detainee registry and is more specific about operationalization of cast tracking system – described as national rather than limited to the Public Ministry. Wording very close to the expected outcome in PAD. Public Ministry / National Police Coordination Mechanism Established (target) Expected outcome: Public Ministry / National Police Working Relationships and Coordination Improved Sub-IR 1.2.3 Interinstitutional coordination improved Similar idea, more flexible in contract (Chemonics was later advised by USAID not to target the DNP) Prosecutor Management Model Designed and Implemented for Santo Domingo Province (target) Expected outcome: Prosecutor Management Model Designed and Implemented for Santo Domingo Province Same expected outcome, sub-IR is broader 33 The six expected outcomes listed here were contained in Annex C to the PAD (Expanded Project Description), rather than in the body of the PAD. Although expressed as outcomes related to the entire project, they are only seen to occur in the CJSSP contract. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 75 CJSSP Outputs and Targets in PAD (and expected outcomes from PAD annex33) Related Outcomes or Results in Contract documents Comments Sub-IR 1.2.1 Effective application of structured and systematized management models National GBV Protocol Implemented to Improve Prosecutor / Police Service Delivery (target) Expected outcome: National GBV Protocol Designed and Implemented to Improve Prosecutor / Police Service Delivery Sub-IR 1.3.2 More responsive systems, structures and procedures for handling and care of vulnerable groups Same expected outcome but not closely reflected in IRs or sub-IRs (Chemonics was later advised by USAID not to target the DNP) Provision of community-level justice services to 125,000 people (target) Expected outcome: Community Access to Justice Services Provided to 125,000 people Sub-IR 1.1.1 Expand and systematize CJH, 1.1.2 Increase and expand use of ADR among community and justice system actors Same expected outcome Alternative Dispute Resolution (ADR) provided to 50,000 people (target) Expected outcome: ADR Services Provided to 50,000 people Sub-IR 1.1.2 Increase and expand use of ADR among community and justice system actors Same expected outcome Public Ministry increases its capacity to manage criminal investigations (expected outcome) Expected outcome: Public Ministry Capacity to Manage Criminal Investigations Improved Same expected outcome, no specifically related IR or sub-IR Reduced percentage of individuals held in pre-trial detention for minor offenses (expected outcome) Expected outcome: Number of individuals held in pre-trial detention for minor offenses reduced Sub-IR 1.1.3 Broaden use of complementary decongestion and abbreviated case processing mechanisms Percentage changed to number, not closely reflected in IRs or sub￾IRs Prosecutors increase their conviction rate (expected outcome) Expected outcome: Conviction rate increased Same expected outcome but does not mention prosecutors, not closely reflected in IRs or sub-IRs The percentage of cases dismissed for insufficient or improper evidence or for exceeding legally mandated Expected outcome: The percentage of cases dismissed for insufficient or improper evidence or for exceeding Same expected outcome, but not closely reflected in IRs or sub-IRs FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 76 CJSSP Outputs and Targets in PAD (and expected outcomes from PAD annex33) Related Outcomes or Results in Contract documents Comments time limitations decreases (expected outcome) legally mandated time limitations decreased. The number of persons not convicted by a court and illegally detained beyond mandated time limits decreases (expected outcome) Expected outcome: The number of persons illegally detained beyond mandated time limits decreased Similar but contract is slightly wider, not closely reflected in IRs or sub-IRs Internal Inspections: USAID will provide training, TA and mentoring to complete implementation of the internal integrity system (SII) within the Public Ministry and establish / strengthen corresponding standards and instruments of internal integrity and accountability within the DNP. (activity) IR 2.2 Improved professional conduct of prosecutor and national police through revitalized ethics and accountability standards Same, including related sub-IRs. (Chemonics was later advised by USAID not to target the DNP) N.B. The relevant PAD text appears in the Public Oversight result section, rather than under Prosecutor Effectiveness. Strengthen FOIA Compliance: USAID will provide training and TA to strengthen Public Ministry and DNP Offices of Public Information to improve the external transparency and accountability of each institution. (activity) Sub IR 2.1.1 Improved responsiveness and handling of citizen complaints Sub IR 2.3.1 Increased citizen knowledge of and access to safe and secure complaints systems Related, but contract does not refer to FOIA nor Offices of Public Information; the focus is on complaints. N.B. The relevant PAD text appears in the Public Oversight result section, rather than under Prosecutor Effectiveness. (Chemonics was later advised by USAID not to target the DNP) Law Enforcement Ethics & Accountability: Conduct organizational development (OD) assessment and policy review for Supreme Oversight Councils of the Public Ministry and DNP. (activity) IR 2.1 Strengthened oversight management by Superior Councils Similar, including sub-IRs and related activities in contract. N.B. The relevant PAD text appears in the Public Oversight result section, rather than under Prosecutor Effectiveness. (Chemonics was later advised by USAID not to target the DNP) Contract Results/Sub-Results with little or no specific reference in the PAD Time to Trial (# Days) is an indicator in the PAD log frame at Purpose level Expected outcome: Reduction of 30 percent in Time to Trial (# Days / Targeted Areas) Related to purpose level indicator in PAD, with a target of 30% added IR 1.4 Strengthened Human Resources, including strengthening of justice training centers and career systems Training and TA to build capacities are mentioned in various places in the PAD, but there is no specific outcome or target FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 77 ICITAP Outputs and Targets in PAD Related Outcomes, Results and Activities in Contract documents (PAPA) Comments Sub-Purpose 2 (1.3.2) Police Transparency and Management Improved Law enforcement transparency and accountability increased (budget, procurement, human resources) (output) Expected outcomes: DNP Change Management and Accountability Strategy Designed and Implemented; DNP Public Information and FOIA Management Capacity Strengthened Result 2.2 activities: Training and TA to strengthen DNP compliance with existing GODR transparency and public oversight laws (ex., FOIA, budget, procurement) Finalize and implement DNP change management and institutional accountability strategy Provide training and TA to strengthen ethics oversight and enforcement within the National Police Provide training and TA to strengthen internal affairs management and oversight Similar, with more detail in the PAPA. Ethics and oversight activities not pursued by ICITAP due to conflict with INL. DNP human resources and administrative management policies and procedures improved (output) Expected outcome: New DNP Human Resource Policy Designed and Implemented Result 2.2 activities: *Training and TA to establish and strengthen internal DNP policy and procedures for workforce planning and oversight *Provide training and TA to strengthen internal affairs management and oversight Similar FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 78 ICITAP Outputs and Targets in PAD Related Outcomes, Results and Activities in Contract documents (PAPA) Comments DNP Information Management and Case Tracking Systems Improved (output) Expected outcome: Provide training and TA to improve DNP internal case management (and related coordination with the Public Ministry coordination) Similar, with added element of coordination with Public Ministry, and nothing included as activity within a results area DNP Community Liaison Cadre Established (Targeted Areas) (output and target) Expected outcomes: DNP Community Liaison Officer Capacity Established (Targeted Areas); Community Crime Prevention Partnerships Established (Targeted Municipalities) Result 2.3 activities: *Provide training, TA and mentoring to establish and assign a cadre of vetted DNP community liaison officers (CLOs) to improve dialogue and collaboration with local communities and civil society Similar. Not pursued by ICITAP due to conflict with INL. National GBV Protocol Implemented to Improve DNP Service Delivery (target) Expected outcome: DNP Investigation and Management of GBV complaints improved Result 2.1 activities: *Gender Violence: provide training and TA to establish and strengthen DNP standards and procedures for family and gender violence (F/GBV) service provision *Provide training, TA and mentoring to strengthen DNP first responder capacity for F/GBV emergency calls under the 911 system *Develop and implement organizational development (OD) assessment for the DNP Directorate of Women (including training and TA to implement OD assessment Similar, but GBV-related activity is much broader in PAPA. Not pursued by ICITAP due to conflict with INL. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 79 ICITAP Outputs and Targets in PAD Related Outcomes, Results and Activities in Contract documents (PAPA) Comments recommendations) New DNP Human Resource Policy Designed and Implemented (target) Expected outcome: New DNP Human Resource Policy Designed and Implemented Result 2.2 activity: *Training and TA to establish and strengthen internal DNP policies and procedures for workforce planning and oversight Same DNP Change Management and Accountability Strategy Designed and Implemented (target) Expected outcome: DNP Change Management and Accountability Strategy Designed and Implemented Result 2.2 activity: *Finalize and implement DNP Change Management and Accountability Strategy Same DNP FOIA, Budget and Procurement Management Capacity Strengthened (target) Expected outcome: DNP Public Information and FOIA Management Capacity Strengthened Result 2.2 activity: *Training and TA to strengthen DNP compliance with existing GODR transparency and public oversight laws (ex., FOIA, budget, procurement) Similar (FOIA not pursued by ICITAP due to understanding that PASJ was working on this) PAPA Results/Sub-Results with little or no specific reference in the PAD Result 2.3 activity: *Training and TA to the DNP, Ministry of Interior and Police and related institutions to establish and strengthen the Dominican National Citizen Security Council PAD refers to Citizen Security Observatory in relation to PASJ, not ICITAP FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 80 PASJ Outputs, Targets and Selected Activities in PAD Related Results and Key Activities in Cooperative Agreement Comments Sub-Purpose 3 (1.3.3) Public Oversight of Crime Prevention Improved. Independent Justice and Citizen Security Forum established to strengthen oversight of GODR law enforcement and crime prevention (target) Result 1.2 activity: Citizen Observatory of Security and Crime Prevention in the provinces of intervention, with the active participation of local organizations. Result 2.1 activity: Observatory on Transparency and Integrity of the Criminal Justice Sector. Similar National police reform legislation passed and implemented (target) Result 2.2 activities: Design strategies for the adoption of the new Organic Law of the National Police and implementation of required activities, Monitor the implementation of the National Police Law and incidence in the process of drafting implementing regulations Similar Justice system leadership and funding to protect vulnerable populations increased (target) Leadership and funding not specifically mentioned, Result 2.4 targets CSOs with provision of training, technical assistance and financial support to civil society organizations that work with vulnerable populations (LGBT, ethnic minorities, and people with disabilities) to increase access to justice and promote equal treatment before the law. No closely related result or activity in the CA Institutional Integrity and Ethics Policy Implemented for Public Ministry (target) Result 1.2 activities: Assessment of the state of affairs of judicial career, Public Ministry, Public Defense and the National Police, as well as the consequences regime; Citizen Monitoring of the Public Ministry and the National Police career system and the consequences regime. Result 1.4 activity: Information campaign and public outreach to inform society of the Institutional Integrity System of the Public Ministry, the Public Defense National Office and the judiciary, as well as mechanisms for citizen’s action. Related but the CA is much broader in terms of targeted institutions, and approaches the issue from citizen side (not directly targeting the Public Ministry as suggested by PAD) Municipal Crime and Violence Prevention Plans Designed and Implemented in targeted areas (target) Result 3.1: Provide training, technical assistance and financial support for the development and implementation of crime prevention action plans in selected municipalities. Result 3.1 activity: Development and implementation of a Regional Plan for Conflict Resolution Result 3.2 activity: Forum for crime prevention and public safety in areas of intervention to reduce violence and improve public safety. Similar. Several related activities in Result 3.1 and 3.2. Community Crime Prevention Partnerships Established (Santiago) (output) Result 1.2 activity: Citizen Observatory of Security and Crime Prevention in the provinces of intervention, with the active participation of local organizations. Result 1.4 activity: State-Society conciliation roundtable, in each province, to design intervention and Various related activities in the CA FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 81 PASJ Outputs, Targets and Selected Activities in PAD Related Results and Key Activities in Cooperative Agreement Comments identification strategies of joint proposals and solutions from all social actors to influence crime incidence and prevention. Citizen campaign to improve community safety and reduce both crime and violence through the implementation of prevention policies, strategies and effective and sustainable action. Result 2.3 activity: Citizenry and Police encounter as a space for dialogue and analysis to identify proposals, intervention strategies and specific commitments for crime prevention. Increased access to justice by poor and marginalized communities (output) None specifically on this issue. Various related results and activities in the CA, but focus is on prevention and oversight Law enforcement transparency and accountability increased (budget, procurement, human resources) (output) Result 1.1: Increased Public Awareness of Criminal Justice and Citizen Security Result 2.1 activities: Strengthen the Citizen Observatory of the National Police; Monitoring the budget implementation of the Judiciary, Public Defender, National Police and the Public Ministry; Technical assistance, training and logistical support to improve understanding of civil society of their right to public information in the criminal justice system and the National Police. Oversight activities throughout the CA, though few specific references to police (this output seems closer to ICITAP) Legal protection of vulnerable groups improved (output) Result 2.4: provision of training, technical assistance and financial support to CSOs that work with vulnerable populations (LGBT, ethnic minorities, and people with disabilities) to increase access to justice and promote equal treatment before the law. Result 1.3 activities: Evaluation of the legal representation system of the rights of victims and vulnerable populations (boys, girls and adolescents, women, ethnic minorities, people with disabilities and LGBT); Technical-legal assistance to strengthen the legal representation of victims and vulnerable groups, to ensure the protection of fundamental rights and improve access to constitutional and criminal justice; Technical and financial assistance to legal defense programs being developed by CSOs for LGBT groups in conflict with the law. Various related results and activities in the CA Establishing and strengthening IPAC model coalition to Result 1.1: Increased Public Awareness of Criminal Justice and Citizen Security - PC aims to build coalitions of civil society organizations, academia and private Similar. PAD activity was expressed in a single Result in the CA. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 82 PASJ Outputs, Targets and Selected Activities in PAD Related Results and Key Activities in Cooperative Agreement Comments strengthen justice system transparency and accountability in preventing crime and protecting vulnerable groups (activity) sector to increase public understanding of access to criminal justice… No mention of IPAC coalition in CA. Conduct OD assessment and strengthen internal management of Supreme Oversight Councils and the development / enforcement of standards and procedures to improve ethics and accountability practices within the PM and DNP (activity) Result 1.2. Increase public awareness of institutional roles and responsibilities of the Public Ministry and the National Police in persecuting and preventing crime. Activities: Assessment of the state of affairs of judicial career, Public Ministry, Public Defense and the National Police, as well as the consequences regime: disciplinary and merits; Citizen Monitoring the implementation process of the Public Ministry and the National Police career system and the consequences regime. PAD item not replicated in CA, though some activities are related (N.B. this item is more closely captured in the CJSSP contract) Implement internal integrity system within the Public Ministry and establish / strengthen corresponding standards and instruments of internal integrity and accountability within the DNP. (activity) Same as above PAD item not replicated in CA, though some activities are related (N.B. this item is more closely captured in the CJSSP contract) Strengthen FOIA Compliance: training and TA to strengthen Public Ministry and DNP Offices of Public Information to improve the external transparency and accountability. Result 1.2. Increase public awareness of institutional roles and responsibilities of the Public Ministry and the National Police in persecuting and preventing crime. PAD item not replicated in CA, though some activities are related (N.B. this item is more closely captured in the CJSSP contract) Provision of training, TA and sub-grants to CSOs serving vulnerable populations (LGBT, ethnic minorities, PWD) to strengthen advocacy for justice and equal treatment under the law (activity) Result 1.3 Improving public understanding of human rights, discrimination reduction and legal protection of vulnerable populations under the criminal justice system. Result 1.4 Developing and implementing national and local crime prevention campaigns in key areas of human vulnerability (family and gender violence, HIV). Similar. Various related results and activities in the CA FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 83 PASJ Outputs, Targets and Selected Activities in PAD Related Results and Key Activities in Cooperative Agreement Comments Informing the public where citizens can file formal complaints about improper behavior of justice sector officials, and advocate to ensure that complaints are reviewed and acted upon appropriately (activity) Result 3.2 Provide training and technical assistance to strengthen community’s capacity to monitor and assess local crime trends and risks and develop independent proposals to reduce crime and improve public safety. Activities: creating and developing mechanisms between citizens and municipal networks and local authorities for filing complaints, monitoring and supporting the dissemination of such complaints’ results. Similar. Various related results and activities in the CA Applied research and policy analysis on the role and performance of the GODR Public Ministry, Police and Judiciary in prosecuting and preventing crime (activity) Result 2.1 Developed research and policy analysis on crime prevention and prosecution and the role and functioning of the Public Ministry, National Police and Judiciary. Same. Several related activities in Result 2.1 Improving the availability of information to victims of crimes, with an emphasis on outreach to historically disadvantaged and vulnerable populations (women, children, and minorities). (activity) Result 3.3. Develop and implement information and community outreach campaigns to raise awareness on family and gender violence and related offenses (such as smuggling and trafficking) and increase access to information and support resources for victims and witnesses. Similar, with several related activities in Result 3.3 Promoting and strengthening partnerships between the Ministry of Interior and Police, AGO and other donors to improve the availability and use of official crime statistics for national policy and strategy development (activity) Result 2.3 Promote and strengthen Interinstitutional coordination between the Ministry of Interior and Police, the National Police, Public Ministry, CSOs and international organizations to improve the availability and use of official crime statistics and the development of a joint intervention strategy. Same. Several related activities in Result 2.3. Providing training, TA and sub-grants to human rights and legal aid organizations to improve civil society capacity to use of strategic litigation and related advocacy tools to reduce Result 2.4 Provision of training, technical assistance and financial support to civil society organizations that work with vulnerable populations (LGBT, ethnic minorities, and people with disabilities) to increase access to justice and promote equal treatment before the law. PAD focus on strategic litigation not replicated in CA, though other advocacy methods are mentioned FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 84 PASJ Outputs, Targets and Selected Activities in PAD Related Results and Key Activities in Cooperative Agreement Comments discrimination and protect vulnerable populations. (activity) CSO / community organization training & TA (crime analysis, community action planning) (activity) Result 3.2 Provide training and technical assistance to strengthen community capacity to monitor and assess local crime trends and risks and develop independent proposals to reduce crime and improve public safety. Same. PAD Outputs and Targets with little or no specific mention in CA Results Provide training, TA and sub-grants to CSOs to improve the understanding and use of FOIA and related transparency laws (ex., municipal budget). None. Mentioned in objectives of CA, but no specific activities in the Results section. CA Results/Sub-Results with little or no specific reference in the PAD None Result 2.5 Establishment and consolidation of partnerships between the Dominican civil society and similar organizations in Latin America and the Caribbean with government stakeholders to strengthen criminal justice and public safety reform in Dominican Republic. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 85 ANNEX C. WRITTEN RESOURCES REVIEWED USAID Documents (general) A Field Guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries, USAID, January 2011 ADS 201 Additional Help: Whole-of-Project Evaluation (WOPE). Washington D.C.: USAID Learning Lab. 2016. Assessment of Caribbean Basin Security Initiative, Final Report, USAID, August 2016 Assessment of USAID Civil Society Programs in the Dominican Republic. Checchi & Company Consulting, Inc., and DevTech Systems, Inc. Santo Domingo: USAID. 2002. Dominican Republic Country Development Cooperation Strategy FY2014-2018, USAID/DR, December 2013 Final report of the USAID Project “Sustainability and Expansion Model of Access to Justice: Community House of Justice”, 2012-2015, undated Informe Final 2003-2008, Proyecto Justicia y Gobernabilidad, USAID/DR, March 2008 Justice System Strengthened Project, USAID/DR, October 2014 Performance Evaluation of Community Justice Houses, USAID/DR, May 2016 Project Appraisal Document for DO 1: Dominican Republic Crime Prevention Program, IR 1.3: Criminal Project Appraisal Document: Criminal Justice System Strengthened Project. USAID/DR. 2014. Report for the Data Quality Assessment/Data Audit of the Criminal Justice System Strengthened Project, Indicators 1 and 5, USAID/DR Monitoring, Evaluation, and Learning Platform, June 2017 USAID/DR Performance Management Plan 2014-2018, USAID/DR, September 2015 CJSSP-related Documents Activity Monitoring and Evaluation Plan, 2016, and AMELP reports 2016-2017 Analysis of Draft Disciplinary Regulation of Judiciary, March 2018 Analysis of Operations of PEPCA, October 2017 Analysis of Prosecution Management Models, October 2016 Assessment and Evaluation of the Community Justice House Program, June 2016 Assessment and proposals for implementing mechanisms for inter-institutional coordination in the criminal system, October 2016 Chemonics International Revised Technical Proposal, May 2015 Citizen Security in DR: conceptual analysis and strategies for prevention of violence, November 2016 CJSSP Weekly Highlights, FY18 Q2 January 1 – March 31, 2018 CJSSP Workplan Narrative and annexes 2017-18 (18 months duration) CJSSP Workplan 2016 Summary table, and modifications table dated October 2016 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 86 COIN subgrant agreement, final report and correspondence related to termination Community Justice House Sustainability Study, November 2016 Concept Paper: Building a functioning model for Integrated Multi-Sectoral Attention to Women Victims and Survivors of Domestic and Gender Based Violence, and their children, May 2017 Concept Paper on Legal Representation for Gender Based Violence Victims, February 2018 Draft Disciplinary Regulation of Judiciary, April 2018 Draft Manual for Prosecution of Public and Private Corruption, undated Draft mapping or matrix of Interagency Notional Delineation of Activities, by Chemonics, December 2017 Draft Memorandum of Understanding between USAID and the Attorney General’s Office, undated Draft protocols for handling of cases by prosecutors (flagrante delicto, conditional suspension, abbreviated trial, conciliation) Evaluations and recommendations of Public Ministry, Judiciary and National Office of the Public Defense Formal Education Programs, February 2016 Feasibility study for Mobile Justice Houses, December 2017 Final report of the technical assistance for the documentation of the procedures of the Monitoring, Evaluation and Learning Unit of the Project, September 2017 IAWJ subgrant agreement, needs assessment and final report Letters from various justice sector institutions requesting assistance from CJSSP, various dates PACAM subgrant agreement, sample of progress reports and report evaluating effect of prosecutor training Participacion Ciudadana subgrant agreement and sample of progress reports Performance Indicator Reference Sheets, approved 2016, and proposed 2018 Proposed revision of CJSSP outcomes and indicators, March 2018 Quarterly and Annual Progress Reports to USAID, July 2015 to December 2017 Report on Characteristics of Dominican National Police Communications, October 2016 Request for Proposals Number SOL-517-15-000003, January 2015 Request for Proposals issued by CJSSP for a coordinated system of attention to victims of GBV and sexual violence, July 2017 Resumen de las actividades de sostenibilidad de las Casas Comunitarias de Justicia, undated Sample of pre and post-training tests, 2016 and 2017 Sample of summaries from Judicial Summit forums, 2016 Summary and update on assessment of prosecution management models, January 2017 Technical Document: Data Quality Assessment & Proposed Redefinition of CJSSP Results Indicator 1. August 2017 Training Database, updated to March 2018 Training needs in ADR and assessment of Citizen Security, October 2017 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 87 UNFPA proposal to CJSSP for grant to support an integrated system of attention to victims of violence, July 2017 USAID DR Task Order AID-517-TO-15-00001 (Contract with Chemonics), June 2015, and four contract modifications PASJ-related Documents Annual Workplans, Years 1, 2 and 3 Activity Monitoring and Evaluation Plan (original 2016, revised 2017 and 2018) Actuaciones irregulares de la Policía Nacional en el ejercicio de sus funciones, November 2016 Condiciones de Trabajo de la Policía Nacional en los Destacamentos del Corredor Duarte y 18 Barrios de la Zona Norte del Distrito Nacional, 2016 Diagnostico introductorio al Marco Presupuestario de la Policía Nacional, Período 2008-2016, 2016 FINJUS quarterly reports to PC, July 2015 to December 2017 Informe Final Consultoría Mesa de diálogo Estado-Sociedad sobre Seguridad Ciudadana en seis provincias del corredor Duarte, undated Memoria Anual 2011, Participación Ciudadana Quarterly Progress Reports to USAID, July 2015 to September 2017 Request for Applications Number: RFA-517-15-000002, February 2015 Revista “Seguridad & Justicia”, Volume 1 August 2017 Participación de la comunidad y sus organizaciones en la PREVENCIÓN DEL DELITO en Cienfuegos, Santiago, Powerpoint presentation, undated USAID Cooperative Agreement Number AID-517-A-15-00006 with Participación Ciudadana, June 2015 Various short publications and speeches by FINJUS ICITAP-related Documents Activity Monitoring and Evaluation Plan (revised 2018) Annual Workplans, Years 1, 2 and 3 Participating Agency Program Agreement Number AID-517-T-15-00001 with ICITAP, July 2017 Portfolio Review, 2016 Progress Reports to USAID, July-December 2015, January-March 2016, April-June 2017, July-September 2017 Other Documents Compromisos del Poder Judicial de la Republica Dominicana en el Marco de la Cumbre Judicial Nacional, 2016 Code of Criminal Procedure of Dominican Republic, August 2007 Cultura política de la democracia en la República Dominicana y en las Américas, 2016/17: Un estudio comparado sobre democracia y gobernabilidad, Rosario Espinal, Ph.D. of Temple University, Jana Morgan, Ph.D. of University of Tennessee, María Fernanda Boidi, Ph.D. Coordinadora Regional de LAPOP y editora del informe, Elizabeth J. Zechmeister, Ph.D. Directora de LAPOP y editora de la serie Vanderbilt University, Noviembre 2017 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 88 Guía para la Prevención de la Violencia desde los Gobiernos Locales, La Federación Dominicana de Municipios (FEDOMU), September 2017 Informe Situación de la Seguridad Ciudadana en República Dominicana 2003-2013, Coalición de ONGs Dominicanas por los Derechos Humanos, March 2013 Kaufmann, Daniel, Andrea Gallina, and Roby Senderowitsch. A Collective Action Approach Against Corruption: The Case of the Dominican Republic. World Bank, Washington, DC, 2015. License: Creative Commons Attribution CC BY 3.0 IGO Regulations Regarding Access to Justice for Vulnerable People, approved by the XIV Ibero-American Judicial Summit, Brasilia 2008 Seguridad Ciudadana en la República Dominicana: Desafíos y Propuestas de Política, Leopoldo Artiles, Secretaría de Estado de Economía, Planificación y Desarrollo, Unidad Asesora de Análisis Económico y Social, August 2009 U.S. civil society promotion and the construction of democratic governance in the Dominican Republic, Rafael A. Boglio Martínez, VI Conferencia Regional de ISTR para América Latina y el Caribe. November 8-11, 2007, Salvador de Bahía, Brasil FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 89 ANNEX D. KEY INFORMANT INTERVIEWS Position Name U.S. Government USAID Youth, Education and Security Office Director Kimberly Ludwig USAID AOR/COR Lissette Dumit INL /DR Timothy Peltier Antonio Arias Chemonics International CJSSP COP Ron Glass CJSSP Deputy COP Rosa Jimenez Chemonics Director of DG Practice Sharon Van Pelt Chemonics Project Director (now at Abt Assoc) Tiernan Mennen Chemonics Manager Ricel Valdes Caldwell CJSSP Senior Institutional Strengthening Activity Manager Thalia Goldberg CJSSP Communications and ICT Projects Manager Gladjorie Rodriguez CJSSP M&E Specialist Germania Estevez CJSSP Access to Justice Specialist Paola Pelletier CJSSP (former) Grants Manager Rosa Maria Gil CJSSP Projects Manager Francina Diaz CJSSP Consultants ADR Consultant (and involved in project design) Brian Treacy Prosecutorial reform consultant Ana Montes Prosecutor Reform Expert Jaime Bautista ADR and citizen security consultant Orlidy Inoa Communications consultant Sandra Fuentes CJSSP Partners EDF Consulting Inc. Chief Executive Officer Natalia Bermudez Participación Ciudadana Vice President José Ceballos FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 90 CJH Project Director Ruth Henriquez Community Justice House Santiago Manager Rosy Colón Community Justice House Las Caobas – Manager Carmen PACAM Director of Projects and Operation Wilmara Vasquez Pelaez, COIN/ODHGV Executive Director Santos Rosario IAWJ Director Silvia Stanga PUCMM/CEPREC Corporate Director Annette Tejada CJSSP GODR Counterparts National Judicial School, Directora Gervasia Valenzuela National Judicial School, Subdirector Jacinto Castillo Moronta National Coordinator Judiciary program lead to promote ADR Walter Cordero Coordinator National Justice Summit 2016 Samuel Arias Judiciary Technical Director Cecilia Cuello Suero Head Prosecutor DA Office SD West Rosalba Ramos Prosecutor, GBV Unit, DA Office SD West Rosa Delia Pared Head Prosecutor DA Office Santiago Luisa Liranzo Coordinator, Public Ministry International Cooperation Department Emily Ramírez Director Public Ministry Statistics Department Vladimir Hernandez Director, AGO Public-Sector Corruption Prosecution Office (PEPCA) Laura Guerrero Pelletier National School of Public Ministry Director Gladys Sánchez AG Deputy/Director, GBV Unit Ana Andrea Villa Camacho Director, Fiscalías GBV Unit, Santo Domingo West Rosalba Diaz Director, National Office of the Public Defense Laura Hernández Inspector General of Judiciary Leonidas Radhámes Peña Public Ministry representative on IMTS working group Alexander Ramón Moya MIP representative on ICMTS working group member Luis Terrero Director of Social Management, Ministry of Interior and Police Carlos Beriguete CJSSP Other FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 91 Bar Association President Miguel Surun Hernández Journalist, ACENTO Fausto Rosario Executive Director, MUDE Petruska Luna Executive Director, CE-Mujer Graciela de la Cruz Executive Director, CNDH Manuel Maria Mercedes Lawyer Fabiola Medina ICITAP Team ICITAP Management Raymond Rivera ICITAP Management German Zuñiga ICITAP Management Luis Zuñiga ICITAP Management Samuel Colon ICITAP Management Carl Risheim ICITAP Consultant Sandra Fuentes ICITAP Consultant Fernando Cardenas ICITAP Consultant (former) Miguel Rodriguez ICITAP Consultant (former) Bolivar Cruz ICITAP Consultant (former) Lizzy Solano ICITAP Consultant (former) Paulo Herrera ICITAP GODR Counterparts DNP Strategic Planning Working Group (Director) Claudio Peguero Castillo DNP Strategic Planning Working Group Sugeidy Bautista Guerrero DNP Strategic Planning Working Group Luz Estrella DNP Strategic Planning Working Group Idalgisa Peña Villamán DNP Directorate of Administration and Finance Edwin Cruz López DNP Directorate of Administration and Finance Leonardo Garcia Alvarez DNP Strategic Planning Working Group Roberto Hernández DNP Human Resources Directorate 2015-2016 Nelson Rosario Guerrero DNP Communications Division Frank Durán Mejía DNP Human Resources Directorate Miguel Jiménez Cruz FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 92 DNP Strategic Planning Working Group Francisca Martínez Pérez DNP Strategic Planning Working Group Robinson Contreras DNP Strategic Planning Working Group Farides Soler DNP Violencia de Genero General Teresa Martinez DNP ex- Director General Gen. Nelson Peguero DNP Human Resources Directorate Buenaventura Gómez DNP Training and Education Department Alejandro Dipre DNP Training and Education Department Orison L. Olivence PASJ (Participacion Ciudadana) Participación Ciudadana (PASJ) Former Executive Director Rosalia Sosa Executive Director Carlos Pimentel Cooordinator Red Regional Justicia y Seguridad Ciudadana Zobeyda Apólito PASJ Project Coordinator Laisa Santana Local Coordinator Santiago Antonio Padilla Local Facilitator Santiago Josefina Mercedes Reyes Local Coordinator Puerto Plata Valentina Tamares Local Coordinator San Francisco de Macoris Ramon David Phipps Local Coordinator La Vega Leocadio Contreras Manager Red Regional Justicia y Seguridad Ciudadana Aracelis De Sena Local Coordinator Santo Domingo Cornelia Adon Vidal FINJUS Executive Director Servio Tulio Castaños Director of Projects Carlos Villaverde Project Manager Maria Dolores Diaz PAJS Coalition Members Executive Director, Herrera Industrial Association Greicy Romero Regional Center for Economic Sustainable Strategies Ernesto Selman Director, American Chamber of Commerce Mary Fernandez Business Action for Education (EDUCA) Darwin Caraballo FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 93 Latin American Council of International Comparative Law Francisco Batlle Director Poder Ciudadano Bartolome Pujals Director, Centro Bono Mario Berges Director PACAM Yanira Lara COIN/ODHGV Executive Director Santos Rosario PAJS Panelists and Experts Asociación de Fiscales Dominicanos Julissa Hernández Asociacion de Jueces Dominicanos Ysis Muñis Policy Advisor on Citizen Security MIP Carolina Ramirez Herrera Independent expert in transparency and corruption Fernando Alvarez Independent expert in judicial reform Jaime Bautista PAJS Local Crime Prevention Initiatives Puerto Plata Director CONANI Richy Almonte Regional Director Womens Ministry Ararca Guerrero Director Regional Office DNP Orison Olivence Vice Mayor Puerto Plata Marisol Almonte Gender Coordinator Muerto Plata Municipality Giselle Capella Santo Domingo West Asociación de Empresas Industriales de Herrera y Provincia Santo Domingo Jose Luis Espinal Consejo Comunitario Buscando Soluciones Ramón Núñez Fundación Voluntad Educativa Andrés Travieso Enc. Área de Salud Mental AntiPandilla Policía Nacional Rolando Mora Moya Casa de la Juventud y la Cultura Gregorio Morillo Casa Comunitaria de Justicia Carmen Victoriana Cienfuegos, Santiago Casa Comunitaria de Justicia Rosie Colon Instituto Politécnico Ramón Dubert Novo Isaac Jiménez Casa de la Juventud y la Cultura Yoryi Stiven Almonte FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 94 Coordinadora Mujeres de Cienfuegos Claudia Almonte Director CDES Reynaldo Peguero PASJ GODR Counterparts Director, Ministry of Interior and Police, Mesa Nacional Maria Sued Staff, Ministry of Interior and Police, Mesa Nacional Carlos Picel MW, Viceministra Politicas de Igualdad Sonia Diaz MW, Enc. Dept Prevencion de Violencia Natividad Lopez MW, Enc. Politicas Migratorias Virginia Rodriguez MW, Tec. NNA huérfanos por Feminicidio Rita Aponte MW, Director of Intersectoral Coordination Violeta Quezada MW, Abogada, Dept. Prevencion de Violencia Mencia Lopez F. Donor Agencies Program Officer, AECID Ana Alvarez Project Officer, UNFPA Jeannie Ferreras FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 95 ANNEX E. DATA COLLECTION INSTRUMENTS This Annex contains the data collection instruments used during the evaluation. These instruments were used as a guideline and adapted as needed to each individual informant. They are: ● Key Informant Interview – CJSSP (General and Tailored to Prosecutorial Authorities) ● Key Informant Interview - ICITAP ● Key Informant Interview - PASJ FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 96 Key Informant Interview questions on CJSSP (general version) As you may know, the Criminal Justice System Strengthened Project is a five-year initiative funded by USAID, whose overall objective is to strengthen the criminal justice system in the Dominican Republic by improving the quality and effectiveness of prosecutions. The project supports activities of several kinds, including: improving case management by prosecutors (Public Ministry), enhancing local community access to justice services, bolstering institutional prosecutor career and integrity systems, and strengthening mechanisms for interinstitutional coordination among various justice sector organizations. The project is also helping various justice sector actors to better manage informal and formal cases involving vulnerable groups. Our team is conducting an independent mid-term evaluation of this project, and this confidential interview is intended to collect information about your experience with the CJSSP and your understanding of the challenges and achievements of the project in implementing its activities. General questions on achievements and challenges 1. What is your role in relation to this project, and how long have you held that position? In which areas of the project have you been closely involved? (this will guide how later questions are framed) 2. What do you consider to be the most important achievements of the project so far? • Ask for specific examples • Are processes underway that are likely to lead to other important achievements? What are those? 3. How do the overall results so far compare with project’s original objectives? • Ask for specifics/examples of areas where expected results were achieved and not yet achieved • If shortfalls mentioned: to what extent were the original expected results realistic at the time the project was designed? Were there changes later that led to difficulties in meeting those expected results? What changes? 4. How have the project and USAID responded to changes/developments/risks arising during implementation that have affected expectations and plans? • If changes made - were changes in project priorities/timing/targets made on a timely basis? Why or why not? How effective were those changes in addressing the changed situation or emerging risk? 5. What do you see as other major challenges or obstacles to achieving the expected results of the project? • Probe for internal factors such as implementation difficulties, staffing, budget constraints, methodology, selection of targeted institutions • Probe for external factors such as coordination, politics, economics, security, etc. that may have limited effectiveness in some way • For each major challenge, ask: What has been the effect on the project? • For each major challenge, ask: How has the project team (and/or others such as the USAID mission) worked to overcome or minimize the effect of that challenge? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 97 Support to government entities (adapt questions to focus on the institution(s) and activities with which the informant is most familiar) 6. What effect has the project had so far on the (AG Office/DA Offices, Superior Councils, Inspector Generals, etc.) and on the work they do? • Which interventions were primarily responsible for this effect? • Has the project had an effect on decongestion of the criminal justice system? If so, through what initiatives? How do you know? If not, why not? • Has the project had an effect on case tracking and data sharing within the system? How do you know? If so, through what initiatives? If not, why not? • Were there any unexpected effects? Any negative effects? • Did the project focus its work with these entities on the most suitable/highest priority areas, or should the focus have been different? 7. To what extent do you think that any positive changes in these entities and their functioning (as noted in previous question) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done (by the project or others) to shore up sustainability? 8. What methods used by the project have been most effective so far with these entities? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and other studies, training, technical assistance/mentoring, development of guidelines, monitoring of progress, etc.) • What is your view of the training of government staff provided or funded so far by the project? (probe for main strengths and weaknesses, depending on how familiar the informant is with the training activities) • What have been the observable effects of that training, both on individual participants and on their employer institutions? To what extent has training contributed to the achievement of the project objectives? 9. To what extent has the project had an effect on inter-institutional relations and coordination in the criminal justice sector? • Which interventions were primarily responsible for this effect? • If there has been no or minimal effect, why? 10. Can you mention any changes in management practices related to prosecutorial effectiveness, that have flowed from the project’s support? • Which interventions were primarily responsible for this effect? • How likely is it that those changes will become models for other similar bodies (ex. other fiscalías) and be replicated? • How likely is it that those changes will be sustained? • What are the challenges to sustainability of the changes that have taken place? • What could be done in the remaining period of the project to ensure longer term impact in the area of management practices and models? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 98 Support to non-government stakeholders/civil society actors choose/adapt questions to focus on the entities and activities with which the informant is most familiar) Community Justice Houses 11. What effect has the project had so far on the Community Justice Houses and the work they do? • Which interventions were primarily responsible for this effect? • Has there been an effect on decongestion of the criminal justice system? If so, how? If not, why not? • Were there any unexpected effects? Any negative effects? • Did the project focus its work with PC and the CJH on the most suitable/highest priority areas, or should the focus have been different? 12. Do you think that the project has contributed to some improvement in the work of the CJHs? • If yes, how likely is it that any improvements will be sustained? • What are the challenges to sustainability of changes? • What can be done (by the project or others) to shore up sustainability? 13. What methods used by the project have been most effective so far with the CJHs? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and studies, funding for operations, training, technical assistance/mentoring, development of guidelines, etc.) Other civil society partners (PACAM, COIN, IAWJ) 14. What effect has so far been achieved by the project by supporting subgrantee ________ to __________________________? • Who has benefited from this subgrant support and how? • Were there any unexpected effects? Any negative effects? • Was this subgrant focused on the most suitable/highest priority areas, or should the focus have been different? 15. To what extent do you think that any positive effects of this subgrant (as noted in previous question) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done (by the project, subgrantee or others) to shore up sustainability? 16. What is your view of the training of civil society actors provided or funded so far by the project? (probe for main strengths and weaknesses, depending on how familiar the informant is with the training activities) • What have been the observable effects of that training, both on individual participants and on their employer organizations? To what extent has training contributed to the achievement of the project objectives? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 99 Relationships and coordination (primarily for management staff) 17. To what extent did the Project make efforts to coordinate and cooperate with other stakeholders, including other US government agencies, Dominican and international bodies involved in the sector? • Did the project complement the work of others, or was there overlap or confusion? Ask for specific examples. 18. How would you describe the project’s relationship to date with the Government of the DR? • How has that relationship evolved since the project began? • To what extent have key/senior government officials (elected and other) been supportive of the project? Ask for examples of support and cooperation, and lack thereof. • How has this affected the project’s implementation and progress towards results? Monitoring of Results (for informants familiar with the monitoring and reporting systems of the project) 19. Is the project on track to meet its indicator targets? • If not, why not? (probe for each indicator mentioned) • Have the indicators proven to be difficult to measure and report on? If so, why? • How well do the indicators measure the results of the project? Do they provide sufficient data in terms of quality and quantity to inform ongoing decision-making about this project? • If not, what else should and could be measured during the remainder of the project? 20. How has the effect of training supported by the project been measured so far? • To what extent have participants’ learning and behavior change been assessed? Other 21. How has the project taken gender issues into account in its approaches and activities? • What specific project strategies or activities have been used to identify and respond to the different needs of men and women? • How effective have those measures been so far? 22. Have the combined activities by USAID in support of justice sector reform contributed to strengthening crime prevention in targeted areas? • Have results to date in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities supported the overall goal of crime prevention? • Why or why not? • If yes, what is the evidence to show the contribution to crime prevention? • If evidence is lacking, how can the project and USAID better measure these results and linkages in future? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 100 Recommendations 23. How could the project be improved for the remainder of its implementation period (early 2020)? 24. What should be the priorities for future support by USAID related to criminal justice system reform, in the longer term? 25. Do you have any final recommendations for the project or USAID to consider? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 101 Key Informant Interview questions on CJSSP (for prosecutorial authorities) As you may know, the Criminal Justice System Strengthened Project is a five-year initiative funded by USAID, whose overall objective is to strengthen the criminal justice system in the Dominican Republic by improving the quality and effectiveness of prosecutions. The project supports activities of several kinds, including: improving case management by prosecutors (Public Ministry), enhancing local community access to justice services, bolstering institutional prosecutor career and integrity systems, and strengthening mechanisms for interinstitutional coordination among various justice sector organizations. The project is also helping various justice sector actors to better manage informal and formal cases involving vulnerable groups. Our team is conducting an independent mid-term evaluation of this project, and this confidential interview is intended to collect information about your experience with the CJSSP and your understanding of the challenges and achievements of the project in implementing its activities. General questions on achievements and challenges 1. What is your current position, and how long have you held that position? • How have you (and/or your institution) interacted with the project? In what activities have you and your institution participated? (this will guide how later questions are framed) 2. What effect has the project had so far on your institution and its work? What kind of changes have resulted? • Ask for specific examples • Which interventions were primarily responsible for this effect? • Are processes underway that are likely to lead to other changes? What are those? • Are there areas where change or progress has been less than you expected? If yes, in what areas and why has progress not met expectations? • Were there any unexpected effects? Any negative effects? 3. Has the project had an effect on _____________? If so, through what initiatives? How do you know? If not, why not? Depending on the informant’s involvement and awareness, and responses to previous question, topics will be selected from the following: • decongestion of the criminal justice system • case tracking and data sharing within the system • management practices related to prosecutorial effectiveness? (N.B. focus on Santo Domingo West fiscalía as pilot) How likely is it that those changes will become models for other similar bodies (ex. other fiscalías) and be replicated? • prosecutor career and integrity systems • inter-institutional relations and coordination in the criminal justice sector • communications by criminal justice sector institutions • legal protection of vulnerable groups • community level access to justice FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 102 4. To what extent do you think that any positive changes in your institution and its work (as noted in previous questions) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done to shore up sustainability during the next two years of project support? (probe for each main area of observed change) 5. What methods used by the project with your institution have been most effective so far? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and other studies, training, technical assistance, development of guidelines, facilitation of dialogue, etc.) 6. What is your view of the training provided (or funded) by the project for your institution? (probe for main strengths and weaknesses, depending on how familiar the informant is with the training activities) • What have been the observable effects of that training on individual participants? • To what extent has training contributed to change in your institution? • What has been the effect on the national training institutions with which the project has collaborated? 7. How have the project and USAID responded to changes/developments/risks that have affected expectations and plans? • How effective were they in addressing the changed situation or emerging risk? 8. What do you see as other major challenges or obstacles to achieving the results that the project is aiming for with your institution? • Probe for internal factors such as project implementation difficulties, staffing, budget constraints, methodology, overlaps with other initiatives • Probe for external factors such as coordination within government and among international actors, politics, economics (justice sector budgets), security, etc. • For each major challenge, ask: What has been the effect on the project and its work with your institution? 9. Have you or your institution been involved with or aware of the civil society organizations with which the project is collaborating? (ex. PC, PACAM, COIN, IAWJ, PUCMM) 10. What is your view of the project’s support to those entities and the work they are doing? (probe in relation to Community Justice Houses, handling of vulnerable populations by the justice system, handling of GBV cases by the system, etc.) • Has there been an effect on the criminal justice system? • If so, through what initiatives? If not, why not? Relationships and coordination (primarily for management staff) 11. To what extent did the project make efforts to coordinate and cooperate with other stakeholders, including other US government agencies, Dominican and international bodies involved in the sector? • Did the project complement the work of others, or was there overlap or confusion? Ask for specific examples. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 103 12. How would you describe the project’s relationship to date with the Government of the DR, in general? And with your institution in particular? • How has that relationship evolved since the project began? • To what extent have key/senior government officials (elected and other) been supportive of the project? Ask for examples of support and cooperation, and lack thereof. • How has this affected the project? Monitoring of results (for informants familiar with the monitoring and reporting systems of the project) 13. How has the project engaged with your institution to measure the results of both the project and your institution’s work? (probe in relation to what the project has offered by way of support, and what the project has expected from the institution) • What effect has this area of engagement had so far on your institution? • Have there been challenges in measuring results and reporting using the project indicators? If so, why? (probe for each relevant indicator – see list below) • How well do the indicators measure the results of your institution’s work? • How could monitoring and data management systems be improved? 1. Prompt response by the Public Ministry for processing cases entering the System. Indicator: Percentage of new cases filed that reach case disposition in targeted District Attorney General´s Office 2. Expansion in the use of Alternative Conflict Resolution, conciliation and mediation of cases. Indicator: Percentage of new cases filed that reach case disposition in targeted District Attorney General´s Offices 3. Criminal Justice System improves management of Gender Based Violence Cases. Indicator: Number of GBV victims assisted by Project supported entities. 14. How has the effect of training supported by the project been measured so far? • To what extent have participants’ learning and behavior change been assessed? • Which specific areas of work have been under or over-funded? Other 15. How has the project taken gender issues into account in its approaches and activities? • What specific project strategies or activities have been used to identify and respond to the different needs of men and women? • How effective have those measures been so far? 16. Have the combined activities by USAID in support of justice sector reform contributed to strengthening crime prevention in targeted areas? • Have results to date in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities supported the overall goal of crime prevention? • Why or why not? • If yes, what is the evidence to show the contribution to crime prevention? • If evidence is lacking, how can the project and USAID better measure these results and linkages in future? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 104 Recommendations 17. How could the project be improved for the remainder of its implementation period (early 2020)? 18. What should be the priorities for future support by USAID related to criminal justice system reform, in the longer term? 19. Do you have any final recommendations for the project or USAID to consider? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 105 Key Informant Interview questions on ICITAP project As you may know, ICITAP is implementing a project funded by USAID, whose objective is to build the capacity of the DNP and its responsiveness to citizens in the DR. The ICITAP project was originally designed to support strengthening the DNP in four areas: (1) investigations coordination with national prosecutors, (2) human resources, (3) community relations, and (4) external oversight. The project was modified in 2017 to further define those four areas to six areas: (1) Planning and Organizational Development, (2) Administration and Police Services Management, (3) Human Resources. (4) Strategic Communications, (5) Police Coordination with National prosecutors, and (6) Police-Community Relations. Our team is conducting an independent mid-term evaluation of this project, and this confidential interview is intended to collect information about your experience with the ICITAP project and your understanding of the challenges and achievements of ICITAP in implementing the project activities, both those as originally defined and those moving into the future. General questions on achievements and challenges 1. What is your role in relation to the ICITAP project, and how long have you held that position? In which areas of activity have you been closely involved? (this will guide how later questions are framed) 2. What do you consider to be the most important achievements of the ICITAP project so far? • Ask for specific examples Are processes underway that are likely to lead to other important achievements? What are those? 3. How do the overall results so far compare with the expected results of the ICITAP project as you understand them? • Ask for specifics/examples of areas where expected results were achieved and not yet achieved • If shortfalls mentioned: to what extent were the original expected results realistic at the time the activity was designed? Were there changes later that led to difficulties in meeting those expected results? What changes? 4. How have the ICITAP project and USAID responded to changes/developments/risks arising during implementation that have affected expectations and plans? • If changes made - were changes in project priorities/timing/targets made on a timely basis? Why or why not? How effective were those changes in addressing the changed situation or emerging risk? 5. What do you see as other major challenges or obstacles to achieving the expected results of the ICITAP project? • Probe for internal factors such as implementation difficulties, staffing, budget constraints, methodology, selection of targeted institutions • Probe for external factors such as coordination, politics, economics, security, etc. that may have limited effectiveness in some way • For each major challenge, ask: What has been the effect on the project? • For each major challenge, ask: How has the project team (and/or others such as the USAID mission) worked to overcome or minimize the effect of that challenge? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 106 Interactions with government entities 6. What effect has the project had so far on reforms to the DNP and on the work it does? • Which project interventions were primarily responsible for this effect? • Were there any unexpected effects? Any negative effects? • Did the project focus its work with the highest priority problems and issues, or should the focus have been different? 7. What methods used by the project have been most effective so far in promoting reform? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and other studies, training, technical assistance/mentoring, development of guidelines, monitoring of progress, etc.) 8. To what extent do you think that any positive changes in the DNP (as noted in previous question) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done (by the project or others) to shore up sustainability? 9. To what extent has the project had an effect on inter-institutional relations and coordination in the criminal justice sector? • Which interventions were primarily responsible for this effect? • If there has been no or minimal effect, why? Relationships and coordination (primarily for management staff) 10. To what extent did the ICITAP project make efforts to coordinate and cooperate with other stakeholders, including other US government agencies, Dominican and international bodies involved in the sector? • Did the project complement the work of others, or was there overlap or confusion? Ask for specific examples. • Did the project coordinate with other USAID projects? Other USG activities? 11. How would you describe the project’s relationship to date with the DNP? • How has that relationship evolved since the ICITAP project began? • To what extent have key/senior government officials (elected and other) been supportive of the ICITAP project? Ask for examples of support and cooperation, and lack thereof. • How has this affected the project’s implementation and progress towards results? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 107 Monitoring of results (for informants familiar with the monitoring and reporting systems of the activity) 12. Is the ICITAP project on track to meet its indicator targets? • If not, why not? (probe for each indicator mentioned) • Have the indicators proven to be difficult to measure and report on? If so, why? • How well do the indicators measure the results of the project? Do they provide sufficient data in terms of quality and quantity to inform ongoing decision-making about this activity? • If not, what else should and could be measured during the remainder of the project? 13. How has the effect of training supported by the project been measured so far? • To what extent have participants’ learning and behavior change been assessed? • What information is available on the effects of training, both on individual participants and on their employer organizations? Vulnerable populations 14. How has the ICITAP project taken gender issues into account in its approaches and activities? • What specific strategies or activities have been used to identify and respond to the different needs of men and women? • How effective have those measures been so far? Overall 15. Have the combined activities by USAID in support of justice sector reform contributed to strengthening crime prevention in targeted areas? • Have results to date in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities supported the overall goal of crime prevention? • Why or why not? • If yes, what is the evidence to show the contribution to crime prevention? • If evidence is lacking, how can the project and USAID better measure these results and linkages in future? Recommendations 16. How could the ICITAP project be improved for the remainder of its implementation period (early 2020)? 17. What should be the priorities for future support by USAID related to criminal justice system reform, in the longer term? 18. Do you have any final recommendations for the project or USAID to consider? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 108 Key Informant Interview questions on PASJ project As you may know, Participación Ciudadana is implementing a Project funded by USAID, whose objective is to promote the participation of civil society organizations in the reform of justice and the police. PC supports activities at three levels: national level policy advocacy to press for reform of laws and regulations and implementation of the reforms approved by the government; support to regional coalitions of CSOs to dialogue with government authorities, the justice sector and the police to develop responses to problems of citizen security; and support for dialogue between CSOs, municipal authorities, justice and police representatives to design and implement local level citizen security plans to better prevent and deter crime. PC also supports CSOs representing vulnerable groups to ensure better access and responsiveness of justice institutions and police. Our team is conducting an independent mid-term evaluation of this project, and this confidential interview is intended to collect information about your experience with the PC project and your understanding of the challenges and achievements of PC in implementing the project activities. General questions on achievements and challenges 1. What is your role in relation to the PC project, and how long have you held that position? In which areas of activity have you been closely involved? (this will guide how later questions are framed) 2. What do you consider to be the most important achievements of the PC project so far? • Ask for specific examples • Are processes underway that are likely to lead to other important achievements? What are those? 3. How do the overall results so far compare with the expected results of the PC project as you understand them? • Ask for specifics/examples of areas where expected results were achieved and not yet achieved • If shortfalls mentioned: to what extent were the original expected results realistic at the time the activity was designed? Were there changes later that led to difficulties in meeting those expected results? What changes? 4. How have the PC project and USAID responded to changes/developments/risks arising during implementation that have affected expectations and plans? • If changes made - were changes in project priorities/timing/targets made on a timely basis? Why or why not? How effective were those changes in addressing the changed situation or emerging risk? 5. What do you see as other major challenges or obstacles to achieving the expected results of the PC project? • Probe for internal factors such as implementation difficulties, staffing, budget constraints, methodology, selection of targeted institutions • Probe for external factors such as coordination, politics, economics, security, etc. that may have limited effectiveness in some way • For each major challenge, ask: What has been the effect on the project? • For each major challenge, ask: How has the project team (and/or others such as the USAID mission) worked to overcome or minimize the effect of that challenge? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 109 Interactions with government entities 6. What effect has the project had so far on reforms to the GODR justice sector institutions and the police, and on the work they do? • Which project interventions were primarily responsible for this effect? • Were there any unexpected effects? Any negative effects? • Did the project focus its work with the highest priority problems and issues, or should the focus have been different? 7. What methods used by the project have been most effective so far in promoting reform in these entities? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and other studies, training, technical assistance/mentoring, development of guidelines, monitoring of progress, etc.) 8. To what extent do you think that any positive changes in these GODR entities (as noted in previous question) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done (by the project or others) to shore up sustainability? 9. To what extent has the project had an effect on inter-institutional relations and coordination in the criminal justice sector? • Which interventions were primarily responsible for this effect? • If there has been no or minimal effect, why? Support to non-government stakeholders/civil society actors (choose/adapt questions to focus on the entities and activities with which the informant is most familiar) 10. Has PC been able to convene the most representative CSOs for its activities to strengthen justice institutions and the police? • Are there some CSOs that should have been convened but were not? • Has PC’s involvement with the Marcha Verde has an impact on its ability to convene CSOs? What specifically has been the impact? 11. What effect has the project had so far on PC’s partner CSOs and the work they do? • Which interventions were primarily responsible for this effect? • Were there any unexpected effects? Any negative effects? 12. Do you think that the activity has contributed to some improvement in the work of the partner CSOs? • If yes, how likely is it that any improvements will be sustained? • What are the challenges to sustainability of changes? • What can be done (by the activity or others) to shore up sustainability? 13. What methods used by the activity have been most effective so far with the partner CSOs? • Why were those methods effective? • Which seemed to be less effective? (Probe re assessments and studies, funding for operations, training, technical assistance/mentoring, development of guidelines, etc.) FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 110 14. Has PC been effective in working with communications media? • Have the media been open to the issues promoted by PC and its partner CSOs? • Can you give examples of achievements or challenges in working with communications media? Subgrantees 15. What effect has so far been achieved by the activity by supporting subgrantee ________ to __________________________? • Who has benefited from this subgrant support and how? • Were there any unexpected effects? Any negative effects? • Was this subgrant focused on the most suitable/highest priority areas, or should the focus have been different? 16. To what extent do you think that any positive effects of this subgrant (as noted in previous question) will be sustained after support by USAID ends? • What are the challenges to sustainability of changes? • What can be done (by the activity, subgrantee or others) to shore up sustainability? Relationships and coordination (primarily for management staff) 17. To what extent did the PC project make efforts to coordinate and cooperate with other stakeholders, including other US government agencies, Dominican and international bodies involved in the sector? • Did the project complement the work of others, or was there overlap or confusion? Ask for specific examples. • Did the project coordinate with other USAID projects? Other USG activities? 18. How would you describe the project’s relationship to date with the Government of the DR? • How has that relationship evolved since the PC project began? • To what extent have key/senior government officials (elected and other) been supportive of the PC project? Ask for examples of support and cooperation, and lack thereof. • How has this affected the project’s implementation and progress towards results? 19. How would you describe the project’s relationship to date with regional and local government institutions? • Have local representatives of justice institutions and police been open to dialogue with CSOs? • Have there been concrete results from the dialogues with justice institutions and police at the regional and local levels? Explain some of the results. Monitoring of results (for informants familiar with the monitoring and reporting systems of the activity) 20. Is the project on track to meet its indicator targets? • If not, why not? (probe for each indicator mentioned) • Have the indicators proven to be difficult to measure and report on? If so, why? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 111 • How well do the indicators measure the results of the project? Do they provide sufficient data in terms of quality and quantity to inform ongoing decision-making about this activity? • If not, what else should and could be measured during the remainder of the project? 21. How has the effect of training supported by the project been measured so far? • To what extent have participants’ learning and behavior change been assessed? • What information is available on the effects of training, both on individual participants and on their employer organizations? Vulnerable populations 22. How has the PC project taken gender issues into account in its approaches and activities? • What specific strategies or activities have been used to identify and respond to the different needs of men and women? • How effective have those measures been so far? 23. How has the PC project supported the work of organizations defending the rights of vulnerable populations? • What has been the objective of PC in working with vulnerable populations? • Has PC been effective in improving access to justice? Overall 24. Have the combined activities by USAID in support of justice sector reform contributed to strengthening crime prevention in targeted areas? • Have results to date in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities supported the overall goal of crime prevention? • Why or why not? • If yes, what is the evidence to show the contribution to crime prevention? • If evidence is lacking, how can the project and USAID better measure these results and linkages in future? Recommendations 25. How could the project be improved for the remainder of its implementation period (early 2020)? 26. What should be the priorities for future support by USAID related to criminal justice system reform, in the longer term? 27. Do you have any final recommendations for the project or USAID to consider? FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 112 ANNEX F. EVALUATION SCOPE OF WORK This annex contains the scope of work provided by USAID/DR (including the complete set of evaluation questions) early in the process. The SOW was further developed in the Inception Report, adding details about team composition, methodology, data collection, and scheduling. Background The U. S. Agency for International Development/Dominican Republic (USAID/DR) has engaged the Monitoring, Evaluation and Learning (MEL) Platform contract implemented by Panagora Group LLC to conduct an external whole-of-project performance evaluation of the Criminal Justice System Strengthened project. This project contributes to the overall goal of the Mission’s Country Development Cooperation Strategy (CDCS) to improve citizen security to promote economic growth, and to IR 1.3 Criminal Justice Institutions Strengthened. This intermediate result has been further developed in a project appraisal document (PAD) which sets out as the purpose of the project, to strengthen the Government of the Dominican Republic’s (GODR) criminal justice system by improving the timeliness and effectiveness of criminal prosecutions. The PAD sub-purposes are as follows ● Sub-Purpose 1.1: Prosecutor Effectiveness Increased. Aimed at strengthening the interagency leadership and case management capacity of the Attorney General and Public Ministry; and improving the timeliness and effectiveness of criminal prosecutions by reducing case backlog and improving the coordination of national and local prosecutors with the DNP. It has two components: o Prosecutor Management Model: USAID will build on the accomplishments of the 2008- 2013 rule of law program by designing and implementing a new case management model for Santo Domingo Province. o Community Justice Houses: USAID will support expansion of the number of Community Justice Houses (“Casas”) from a current total of five to 15 by program completion in 2018. Certified F/GBV counseling and referral services will be incorporated into each new casa as part of the expansion strategy. ● Sub-Purpose 2.1: Police Management and Transparency Improved. Provide management training and TA for the police with a specific focus on leadership, operations management, and transparency to strengthen internal demand for reform. This will include targeted training and assistance to strengthen DNP coordination with the Public Ministry, as well as strengthen DNP performance and accountability in key areas of crime prevention such as Family and Gender Violence (F/GBV) and community relations. USAID will provide training and TA to strengthen DNP compliance with existing transparency laws such as FOIA, Budget and Procurement, as well as to improve coordination with national prosecutors and strengthen community relations capacity. ● Sub-Purpose 3.1: Public Oversight in Crime Prevention Improved. Strengthen the watchdog role of Dominican civil society organizations (CSOs) in monitoring GODR law enforcement and crime prevention efforts. Specific priorities will include the establishment of coalitions to advocate for improved government performance and accountability on critical matters such as FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 113 police transparency, gender violence, and judicial corruption; and the use of new tools such as social audit and electronic media to strengthen civil society advocacy in these areas. It has two components: o Community – Police Relations: Establish and strengthen partnerships between community organizations, municipal authorities, and police as a key element of public oversight. And leverage ongoing access to justice and dispute mediation partnerships between the Attorney General and civil society under the USAID/DR Community Justice House program by incorporating trained and vetted DNP officers into broader community outreach activities led by civil society. o Rights-Based Advocacy: USAID support for civil society will be targeted to leverage existing transparency laws such as FOIA, budget and procurement to increase the effectiveness of public oversight of GODR law enforcement and crime prevention efforts. This will include training and TA to strengthen CSO capacity to access and analyze publicly available information such as arrest data, prosecution outcomes, and disciplinary action against justice system personnel; as well as to conduct public debate of GODR performance in prosecuting and preventing crime. This project is being implemented under three separate mechanisms: Criminal Justice System Strengthened (CJSS) activity. Contract with Chemonics International; $21,536,654, starting on June 8, 2015 and ending on June 7, 2020. The objective is to strengthen the criminal justice system in the Dominican Republic by improving the quality and effectiveness of prosecution. The activity will strengthen the Public Ministry’s management and technical capacities to process critical cases by expanding and adapting new management models; and increase the transparency and accountability of Dominican law enforcement and crime prevention efforts by strengthening internal controls and external oversight for both the Public Ministry and National Police. Institutional Capacity and Transparency Strengthening for Police Reform (ICITAP). Participating Agency Program Agreement (PAPA) with the United States Department of Justice; $2,800,000, starting on July 20, 2015 and ending on July 19, 2019. The objective is to strengthen institutional capacity and transparency within the Dominican National Police (DNP). The activity will improve police – prosecutor coordination throughout the criminal investigations process; strengthen DNP human resources planning and management; strengthen DNP community relations capacity; and strengthen external oversight mechanisms for the Dominican National Police (DNP) Civil Society Action for Accountable Justice and Security activity. Cooperative Agreement with Participacion Ciudadana; $6,400,000, starting on June 22, 2015 and ending June 21, 2018. The objective is to increase public awareness on criminal justice and public safety, especially in the institutional roles and responsibilities of the Public Ministry and the National Police in crime persecution and prevention. The activity will improve public awareness of crime and criminal justice issues; increase access and availability of information and support resources for victims and witnesses; increase use of the Dominican Freedom of Information Act (FOIA) and related legislation to exert social control actions and monitor criminal justice system’s actors, including the police; and support civil society organizations in pressing for reforms in the criminal justice and public safety systems. Concentration and coordination of activities. The three activities will all be concentrated geographically on the Duarte corridor prioritized in the Mission CDCS. The corridor goes from Santo Domingo north to Puerto Plata. It includes six of the nation’s 32 provinces, 70 percent of the Dominican FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 114 population, and 63 percent of youth. The cities where crime is the highest in the Dominican Republic are all located within this corridor. The implementing partners will coordinate actions in furtherance of the Mission’s DO 1 and the intermediate results (IRs). ● CJSS will ensure that government partners and institutions are linked to activities led by other CSOs working under USAID’s Civil Society Action for Accountable Justice and Security program to reinforce complementary and avoid overlap. ● ICITAP training and TA to strengthen DNP community relations capacity will be coordinated with other USAID-funded assistance to establish and strengthen community crime prevention partnerships with local civil society organizations, the private sector, and municipal authorities. ● PC aims to identify initiatives that contribute to establish synergy with other actions supported by USAID, such as monitoring and social control of the justice system institutions (CJSS), the Community House of Justice program, the Project Youth at Risk, the Project HIV/AIDS and the Program of Global Climate Change Adaptation. Evaluation Questions Overall Results 1. Taken as a whole, have the three activities produced the results set out in the CDCS, “Criminal justice institutions strengthened”, and in the Justice PAD, “to strengthen the GODR criminal justice system by improving the timeliness and effectiveness of criminal prosecutions”? Has the design of the mechanisms (contracts and agreements) responded fully to the CDCS and PAD results framework? 2. Taken as a whole, have the activities in support of justice sector reform supported the expected links between the overall goal of crime prevention and the intermediate results of improvement in prosecutor effectiveness, better police management, and community engagement with security sector institutions in these communities? What data do we have? What data do we need / how can we better measure results and linkages going forward? Are there specific evaluation exercises that might be conducted to demonstrate these links? Effectiveness 3. Have these three activities achieved the expected results of their contracts or agreements for the first half of the implementation period? If not, why not and what are the recommendations for course-correction for the second half of the award period? If the expected results have been achieved, are there implementation challenges we should be aware of for the remainder of the award? 4. What have been the major achievements and challenges of the activities conducted to increase prosecutor effectiveness? Has the activity met its targets? Have interventions been designed and implemented such that they specify, generate and document changes in management practices that can provide models for reform? If not, what changes need to be made to the activities to achieve this? 5. What have been the major achievements and challenges of the activities conducted to strengthen police management and transparency? Are the activities meeting all the FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 115 objectives set out in the revised Participating Agency Program Agreement? Which of the objectives are not being met? Why? Give specific examples. What changes to the activities are needed to ensure that they meet all their objectives? 6. How effective have the civil society activities been in creating demand for justice and monitoring law enforcement and crime prevention efforts, both at the national and local levels? Beyond the local implementing partners themselves, are other civil society organizations or leaders involved and invested in these activities? Is there a coalition of CSOs working to improve performance and accountability (e.g. transparency, GBV, judicial corruption)? How has the private sector participated in these coalitions? What are the obstacles to bringing more CSOs/private sector organizations into these activities? Do the grantees and sub-grantees have the capacity to achieve results? Management 7. Has there been adequate coordination among the different justice sector activities? How have USAID/DR and IPs ensured coordination? Give specific examples of effective coordination (or challenges to coordination) between implementing partners (IPs) and/or between GODR counterparts. 8. Has the Mission been opportune in identifying emerging challenges and making course corrections in the different activities? Are there quality performance data for decision￾making by USAID? Do the IPs generate quality performance data for ensuring effective management and monitoring? Sustainability 9. Has USAID/DR and the IPs been effective in engaging the GODR in the justice sector activities, and ensuring buy-in and support for proposed reforms? Give specific examples of GODR buy-in (or not) from each of the activities and their respective components including subgrantees activities. 10. Which of the justice sector activities (and individual components) supported by USAID/DR are producing sustainable results? Give examples of the main program interventions that will continue (or not) producing results in the long-term. What measures need to be taken to ensure greater sustainability of results in each of the activities? Methodology Following USAID Evaluation Policy and technical guidance, the evaluation will use a mixed-methods approach as follows: ● It will rely on the review of project documents and other contextual documents produced by implementing partners (IPs), USAID and other USG agencies, Government of the Dominican Republic (GODR), other development agencies, and research institutions. ● Secondly, it will utilize structured interviews with key informants in the justice sector, including the Public Ministry ( the statistics department, the international cooperation department, the FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 116 prosecutor training school, the Gender Based Violence director, the districts attorneys for Santiago, Santo Domingo West and the National District), the Community Justice Houses, the Judiciary (inspector general, judicial school, technical director of the Judiciary and human resources department), the National Public Defense, the inter-institutional case tracking-case management roundtable, the Ministry of Interior and Police and the DNP. IT will also interview municipal authorities, and non-governmental actors (human rights organizations and other CSOs, universities and research institutions, lawyers, and journalists), illustrative examples are INTEC (mesa de Gobernanza), and PUCAMAIMA University (Center for alternative dispute resolution). These KIIs will be conducted in the different wards of Santo Domingo, Santiago and other cities in the Duarte corridor. The Board of the Community Justice Houses in Moca and La Vega are also good examples of stakeholders to interview. ● Third, the evaluation team will design and conduct small focus group discussions with four stakeholder groups: public prosecutors; DNP officers; trial lawyers; and civil society activists. The FGDs will be based on structured discussion guides and facilitated by evaluation team members. ● Last, if possible, the evaluation will incorporate a short email survey of selected justice sector officials (Public Ministry, Judiciary, Public Defense, Ministry of Interior) focusing on the evaluation questions about the effectiveness and sustainability of the CJSS interventions. The feasibility of this component of data collection will be evaluated with CJSS and the Public Ministry’s National School. ● Other donors such as the Spanish Cooperation, the World Bank, the European Union and the UNDP. Team Composition The evaluation team will be composed of the following positions: Team Leader: ● A graduate degree (LLD, PhD, MA, MSC or similar) in a related field, such as Law, Political Science, Criminology, Sociology or similar. ● At least 10 years of professional experience in designing, managing or evaluating international development programs in the fields of rule of law, justice sector reform, and policing. ● Significant experience as Team Leader of evaluations of donor funded international development programs, especially USAID projects and activities ● Experience evaluating rule of law and/or justice sector reform programs ● Practical skills in the use of mixed methods for evaluations ● Proven experience in managing evaluation teams and analytical and report drafting skills ● Fluent English and Spanish Team Member I: ● A graduate degree (PhD, MA, MSC or similar) in a related field, such as Law, Political Science, Criminology, Sociology or similar. ● At least 5 years of professional experience in designing, managing or evaluating international development programs. Ideally, specialized in either justice sector or police/security. FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 117 ● Experience as Team Member of evaluations of donor funded international development programs, especially USAID projects and activities ● Practical skills in the use of mixed methods for evaluations ● Fluent English and Spanish Team Member II: ● Same requirements as Team Member I. In this case, the team member will be the MEL Platform Chief of Party, who will provide focused, limited support for KIIs and FGDs relating to the Civil Society Action for Accountable Justice and Security activity implemented by PC. Team Member III: ● Same requirements as Team Member I. In this case, the team member will be the MEL Platform Monitoring and Evaluation Specialist, who will provide focused, limited support for the conduct of FGDs. Logistics Coordinator: ● This position will support scheduling of field work, and travel logistics. It will be filled by the MEL Platform Administrative Coordinator. Scheduling and Field Work The evaluation will take place between February and April 2018 and will require up to 45 days of effort during that period, including three weeks of field work in the Dominican Republic. The notional schedule is as follows: ● February, week 1: finalization of team members ● February, weeks 2 and 3: document review and field work planning ● February, week 4: mobilization to Santo Domingo, inbrief and team meeting ● March, weeks 1, 2 and 3: KIIs and FGDs and distribution of email survey ● March, week 4: team meeting for data analysis and outbrief ● April, weeks 1 and 2: report drafting ● April, week 3: submission of report draft to USAID FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 118 ANNEX G. CONFLICT OF INTEREST DECLARATIONS FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 119 FINAL REPORT – WHOLE-OF-PROJECT MID-TERM PERFORMANCE EVALUATION CRIMINAL JUSTICE SYSTEMS STRENGTHENED PROJECT 120