PROMOTING JUSTICE PROGRAM FINAL MONITORING & EVALUATION REPORT JUNE 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International (MSI), A Tetra Tech Company. FINAL MONITORING AND EVALUATION REPORT PROMOTING JUSTICE PROJECT JUNE 2019 Management Systems International Corporate Offices 200 12th Street, South Arlington, VA 22202 USA Tel: + 1 703 979 7100 Contracted under AID-523-C-14-00003 USAID/Mexico Promoting Justice Project DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/MEXICO PROMOTING JUSTICE PROJECT i FINAL MONITORING & EVALUATION REPORT TABLE OF CONTENTS ACRONYMS...........................................................................................................................................III KEY RESULTS SUMMARY ...................................................................................................................1 TASK 1: INCREASED LEGITIMACY OF THE CRIMINAL JUSTICE SYSTEM.........................2 1.1 REFORMED JUSTICE SECTOR INSTITUTIONS ...........................................................................2 1.1.1 # OF TARGETED JSIS WITH PROCEDURES AND STRUCTURES ALIGNED TO THE NCPC AND SECONDARY LEGISLATION ........................................................................2 1.2 PRE-TRIAL SERVICES ............................................................................................................................3 1.2.1 PERCENT OF RECOMMENDATIONS OF PRE-TRIAL DETENTION BY PTS UNIT OPERATORS .............................................................................................................................4 1.2.2 RATE OF COMPLIANCE WITH PRE-TRIAL MEASURES...............................................5 CONTEXT INDICATOR: PRE-TRIAL DETENTION RATES IN 2018 ..................................8 1.3 CRIMINAL CASE DISPOSITION RATES....................................................................................... 11 1.3.1 CRIMINAL CASE DISPOSITION RATES IN TARGETED COURTS ......................... 12 CONTEXT INDICATORS: ENJUICIAMIENTO COURTS AND GENERAL EFFECTIVENESS.................................................................................................................................. 15 CONTEXT INDICATOR: DISAGGREGATION OF TYPES OF CASE RESOLUTIONS 17 1.4 PROSECUTION RATES..................................................................................................................... 19 1.4.1 PROSECUTION RATES IN TARGETED INVESTIGATION AND PROSECUTION UNITS ................................................................................................................................................... 19 CONTEXT INDICATOR: HOMICIDE CRIME INCIDENCE ................................................ 23 1.4.2 % OF PLEA AGREEMENTS IN TARGETED SPECIALIZED INVESTIGATION UNITS ................................................................................................................................................... 25 1.4.3 PERCENTAGE OF CASES WITH SENTENCES GREATER THAN 3 YEARS ......... 28 1.5 EFFECTIVENESS OF PUBLIC DEFENSE ........................................................................................ 31 1.5.1 RATE OF EFFECTIVE DEBATES EXECUTED BY TARGETED PUBLIC DEFENDERS........................................................................................................................................ 31 CONTEXT INDICATOR: ACCELERATED PROGRAM FOR IMPROVED PERFORMANCE OF PDS................................................................................................................ 33 TASK 2: INCREASED CAPACITY FOR MONITORING AND EVALUATION OF CRIMINAL JUSTICE SYSTEM OPERATION .................................................................................34 2.1 MONITORING PERFORMANCE.................................................................................................... 34 2.1.1 NUMBER OF TARGETED JSIS THAT HAVE QUALITY CONTROLS TO MONITOR AND MEASURE PERFORMANCE.......................................................................... 34 USAID AND FOREIGN ASSISTANCE INDICATORS ...............................................................36 ANNEX I: LAWS ALIGNED WITH THE ACCUSATORY SYSTEM PASSED DURING FY 2015 TO 2019 .......................................................................................................................................38 ANNEX II: UPDATED PERFORMANCE MONITORING PLAN (PMP) ................................41 ANNEX III: UPDATED PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS) .......47 ANNEX IV: METHODOLOGICAL NOTE....................................................................................65 ANNEX V: TOTAL PROJUST TRAINEES FOR ALL YEARS.....................................................67 USAID/MEXICO PROMOTING JUSTICE PROJECT ii FINAL MONITORING & EVALUATION REPORT INDEX OF TABLES AND FIGURES Table 1: Justice Sector Institutions with Procedures and Structures Aligned to the NCPC and Secondary Legislation....................................................................................................................................3 Table 2: Recommendations of Pretrial Detention ..........................................................................................5 Table 3: Pretrial Detention Rates, 2014–2018, for Adults Prosecuted for Nonserious Crimes........8 Table 4: Pretrial Detention Rates 2016-2018 for Juveniles........................................................................ 10 Table 5: Case Disposition Rates in Control Courts, 2014-2018.............................................................. 14 Table 6: General Case Disposition Rates, 2014–2018................................................................................ 16 Table 7: Case Disposition Rates in Enjuiciamiento Courts, 2014–2018................................................. 17 Table 8: Types of Resolutions in Control Courts, 2014-2018.................................................................. 18 Table 9: Homicide Prosecution Rates Before and During PROJUST’s support ................................... 20 Table 10: Homicide incidence per subtype of crime, 2018........................................................................ 25 Table 11: Percentage of Convictions Obtained in Plea Bargaining. .......................................................... 26 Table 12: Percentage of Convictions Obtained in Trial. ............................................................................. 27 Table 13: Sentencing Length for Convictions in Plea Bargaining and in Trial, 2018 ........................... 30 Table 14: Rate of Effective Debates by Public Defenders .......................................................................... 32 Table 15: Accelerated Program for Improved Performance of PDs, per state..................................... 33 Table 16: Progress on Establishment of M&E Units as of FY 2019.......................................................... 35 Figure 1. Project Coverage Map with Results per Target State...................................................................1 Figure 2: Rates of Compliance with Precautionary Measures and Conditional Release Measures for Adults, 2015–2018.........................................................................................................................................6 Figure 3: Rates of Compliance with Precautionary Measures and Conditional Release Measures for Juveniles, 2015–2018.....................................................................................................................................7 Figure 4: Pretrial Detention Rates, 2014–2018, for Adults Prosecuted for Nonserious Crimes......9 Figure 5: Pretrial Detention Rates, 2014–2018, for Juveniles Prosecuted for Nonserious Crimes. 11 Figure 6: Case Flow Through Control and Enjuiciamiento Courts ......................................................... 13 Figure 7: Case Disposition Rate Trends per District.................................................................................. 13 Figure 8: Indictments 2015–2018, Analysis of Cases Through Time....................................................... 21 Figure 9: Evolution of Prosecution Rates for a Given Cohort of Homicide Cases............................ 23 Figure 10: Homicide Incidence, 2015–2018................................................................................................... 24 Figure 11: Plea Bargain Rate, Baseline vs. Final Result................................................................................. 28 USAID/MEXICO PROMOTING JUSTICE PROJECT iii FINAL MONITORING & EVALUATION REPORT ACRONYMS AG Attorney General AJC Alternative Justice Center CSO Civil Society Organization FY Fiscal Year JSI Justice Sector Institution M&E Monitoring and Evaluation NCPC National Criminal Procedure Code PD Public Defender PROJUST Promoting Justice Project PTS Pre-Trial Services SOP Standard Operating Procedure USAID United States Agency for International Development USAID/MEXICO PROMOTING JUSTICE PROJECT 1 FINAL MONITORING & EVALUATION REPORT KEY RESULTS SUMMARY The Promoting Justice Project (PROJUST) was a USD $68.2 million, five-year initiative funded by the U.S. Agency for International Development (USAID) to provide comprehensive technical assistance to Mexican state and federal authorities to support the effective implementation and consolidation of the transition from an inquisitorial to an accusatorial criminal justice system. This Final Monitoring and Evaluation Report provides final results on project indicators and provides data and context around these gains. Figure 1 provides an overview of PROJUST coverage in target states, as well as a selection of high￾level results achieved under key project indicators in 2018. FIGURE 1. PROJECT COVERAGE MAP WITH RESULTS PER TARGET STATE USAID/MEXICO PROMOTING JUSTICE PROJECT 2 FINAL MONITORING & EVALUATION REPORT TASK 1: INCREASED LEGITIMACY OF THE CRIMINAL JUSTICE SYSTEM 1.1 REFORMED JUSTICE SECTOR INSTITUTIONS Targeted justice sector institutions (JSIs) have reformed structures, procedures and regulations to align with the National Criminal Procedure Code (NCPC) and secondary legislation, following identified best practices. PROJUST collaborates with state JSIs to codify and institutionalize the reforms necessary to align with the NCPC. In addition to supporting the alignment of organic regulations and other secondary laws into the framework of the accusatory criminal justice reform, PROJUST supports the development and implementation of institutional protocols, manuals and procedures to further operationalize the legislation. This work supports consolidation of the new criminal justice system, responding directly to a need to institutionalize structural and operational changes promoted under PROJUST’s various technical areas that will carry into the future. The single indicator under Result 1.1 does not measure strictly legal reforms, but rather is oriented toward reformed structures. 1.1.1 # OF TARGETED JSIs WITH PROCEDURES AND STRUCTURES ALIGNED TO THE NCPC AND SECONDARY LEGISLATION Definition: Total JSIs in targeted states that modify or create procedures, structures and regulations (e.g., organic regulations, organizational manuals, operating manuals standard operating procedures [SOP] or protocols, etc.). This indicator considers JSIs that: 1) Have modified their structure or created a new structure (for Pre-Trial Services Units); 2) Are operating according to new processes; and 3) Have an operating organizational manual or procedure manual in use (although not necessarily officially approved, but in use) or approved organic regulations. The institution needs to meet the three requisites for being counted toward this indicator with PROJUST support to achieve at least one of the requisites. Unit: Justice sector institutions. Disaggregated by: State, JSI; type of procedure or regulation. Type of indicator: Output Key data: Baseline Life of Project Target Final Result 9 20 22 A total of 22 JSIs have modified or created procedures, structures and regulations in line with the accusatory system with PROJUST support since 2015, including four in fiscal year (FY) 2019. Collaboration at the state level in 2019 was equally divided among state court and Attorneys General’s (AG) Offices, this being focused largely on initiatives to transition to an autonomous AG USAID/MEXICO PROMOTING JUSTICE PROJECT 3 FINAL MONITORING & EVALUATION REPORT Office (i.e., a Fiscalía instead of a Procuraduría). Table 1 provides an overview of PROJUST results in these efforts per fiscal year. TABLE 1: JUSTICE SECTOR INSTITUTIONS WITH PROCEDURES AND STRUCTURES ALIGNED TO THE NCPC AND SECONDARY LEGISLATION State Institution Baseline FY 2015 – 2016 Baja California Sur Pre-Trial Services Unit for Adults Oaxaca Pre-Trial Services Unit for Adults Nuevo Leon Pre-Trial Services Unit for Adults Nuevo Leon Pre-Trial Services Unit forJuveniles Queretaro Pre-Trial Services Unit for Adults Queretaro Pre-Trial Services Unit forJuveniles San Luis Potosi Pre-Trial Services Unit for Adults San Luis Potosi Pre-Trial Services Unit for Juveniles Zacatecas Pre-Trial Services Unit for Adults FY 2017 Coahuila State Court Tabasco Attorney General’s Office Coahuila Pre-Trial Services Unit for Adults Sonora Pre-Trial Services Unit for Adults FY 2018 Chihuahua Pre-Trial Services Unit Zacatecas Attorney General’s Office San Luis Potosi Attorney General’s Office Zacatecas State Court Nuevo Leon Public Defender’s Office FY 2019 Chihuahua State Court Nayarit State Court Nayarit Attorney General’s Office Sonora Attorney General’s Office Total 22 justice sector institutions 1.2 PRE-TRIAL SERVICES Targeted state justice system actors reduce use of pre-trial detention, especially for youth and non-violent defendants. PROJUST reports on two indicators for work with Pre-Trial Services (PTS) Units: the rate at which PTS operators recommend pre-trial detention (seeking a reduction), and the rate of compliance by defendants with precautionary measures while under supervision (seeking an increase). In addition, PROJUST reports on overall pre-trial detention rates, although this indicator is no longer contractual and is provided here for context only. USAID/MEXICO PROMOTING JUSTICE PROJECT 4 FINAL MONITORING & EVALUATION REPORT To uphold human rights and reduce overcrowding in detention facilities, it is essential to limit the use of pretrial detention to cases where it is appropriate. Alternatives to pre-trial detention, such as precautionary measures, allow eligible defendants to continue working, participate in treatment programs, attend classes and care for their families while they await trial. This is more cost-effective and it keeps nonviolent and first-time offenders, and those who may later be proven innocent, out of the prison system. 1.2.1 PERCENT OF RECOMMENDATIONS OF PRE-TRIAL DETENTION BY PTS UNIT OPERATORS Definition: The operator that this indicator refers to is the risk evaluator that recommends precautionary measures to be imposed according to the risks detected. This indicator is calculated by dividing the total number of recommendations of pre-trial detention by the total number of precautionary measures recommendations issued. Unit: Recommendations. Disaggregated by: State. Type of indicator: Outcome Key Data: Baseline States Measured Life of Project Target Final Result States Measured 60.0% Coahuila 50.0% 49.3% Coahuila This indicator measures the decrease in recommendations of pre-trial detention specifically for crimes in which detention is not mandatory.1 A lower percentage suggests a greater capacity of PTS officers to collect and evaluate risk data and develop appropriate precautionary measures as alternatives to pre-trial detention in cases of low risk. Currently, most of the PTS Units focus their efforts on evaluating and supervising. The evaluation process ends with the production of a technical opinion, which presents all relevant information and categorizes it into a level of risk, considering both the chance that defendants may attempt to flee prior to completion of their trial and any risk they may pose to the victim(s), witness(es) or the community in question. However, PTS Units generally do not issue technical opinions that explicitly recommend specific precautionary measures for each case. Coahuila’s was the first PTS Unit that started issuing such recommendations as part of their technical opinions. Identifying these recommendations as a best practice in PTS, PROJUST started to work with PTS units in other states to incorporate recommendations into their technical opinions with the expectation that the practice would be readily adopted. However, units were largely unreceptive to this suggestion. In some cases, this was due to pressure from AG Offices and the State Secretariats of Public Security to impose pre-trial detention regardless of risk levels, in an attempt to show a hard stance against criminals. As a result, PROJUST was unable to measure this indicator in other states, absent recommendations either for or against pre-trial detention. 1 Serious crimes, such as homicide, femicide, kidnapping or rape, have mandatory pre-trial detention, according to the National Criminal Procedure Code. USAID/MEXICO PROMOTING JUSTICE PROJECT 5 FINAL MONITORING & EVALUATION REPORT Coahuila’s baseline data is for the 2016 calendar year and measures recommendations in cases for which charges were pressed. Within two years of intensive work, a significant drop can be observed, of 10.7 percentage points, or 17.8 percent, relative to baseline. TABLE 2: RECOMMENDATIONS OF PRETRIAL DETENTION District, State Year Recommendations of pre-trial detention (A) Total precautionary measures recommendations (B) Rate % (A/B) Saltillo, Coahuila 2016 (Baseline) 39 65 60.0% 2017 40 73 54.8% 2018 532 1,080 49.3% 1.2.2 RATE OF COMPLIANCE WITH PRE-TRIAL MEASURES Definition: The compliance rate measures compliance by defendants supervised by PTS units with precautionary measures and conditions for release imposed by courts. The rate is calculated by dividing the total number of defendants in full compliance divided by the total number of defendants supervised by the unit (active cases) in the corresponding year. Unit: Compliance rate reported by target PTS units. Disaggregated by: State. Type of indicator: Outcome. Key Data: Original Baseline States Measured Life-of-Project Target 70.4% Baja California Sur Michoacan Oaxaca Queretaro Sonora Tlaxcala 85.0% Adjusted Baseline States Measured Final Result 77.1% Baja California Sur Coahuila Nuevo Leon San Luis Potosi Zacatecas 80.6% Defendants granted precautionary measures instead of pre-trial detention must accept and comply with those measures under supervision by the PTS Unit. This indicator measures the extent to which defendants successfully comply with the precautionary measures (and conditions for release, in cases of pre-trial diversion) that a judge imposes. A high rate of compliance helps legitimize alternatives to both pre-trial detention and prison sentences by assuring the public that it neither impedes justice nor exposes victims or communities to elevated risks via recidivism. USAID/MEXICO PROMOTING JUSTICE PROJECT 6 FINAL MONITORING & EVALUATION REPORT Over the past few years of measuring this indicator, first as a context indicator and now as a contractual indicator, PROJUST has changed its geographic focus, resulting in the replacement of some states that the project initially measured but no longer supports; new states have received support in subsequent years. PROJUST’s target for this indicator was established at 85 percent compliance or higher on average across all target states receiving support for PTS and achieved a final result of 80.6 percent. Figure 2 gives a more detailed breakdown of compliance rates per state. Most target states presented an increase during the last year of intervention (between 3 and 9 percentage points), except for Coahuila. Additionally, during 2018, Chihuahua (Morelos District) established a new PTS Unit with PROJUST’s support, reporting a 96.7 percent compliance rate. Because 2018 was the first and only year with measurements for this unit, it was not possible to set a baseline and it was not included in PROJUST’s final measurement of global average/final result against LOP target, however the yearly average for just 2018 would have risen to 83.3 percent, were Chihuahua included. FIGURE 2: RATES OF COMPLIANCE WITH PRECAUTIONARY MEASURES AND CONDITIONAL RELEASE MEASURES FOR ADULTS, 2015–2018 End-of-Project Result 80.6% Note: Measurements consider the following states and districts: Coahuila (Saltillo), Baja California Sur (baseline for Comondú, first district to open a PTS Unit; subsequent measurements include La Paz, Mulege, Los Cabos and Santa Rosalia), Nuevo Leon (statewide), San Luis Potosí (San Luis Potosí) and Zacatecas (statewide). 81.3% 84.3% 82.8% 90.4% 82.3% 88.8% 93.4% 88.7% 84.9% 44.3% 82.1% 83.3% 93.7% 77.6% 91.8% 38.7% 86.0% 90.4% 96.1% 80.6% Baja California Sur Coahuila Nuevo León San Luis Potosí Zacatecas Yearly average 2015 2016 2017 2018 USAID/MEXICO PROMOTING JUSTICE PROJECT 7 FINAL MONITORING & EVALUATION REPORT PROJUST also facilitated some activities for PTS Units that serve juveniles. These units reported an average increase of 2.2 percentage points. FIGURE 3: RATES OF COMPLIANCE WITH PRECAUTIONARY MEASURES AND CONDITIONAL RELEASE MEASURES FOR JUVENILES, 2015–2018 End-of-Project Result 91.4% Note: Measurements consider the following states and districts: Chihuahua (Chihuahua and Cd. Juárez, Nuevo Leon (statewide) and San Luis Potosí (statewide). Chihuahua’s 2018 reported only from 9th April (begin support) to September 16 (available information). 84.3% 94.1% 89.2% 92.0% 82.3% 100.0% 91.4% Chihuahua Nuevo León San Luis Potosí Yearly average 2017 2018 USAID/MEXICO PROMOTING JUSTICE PROJECT 8 FINAL MONITORING & EVALUATION REPORT CONTEXT INDICATOR: PRE-TRIAL DETENTION RATES IN 2018 Trends are not stable; important externalities, such as political context, citizen perception and crime incidence trends, affect this indicator. TABLE 3: PRE-TRIAL DETENTION RATES, 2014–2018, FOR ADULTS PROSECUTED FOR NONSERIOUS CRIMES2 End-of-Project Result 21.0% Indicator Baja California Coahuila Nuevo Leon Chihuahua San Luis Potosi Zacatecas Sonora Average 2014 Adults in pre-trial detention (N) 417 64 67.7% Adults under prosecution (NAP) 417 181 2014 Rate % (N/NAP) 100% 35.4% 2015 Adults in pre-trial detention (N) 136 6,468 29 86 1,221 53.7% Adults under prosecution (NAP) 186 10,366 89 284 1,745 2015 Rate % (N/NAP) 73.1% 62.4% 32.6% 30.3% 70.0% 2016 Adults in pre-trial detention (N) 4 264 2,257 6 119 274 26.3% Adults under prosecution (NAP) 35 1,198 7,611 37 609 467 2016 Rate % (N/NAP) 11.4% 22.0% 29.7% 16.1% 19.5% 58.7% 2017 Adults in pre-trial detention (N) 0 546 3,433 2,901 14 195 26.3% Adults under prosecution (NAP) 31 1,671 11,048 16,500 27 794 2017 Rate % (N/NAP) 0.0% 32.7% 31.1% 17.6% 51.9% 24.6% 2018 Adults in pre-trial detention (N) 3 352 2,958 3,608 30 170 21.0% Adults under prosecution (NAP) 67 3,996 11,263 13,811 64 1,263 2018 Rate % (N/NAP) 4.5% 8.81% 26.3% 26.1% 46.9% 13.5% *Note: Baja California Sur (Comondu Judicial District); Coahuila (statewide); Chihuahua (statewide); Nuevo Leon (statewide); San Luis Potosi (Matehuala and Rioverde); Zacatecas (statewide) and Sonora (Hermosillo). For Baja California Sur and San Luis Potosi, 2018 is the fourth measurement, as the baseline was established at 2014. For Nuevo Leon and Zacatecas, 2018 is the third measurement, as baseline was established at 2015. For Coahuila, 2018 is the second measurement. For Chihuahua, 2017 is the baseline and 2018 is the first measurement. 2 Crimes where pre-trial detention is not mandatory. USAID/MEXICO PROMOTING JUSTICE PROJECT 9 FINAL MONITORING & EVALUATION REPORT FIGURE 4: PRE-TRIAL DETENTION RATES, 2014–2018, FOR ADULTS PROSECUTED FOR NONSERIOUS CRIMES3 *Note: Baja California Sur (Comondu Judicial District); Coahuila (statewide); Chihuahua (statewide); Nuevo Leon (statewide); San Luis Potosi (Matehuala and Rioverde); Zacatecas (statewide) and Sonora (Hermosillo). For Baja California Sur and San Luis Potosi, 2018 is the fourth measurement as baseline was established at 2014. For Nuevo Leon and Zacatecas, 2018 is the third measurement as baseline was established at 2015. For Coahuila, 2018 is the second measurement. For Chihuahua, 2017 is the baseline and 2018 the first measurement. 3 Crimes where pre-trial detention is not mandatory. 100.0% 35.4% 67.7% 73.1% 62.4% 32.6% 30.3% 70.0% 53.7% 11.4% 22.0% 29.7% 16.2% 19.5% 58.7% 26.3% 32.7% 31.1% 17.6% 51.9% 24.6% 26.3% 4.5% 8.8% 26.3% 26.1% 46.9% 13.5% 21.0% Baja California Coahuila Nuevo León Chihuahua San Luis Potosí Zacatecas Sonora Yearly Average 2014 2015 2016 2017 2018 USAID/MEXICO PROMOTING JUSTICE PROJECT 10 FINAL MONITORING & EVALUATION REPORT TABLE 4: PRETRIAL DETENTION RATES 2016-2018 FOR JUVENILES End-of-Project Result 40.7% Indicator Morelos Nuevo Leon Chihuahua Average 2015 (baseline) Juveniles in pre-trial detention (N) 24 905 464.5 Juveniles under prosecution (NAP) 170 974 572 2015 Rate % (N/NAP) 14.1% 92.9% 54% 2016 Juveniles in pre-trial detention (N) 19 172 95.5 Juveniles under prosecution (NAP) 52 216 134 2016 Rate % (N/NAP) 36.5% 79.6% 58% 2017 Juveniles in pre-trial detention (N) 21 205 43 90 Juveniles under prosecution (NAP) 61 252 405 293 2017 Rate % (N/NAP) 34.4% 81.3% 10.61% 42.1% 2018 Juveniles in pre-trial detention (N) 81 32 57 Juveniles under prosecution (NAP) 135 150 143 2018 Rate % (N/NAP) 60.0% 21.3% 40.7% *Note: Morelos, Chihuahua and Nuevo Leon are at a state level (statewide). The baseline for Morelos and Nuevo León was established in 2015. For Coahuila, 2017 is the baseline and 2018 is the first measurement. Nuevo Leon saw a minor increase (almost 2 percentage points). The PTS unit for juveniles in San Luis Potosi has not provided data on the pre-trial detention rates indicator since 2017. Lack of interest and other political reasons prompted PROJUST to cease further work on this state for PTS Units (for both juveniles and adults). USAID/MEXICO PROMOTING JUSTICE PROJECT 11 FINAL MONITORING & EVALUATION REPORT FIGURE 5: PRE-TRIAL DETENTION RATES, 2014–2018, FOR JUVENILES PROSECUTED FOR NONSERIOUS CRIMES4 *Note: Morelos, Nuevo León and Chihuahua results are statewide. 1.3 CRIMINAL CASE DISPOSITION RATES Increase criminal case disposition rates in targeted courts. Case disposition, defined as the sentencing or other final settlement of a criminal case, is an important means of measuring how State Courts (and the wider state justice system) resolve cases, relative to the number of cases they receive. In 2016, this indicator’s definition was modified to expand the types of case resolutions counted. The baseline (2014) and first measurement (2015) were revised to reflect this and maintain consistency with the subsequent second and third measurements (2016 and 2017). PROJUST’s target states measured for this indicator have also shifted to reflect the states receiving direct support for State Courts. For the 2017 calendar year, PROJUST ceased support to Baja California and Morelos, 5 but in 2018 it started support to Chihuahua and Nuevo Leon. Baseline for these states is provided below. Additional data collected from targeted State Courts is also provided in this section as further context. 4 Crimes where pre-trial detention is not mandatory. 5 Both courts showed little interest in moving forward with activities. For Morelos, almost all 2017 remained without significant intervention, due to the process of appointing a new State Court President. In Baja California, on the other hand justices and judges remained reluctant to make changes in their internal processes such as the standardization of criteria or a new method to schedule hearings. 14.1% 92.9% 53.5% 36.5% 79.6% 58.1% 10.6% 34.4% 81.3% 57.9% 10.7% 21.3% 16.0% Chihuahua Morelos Nuevo León Yearly Average 2015 2016 2017 2018 USAID/MEXICO PROMOTING JUSTICE PROJECT 12 FINAL MONITORING & EVALUATION REPORT 1.3.1 CRIMINAL CASE DISPOSITION RATES IN TARGETED COURTS Definition: The case disposition rate is calculated as: the number of resolved or closed cases divided by the number of new cases in the relevant year. Case disposition in targeted courts are actions taken as the result of an appearance in court by a defendant where outcomes can include: non-probable cause ruling (no vinculación a proceso), dismissal, acquittal, conviction (in oral trials and as a result of plea bargaining), pre-trial diversion and settlement (between victim and defendant). Unit: Criminal case disposition rate reported per court. Disaggregated by: state, court type of resolution (e.g., non-probable cause ruling, dismissal, acquittal, conviction, pre-trial diversion, settlement); type of crime (e.g., domestic violence, homicide, rape, kidnapping). Type of indicator: Outcome Key Data: Original Baseline States Measured Life-of-Project Target 69.3% Baja California Morelos Zacatecas 70.0% Adjusted Baseline States Measured6 Final Result 57.7% Chihuahua Coahuila Nayarit Nuevo Leon San Luis Potosi Zacatecas 71.3% Data collected for this indicator corresponds to full calendar years and differentiates between the two types of courts within the State Court: juzgados de control (“Control Courts”) and tribunales de enjuiciamiento (“Enjuiciamiento Courts,” or prosectution courts). Control Courts handle the initial and intermediate hearings in criminal cases, during which the majority of crimes are resolved. More serious crimes that proceed to oral trials are transferred to the Enjuiciamiento Courts. Except for in Coahuila, PROJUST only supported Control Courts, thus case disposition rates reported below reflect Control Courts only. Coahuila’s case disposition rate remains unchanged, even when considering the Enjuiciamiento Court (related data is provided under the context indicators below). 6 Support for Morelos and Baja California ceased in the middle of 2017. USAID/MEXICO PROMOTING JUSTICE PROJECT 13 FINAL MONITORING & EVALUATION REPORT FIGURE 6: CASE FLOW THROUGH CONTROL AND ENJUICIAMIENTO COURTS Note: A court order denotes that a case was not legally resolved; rather, it was was diverted to the other court. FIGURE 7: CASE DISPOSITION RATE TRENDS PER DISTRICT 68.0% 68.9% 68.7% 66.8% 64.8% 52.8% 83.3% 82.1% 88.7% 44.1% 44.6% 42.2% 75.7% 76.0% 67.2% 62.9% 37.8% 66.7% 2014 2015 2016 2017 2018 Zacatecas, Zacatecas Saltillo, Coahuila San Luis Potosí, San Luis Potosí Chihuahua, Chihuahua Tepic, Nayarit Monterrey, Nuevo León USAID/MEXICO PROMOTING JUSTICE PROJECT 14 FINAL MONITORING & EVALUATION REPORT TABLE 5: CASE DISPOSITION RATES IN CONTROL COURTS, 7 2014-2018 End-of-Project Result 71.3% CONTROL COURTS Year District, State New Cases in the Relevant Year (A) Number of Resolved Cases (B) Rate % (B/A) Yearly Average 20148 Zacatecas, Zacatecas (baseline) 375 255 68.0% 68.0% 2015 Saltillo, Coahuila (baseline) 195 104 53.3% 63.6% Zacatecas, Zacatecas 379 261 68.9% 2016 Saltillo, Coahuila 762 636 83.5% 75.1% San Luis Potosí, San Luis Potosí (baseline) 127 56 44.1% Zacatecas, Zacatecas 390 268 68.7% 2017 Saltillo, Coahuila 1,794 1,475 82.2% 57.2% Chihuahua, Chihuahua (baseline) 2,903 2,197 75.7% Tepic, Nayarit (baseline) 1,227 824 67.2% Monterrey, Nuevo León (baseline) 5,847 2,211 37.9% San Luis Potosí, San Luis Potosí 484 216 44.6% Zacatecas, Zacatecas 858 573 66.78% 2018 Saltillo, Coahuila 2,758 2,445 88.7% 71.3% Chihuahua, Chihuahua 2,644 2,010 76.0% Tepic, Nayarit 2,283 1,437 62.9% Monterrey, Nuevo León 3,963 2,645 66.7% San Luis Potosí, San Luis Potosí 625 264 42.2% Zacatecas, Zacatecas 867 562 64.8% Efficiency gains spiked in two main districts: Saltillo (Coahuila) and Monterrey (Nuevo Leon). A more discrete improvement occurred in Chihuahua. Saltillo’s remarkable improvement from 2015 to 2016 (35.4 percentage points) was sustained through the remaining years of the project, even though workload nearly doubled while the number of judges and courtrooms remained unchanged. Monterrey showed an important increase of 28.8 percentage points and Chihuahua a modest 0.3. 7 For Control Courts, a court order is considered a resolved case. 8 Cuernavaca (Morelos) and Mexicali (Baja California) also received support in 2015 (with a 2014 baseline); however, support ceased by the end of 2016 due to political reasons. USAID/MEXICO PROMOTING JUSTICE PROJECT 15 FINAL MONITORING & EVALUATION REPORT Control courts are most affected by activities regarding the implementation of PROJUST’s block￾scheduling methodology, standardized judging criteria and optimized administrative processes. Tepic (Nayarit), Zacatecas (Zacatecas), and San Luis Potosi presented a small decrease relative to baseline of 4.3 percent, 3.2 percent and 1.9 percent, respectively. However, these courts received more than twice the number of cases (five times as many in San Luis Potosi), signaling improved efficiency relative to workload. CONTEXT INDICATORS: ENJUICIAMIENTO COURTS AND GENERAL EFFECTIVENESS In response to recommendations from PROJUST’s Midterm Evaluation, the project also measured case disposition for “general effectiveness in state courts” in 2017 and 2018 as a context indicator to monitor the overall performance of the State Court system, without differentiating between Control Courts and Enjuiciamiento Courts. While the disaggregated indicators for control and enjuiciamiento help identify specific bottlenecks in the administration of criminal justice, the general effectiveness measurement is intended to gauge the overall impact of PROJUST. Most of the impact is observed in Control, since more than 95 percent of the full workload is resolved under this court. Predictably, the efficiency rate drops slightly when adding the Enjuicimiento Court cases,9 as oral trials are more complex and thus lengthier, so it is likely that most cases did not reach a resolution within the one-year timeframe measured. (See Figure 6.) 9 Numbers are almost the same as in Control Courts. USAID/MEXICO PROMOTING JUSTICE PROJECT 16 FINAL MONITORING & EVALUATION REPORT TABLE 6: GENERAL CASE DISPOSITION RATES, 2014–2018 GENERAL EFFECTIVENESS Year District, State New Cases in the Relevant Year (A) Number of Resolved Cases (B) Rate % (B/A) Yearly Average 201410 Zacatecas, Zacatecas (baseline) 375 255 68.0% 68.0% 2015 Saltillo, Coahuila (baseline) 195 103 52.8% 63.4% Zacatecas, Zacatecas 379 261 68.9% 2016 Saltillo, Coahuila 762 635 83.3% 74.8% San Luis Potosí, San Luis Potosí (baseline) 127 56 44.1% Zacatecas, Zacatecas 390 266 68.2% 2017 Saltillo, Coahuila 1,794 1,472 82.1% 57.0% Chihuahua, Chihuahua (baseline) 2,903 2,170 74.8% Tepic, Nayarit (baseline) 1,227 824 67.2% Monterrey, Nuevo León (baseline) 5,847 2,229 38.1% San Luis Potosí, San Luis Potosí 484 214 44.2% Zacatecas, Zacatecas 858 567 66.1% 2018 Saltillo, Coahuila 2,758 2,446 88.7% 71.0% Chihuahua, Chihuahua 2,644 1,985 75.1% Tepic, Nayarit 2,283 1,437 62.9% Monterrey, Nuevo León 3,963 2,642 66.7% San Luis Potosí, San Luis Potosí 625 259 41.4% Zacatecas, Zacatecas 867 555 64.0% 10 Cuernavaca (Morelos) and Mexicali (Baja California) also received support during 2015 (with a 2014 baseline); however, support ceased by the end of 2016 due to political reasons. USAID/MEXICO PROMOTING JUSTICE PROJECT 17 FINAL MONITORING & EVALUATION REPORT TABLE 7: CASE DISPOSITION RATES IN ENJUICIAMIENTO COURTS, 2014–2018 ENJUICIAMIENTO COURTS Year District, State New Cases in the Relevant Year (A) Number of Resolved Cases (B) Rate % (B/A) Yearly Average 201411 Zacatecas, Zacatecas (baseline) 1 1 100.0% 100.0% 2015 Saltillo, Coahuila (baseline) 5 4 80.0% 85.7% Zacatecas, Zacatecas 2 2 100.0% 2016 Saltillo, Coahuila 5 4 80.0% 57.1% San Luis Potosí, San Luis Potosí (baseline) 0 0 - Zacatecas, Zacatecas 2 0 0.0% 2017 Saltillo, Coahuila 3 0 0.0% 60.7% Chihuahua, Chihuahua (baseline) 0 0 - Tepic, Nayarit (baseline) 0 1 - Monterrey, Nuevo León (baseline) 10 9 90.0% San Luis Potosí, San Luis Potosí 8 6 75.0% Zacatecas, Zacatecas 7 1 14.3% 2018 Saltillo, Coahuila 7 4 57.1% 34.6% Chihuahua, Chihuahua 31 3 9.7% Tepic, Nayarit 0 0 - Monterrey, Nuevo León 75 36 48.0% San Luis Potosí, San Luis Potosí 5 0 0.0% Zacatecas, Zacatecas 9 1 11.1% CONTEXT INDICATOR: DISAGGREGATION OF TYPES OF CASE RESOLUTIONS General theory and recommendations for an optimal functioning of State Courts suggest resolving most cases through alternatives to trial to unclog the system of less serious crimes. Trends and effectiveness in the use of alternative types of resolutions can be better understood with an analysis of the types of case resolutions. Most cases measured in case disposition rates were resolved as pre-trial diversion, which is considered reasonable given that most common crimes are nonserious and nonviolent, and can be negotiated or treated through alternatives to trial. 11 Cuernavaca (Morelos) and Mexicali (Baja California) also received support during 2015 (with a 2014 baseline); however, support ceased by the end of 2016 due to political reasons. USAID/MEXICO PROMOTING JUSTICE PROJECT 18 FINAL MONITORING & EVALUATION REPORT Non-probable cause and illegal detention are two types of resolutions that demonstrate the overall effectiveness of the justice system during initial hearings. PROJUST considers that a reasonable standard for probable cause is around 90% of all cases and supports improved performance of AG Offices to reduce the instance of non-probable cause rulings, as they can indicate either poor standards for arrests and investigations by police and prosecutors, or instances of impunity for genuinely guilty parties. PROJUST also provides targeted support to PDs to increase the effectiveness of their debates in cases of legitimate lack of probable cause and/or illegal detention, as part of their defense of due process. In 2018, only San Luis Potosi and Saltillo had increasing numbers of illegal detentions but had fairly good numbers for non-probable cause. This information is relevant for further analysis since most of the cases resulting in this solution might be the consequence of poor investigations or detention processes. TABLE 8: TYPES OF RESOLUTIONS IN CONTROL COURTS, 2014-2018 Timeframe District, State Illegal Detention Non￾Probable Cause Restitution Agreement Plea Agreement Dismissal Pre-trial Diversion Court Order Suspended Cases12 2014 Zacatecas 0.8% 3.9% 5.1% 11.0% 32.5% 40.0% 0.4% 6.3% 2015 Coahuila 1.9% 20.2% 2.9% 11.5% 38.5% 17.3% 4.8% 2.9% Zacatecas 2.3% 5.7% 4.2% 7.3% 31.4% 37.2% 0.8% 11.1% 2016 Coahuila 3.8% 4.7% 1.7% 5.3% 21.4% 60.7% 0.8% 1.6% San Luis Potosí 7.1% 19.6% 12.5% 7.1% 8.9% 42.9% 0.0% 1.8% Zacatecas 2.2% 10.8% 3.7% 3.4% 33.2% 34.0% 0.7% 11.9% 2017 Coahuila 4.5% 5.8% 8.6% 2.8% 0.7% 77.1% 0.1% 0.3% Chihuahu a 8.0% 12.2% 12.6% 12.4% 3.8% 40.7% 1.2% 9.2% Nayarit 4.20% 14.30% 1.10% 22.30% 14.70% 42.40% 1.00% 0.00% Nuevo León 6.5% 8.7% 20.7% 4.6% 13.4% 37.4% 0.0% 8.6% San Luis Potosí 11.57% 14.35% 8.80% 4.63% 16.67% 39.35% 3.70% 0.93% Zacatecas 1.18% 6.94% 7.28% 4.57% 34.52% 36.21% 1.18% 8.12% 2018 Coahuila 27.4% 6.3% 2.7% 1.3% 0.5% 61.1% 0.1% 0.4% Chihuahu a 5.9% 13.3% 12.9% 15.6% 5.5% 45.4% 1.4% 0.0% Nayarit 1.5% 8.1% 1.3% 23.7% 9.5% 55.9% 0.0% 0.0% Nuevo León 11.6% 8.9% 12.6% 7.4% 9.5% 47.6% 1.5% 0.9% San Luis Potosí 19.7% 17.8% 8.3% 4.2% 13.6% 33.3% 1.9% 1.1% Zacatecas 0.2% 7.8% 19.6% 3.7% 10.9% 53.9% 1.4% 2.5% *Note: Only for capital districts. Coahuila (Saltillo); Chihuahua (Chihuahua); Nayarit (Tepic); Nuevo León (Monterrey); San Luis Potosí (San Luis Potosí); Zacatecas (Zacatecas) 12 According to the National Criminal Procedure Code (Art. 331), a case can be considered suspended under certain circumstances such as temporary mental illness. USAID/MEXICO PROMOTING JUSTICE PROJECT 19 FINAL MONITORING & EVALUATION REPORT 1.4 PROSECUTION RATES Increase prosecution rates for targeted state-level prosecutors. PROJUST monitors the impact of its work with AG Offices by tracking the performance of Specialized Investigation and Prosecution Units for Homicides. Three indicators for this target group of units are measured: prosecution rates, plea bargain rates and the rate at which cases have sentences of greater than three years. As AG Offices do not record this data, PROJUST manually collects data for these indicators, reviewing casefiles individually to ensure data quality. Because of the sensitive nature of homicide casefiles, PROJUST is able to collect data for only units receiving active project assistance. For this reason, data is unavailable for units that did not receive direct support throughout the duration of the project, and baselines were adjusted to reflect units that could be measured through the final calendar year of 2018. 1.4.1 PROSECUTION RATES IN TARGETED INVESTIGATION AND PROSECUTION UNITS Definition: Annual prosecution rates for homicides are calculated by obtaining the total number of indictments (acusaciones), divided by the total number of investigations conducted by targeted investigation and prosecution units. Unit: Prosecution rate reported by investigation and prosecution units. Disaggregated by: Prosecution rates per targeted investigation and prosecution unit. Type of indicator: Outcome. Key Data: Original Baseline States Measured13 Life-of-Project Target 11.6% Baja California Tabasco Nayarit Zacatecas +20.0% Adjusted Baseline States Measured Final Result 3.8% Baja California Chihuahua Coahuila Nayarit Tabasco +8.79% Prosecution rates in targeted investigation and prosecution units were measured since project start￾up as the principal indicator for the work with AG Offices. By 2018, assistance was active in seven Homicide Units in the districts of Ciudad Juarez, Mexicali, Saltillo, Nayarit, Tijuana, Torreon and Villahermosa, with only two of those in existence since 2015. This indicator’s results increased an average of 8.79 percent over baseline across those target units. 13 Only specific districts: Mexicali (Baja California), Macuspana (Tabasco), Tepic (Nayarit) and Zacatecas (Zacatecas). USAID/MEXICO PROMOTING JUSTICE PROJECT 20 FINAL MONITORING & EVALUATION REPORT Reductions in arrests in flagrante, the rise of organized crime, and exponential growth in crime incidence generally have impacted prosecution rates more than growth in capacity of the target units. This resulted in less dramatic gains in prosecution rates than anticipated and in ways that were beyond the reach of project technical assistance. Homicides attributed to organized crime have a distinct methodology of investigation (one that is not part of PROJUST’s model for target units) and exponential growth in crime rates have outweighed the capacity of units to respond due to lack of personnel, as well as limited time and resources. Nayarit specifically was a significant outlier, facing an overwhelming increase in crime incidence, the highest rate of homicides attributed to organized crime, with only four prosecutors handling approximately 49 cases each, all of which resulted in a lower capacity to improve prosecution rates. This pulled the global average for this indicator down dramatically. Without Nayarit, PROJUST’s average increase in prosecution rates across the other six units would have been 44.6 percent. TABLE 9: HOMICIDE PROSECUTION RATES BEFORE AND DURING PROJUST’S SUPPORT14 End-of-Project Result +8.79% District, State Baseline Result as of 201815 Percentage Change Total # o 16 f Indictments (A) Total # of Cases Investigated (B) Prosecution Rates (A/B) Total # of Indictments (C) Total # of Cases Investigated (D) Prosecution Rates(C/D) Cd. Juárez, Chihuahua (2017) 36 609 5.9% 70 1,480 4.7% -20.0% Mexicali, Baja California (2015) 14 114 12.3% 58 487 11.9% -3.0% Nayarit (statewide) (2016) 21 75 28.0% 11 363 3.0% -89.2% Saltillo, Coahuila17 (2015) 0 25 0% 19 90 21.1% +21.1% Tijuana, Baja California (2017) 15 1,578 1.0% 76 3,717 2.0% +115.1% Torreon, Coahuila (2017) 9 95 9.5% 26 156 16.7% +75.9% Villahermosa, Tabasco (2017) 5 121 4.1% 11 226 4.9% +17.8% Total 100 2,617 3.82% 271 6,519 4.16% +8.79% 14 Baseline year changes for each city due to the program’s flexibility to incorporate new units. 15 Includes all cases, independently of the year the investigation began. 16 Based on percentual change formula: (final result-baseline)/baseline. 17 PROJUST’s activities started in the second half of 2016. USAID/MEXICO PROMOTING JUSTICE PROJECT 21 FINAL MONITORING & EVALUATION REPORT Results from prior years: District, State Results Total # of Indictments (A) Total # of Cases Investigated (B) Prosecution Rates (A/B) Mexicali, Baja California (2017) 33 370 8.91% Nayarit (statewide) (2017) 17 244 6.97% Saltillo, Coahuila (2017) 17 74 22.97% Mexicali, Baja California (2016) 22 230 9.56% Saltillo, Coahuila (2016) 9 47 19.14% While prosecution rates did not rise to meet the target, significant improvement in the performance of target units occur when considering absolute numbers. Significant increases in the number of cases investigated took place across the board, more than fourfold in Nayarit and Mexicali, reflecting the exponential growth in workloads these units faced. Even with the rise in homicide incidence, however, human resources within the units remained constant. The results show an important increase in the number of indictments achieved by units in most cases, demonstrating an increase in productivity that, while not enough to increase prosecution rates more significantly, is an important result to consider when assessing the impact of PROJUST’s technical assistance. Figure 8 further demonstrates the rise in indictments over the course of project support for all units except Nayarit. FIGURE 8: INDICTMENTS 2015–2018, ANALYSIS OF CASES THROUGH TIME 36 70 14 22 33 58 0 9 17 21 19 17 11 15 76 9 26 5 11 2015 2016 2017 2018 Number of Indictments per unit Juárez Mexicali Saltillo Nayarit Tijuana Torreón Villahermosa USAID/MEXICO PROMOTING JUSTICE PROJECT 22 FINAL MONITORING & EVALUATION REPORT The U.S. concept of an indictment is different from Mexico’s. Likened to acusación in the Mexican system, in reality, this is not the same point in the criminal justice process as an indictment in the U.S. Acusaciones occur much later in the Mexican process, requiring far more time to achieve than just a calendar year. This timing alters the significance of a prosecution rate calculated on the basis of achieving an acusación. Given the much lengthier time on average that it takes to reach an indictment in this sense, especially for serious crimes like homicides, measuring prosecution rates annually (as the indicator was originally established) tends to capture only cases that require less investigation, such as those where the suspect was captured in flagrante. 18 They are more an indicator of police work, rather than AG work. Consequently, PROJUST measured how prosecution rates for a given cohort of homicide cases changed over longer periods to capture how prosecution rates rise significantly when considered over at least two years after the case is opened. Each line in Figure 9 represents the cohort of homicide cases that entered the system in a given year and tracks the prosecution rate of that cohort over time. 18 In the act of committing the crime. USAID/MEXICO PROMOTING JUSTICE PROJECT 23 FINAL MONITORING & EVALUATION REPORT FIGURE 9: EVOLUTION OF PROSECUTION RATES FOR A GIVEN COHORT OF HOMICIDE CASES CONTEXT INDICATOR: HOMICIDE CRIME INCIDENCE A spike in homicide incidence can be seen across the board in 2017. The following figures demonstrate the same measurement, but the significantly higher incidence in Tijuana and Ciudad Juarez merit a separate graph. Tijuana alone has over 10 times more homicides than most targeted districts. 2015 2016 2017 2018 Juárez 2017 5.91% 13.96% Mexicali 2015 12.28% 30% 33.33% 40.35% Mexicali 2016 1.54% 17.69% 27.69% Mexicali 2017 4.94% 23.46% Nayarit 2016 28% 36.00% 36.00% Nayarit 2017 5.79% 10.00% Saltillo 2015 0.00% 16% 20.00% 28.00% Saltillo 2016 22.73% 45.45% 45.45% Saltillo 2017 33.33% 63.89% Tijuana 2017 0.95% 4.63% Torreón 2017 9.47% 23.16% Villahermosa 2017 4.13% 9.92% 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 PROSECUTION RATE Juárez 2017 Mexicali 2015 Mexicali 2016 Mexicali 2017 Nayarit 2016 Nayarit 2017 Saltillo 2015 Saltillo 2016 Saltillo 2017 Tijuana 2017 Torreón 2017 Villahermosa 2017 USAID/MEXICO PROMOTING JUSTICE PROJECT 24 FINAL MONITORING & EVALUATION REPORT FIGURE 10: HOMICIDE INCIDENCE, 2015–201819 Table 10 provides further context, disaggregating incidence into three categories of homicides: organized crime, gender-based violence and criminal gangs, or “pandillas.”20 Nayarit and Torreon have the highest incidence of homicides by organized crime. 19 Tijuana and Ciudad Juarez are excluded from this table since its incidence is more than seven times the highest incidence reported in this graph, hence the trends are not visible. 20 To categorize each case, it was necessary to look over certain characteristics of the crime and the crime scene. 114 130 162 150 25 22 36 33 75 190 136 95 70 83 121 110 2015 2016 2017 2018 Mexicali Saltillo Nayarit Torreón Villahermosa 609 907 1578 2154 2015 2016 2017 2018 Ciudad Juarez Tijuana USAID/MEXICO PROMOTING JUSTICE PROJECT 25 FINAL MONITORING & EVALUATION REPORT TABLE 10: HOMICIDE INCIDENCE PER SUBTYPE OF CRIME, 2018 District, State Organized Crime Gender Violence Criminal Gangs 2018 Cd. Juárez, Chihuahua 6.5% 0.3% 1.5% Mexicali, Baja California 7.7% 7.6% 0.0% Nayarit (statewide) 59.1% 5.2% 0.0% Saltillo, Coahuila 0.0% 0.9% 0.0% Tijuana, Baja California 0.1% 0.2% 0.0% Torreon, Coahuila 36.4% 0.0% 0.0% Villahermosa, Tabasco 0.6% 0.0% 0.0% 1.4.2 % OF PLEA AGREEMENTS IN TARGETED SPECIALIZED INVESTIGATION UNITS Definition: Percentage of cases that reach a plea agreement (also known as plea bargain) in targeted specialized investigation units within AG Offices. This indicator is calculated by dividing the number of plea agreements by the total number of convictions obtained. Unit: Plea agreements reported by targeted investigation and prosecution units. Disaggregated by: State. Type of indicator: Outcome. Key Data: Original Baseline States Measured21 Life of Project Target 100.0% Zacatecas Nayarit 90.0% Adjusted Baseline States Measured Final Result 84.6% Chihuahua Nayarit Baja California Coahuila Tabasco 82.1% Increasing the use of plea agreements/plea bargains is an objective of PROJUST’s work with target units because they unclog the system from unnecessary lengthy trials, provide prompt and expedited justice for all parties involved and indicate a strong investigation and prosecution by an AG Office when defendants are willing to accept responsibility for their crimes prior to a full trial. A plea bargain ensures a conviction while providing both the defense and the prosecution with influence over the final sentence to arrive at a resolution that is acceptable to the victim and the defendant. 21 Only specific districts: Zacatecas (Zacatecas) and Tepic (Nayarit). USAID/MEXICO PROMOTING JUSTICE PROJECT 26 FINAL MONITORING & EVALUATION REPORT Results under this indicator fell short of the target, but in absolute numbers, the use of plea agreements increased significantly. It rose threefold from a baseline of 33 plea agreements to 101 across all target units in 2018, an increase of 206 percent. In nearly all target units, total convictions increased, via both plea agreements and oral trials. While this is a positive result for the units and a positive reflection on PROJUST’s impact to increase prosecution of homicides, it negatively affected the results under this indicator because the relative increase in convictions via trial outweighed that of convictions via plea agreements, resulting in an apparent drop in the rate of agreements. TABLE 11: PERCENTAGE OF CONVICTIONS OBTAINED IN PLEA BARGAINING. End-of-Project Result 82.1% District, State Baseline Result as of 201822 Total # of Cases Closed with a Conviction (A) Conviction in Plea Bargaining (B) % of Convictions Through Pleas (B/A) Total # of Cases Closed with a Conviction (C) Conviction in Plea Bargaining (D) % of Convictions through Pleas (D/C) Cd. Juárez, Chihuahua (2017) 3 3 100.0% 35 30 85.7% Nayarit (statewide) (2016) 20 18 90.0% 8 7 87.5% Mexicali, Baja California (2015) 5 5 100.0% 17 12 70.6% Saltillo, Coahuila23 (2015) 0 0 0% 20 16 80.0% Tijuana, Baja California (2015) 4 4 100.0% 24 23 95.8% Torreon, Coahuila (2017) 0 0 0% 9 6 66.7% Villahermosa, Tabasco (2017) 7 3 43% 10 7 70.0% Total 39 33 84.6% 123 101 82.1% 22 Includes all cases, independently of the year the investigation was initiated. 23 PROJUST’s activities started in the second half of 2016. USAID/MEXICO PROMOTING JUSTICE PROJECT 27 FINAL MONITORING & EVALUATION REPORT Results from prior years: District, State Results Total # of Cases Closed with a Conviction (A) Conviction in Plea Bargaining (B) % of Convictions Through Pleas (B/A) Mexicali, Baja California (2016) 15 12 80% Mexicali, Baja California (2017) 10 10 100% Saltillo, Coahuila (2016) 9 6 67% Saltillo, Coahuila (2016) 18 14 78% Nayarit (statewide) (2017) 18 17 94% In general, eight of 10 sentenced cases are convicted, and most homicide cases reach a settlement before going to trial, demonstrating efficient resource management within the specialized units and well-structured investigations that achieve conviction prior to a full trial. Ciudad Juarez, Saltillo and Tijuana show the most convictions during PROJUST’s assistance. TABLE 12: PERCENTAGE OF CONVICTIONS OBTAINED IN TRIAL. District, State Baseline Result as of 201824 Total # of Cases Closed with a Conviction (A) Conviction in Trial (B) % of Convictions through Pleas (B/A) Total # of Cases Closed with a Conviction (C) Conviction in Trial (D) % of Convictions through Trial (D/C) Cd. Juárez, Chihuahua (2017) 3 0 0% 35 5 14.0% Nayarit (statewide) (2016) 20 2 10.0% 8 1 13.0% Mexicali, Baja California (2015) 5 0 0% 17 5 29.0% Saltillo, Coahuila25 (2015) 0 0 0% 20 4 20.0% Tijuana, Baja California (2015) 4 0 0% 24 1 4.0% 24 Includes all cases, independently of what year the investigation was initiated. 25 PROJUST’s activities started in the second half of 2016. USAID/MEXICO PROMOTING JUSTICE PROJECT 28 FINAL MONITORING & EVALUATION REPORT District, State Baseline Result as of 201824 Total # of Cases Closed with a Conviction (A) Conviction in Trial (B) % of Convictions through Pleas (B/A) Total # of Cases Closed with a Conviction (C) Conviction in Trial (D) % of Convictions through Trial (D/C) Torreon, Coahuila (2017) 0 0 0% 9 3 33.3% Villahermosa, Tabasco (2017) 7 4 57.0% 10 3 30.0% Total 39 6 15.0% 123 22 18.0% FIGURE 11: PLEA BARGAIN RATE, BASELINE VS. FINAL RESULT 1.4.3 PERCENTAGE OF CASES WITH SENTENCES GREATER THAN 3 YEARS Definition: Percentage of convictions in trial and plea bargaining (measured and reported separately) in which the time of the sentence is more than three years. This indicator is calculated by dividing the number of convictions for which more than three years sentence is applied by the total number of convictions (in both trials and plea bargaining) in targeted districts receiving project support to improve the prosecution of crimes. Unit: Convictions issued in all types of crimes in targeted districts receiving project support to improve the prosecution of crimes. Disaggregated by: Convictions in plea bargaining and trial. Type of indicator: Outcome. This indicator is a hybrid between context and performance/custom indicator, given the numerous factors outside the project’s control that could influence achievement or underachievement of the target for this indicator. As a result, a life-of-project target could not be set. Key Data: 100.0% 90.0% 100.0% 100.0% 43.0% 85.7% 87.5% 84.6% 70.6% 80.0% 95.8% 66.7% 70.0% 82.1% Cd. Juárez, Chihuahua Nayarit (statewide) Mexicali, Baja California Saltillo, Coahuila Tijuana, Baja California Torreon, Coahuila Villahermosa, Tabasco Total Baseline Result as of 2018 USAID/MEXICO PROMOTING JUSTICE PROJECT 29 FINAL MONITORING & EVALUATION REPORT Original Baseline States Measured Life-of-Project Target N/A N/A N/A Adjusted Baseline States Measured Final Result 31.0% Chihuahua Nayarit Baja California Coahuila Tabasco 27.0% This indicator measures the convictions in trial and via plea bargaining in which the time of the sentence is more than three years. Added to PROJUST’s M&E Plan in November 2017 at the request of USAID, this indicator is a hybrid between context and custom indicator given that the data reflects the overall performance of the justice system in a given district and there are a number of factors outside PROJUST’s control. Under these circumstances, a target could not be stablished, since it is influenced by the type of crime committed, the circumstances and difficulties to achieve convictions, and the Criminal Codes for each state (years of sentence differ greatly). It is important to keep in mind that the data is reported for all types of crimes, not just homicides as in the above indicators, and includes cases that reached a conviction in a given year regardless of the year the case was opened. Additionally, although this indicator is measured for those states receiving support for improved prosecution rates, the data is provided by the State Court. USAID/MEXICO PROMOTING JUSTICE PROJECT 30 FINAL MONITORING & EVALUATION REPORT TABLE 13: SENTENCING LENGTH FOR CONVICTIONS IN PLEA BARGAINING AND IN TRIAL, 201826 End-of-Project Result 27.0% Plea Bargain 72.9% Trial District, State Total # of Convictions in Plea Bargaining (A) Total # of Convictions in Plea Bargaining with Sentences >3 Years (B) % (B/A) Global Average Total # of Convictions in Trial (C) Total # of Convictions in Trial with Sentences >3 Years (C) % (C/D) Global Average 2017 (baseline) Cd. Juárez, Chihuahua 756 267 35.3% 31.0% 163 128 79.0% 69.2% Mexicali, Baja California 1,160 251 4.1% 47 24 51.1% Nayarit (statewide) 664 239 36.0% 3 0 0.0% Saltillo, Coahuila 41 35 85.4% 6 6 100.0% Tijuana, Baja California 458 130 28.4% 6 2 33.3% Torreon, Coahuila 16 10 62.5% 0 0 0.0% Villahermosa, Tabasco 88 49 55.7% 93 57 61.3% 2018 Cd. Juárez, Chihuahua 864 284 32.9% 27.0% 256 189 73.8% 72.9% Mexicali, Baja California 2,052 442 5.1% 105 85 81.0% Nayarit (statewide) 453 117 25.8% 4 3 75.0% Saltillo, Coahuila 30 24 80.0% 11 9 81.8% Tijuana, Baja California 750 219 29.2% 28 25 89.3% Torreón, Coahuila 49 37 75.5% 3 3 0.0% Villahermosa, Tabasco 563 166 29.5% 178 120 67.4% To a point, results reflect the “recommended” behavior of convictions in court. Most cases with a scheduled hearing are nonserious, and as a result can be solved during initial or intermediate hearing (via a plea bargain), with a smaller number of cases taken to trial. Ciudad Juarez’s most common 26 This data is for all cases that reached a conviction in 2017, regardless of what year the case began. USAID/MEXICO PROMOTING JUSTICE PROJECT 31 FINAL MONITORING & EVALUATION REPORT crime presented in court is possession of narcotics (a crime with a high plea bargain rate), while in the rest of target districts, the most common crime was robbery. 1.5 EFFECTIVENESS OF PUBLIC DEFENSE Increase effectiveness of the public defense. Part and parcel of the accusatory system is the premise that defendants enjoy the right to a competent defense that upholds due process and limits unnecessary restrictions on their liberty while awaiting a final sentence. Accordingly, PROJUST provided direct support to public defenders (PDs) to improve their performance when debating in initial hearings, a key moment in criminal proceedings when probable cause is established and precautionary measures are imposed. 1.5.1 RATE OF EFFECTIVE DEBATES EXECUTED BY TARGETED PUBLIC DEFENDERS Definition: Percentage of effective debates in initial hearings presented by public defenders. This indicator is calculated by taking the number of debates that received a favorable ruling over the total number of debates presented by public defenders. Unit: Debates by public defender. Disaggregation: Type of resolution and gender of defendants. Type of indicator: Outcome. Key Data: Original Baseline States Measured Life-of-Project Target 57.0%27 Zacatecas 55.0% Adjusted Baseline States Measured Final Result 48.3% Coahuila Zacatecas 53.1% Improvements in inherently subjective services, such as public defense, are not easily quantifiable, but in 2017 PROJUST proposed a new indicator to gauge its impact. This indicator measures the number of debates in initial hearings opened by target PDs for which they received a favorable ruling, divided by the total number of debates opened by the PD. To enable a more granular assessment, the data is disaggregated by type of ruling and by state. Given the recent introduction of this indicator combined with the relatively short timeframe left in the project, PROJUST established the baseline and took measurements semi-annually in order to have more data points to chronicle incremental changes. As of this reporting, PROJUST has measured this indicator in two target states where data is currently available and with quality data. The target was established at 55 percent, with final results reaching 53.1 percent across the two PD Offices. In other target PD Offices, no record of debate performance is monitored, and although PROJUST worked with Monitoring and Evaluation Units in PD Offices, data was insufficient to report 27 Baseline was higher than the target established because baseline data could only be collected in one of six target PD Offices. It was anticipated that more Offices would be measured in subsequent years and likely lower the baseline. USAID/MEXICO PROMOTING JUSTICE PROJECT 32 FINAL MONITORING & EVALUATION REPORT under this indicator. The original reported baseline was calculated with the support of written registries of Zacatecas’ PD Office, with unreliable data quality at the time, 28 for the second period reported (July to December 2017), a more precise method was implemented and the original baseline adjusted to include Coahuila. TABLE 14: RATE OF EFFECTIVE DEBATES BY PUBLIC DEFENDERS End-of-Project Result 53.1% Type of Hearing Coahuila Zacatecas Debates Average Opened Effective Debates Rate of Effective Debates Debates Opened Effective Debates Rate of Effective Debates Jan-Jun 2017 (Original Baseline) Probable cause 81 20 25.0% Precautionary 57.0% measures 106 87 82.0% Overall 187 107 57.0% Jul-Dec 2017 (Adjusted Baseline) Probable cause 97 37 38.1% 45 18 40.0% Precautionary 48.3% measures 77 50 64.9% 15 8 53.3% Overall 174 87 50.0% 60 26 43.3% Jan-Jun 2018 Probable cause 51 25 49.0% 70 24 34.3% Precautionary 50.3% measures 47 27 57.4% 29 23 79.3% Overall 98 52 53.1% 99 47 47.5% Jul-Dec 2018 Probable cause 94 47 50.0% 57 22 38.6% Precautionary 53.1% measures 80 46 57.5% 61 40 65.6% Overall 174 93 53.4% 118 62 52.5% Within criminal justice proceedings, initial hearings determine two key rulings that are relevant to quality public defense: probable cause and determination of precautionary measures. As a whole, the rate of effectiveness of debates improved from the second measurement to the end of the project, by 4.7 percentage points. For Coahuila, probable cause’s effectiveness of debates 28 This first outcome was envisioned as a pilot. USAID/MEXICO PROMOTING JUSTICE PROJECT 33 FINAL MONITORING & EVALUATION REPORT improved by almost 10 percentage points, while in Zacatecas, greater impact can be seen in precautionary measures (12.3 points). CONTEXT INDICATOR: ACCELERATED PROGRAM FOR IMPROVED PERFORMANCE OF PDs During the Q2 FY2019, target PD Offices took part in a three-month training program titled, “Accelerated Program to Improve Performance in Hearings,” an effort to boost and improve the methodology of implementation for all previously referred activities. A flexible, online method of training, combined with onsite mentoring and observation of hearings, proved efficient in increasing technical knowledge, strategic abilities during hearings and registry of information for cohorts of 10 PDs in six cities: Chihuahua, Saltillo, San Luis Potosí, Tijuana, Torreon and Zacatecas (see Table 15). TABLE 15: ACCELERATED PROGRAM FOR IMPROVED PERFORMANCE OF PDs, PER STATE29 Type of Hearing Baseline (September-December 2018) Final Result (January-March 2019) Impact Debates Opened Effective Debates Rate of Effective Debates Debates Opened Effective Debates Rate of Effective Debates Saltillo Probable cause 23 9 39.1% 23 10 43.5% Precautionary +33.2% measures 20 8 40.0% 15 10 66.7% Overall 43 17 39.5% 38 20 52.6% Tijuana Probable cause 8 3 37.5% 2 1 50.0% Precautionary +26.6% measures 27 12 44.4% 11 6 54.5% Overall 35 15 42.9% 13 7 53.9% Torreon Probable cause 23 1 4.35% 12 1 8.33% Precautionary +65.7% measures 15 3 20.0% 11 3 27.7% Overall 38 4 10.5% 23 4 17.4% Zacatecas Probable cause 25 7 28.0% 42 16 38.1% Precautionary +41.5% measures 24 8 33.3% 18 10 55.6% Overall 49 1 30.6% 60 26 43.3% 29 Only in the district of Saltillo. USAID/MEXICO PROMOTING JUSTICE PROJECT 34 FINAL MONITORING & EVALUATION REPORT Table 15 shows the improvement in the two types of rulings. All cities demonstrated a significant amount of effective use of debates, with the highest increases in Zacatecas and Coahuila (for precautionary measures) and Torreon (for probable cause). Even though the number of debates opened decreases, the rate of success increases, meaning that not only are the arguments during a hearing stronger, but the decision by public defenders to open debate was also more strategic. TASK 2: INCREASED CAPACITY FOR MONITORING AND EVALUATION OF CRIMINAL JUSTICE SYSTEM OPERATION Task 2 of PROJUST increases the capacity for monitoring and evaluation of institutional performance in the criminal justice system. The project focuses efforts under this Task on the establishment of Monitoring and Evaluation (M&E) Units in targeted state JSIs. 2.1 MONITORING PERFORMANCE Quality controls to monitor and measure performance established in justice sector institutions. 2.1.1 NUMBER OF TARGETED JSIs THAT HAVE QUALITY CONTROLS TO MONITOR AND MEASURE PERFORMANCE Definition: Total number of justice institutions that have specialized units in charge of monitoring and evaluating institutional performance (such as through performance indicators, satisfaction surveys, measurement of process discipline) with the objective of identifying performance gaps and generating responses for continuous improvement. An M&E Unit is considered to be established when Phase 5 of the implementation methodology is reached. Unit: Targeted JSIs. Disaggregated by: JSIs and state. Type of indicator: output Key Data: Baseline Life-of-Project Target Final Result 0 18 20 This indicator is defined as the number of targeted JSIs that have established M&E Units, which are designed to produce statistical data against performance measurement indicators, identify performance gaps and generate responses for continuous improvement. An M&E Unit is “established” when it has reached Phase 5 of the project’s implementation methodology: Phase 1: Feasibility assessment Phase 2: Formalization (Establishing the unit’s legal and financial security through the amendment of laws and regulations) Phase 3: Alignment (Customizing the M&E Unit Model to local context) Phase 4: Capacity Building (Training M&E Unit operators) Phase 5: Start-Up (Coaching unit operators in defining indicators and targets, collecting baseline data, etc.) USAID/MEXICO PROMOTING JUSTICE PROJECT 35 FINAL MONITORING & EVALUATION REPORT Phase 6: Consolidation (Continuous improvement) Phase 2 in this methodology is of particular significance, given that formalization of an M&E Unit via reformed laws, regulations or formal agreements lays the foundation for the sustainability and permanence of the unit beyond the scope of the project and changing political will of the institution. Since the methodology does not require a unit to progress through the phases in a linear fashion, some units may not have achieved formalization, but are considered established and operating. While this indicator counts only units that have been “established” in terms of the M&E Unit methodology, an additional two units were created in FY2019 and received extensive project support to begin operations. Both were located in the AG Offices of Nuevo Leon and Sonora, for a total of 22 M&E Units created. As of May 2019, a total of 20 M&E Units across 10 target states had been established. TABLE 16: PROGRESS ON ESTABLISHMENT OF M&E UNITS AS OF FY 2019 State Justice Institution Implementation Phases 1 2 3 4 5 6 Baja California Public Defender's Office X X X X X State Court X X X X X Baja California Sur Public Defender's Office X X X X Pre-Trial Services Unit X X X X X X Chihuahua Public Defender's Office X X X X X State Court X X X X X Coahuila State Court X X X X X X Public Defender's Office X X X X X X Attorney General's Office X X X X X Jalisco Attorney General's Office X X X X X X Nayarit State Court X X X X X Attorney General’s Office X X X X X X Nuevo Leon Public Defender's Office X X X X X X State Court X X X X X San Luis Potosi Attorney General’s Office X X X X Public Defender’s Office X X X X X Tabasco Attorney General's Office X X X X X X Zacatecas Attorney General’s Office X X X X X X State Court X X X X X X Public Defender´s Office X X X X X Total Units Formalized: 10 Total Units Established: 20 USAID/MEXICO PROMOTING JUSTICE PROJECT 36 FINAL MONITORING & EVALUATION REPORT USAID AND FOREIGN ASSISTANCE INDICATORS In addition to project-specific indicators, PROJUST also contributes to USAID/Mexico Development Objective 2 (DO2) and Foreign Assistance (FA) Indicators. This section summarizes our activity under these indicators. # Indicator Reporting in Detail: # Achieved FY 2019 DO2c Number of states receiving technical assistance on comprehensive criminal justice reform. (Note that the definition of this indicator specifically refers to prosecutors, judges and public defenders.) 11 states30 received PROJUST’s technical assistance (TA) on criminal justice reform specifically targeting judges, prosecutors and public defenders. Overall, TA included activities to strengthen the performance of State Courts, Attorneys General, Public Defenders’ Offices, and Pre￾Trial Services Units to improve the criminal justice system’s efficiency. IR 2.1a Number of laws and regulations supporting the implementation of the new criminal justice sector reforms passed A new law was passed in FY 2019 with project support. (See Annex I for a list of laws passed with project support in FY 2015-19). IR 2.2c & 2.1.2-7 Number of judges and judicial personnel trained (disaggregated by sex) 39331 judicial personnel32 (220 women, 173 men) were trained by USAID’s PROJUST. 30 Baja California, Baja California Sur, Chihuahua, Coahuila, Jalisco, Nayarit, Nuevo Leon, San Luis Potosi, Sonora, Tabasco and Zacatecas. 31 These figures do not include data from DPK-PROJUST and pertain only to training conducted by MSI-PROJUST for FY 2019. 32 Judicial personnel include judges, magistrates, prosecutors, advocates, inspectors and court staff. USAID/MEXICO PROMOTING JUSTICE PROJECT 37 FINAL MONITORING & EVALUATION REPORT # Indicator Reporting in Detail: # Achieved FY 2019 Sub-IR 2.2.1b Number of justice sector representatives with increased knowledge based on pre- and post-tests Of the 393 justice sector operators trained by PROJUST, 157 were evaluated or tested pre- and post-training. Of these 157, 48.4% (76) obtained a higher score on the post-training test or evaluations. Disaggregation by justice sector operator: Justice Sector Operator # of trainees # of trainees that increased knowledge Female Male Female Male Court clerk 83 43 1 0 Judge 24 24 4 1 Justice 3 13 3 6 Prosecutor 34 21 17 15 PTS staff 40 36 12 4 Public defender 36 36 9 4 Total 220 173 46 30 USAID/MEXICO PROMOTING JUSTICE PROJECT 38 FINAL MONITORING & EVALUATION REPORT ANNEX I: LAWS ALIGNED WITH THE ACCUSATORY SYSTEM PASSED DURING FY 2015 TO 2019 PROJUST tracks the number of laws aligned with the accusatory system that are passed with project support during a given fiscal year at both the state and national levels, as part of reporting against USAID/Mexico Foreign Assistance Indicator IR 2.1a. “Number of laws and regulations supporting the implementation of the new criminal justice sector reforms passed.” Legal frameworks necessary for the successful implementation of the accusatory criminal justice system have been developed with PROJUST support in 28 states since 2015. This support contributed to the modification and/or passage of 83 laws at the state level, which are now harmonized with the new system, and three key national laws, the National Juvenile Justice Law, the National Execution of Sanctions Law and the Organic Law for the Federal Attorney General’s Office, for a total of 86 laws from 2015-2019. # State Laws Passed FY 2015 States that received PROJUST’s direct technical assistance 1 Baja California Organic Law for the State Secretariat of Public Safety 2 Execution of Sanctions Law 3 Victims’ Assistance Law 4 Juvenile Justice Law 5 Organized Crime Law 6 Alternative Dispute Resolution Law 7 Organic Law for the State Court 8 Organic Law for the Attorney General’s Office 9 Organic Law for the Public Defender’s Office 10 Baja California Sur Organic Law for the State Court 11 Organic Law for the Attorney General’s Office 12 Colima Victims’ Assistance Law 13 Morelos Organic Law for the State Court 14 Organic Law for the Attorney General’s Office 15 Nuevo Leon Organic Law for the State Secretariat of Public Safety 16 Execution of Sanctions Law 17 Oaxaca Organic Law for the State Secretariat of Public Safety 18 Organic Law for the State Court 19 Witness Protection Law 20 Asset Seizure Law 21 Organic Law for the Attorney General’s Office 22 Puebla Organic Law for the State Court 23 Queretaro Organic Law for the State Court States using SETEC-PROJUST’s legislation models (indirect assistance) 24 Campeche Organic Law for the State Court 25 Organic Law for the Attorney General’s Office 26 Organic Law for the Public Defender’s Office USAID/MEXICO PROMOTING JUSTICE PROJECT 39 FINAL MONITORING & EVALUATION REPORT # State Laws Passed 27 Organic Law for the State Secretariat of Public Safety 28 Witness Protection Law 29 Victims’ Assistance Law 30 Chiapas Organic Law for the State Court 31 Organic Law for the Attorney General’s Office 32 Organic Law for the Public Defender’s Office 33 Organic Law for the State Secretariat of Public Safety 34 Witness Protection Law 35 Victims’ Assistance Law 36 Durango Organic Law for the State Court 37 Organic Law for the Attorney General’s Office 38 Organic Law for the Public Defender’s Office 39 Organic Law for the State Secretariat of Public Safety 40 Witness Protection Law 41 Victims’ Assistance Law 42 State of Mexico Organic Law for the State Court 43 Organic Law for the Attorney General’s Office 44 Organic Law for the Public Defender’s Office 45 Organic Law for the State Secretariat of Public Safety 46 Witness Protection Law 47 Victims’ Assistance Law 48 Guerrero Organic Law for the State Court 49 Organic Law for the Attorney General’s Office 50 Organic Law for the Public Defender’s Office 51 Organic Law for the State Secretariat of Public Safety 52 Witness Protection Law 53 Victims’ Assistance Law 54 Tamaulipas Organic Law for the State Court 55 Organic Law for the Attorney General’s Office 56 Organic Law for the Public Defender’s Office 57 Organic Law for the State Secretariat of Public Safety 58 Witness Protection Law 59 Victims’ Assistance Law FY 2016 States that received PROJUST’s direct technical assistance 1 Federal Execution of Sanctions Law 2 Juvenile Justice Law 3 Nuevo Leon Organic Law for the State Court 4 Sonora Witness Protection Law 5 Tabasco Organic Law for the Public Defender’s Office 6 Veracruz Organic Law for the State Secretariat of Public Safety 7 Organic Law for the Attorney General’s Office 8 Organic Law for the Attorney General’s Office USAID/MEXICO PROMOTING JUSTICE PROJECT 40 FINAL MONITORING & EVALUATION REPORT # State Laws Passed 9 Zacatecas Organic Law for the State Secretariat of Public Safety 10 Michoacan Organic Law for the State Secretariat of Public Safety 11 Aguascalientes Organic Law for the Attorney General’s Office 12 Coahuila Organic Law for the Attorney General’s Office 13 Tlaxcala Organic Law for the State Court 14 Organic Law for the Attorney General’s Office 15 Jalisco Organic Law for the State Court 16 Organic Law for the Attorney General’s Office States using SETEC-PROJUST’s legislation models (indirect assistance) 17 Guanajuato Organic Law for the State Court 18 Organic Law for the State Secretariat of Public Safety 19 Hidalgo Organic Law for the Attorney General’s Office 20 San Luis Potosi Organic Law for the State Court 21 Yucatan Organic Law for the State Secretariat of Public Safety 22 Chihuahua Witness Protection Law FY 2017 No new or revised law was passed in FY 2017 with project support. FY 2018 States that received PROJUST’s direct technical assistance 1 Zacatecas Organic Law for the Attorney General’s Office 2 Organic Law for the State Court 3 San Luis Potosi Organic Law for the Attorney General’s Office 4 Nuevo Leon Organic Law for the Attorney General’s Office FY 2019 States that received PROJUST’s direct technical assistance 1 Federal Organic Law for the Federal Attorney General’s Office USAID/MEXICO PROMOTING JUSTICE PROJECT 41 FINAL MONITORING & EVALUATION REPORT ANNEX II: UPDATED PERFORMANCE MONITORING PLAN (PMP) RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 Task 1. Promote increased legitimacy of the criminal justice system 1.1 Targeted justice sector institutions (JSIs) have reformed structures, procedures and regulations to align with the National Criminal Procedure Code (NCPC) and secondary legislation, following identified best practices 1.1.1 # of targeted JSIs with procedures and structures aligned to the NCPC and secondary legislation Definition: Total JSIs in targeted states that modify or create procedures, structures and regulations (e.g., organic regulations, organizational manuals, operating manuals standard operating procedures (SOP) or protocols, etc.). This indicator considers JSIs that: 1) have modified its structure or created a new structure (case of PTS units); 2) are operating according to new processes; and 3) have an operating organizational manual or procedure manual in use (not necessarily ‘officially’ approved, but in use); or approved organic regulations. The institution needs to meet the three requisites for being counted toward this indicator with PROJUST support to achieve at least one of the requisites. Unit: Justice sector institutions. Disaggregated by: State, JSI; type of procedure or regulation. Type of indicator: Output 9 N/A N/A N/A 13 18 20 22 N/A USAID/MEXICO PROMOTING JUSTICE PROJECT 42 FINAL MONITORING & EVALUATION REPORT RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 1.2 Targeted state justice system actors reduce use of pretrial detention, especially for youth and non-violent defendants 1.2.1 % of recommendati ons of pre-trial detention by PTS unit operators Definition: The operator that this indicator refers to is the risk evaluator that recommends precautionary measures to be imposed according to the risks detected. This indicator is calculated by dividing the total number of recommendations of pre-trial detention by the total number of precautionary measures recommendations issued. Unit: Recommendations. Disaggregated by: State. Type of indicator: Outcome 60.0% N/A N/A N/A N/A 54.8% 50.0% 49.3% This indicator seeks a reduction from baseline to final result and could only be measured in Coahuila. 1.2.2 Rate of compliance with pre-trial measures Definition: The compliance rate measures compliance by defendants supervised by PTS units with precautionary measures and conditions for release imposed by courts. The rate is calculated by dividing the total number of defendants in full compliance divided by the total number of defendants supervised by the unit (active cases) in the corresponding year. Unit: Compliance rate reported by target PTS units. Disaggregated by: State. Type of indicator: Outcome 70.4% 77.1% N/A N/A 88.7% 77.6% 85.0% 80.6% N/A USAID/MEXICO PROMOTING JUSTICE PROJECT 43 FINAL MONITORING & EVALUATION REPORT RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 1.3 Increase criminal case disposition rates in targeted courts 1.3.1 Criminal case disposition rates Definition: The case disposition rate is calculated as: the number of resolved or closed cases divided by the number of new cases in the relevant year. Case disposition in targeted courts are actions taken as the result of an appearance in court by a defendant where outcomes can include: non￾probable cause ruling (no vinculación a proceso), dismissal, acquittal, conviction (in oral trials and as a result of plea bargaining), pre-trial diversion and settlement (between victim and defendant). Unit: Criminal case disposition rate reported per court. Disaggregated by: state, court type of resolution (e.g., non-probable cause ruling, dismissal, acquittal, conviction, pre-trial diversion, settlement); type of crime (e.g., domestic violence, homicide, rape, kidnapping). Type of indicator: Outcome 69.3% 57.7% N/A 63.4% 75.0% 57.1% 70.0% 71.3% Baseline adjusted to reflect changes in target districts assisted. USAID/MEXICO PROMOTING JUSTICE PROJECT 44 FINAL MONITORING & EVALUATION REPORT RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 1.4 Increased prosecution rates for targeted state-level prosecutors 1.4.1 Prosecution rates in targeted investigation and prosecution units Definition: Annual prosecution rates for serious crimes (such as homicides) is calculated by obtaining the total number of indictments (acusaciones) divided by the total number of investigations conducted by targeted investigation and prosecution units. For other types of crimes, such as robbery or domestic violence, the calculation is the number of cases disposed through alternative resolutions (i.e. agreements or pre-trial diversion) plus those that reach indictments divided by the total number of investigations conducted by the unit. Unit: Prosecution rate reported by investigation and prosecution units. Disaggregated by: Prosecution rates p/ targeted investigation and prosecution unit; type of crime (e.g., homicide, rape, kidnapping, aggravated robbery, domestic violence). Type of indicator: Outcome 11.6% 3.8% +10. 1% +14.8% +4.3% +4.2% +20% +8.79% Baseline adjusted to reflect changes in target units assisted. Previously reported measurements may also differ for this reason, and to reflect adjustments to how results were measured. All adjustments were discussed with USAID prior to reporting. USAID/MEXICO PROMOTING JUSTICE PROJECT 45 FINAL MONITORING & EVALUATION REPORT RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 1.4.2 % of plea agreements in targeted specialized units Definition: Percentage of cases that reach a plea agreement (also known as plea bargain) in targeted specialized investigation units within AG Offices. This indicator is calculated by dividing the number of plea agreements by the total number of convictions obtained. Unit: Plea agreements reported by targeted investigation and prosecution units. Disaggregated by: State. Type of indicator: Outcome 100.0% 84.6% N/A N/A 82.5% 85.3% 90.0% 82.1% Baseline adjusted to reflect changes in target units assisted. 1.4.3 Percentage of cases with sentences grater than 3 years Definition: Percentage of convictions in trial and plea bargaining (measured and reported separately) in which the time of the sentence is more than 3 years. This indicator is calculated by dividing the number of convictions for which more than three years sentence is applied by the total number of convictions (in both trials and plea bargaining) in targeted districts receiving project support to improve the prosecution of crimes. Unit: Convictions issued in all types of crimes in targeted districts receiving project support to improve the prosecution of crimes. Disaggregated by: Convictions in plea bargaining and trial. Type of indicator: Outcome N/A 31.0% N/A N/A N/A N/A N/A 27.0% Established as a hybrid context/contra ctual indicator. Baseline could not be established until FY2019 due to data access and quality issues. For this reason, no target could be set. USAID/MEXICO PROMOTING JUSTICE PROJECT 46 FINAL MONITORING & EVALUATION REPORT RFP Result by Task Indicator Definition and Unit of Measure Original Baseline Adjusted Baseline RESULTS PER YEAR LOP TARGET RESULT COMMENTS 1 2 3 4 1.5 Increased effectiveness of the public defense 1.5.1 Increased effectiveness of the public defense Definition: Percentage of effective debates in initial hearings presented by public defenders. This indicator is calculated by taking the number of debates that received a favorable ruling over the total number of debates presented by public defenders. Unit: Debates by public defender. Disaggregation: Type of resolution and gender of defendants. Type of indicator: Outcome 57.0% 48.3% N/A N/A N/A 50.3% 55.0% 53.1% This indicator could only be measured in Coahuila and Zacatecas. Task 2: Support building increased capacity for monitoring and evaluation of criminal justice system operation 2.1 Quality controls to monitor and measure performance established in justice sector institutions 2.1.1 # of targeted JSIs that have quality controls to monitor and measure performance Definition: Total number of justice institutions that have specialized units in charge of monitoring and evaluating institutional performance (through performance indicators, satisfaction surveys, measurement of process discipline, for example) with the objective of identifying performance gaps and generating responses for continuous improvement. An M&E Unit is considered to be established when step 5 of the implementation methodology is reached. Unit: Targeted JSIs. Disaggregated by: JSIs and state. Type of indicator: output 0 N/A 0 9 12 15 18 20 N/A USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 47 ANNEX III: UPDATED PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS) PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.1.1 # of targeted JSIs with procedures and structure aligned to the NCPC and secondary legislation DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Total JSIs in targeted states that modify or create procedures, structures and regulations (e.g., organic regulations, organizational manuals, operating manuals standard operating procedures (SOP) or protocols, etc.). Unit of Measure: Justice sector institutions Disaggregated by: State, JSI; type of procedure or regulation Rational or Justification for Indicator: Legislative work under PROJUST has taken significant efforts to design, coach and approve secondary legislation. In many cases, achieving legislative changes can improve implementation results. PLAN FOR DATA COLLECTION BY USAID Data Source: JSI records/reports and PROJUST records. Method of Data Collection and Construction: Methods: desk review, content analysis. Construction: This indicator will consider JSIs that: 1) have modified its structure or created a new structure (case of PTS units); 2) are operating according to new processes; and 3) have an operating organizational manual or procedure manual in use (not necessarily ‘officially’ approved, but in use); or approved organic regulations. The institution needs to meet the three requisites for being counted toward this indicator with PROJUST support to achieve at least one of the requisites. Reporting Frequency: Annually (fiscal year) Individual Responsible for Acquiring the Data: PROJUST Technical Specialist in Legislation Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 48 Known Data Limitations: None Actions Taken or Planned to Address Data Limitations: N/A CHANGES TO INDICATOR Changes to Indicator: Minor change to indicator name; change to definition and construction (May 2017). Originally, this indicator was defined as “Total states that modify or create basic secondary laws. USAID considers the following six laws to be critical: (a) organic laws for four justice sector institutions (JSI) including the State Court, Attorney General (AG) and Public Defender (PD) Offices and Secretariat of Public Safety (SSP, acronym in Spanish); (b) Witness Protection; and (c) Victims’ Assistance. These laws must be drafted to comply with three key standards: the 2008 constitutional reform, HR and best practices. The revision or drafting of laws will be informed by and aligned to the legislative models of the Technical Secretariat of the Criminal Justice Implementation Commission (SETEC, acronym in Spanish).” The unit measured was originally target states (as opposed to institutions) and the indicator tracked laws (as opposed to “procedures, structures, and regulations.” These changes were made following the 2016 Constitutional Deadline, at which point all Mexican states had achieved modifications to all secondary laws outlined in the original definition. Since no further progress would be made to this indicator as it was originally defined, PROJUST proposed the abovementioned changes to reflect the new focus of its legislative work for the remaining years of the project. Other Notes: None. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 49 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.2.1 % of recommendations of pre-trial detention by Pre-Trial Services (PTS) unit operators. DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): The operator that this indicator refers to is the risk evaluator that recommends precautionary measures to be imposed according to the risks detected. This indicator will be calculated by dividing the total number of recommendations of pre-trial detention by the total number of precautionary measures recommendations issued. Unit of Measure: Recommendations. Disaggregated by: State. Rational or Justification for Indicator: The main objective of PTS Units is to gather, analyze and determine the level and type of risk the defendant presents to victims, witnesses, and the community. These risk evaluations are presented in a technical opinion that is considered by public defenders, prosecutors and judges during initial hearings to determine pre-trial measures. As a new institution under the accusatory criminal justice system, PTS Units are intended to promote alternatives to pre-trial detention, especially in non￾serious cases and for nonviolent defendants and first-time offenders. PLAN FOR DATA COLLECTION BY USAID Data Source: PTS units records. Method of Data Collection and Construction: Methods: Desk review of pre-trial recommendations in technical opinions issued by the PTS Unit. Construction: This indicator will be calculated by dividing the total number of pre-trial detention recommendations in technical opinions by the total number of technical opinions issued. Given that PTS Units do not track this information, data is collected via staff visits to each district receiving project support. Reporting Frequency: Annually (calendar year) Individual Responsible for Acquiring the Data: PROJUST technical staff specialized in PTS. Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 50 Date of Next Data Quality Assessments: N/A Known Data Limitations: Most of the information needs to be revised manually, on a case-by-case basis and on site for quality assurance requirements. Actions Taken or Planned to Address Data Limitations: The program assigned resources to ensure data collection for this indicator by project staff. CHANGES TO INDICATOR Changes to Indicator: N/A Other Notes: PTS Units generally do not issue technical opinions that explicitly recommend specific precautionary measures for each case. Coahuila’s was the first PTS Unit that started issuing such recommendations as part of their technical opinions. Identifying these recommendations as a best practice in PTS, PROJUST started to work with PTS units in other states to incorporate recommendations into their technical opinions with the expectation that the practice would be readily adopted. However, units were largely unreceptive to this suggestion. In some cases, this was due to pressure from AG Offices and the State Secretariats of Public Security to impose pre-trial detention regardless of risk levels, in an attempt to show a hard stance against criminals. As a result, PROJUST was unable to measure this indicator in other states, absent recommendations either for or against pre-trial detention. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 51 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.2.2 Rate of compliance with pre-trial measures DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): The compliance rate measures compliance by defendants supervised by PTS units with precautionary measures and conditions for release imposed by courts. The rate is calculated by dividing the total number of defendants in full compliance divided by the total number of defendants supervised by the unit (active cases) in the corresponding year. Full compliance refers to all the supervised accuser that did not committed a full non-compliance. Full non-compliance refers to the lack of compliance in all supervision activities. Unit of Measure: Compliance rate reported by target PTS units. Disaggregated by: State. Rational or Justification for Indicator: An important task for UMECAs is supervision. It Is their duty to give follow-up to the compliance of every accused under a precautionary measure in freedom, having the responsibility of issuing a non-compliance report should it be necessary. PLAN FOR DATA COLLECTION BY USAID Data Source: PTS unit records. Method of Data Collection and Construction: Methods: Desk review of PTS unit records. All PTS Unit’s provide information via a data matrix that was designed by PROJUST. Once this information is reviewed for data quality, project staff count final numbers manually. Construction: The rate is calculated by dividing the total number of defendants in full compliance divided by the total number of defendants supervised by the unit (active cases) in the corresponding year. Reporting Frequency: Annually Individual Responsible for Acquiring the Data: PROJUST technical staff specialized in PTS. Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 52 Known Data Limitations: PTS Units generally do not collect this data. A matrix was provided by PROJUST to each target unit for data collection and reviewed for quality following submission. Actions Taken or Planned to Address Data Limitations: The program has assigned resources to ensure data collection for this indicator. CHANGES TO INDICATOR Changes to Indicator: Definition of full compliance.(May 2017) Other Notes: Data also includes information from PTS Units specialized in Juveniles. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 53 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.3.1 Criminal case disposition rates in targeted courts DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Case disposition rates are the resolution of a case file taken by a judge in order to close or suspend a case. Closed cases are considered when the following results are met: illegal detention, non-probable cause, dismissal, conviction (by plea-agreement or trial). Suspended cases are considered when the following results are met: pre-trial diversion settlement or restitution agreement. Unit of Measure: Criminal case disposition rate reported per court. Disaggregated by: State, court type of resolution illegal detention, non-probable cause, dismissal, conviction (by plea-agreement or trial). pre-trial diversion settlement or restitution agreement; type of crime (e.g., domestic violence, homicide, rape, kidnapping). Rational or Justification for Indicator: Case disposition is the final result expected in justice system which is to reach the end of a judicial process, or access alternative justice mechanisms in order to solve a problem. PLAN FOR DATA COLLECTION BY USAID Data Source: State Courts records; full databases, per case file number for relevant year. Method of Data Collection and Construction: Methods: Desk review; content analysis and standardization of databases; physical verification Construction: The case disposition rate is calculated as: the number of resolved or closed cases divided by the number of new cases in the relevant year. Other considerations: a) For restitution agreement and pre-trial diversion, data is considered from the moment an agreement is settled by defendant and victim, regardless of whether all conditions are met within the relevant year. b) Since the cases considered are only those admitted and solved or suspended per relevant year, and each case file may contain more than one related defendant, variables per case file are taken as follows: i. Opened case file: given a relevant year, a case file that entries state court in relevant year and at least one defendant assisted to first hearing. ii. Closed case file: given a relevant year, a case file in which all defendants reached a settlement, received a conviction or its case suspended according to close file definition. For the disaggregation by court, a court order is also considered a closed case file (see figure below). USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 54 Reporting Frequency: Annually (calendar year) Individual Responsible for Acquiring the Data: PROJUST Monitoring and Evaluation Team Data Analyst Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A Known Data Limitations: Most court databases are not standardized, a significant amount of time must be invested in database review in order to standardize data collection. Information should be identifiable by case file number, defendant and crime. Actions Taken or Planned to Address Data Limitations: The program has assigned resources to ensure data collection for this indicator. CHANGES TO INDICATOR Changes to Indicator: Changed the disaggregation for this indicator (May 2017). Other Notes: None Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 55 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.4.1 Prosecution rates in targeted investigation and prosecutors units DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Policies and laws developed to institutionalize effective investigation and prosecutorial approaches Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Annual prosecution rates for serious crimes (homicides) will be calculated by obtaining the total number of indictments (acusaciones) divided by the total number of investigations conducted by targeted investigation and prosecution units. For other types of crimes, such as robbery or domestic violence, the calculation will be the number of cases disposed (through alternative resolutions (i.e. agreements or pre-trial diversion) plus those that reach indictments divided by the total number of investigations conducted by the unit. Unit of Measure: Prosecution rates of targeted investigation and prosecution units Disaggregated by: Prosecution rates p/ targeted investigation and prosecution unit; type of crime (e.g., homicide, rape, kidnapping, aggravated robbery, domestic violence). Rational or Justification for Indicator: PROJUST monitors the impact of its work with Attorneys General (AG) Offices by tracking the performance of Specialized Investigation and Prosecution Units for Homicides. PLAN FOR DATA COLLECTION BY USAID Data Source: AG Office records Method of Data Collection and Construction: Methods: Desk review; content analysis; physical verification. Construction: For serious crimes: Total number of indictments (acusaciones) divided by the total number of investigations conducted by targeted investigation and prosecution units. For other types of crimes: the calculation will be the number of cases disposed (through alternative resolutions (i.e. agreements or pre-trial diversion) plus those that reach indictments divided by the total number of investigations conducted by the unit. Reporting Frequency: Annually Individual Responsible for Acquiring the Data: PROJUST Task 1 Team Leader Role of this Individual on DO Team: NA DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 56 Known Data Limitations: AG Offices do not record data on prosecution rates so PROJUST manually collects data for this indicator. Because of the sensitive nature of homicide casefiles, PROJUST is able to collect data for only units receiving active project assistance. For this reason, data is unavailable for units that did not receive direct support throughout the duration of the project, and baselines were adjusted to reflect units that could be measured through the final calendar year of 2018. Actions Taken or Planned to Address Data Limitations: Given the absence of AG Office records, PROJUST manually collects data for this indicator, reviewing casefiles individually to ensure data quality. CHANGES TO INDICATOR Changes to Indicator: N/A Other Notes: Other crimes besides homicides could not be measured since project support to other specialized units was limited to less than a year and data on other crimes was not made available to the project. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 57 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.4.2 % of plea agreements in targeted specialized investigation units DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Policies and laws developed to institutionalize effective investigation and prosecutorial approaches Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Percentage of cases that reach a plea agreement (also known as plea bargain) in targeted specialized investigation units within AG Offices. This indicator is calculated by dividing the number of plea agreements by the total number of convictions obtained. Unit of Measure: Plea agreements reported by targeted investigation and prosecution units Disaggregated by: State. Rational or Justification for Indicator: Increasing the use of plea agreements/plea bargains is an objective of PROJUST’s work with target units because they unclog the system from unnecessary lengthy trials, provide prompt and expedited justice for all parties involved and indicate a strong investigation and prosecution by an AG Office when defendants are willing to accept responsibility for their crimes prior to a full trial. A plea bargain ensures a conviction while providing both the defense and the prosecution with influence over the final sentence to arrive at a resolution that is acceptable to the victim and the defendant. PLAN FOR DATA COLLECTION BY USAID Data Source: AG Office records/reports Method of Data Collection and Construction: Methods: Desk review; content analysis; physical verification. Construction: This indicator is calculated by dividing the number of plea agreements by the total number of convictions obtained. Reporting Frequency: Annually Individual Responsible for Acquiring the Data: PROJUST Task 1 Team Leader Role of this Individual on DO Team: NA DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A Known Data Limitations: AG Offices do not record data on plea bargains so PROJUST manually collects data for this indicator. Because of the sensitive nature of homicide casefiles, PROJUST is able to collect data for only units receiving active project assistance. For this reason, data is unavailable for units that USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 58 did not receive direct support throughout the duration of the project, and baselines were adjusted to reflect units that could be measured through the final calendar year of 2018. Actions Taken or Planned to Address Data Limitations: Given the absence of AG Office records, PROJUST manually collects data for this indicator, reviewing casefiles individually to ensure data quality. CHANGES TO INDICATOR Changes to Indicator: None Other Notes: Plea bargains on other crimes besides homicides could not be measured since project support to other specialized units was limited to less than a year and data on other crimes was not made available to the project. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 59 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.4.3 Percentage of cases with sentences greater than 3 years DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Policies and laws developed to institutionalize effective investigation and prosecutorial approaches Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Percentage of convictions in trial and plea bargaining (measured and reported separately) in which the time of the sentence is more than 3 years. This indicator will be calculated by dividing the number of convictions for which more than three years sentence is applied by the total number of convictions (in both trials and plea bargaining) in targeted districts receiving project support to improve the prosecution of crimes. Unit of Measure: Convictions issued in all types of crimes in targeted districts receiving project support to improve the prosecution of crimes. Disaggregated by: Convictions in plea bargaining and trial. Rational or Justification for Indicator: This indicator measures the convictions in trial and via plea bargaining in which the time of the sentence is more than three years. Added to PROJUST’s M&E Plan in November 2017 at the request of USAID, this indicator is a hybrid between context and custom indicator given that the data reflects the overall performance of the justice system in a given district and there are a number of factors outside PROJUST’s control. PLAN FOR DATA COLLECTION BY USAID Data Source: State Court records Method of Data Collection and Construction: Methods: Desk review; content analysis; physical verification. Construction: This indicator will be calculated by dividing the number of convictions for which more than three years sentence is applied by the total number of convictions (in both trials and plea bargaining) in targeted districts receiving project support to improve the prosecution of crimes. The steps to data collection and constructions are the following: a) Give formal notice and request to state coordinator, who will approach the liaison to ensure the quality and compliance of the data. Generally, the request if for a full database, in order to obtain all insight and have better source of information. b) Once the information is received, a process of cleaning and normalization is held by a consultant. This ensures that all information is comparable and in a friendly database for further use. c) Once the database is finalized, all calculations are made. Indicator’s considerations: a) All information is per person, with a sentence meaning that is being carried out by a sole defendant. b) Reported by: 0-3 years, or (0,3]; and (3 or more) Reporting Frequency: Annually USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 60 Individual Responsible for Acquiring the Data: PROJUST Task 1 Team Leader Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A Known Data Limitations: A target could not be stablished, since it is influenced by the type of crime committed, the circumstances and difficulties to achieve convictions, and the Criminal Codes for each state (years of sentence differ greatly). The data is reported for all types of crimes, not just homicides as in the above indicators, and includes cases that reached a conviction in a given year regardless of the year the case was opened. Additionally, although this indicator is measured for those states receiving support for improved prosecution rates, the data is provided by the State Court. Actions Taken or Planned to Address Data Limitations: CHANGES TO INDICATOR Changes to Indicator: This indicator is a hybrid between context and performance/custom indicator, given the numerous factors outside the project’s control that could influence achievement or underachievement of the target for this indicator. As a result, a life-of-project target could not be set. Other Notes: Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 61 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 1.5.1 Rate of effective debates executed by targeted public defenders DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Percentage of effective debates in initial hearings presented by public defenders. This indicator will be calculated by taking the number of debates that received a favorable ruling over the total number of debates presented by public defenders. Only for initial hearings. Unit of Measure: Debates by public defender Disaggregated by: Type of resolution and gender of defendants. Rational or Justification for Indicator: Part and parcel of the accusatory system is the premise that defendants enjoy the right to a competent defense that upholds due process and limits unnecessary restrictions on their liberty while awaiting a final sentence. Accordingly, PROJUST provided direct support to public defenders to improve their performance when debating in initial hearings, a key moment in criminal proceedings when probable cause is established and precautionary measures are imposed. PLAN FOR DATA COLLECTION BY USAID Data Source: JSI records (Public Defenders’ Offices) Method of Data Collection and Construction: Methods: Case files review Construction: This indicator is calculated by taking the number of debates that received a favorable ruling over the total number of debates presented by public defenders. Considerations: a) A presented debate is that in which a public defendant states a counterargument during a hearing. b) A successful debate is when that counterargument achieves the solicited change. Data collection method: a) Calculation of a representative sample amongst all initial hearings in which a public defender is present. b) Analysis and data entry for each hearing in the sample via a trained consultant (using a specialized excel template). c) Once all information is processed, final calculations are obtained. Reporting Frequency: Annually, however measurements are calculated for six-month periods. Individual Responsible for Acquiring the Data: PROJSUT Specialist in Public Defense USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 62 Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A Known Data Limitations: Public Defenders’ Offices do not collect this data. Actions Taken or Planned to Address Data Limitations: PROJUST manually collected data via a specialized format designed for this purpose. Data quality was ensured through a review of a representative sample of hearing recordings. CHANGES TO INDICATOR Changes to Indicator: None Other Notes: Given the late introduction of this indicator, PROJUST established the baseline and took measurements semi-annually in order to have more data points to chronicle incremental changes. Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 63 PROJUST PERFORMANCE INDICATOR REFERENCE SHEET Indicator Number and Name: 2.1.1 # of targeted JSIs that have quality controls to monitor and measure performance DO Number and Name: DO 2: GOM Legitimacy Increased IR Number and Name: Effectiveness of justice sector institutions enhanced Sub-IR Number and Name: Promote efficiency of justice sector processes Performance Plan and Report (PPR) Indicator? No Standard or Custom? Custom DESCRIPTION Precise Definition(s): Total number of justice institutions that have specialized units in charge of monitoring and evaluating institutional performance (through performance indicators, satisfaction surveys, measurement of process discipline, for example) with the objective of identifying performance gaps and generating responses for continuous improvement. An M&E Unit is considered to be established when step 5 of the implementation methodology is reached.33 Unit of Measure: Targeted JSIs. Disaggregated by: JSIs and state. Rational or Justification for Indicator: Task 2 of PROJUST increases the capacity for monitoring and evaluation of institutional performance in the criminal justice system. The project focuses efforts under this Task on the establishment of Monitoring and Evaluation (M&E) Units in targeted state JSIs. PLAN FOR DATA COLLECTION BY USAID Data Source: JSI records/reports; PROJUST records /reports Method of Data Collection and Construction: Methods: Desk review; content analysis Construction: count Reporting Frequency: Annually Individual Responsible for Acquiring the Data: PROJUST Task 2 Team Leader Role of this Individual on DO Team: N/A DATA QUALITY ISSUES Dates of Past Data Quality Assessments and Name of Reviewer: None Date of Next Data Quality Assessments: N/A Known Data Limitations: None 33 The methodology to establish M&E Units and Quality Control Consultative Committees, a later stage of the process, consists of 10 steps. Step 5 (Planning for improvement plan: selecting results targets) is the point of establishment of an M&E unit. USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 64 Actions Taken or Planned to Address Data Limitations: N/A CHANGES TO INDICATOR Changes to Indicator: Minor change to indicator name; revised definition Other Notes: Date Sheet was Last Updated: May 2019 by: PROJUST and was last reviewed & approved on: June 21, 2019 by PO staff member Clare McInerney USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 65 ANNEX IV: METHODOLOGICAL NOTE This methodological note serves to clarify three key specifications regarding how PROJUST measures its indicators: Calendar year vs. Fiscal year Of the eight contractual indicators reported by PROJUST, all but two are measured on a calendar year basis (January through December) as opposed to a fiscal year basis (October through September). The two indicators measured on a fiscal year basis are the following output indicators: 1.1.1-- # of targeted JSIs with procedures and structures aligned to the NCPC and secondary legislation 2.1.1-- # of targeted JSIs that have quality controls to monitor and measure performance Additionally, all USAID Development Objective 2 and Foreign Assistance Indicators (also at the output level) were also measured on a fiscal year basis. These include: DO2c-- Number of states receiving technical assistance on comprehensive criminal justice reform. (Note that the definition of this indicator specifically refers to prosecutors, judges and public defenders.) IR 2.1a--Number of laws and regulations supporting the implementation of the new criminal justice sector reforms passed IR 2.2c & 2.1.2-7-- Number of judges and judicial personnel trained (disaggregated by sex) Sub-IR 2.2.1b-- Number of justice sector representatives with increased knowledge based on pre￾and post-tests The calendar year measurement period is due to the fact that Mexican justice institutions collect and report data on institutional performance on a calendar year basis and this information is used to calculate outcome level indicators. PROJUST’s reporting on progress against indicators was submitted on a fiscal year basis in conjunction with Annual Reports, however data for January through September of a given year was not available from target institutions at the time of those reports. Original baselines vs. adjusted baselines Baselines for all contractual indicators were measured using the average of the first-year measurement for each target institution, unit, or district. In the majority of cases, the first year measured was the year prior to the start of PROJUST support. Those instances where this is not the case are detailed previously in this report under the corresponding indicator. For half of the contractual indicators (four of eight), original baselines established and reported in 2015 or 2017 were adjusted to reflect the changes in target institutions/units/districts over the final years of the project. These adjusted baselines are provided where relevant in Annex II: “Updated Performance Monitoring Plan (PMP)” and reflected under the corresponding section for those indicators throughout the report. Responding to both contract modifications and guidance from USAID, following 2016, PROJUST focused its activities on fewer states and only those justice USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 66 institutions where sufficient political will and institutional buy-in could be established for collaboration with technical assistance. This adaptability to changing political climates resulted in decisions each year to withdraw project support from some institutions/units/districts and redirect it towards others where conditions were more favorable. Given that in some cases, the baselines established did not reflect any of the target institutions/units/districts supported in the final year(s) of the project, PROJUST felt it appropriate to provide adjusted baselines that could be accurately compared to final measurements by considering the same target institutions/units/districts measured. Prosecution Rate Changes to the methodology for measuring prosecution rates occurred over the life of the project as PROJUST and USAID sought to best capture the results of technical assistance to targeted Specialized Investigation and Prosecution Units within Attorneys General Offices. The definition of the indicator remained unchanged: “Annual prosecution rates for homicides are calculated by obtaining the total number of indictments (acusaciones), divided by the total number of investigations conducted by targeted investigation and prosecution units.” However, the interpretation of this definition underwent one primary change and one secondary change (which is no longer relevant at the time of this reporting and is not reflected in this report). The primary change was that while PROJUST originally understood “the total number of investigations conducted” to refer only to those cases that entered target units in a given year, technical reviews with USAID later clarified (in 2018) that the focus was rather on all open cases, regardless of the year they entered. In Spanish, the terminology of “con pendencia,” (with backlog/pending cases),” versus “sin pendencia” (without backlog), is the imperative term, and underscores the fact that to not consider those cases backlogged does not give a full representation of the workload faced by a target unit. The secondary change was that, under the original interpretation of the definition, PROJUST began reporting prosecution rates for target units over a two-year measurement period. Based on the understanding that most homicide cases cannot be solved within a single calendar year, given the complexity of such cases, a certain year’s cases were considered a cohort that was then tracked over multiple years to show how most cases were eventually prosecuted, just not within the same calendar year they entered a target unit. These multi-year measurements for prosecution rates are provided in previous Indicator Progress Reports submitted by PROJUST with Annual Reports to USAID but are not included in this final report based on the current understanding of the indicator. End of Project Result vs Final Year Result For all indicators, the end of project result is the same as the final year result (calendar year 2018 or fiscal year 2019), given that all outcome level indicators are cumulative and all outcome indicators had life-of-project targets whose final measurements correspond to calendar year 2018. USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 67 ANNEXV: TOTAL PROJUST TRAINEES FOR ALL YEARS During PROJUST’s implementation, which covers the period from November 1, 2014 to May 31, 2019, the program delivered a total of 300 training courses (totaling 15,439 hours of instruction) that benefited 4,191 justice sector operators (an average of 76 trainees per month). From these 4,191, a number of trainees attended multiple courses, in some cases as many as eight. These participants have only been counted once in all data on trainees presented in this annex. For each training course, PROJUST reports the number of people that meet the course attendance requirement of at least 80% of all modules. In the reporting period, more than 90% of the participants met this criterion (3,788 trainees). From the total trainees, all of whom were justice sector operators, PROJUST’s training courses benefited alternative justice center staff, court clerks, forensic experts, judges, justices, mediators, police investigators, prosecutors, PTS staff, public defenders and victim’s assistance staff, from all 32 states as shown in Table 1 below. Female participants constituted 51.64% of all trainees. Table 1 on the following page details the number and percentage of justice sector operators trained in the beneficiary state. During PROJUST’s activities, the highest proportion of the trainees were prosecutors (29.3%), followed by PTS staff (20.5%). USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 68 Table 1: Justice Sector Operators - Number of Trainees by State ALTERNATIVE JUSTICE CENTER % COURT CLERKS % FORENSIC EXPERTS % JUDGES % JUSTICE % MEDIATOR % POLICE INVESTIGATORS % PROSECUTOR % PTS OPERATORS % PUBLIC DEFENDER % VICTIM'S ASSISTANCE STAFF % TOTAL % STATE TITLE Aguascalientes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 15 2 3 1 0 0 19 1 Baja California 0 0 14 3 2 7 3 1 0 0 0 0 23 14 51 5 1 0 58 10 0 0 152 4 Baja California Sur 0 0 23 4 0 0 3 1 0 0 0 0 0 0 44 4 52 7 10 2 0 0 132 3 Campeche 0 0 1 0 0 0 6 1 0 0 0 0 0 0 5 0 6 1 5 1 0 0 23 1 Chiapas 0 0 8 1 0 0 0 0 0 0 0 0 0 0 0 0 5 1 0 0 0 0 13 0 Chihuahua 1 0.002 97 18 0 0 134 29 0 0 0 0 1 1 150 14 104 13 43 7 1 2 531 14 Coahuila 0 0 55 10 7 24 38 8 1 3 4 40 5 3 121 11 30 4 54 9 0 0 315 8 Colima 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 1 0 0 0 0 5 0 Durango 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 1 20 3 0 0 0 0 33 1 Federal District 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 5 1 0 0 0 0 6 0 Guerrero 1 0 9 2 0 0 0 0 0 0 0 0 0 0 12 1 0 0 14 2 0 0 36 1 Guerrero 0 0 20 4 0 0 0 0 0 0 0 0 0 0 25 2 3 0 18 3 0 0 66 2 Hidalgo 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 54 7 0 0 0 0 57 2 Jalisco 0 0 5 1 0 0 9 2 0 0 0 0 0 0 21 2 3 0 5 1 0 0 43 1 Michoacan 0 0 21 4 0 0 0 0 0 0 0 0 0 0 0 0 75 10 0 0 4 7 100 3 Morelos 0 0 0 0 0 0 19 4 1 3 0 0 3 2 34 3 19 2 18 3 0 0 94 2 Nayarit 0 0 78 14 4 14 20 4 21 60 3 30 2 1 68 6 4 1 7 1 0 0 207 5 Nuevo Leon 0 0 67 12 0 0 35 8 1 3 0 0 3 2 66 6 57 7 54 9 4 7 287 8 USAID/MEXICO PROMOTING JUSTICE PROJECT –MONITORING AND EVALUATION REPORT 69 ALTERNATIVE JUSTICE CENTER % COURT CLERKS % FORENSIC EXPERTS % JUDGES % JUSTICE % MEDIATOR % POLICE INVESTIGATORS % PROSECUTOR % PTS OPERATORS % PUBLIC DEFENDER % VICTIM'S ASSISTANCE STAFF % TOTAL % STATE TITLE Oaxaca 0 0 0 0 0 0 25 5 0 0 0 0 0 0 68 6 35 5 51 9 0 0 179 5 Puebla 0 0 1 0 0 0 4 1 5 14 0 0 0 0 34 3 2 0 32 5 0 0 78 2 Queretaro 0 0 1 0 0 0 8 2 0 0 0 0 0 0 7 1 63 8 14 2 0 0 93 2 Quintana Roo 0 0 0 0 0 0 0 0 1 3 0 0 0 0 10 1 7 1 0 0 0 0 18 0 San Luis Potosi 0 0 37 7 0 0 19 4 3 9 0 0 13 8 71 6 35 5 67 11 18 32 263 7 Sinaloa 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 1 0 0 0 0 4 0 Sonora 16 0.029 0 0 2 7 4 1 0 0 3 30 6 4 67 6 48 6 28 5 2 4 176 5 State of Mexico 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 1 0 0 0 0 6 0 Tabasco 0 0 0 0 7 24 20 4 2 6 0 0 67 41 120 11 29 4 23 4 20 36 288 8 Tamaulipas 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 3 0 Tlaxcala 0 0 0 0 0 0 10 2 0 0 0 0 0 0 13 1 24 3 15 3 0 0 62 2 Veracruz 0 0 0 0 0 0 10 2 0 0 0 0 13 8 22 2 5 1 19 3 0 0 69 2 Yucatan 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 1 0 0 0 0 4 0 Zacatecas 0 0 110 20 7 24 91 20 0 0 0 0 28 17 85 8 53 7 45 8 7 13 426 11 Total 18 0.03 548 14.5 29 0.8 459 12.1 35 0.9 10 0.3 164 4.3 1,110 29.3 776 20.5 583 15.4 56 1.5 3,788 100