July 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Tetra Tech DPK PROGRAM AGAINST VIOLENCE AND IMPUNITY IN GUATEMALA MONITORING AND EVALUATION FINAL REPORT JUNE 2009 - JUNE 2012 PROGRAM AGAINST VIOLENCE AND IMPUNITY IN GUATEMALA MONITORING AND EVALUATION FINAL REPORT JUNE 2009 - JUNE 2012 PROGRAM: PROGRAMA DE USAID JUSTICIA CONTRA LA VIOLENCIA USAID OFFICE: USAID/GUATEMALA TASK ORDER: AID-DFD-I-08-04-00173 CONTRACTOR: TETRA TECH DPK DATE: JULY 2012 The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. July 2012 . TABLE OF CONTENTS LIST OF ACRONYMS............................................................................................................................... 1 I. INTRODUCTION................................................................................................................................... 2 A. The Program Against Violence and Impunity (PAVI) ...................................................................... 2 B. PAVI’S Approach to Monitoring and Evaluation .............................................................................. 3 C. Monitoring and Evaluation Methodology ........................................................................................... 4 D. M&E Lessons Learned ............................................................................................................................ 6 II. ANALYIS OF M&E RESULTS ......................................................................................................... 8 A. Overall Project Results .......................................................................................................................... 8 B. Results by Sub-IR and Indicators ........................................................................................................14 Sub-IR1: Improve Justice System Capacity to Prosecute and Try Serious Crimes ...............14 Sub-IR2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence ..........19 Sub-IR3: Increased Internal Accountability and Oversight Within the Justice Sector .........24 Special Activities ...................................................................................................................................29 III. LESSONS LEARNED........................................................................................................................32 A. Activities in Petén ..................................................................................................................................32 B. Grants Programs ....................................................................................................................................33 ANNEXES A. SUCCESS STORIES/CASE STUDIES - Developing Teamwork in the Justice System’s Performance Assessment Unit - Fostering Emotional Resilience in Petén’s Office of Victims Attention - Bringing Environmental Justice to the Mayan Biosphere Reserve - Helping At-Risk Youth Create a Better Life - Providing 24-Hour Judicial Services to Mixco, Guatemala - Training Guatemala’s Criminal Prosecutors - Reducing Gender-based Violence in Izabal B. FEMICIDE CASE STUDIES C. 2011 M&E PLAN (Revised and Final Version) D. 2010 M&E ANNUAL REPORT (Attached Separately) E. 2011 M&E ANNUAL REPORT (Attached Separately) 1 LIST OF ACRONYMS COR CSO IDPP Contracting Officer’s Representative Civil Society Organization Instituto de la Defensa Pública Penal- Public Defense Institute IIM Institutional Integrity Model MP Ministerio Público – Public Ministry M&E Monitoring and Evaluation OAV Oficina de Atención a la Víctima del Ministerio Público – Public Ministry’s Office of Victims Assistance OJ Organismo Judicial – Judiciary PAVI Program Against Violence and Impunity PDC Partners for Democratic Change SICOMP Sistema de Información y Control del Ministerio Público – Public Ministry’s Control and Information System Sub IR Sub-Intermediate Result Tt DPK Tetra Tech DPK TTC Information Technology and Telecommunications Center USAID United States Agency for International Development 2 I. INTRODUCTION The following Final Monitoring and Evaluation (M&E) Report covers all the M&E activities of the Program Against Violence and Impunity (PAVI or Project). PAVI is a USAID-funded initiative in Guatemala, implemented by contractor Tetra Tech DPK (Tt DPK) from September 1st 2009 through June 28th 2012. The project aimed to strengthen the rule of law by improving the delivery of judicial and prosecutorial services, heightening coordination among justice sector institutions, and building civil society’s capacity to monitor and report on the performance of the justice operators. In order to produce this Final M&E Report, the Tt DPK team gathered and consolidated data on performance and impact indicators documented in two Annual M&E Reports and six quarterly reports produced throughout the life of the Project, interviewed key PAVI counterparts, and met with USAID/Guatemala and PAVI’s staff. 1 This final report provides an overview of the implementation of the M&E strategy, an analysis of the overall Project results (based on performance and impact indicators consolidated data) and lessons learned for future interventions. A. THE PROGRAM AGAINST VIOLENCE AND IMPUNITY IN GUATEMALA (PAVI) The Project combined five integrated Sub-Intermediate Results (Sub-IRs) and Special Activities2 : a) Sub-IR 1: Improved Justice System Capacity to Prosecute and Try Serious Crime b) Sub-IR 2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence c) Sub-IR 3: Increased Internal Accountability and Oversight within the Justice Sector d) Special Activity 1: Supporting High-Impact Courts e) Special Activity 2: Strengthen Justice Sector Capacity to Combat Illegal Activities in Petén 1. Sub Intermediate Result 1(Sub IR 1) – Improved Justice System Capacity to Prosecute and Try Serious Crimes: Using newly passed laws on prosecution and working with already established crime prevention units, activities under this component sought to aid in setting up unified criteria for how to investigate and prosecute serious crimes. Key activities included helping to develop effective information management methodologies and tools within the Public Ministry and helping to implement key legislation to improve the prosecution of serious crimes. 1 Meetings conducted by Partners for Democratic Change (PDC) Director Lelia Mooney and PDC Manager Jessica Varat included USAID/Guatemala Contracting Officer’s Representative (COR) Cassandra Wheeler, President of the High Impact Court ―A‖ Justice Jazmin Gálvez Barrios, Chief Justice of the High Impact Court ―B‖ Justice Miguel Angel Galvez, Director of Guatemaltecos Extraordinarios Juan Carlos Molina, Prosecutor Ingrid Xuc from the Public Ministry’s Analysis Unit, Chief of the Judicial Section of Villa Nueva, Saul Estuardo Pensamiento and Prosecutor Hugo Rosales from the Litigation Unit of the Prosecutor for Crimes against Life, and the PAVI team. 2 On February 3rd, 2011, USAID informed Tt DPK about the cancellation of Special Activity 3 ―Millennium Challenge Indicators Promoted‖, which was included in the original Task Order scope of work. 3 2. Sub IR2 – Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence: The Project helped increase availability of effective governmental and non￾governmental legal aid services for victims while also helped to improve coordination among these services. 3. Sub IR3 – Increase Internal Accountability and Oversight within the Justice Sector: By setting up the Institutional Integrity Model, the Project contributed to the continued improvement of accountability and transparency in justice sector institutions. Other key activities included helping to complete the Public Defender Institute’s evaluation system, working with judiciary staff to analyze recruitment, selection, and promotion systems, and assisting in creating a network of positive leadership within the institutions. 4. Special Activity 1 – Support High Impact Courts: The Project helped judicial and prosecutorial leadership to design and establish high impact courts that focus on combating organized crime, kidnapping, and drug and human trafficking cases. This included providing guidance and support for the implementation of successful court models and their infrastructure with clear definitions to what falls into their jurisdictions. 5. Special Activity 2 – Strengthen Justice Sector Capacity in Petén: The Project assisted in conducting gap analysis of court and prosecutorial current practices for the investigation and preparation of cases. The caseload in Petén was linked to other project￾supported activities, such as the high-impact courts, for effective handling of the more serious crime cases. B. PAVI’S APPROACH TO MONITORING AND EVALUATION PAVI’s approach to M&E was a combination of internal or self-evaluations (conducted by the implementing project team), external (conducted by Partners for Democratic Change, an outside organization not directly associated with the project activities3 ), and participatory evaluations (conducted with the participation of key Guatemalan government institutions, civil society organizations, and other key stakeholders). PAVI had two dedicated staff personnel working on M&E: the Deputy Project Director and the Planning and Supervision Assistant. They were in charge of supporting M&E activities (management, data collection, and information processing) in Guatemala and collaborating with the external evaluator on the development of PAVI’s M&E reports. Tt DPK engaged PDC to support with the development of PAVI’s M&E Plan, design of data collection methodology and instruments, and drafting of PAVI’s M&E reports on a quarterly and annual basis. The following chart summarizes the Project’s M&E approach: 3 Tt DPK executed a subcontract with PDC for the provision of M&E technical support to the PAVI team and to act as an external evaluator (not independent) of the Program. 4 PAVI’S MONITORING AND EVALUATION APPROACH C. MONITORING AND EVALUATION METHODOLOGY Conceptual Framework Although the terms ―Monitoring‖ and ―Evaluation‖ are used together, these are two different functions, with different conceptual and operational implications. Performance monitoring is an on-going process intended to provide information to managers about the progress of specific project outputs according to fixed objectives and targets. PAVI’s M&E Plan set a series of indicators to monitor the performance of project activities against agreed targets. Evaluation is a more comprehensive concept. It looks at the effects and impact of activities to help establish lessons learned and prioritize development strategies. PAVI’s M&E Plan also set a Data Evaluation Responsible parties:  External = PDC  Internal = PAVI Coordination and Technical Support Responsible party: Tt DPK Home Office Data Collection Responsible party: PAVI Data Reporting and Analysis Responsible parties: PDC & PAVI Report Evaluation Responsible party: USAID 5 series of indicators and targets to measure, to a limited extent4 , the impact of Project activities. Indicator Selection Determining adequate and appropriate indicators is not an exact science and the precision of a laboratory experiment is not required, especially when operating in a data-poor development setting such as Guatemala. Notwithstanding these shortcomings, the indicators were carefully selected to reflect USAID’s criteria of integrity, accuracy, reliability, and timeliness in its M&E exercises. The indicators used in the PAVI M&E Plan were crafted to reflect objectivity, practicality, validity, and attribution to the Project. The objective of the M&E Plan was to specify a full set of indicators from the outset, acknowledging that some would need to be adjusted, revised, or eliminated as the Project activities unfolded. Over the life-cycle of the Project, different types of indicators proved more useful than others. In the first six months of the Project, process (output) indicators were the primary source of performance information, while during the second and third years the impact indicators became more revealing. Indicator Measurement Schedules The two sets of performance and impact indicators required separate data reporting schedules. The performance indicators were measured on a project-year basis in order to encompass as much of the project activities as possible.5 Impact indicators were measured using data that was collected primarily from the Public Ministry’s SICOMP database, which issues reports at the end of the calendar year. As a result, progress on the impact indicators was reported on a calendar-year basis. Due to this fact, the information collected for 2012 was partial (through April or May) and could not be equated with previous, complete impact indicator reporting periods. For this reason, general data projections have been estimated for the remainder of 2012, based on previous years’ project experience, to suggest possible trends that could be comparable with 2010 and 2011 figures. Data Collection Many of the quantitative indicators required Project staff to obtain official data from the government institutions with whom they were working. Data collection methods within the judicial branch and legal system in Guatemala are less than perfect, and in fact, the Project offered technical assistance to improve justice system internal information systems. Monitoring activities assisted PAVI staff in conducting brief spot checks to verify data reporting within a sampling of the Public Ministry units whose data was used for the Project’s indicators (and other institutions offering victim’s assistance as well). As the project progressed and data collection and reporting systems became more reliable and routine, these activities became less necessary. 4 The monitoring and reporting of impact indicators were intended to provide general information about the potential impact of the Project and inform the implementation of activities. This work did not follow nor constitute an in-depth impact evaluation effort as defined by USAID’s 2011 Evaluation Policy. 5 The Project Year 1 corresponds to the period from July 29, 2009 to September 30, 2010. The Project Year 2 corresponds to the period from October 1, 2010 to September 30, 2011. The Project Year 3 corresponds to the period from October 1, 2011 to June 28, 2012. 6 Other data was collected directly through internal Project documentation and records, including participant surveys applied by the PAVI staff, personal interviews conducted with project stakeholders, and information to build case studies and success stories. The following articles of the Guatemalan Criminal Code define the crimes that were tracked by the PAVI staff under the different indicators:  Crimes against life: (a) Homicide: Articles 123 and 124, (b) Infanticide: Article 129, (c) Parricide: Article 131, (d) Murder: Article 132; and Femicides: Article 6, Law Against Femicides and Other Forms of Violence Against Women (Decree 22.2008)  Crimes against cultural patrimony: Articles 332 A, B, and C.  Crimes against the environment: Articles 346 and 347. M&E Plan Revisions The data and analysis presented in this final project report reflect the final iteration of an M&E system that has evolved over the course of the project. At various points, the M&E Plan – originally approved in March 2010- was revised and adjusted to respond to changing realities and lessons learned throughout Project implementation. Any and all amendments to the M&E plan were made by the PAVI project team and approved by the USAID COR. These adjustments included the following:  In September 2011, the original M&E Plan was revised to set 2008 as the baseline for all Project indicators—replacing the previously established baseline of 20096 , eliminate certain indicators7 , and adjust several of the targets  Certain indicators are measured by percentage increases. Previous M&E plans and reports calculated these increases relative to the baseline. Upon USAID COR’s request on July 11, 2012, this report measures target rates of improvement or change relative to the previous year, as opposed to the project baseline. The cut-off date to prepare this Final Report was May 31, 2011. Hence the report presents data through that date. For certain impact indicators that measure progress through percentage rate changes related to previous calendar years, the report uses the January-May data to estimate projections through the end of 2012. This facilitates the analysis of impact indicators. D. M&E LESSONS LEARNED As previously noted, Tt DPK has offered a unique evaluation strategy by bringing in an external team of evaluation specialists early in the execution of the Project in order to ensure an objective yet collaborative approach to M&E that involved PAVI’s technical staff and counterparts in all phases of monitoring performance and evaluation of this Project. 6 Data for 2009 have been presented in previous reports. Due to the fact that they previously served as the Project baseline, targets were not established for this year 7 Upon discussions with USAID, the following indicators were eliminated: Indicator 1, Number of Institutional Improvement Plans developed; Indicator 10, Number of Institutional Plans of Action to implement the Institutional Integrity Model; Indicator 12, Level of progress in the implementation of the Institutional Plans of Action within target justice sector units that result from the Institutional Integrity Model workshops; and Indicator 16, Number of MCC stakeholders who receive outreach and training in data generation and their reporting on Rule of Law indicators. 7 The lessons learned from this approach to evaluation are the following: • Involvement of field office personnel in M&E efforts ensures accurate readings and reporting of project activities and results. A collaborative process whereby the external evaluation specialists elicited the PAVI technical staff’s input on the revision of the M&E Plan ensured that indicators accurately reflected project outputs. This allowed the external evaluation specialists to account for changing realities on the ground and avoid a cookie-cutter approach to monitoring and evaluation • Regular visits to the field guarantee a more integral and richer M&E analysis. Quarterly monitoring trips and close communication with the PAVI technical staff in charge of data collection allowed the external evaluation specialists to not only focus on project outputs, but also on the processes that led to these outputs. As a result, the external evaluation team was able to capture additional outcomes and produce a richer analysis of the data. For example, on-site interviews with PAVI’s grantee organizations in Peten were fundamental to understanding the linkages between the Project’s grants program and its overall project objectives. This allowed for a better assessment of the Project’s impact. • M&E collaborative approaches require close and constant communication among the involved parties to ensure an effective and efficient work. Tt DPK HO played a relevant role coordinating the communication and collaboration between PDC and PAVI staff. Thanks to this support, clarifications from the field were promptly provided and observations to draft reports adequately addressed 8 II. ANALYSIS OF M&E RESULTS A. OVERALL PROJECT RESULTS PAVI showed overall strong results in performance indicators. Its training activities across Sub￾IRs met demonstrated needs for capacity development, reached a wide range of participants from the target sectors and institutions, and generated high levels of satisfaction among training participants. Impact indicators demonstrated more varied progress over the lifetime of the project. Results in impact indicators can be characterized as generally positive, particularly in PAVI’s early gains over baseline levels, while the impact of external conditions and prevailing trends in Guatemala’s challenging setting were apparent in some indicators. The analysis of certain impact indicator results and accompanying qualitative data demonstrated the extent to which successful PAVI interventions in developing justice system capacity complicated the task of measuring certain impact indicators. For example, PAVI’s intervention improving Guatemalan prosecutors’ strategic development of cases and complex litigation skills could not be measured simply by the number of discrete cases brought to trial. As the prosecutors’ skills improved, individual cases were linked under more complex ones—thus reducing the total number of cases brought to trial—that sought to dismantle criminal organizations. These trends and nuances of PAVI’s monitoring and evaluation are further elaborated later in the report. The following sections provide a general analysis of key areas of PAVI’s activity. These syntheses and conclusions emerge from the quantitative data represented by the indicators and are further refined by the qualitative data gathered over the course of the project. Specific, data breakdowns of each indicator follow in the analysis by Sub-IR and Indicator section. Advancing Investigative and Prosecutorial Capacity Monitoring and evaluation of PAVI’s work under Sub-IR1, ―Improve Justice System Capacity to Prosecute and Try Serious Crimes,‖ focused on PAVI’s prosecutorial training programs. Following a participatory needs assessment conducted in collaboration with justice system prosecutors and administrators, PAVI designed and conducted a series of workshops and trainings relating to strategic litigation of complex cases involving analysis, organization and systematization of information, establishing connections between distinct cases, and collaborative case development. PAVI exceeded performance indicator targets in the implementation of these trainings, workshops, and coordination meetings, in the number of participants in these events, and in the level of participant satisfaction with these events. Participant feedback was overwhelmingly positive, noting that the content responded directly to weaknesses in participants’ skill sets and the presentation was clear. The most frequently 9 registered suggestion was that more trainings be held and with greater regularity, integrating them into the work life of Guatemalan prosecutors. PAVI responded to this high demand by increasing training activities, presenting more than double the target number of trainings in project years 2 and 3. A series of post-training interviews with Hugo Rosales, prosecutor for the Litigation Unit of the Prosecutor’s Office for Crimes against Life, provided specific examples of the trainings’ impact on his prosecutorial skills. Lic. Rosales explained that over the course of his 3-day training he learned concrete litigation skills and legal techniques that allowed him to successfully manage the increasingly complex cases that were being assigned to him. He noted that the need for complex litigation training is widespread, as laws concerning organized crime and related investigative techniques have not been in place for long in Guatemala so justice sector professionals have had little time to respond to their new demands. PAVI trainings effectively responded to this issue, not only by addressing case theory, evidence preparation, and litigation skills, but also through briefings on the use of illicit association charges in cases involving organized crime and multiple defendants. Lic. Rosales has been successful in the use of these skills and techniques, noting that in one case involving conspiracy to commit murder, international trafficking, money laundering, and illicit association, he successfully secured guilty verdicts and maximum sentences for 48 defendants. As a result of these efforts, impact indicators generally demonstrated significant increases in crimes brought to trial and guilty verdicts issued in the Department of Guatemala. It is worth noting that following initially robust growth in these indicators in 2009 and 2010, this rate of growth began to slow and even decrease in 2011 and 2012. The reason for this pattern can be partly attributed to the nature of PAVI’s trainings, as prosecutors gained new skills and capacities in handling complex cases involving multiple defendants, cases were consolidated and more selectively prepared for greatest impact, leading to a lower number of cases even as cases began to yield stronger results. A related and essential PAVI intervention in this area was its work to upgrade and implement the Sistema de Información y Control del Ministerio Público (SICOMP), a nation-wide database used by justice system officials to store, monitor, exchange, and analyze data. Over the course of the project, SICOMP has become increasingly integral to daily justice system operations, in addition to broader internal monitoring and evaluation efforts. PAVI’s specific activities in this area focused on the upgrade from SICOMP I to SICOMP II, as well as supporting the transfer of SICOMP II to the Petén region. According to a member of the technical expert team involved in the SICOMP II upgrade, the SICOMP II program surpasses the technical capabilities of SICOMP I on many levels. Since its inception, the system has constantly undergone upgrades and improvements. The technical team designed these improvements in response to the needs of various users with the goal of making the program more user-friendly. New features include photo albums for images of crime scenes, the use of an internet server so that information can be accessed in real time, and access to audio files recorded from court hearings. There have also been improvements in the search function, the ability to cross-reference cases, and in system security. Another important improvement in 10 SICOMP II is that the centralized server allows updates to be made to the program from the main server; in the previous system, updates were made to individual computers. On the other hand, the process of upgrading the system within the MP has been slowed down by the computers’ insufficient memory to fully implement the upgrades. To address this situation, PAVI procured a significant number of RAM memories for the SICOMP, though not all computers were able to be upgraded in this manner, leading some prosecutors to continue to report challenges with their computers’ processing speed. Lic. Rosales illustrated the concrete value of the SICOMP upgrades, reporting that his colleagues in the Analysis Unit have begun to use SICOMP’s cross-referencing capability to link cases, thereby performing better analysis. In Petén, PAVI struggled to meet impact indicator targets due to a number of challenging conditions in the remote department. While the PAVI project made some early progress, meeting targets for crimes against life brought to trial in Petén, indicators reflected a general downward trend following a highly publicized massacre committed by an illicit criminal group in May of 2011. In addition to terrorizing residents and solidifying perceptions of rampant criminal activity and impunity in the region, it precipitated the government’s declaration of a state of siege that lasted through January of 2012. These challenges were further compounded by a change in leadership of the prosecutor’s office in Petén and the resulting period of re￾orientation and adjustment. Impact indicators relating to femicide in Petén illustrated the added difficulty of addressing a social and legal challenge that has been growing at an alarming rate in recent years. Numerous recent studies and the government’s recent launching of a dedicated taskforce to combat femicide have reinforce PAVI’s experience in this area: the issue of femicide is one of the most challenging citizen security issues confronting Guatemala due to the combination of criminal, social, economic, and institutional factors involved. Victims Assistance: An Inter-institutional Analysis Monitoring and evaluation of PAVI’s work under Sub-IR2, ―Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence,‖ focused on improving victim assistance services and promoting coordination between institutions working with victims and witnesses of serious crimes. In collaboration with justice sector officials, PAVI developed the Protocol for Persons Involved as Witnesses in Criminal Trials (the Protocol), a uniform set of procedures and standards meant to define performance standards for personnel working with victims and establish standards and procedures for the safe and secure provision of services to victims and witnesses throughout all stages of a case. Performance indicator targets relating to this activity included number of workshops, coordination meetings, and trainings for institutions offering victims assistance, number of participants in these events, and participant satisfaction rates. PAVI exceeded all targets in these performance indicators, providing extensive trainings that generated strong levels of participant satisfaction. Participants reported that the trainings offered concrete examples and strategies 11 for their work and recommended more comprehensive trainings at all levels and throughout the various institutions of the justice sector. PAVI met all targets in its performance indicator measuring levels of improved coordination between organizations offering victim assistance through the implementation of the Protocol. This reflected PAVI’s effective involvement of all relevant institutions and stakeholders in the development of the Protocol, ensuring local ownership of the process and investment in its success. Nonetheless, results in indicator 7, which measured increases in numbers of victims assisted by four participating institutions, were more varied. While the total number of people assisted rose over the course of the project, demonstrating progress in extending services to a larger population of victims, differences in service provision within the institutions led to missing some of the targets while others continued to see large increases in victims served. Specifically, trends over the project reporting periods revealed diminishing numbers of victims served by the Public Ministry Office of Victims Attention while the Public Defender’s Office of Free Legal Aid made strong strides in increasing service provision. Users reported preferring the Public Defender’s Office because it provided more efficient and immediate service. So while some institutions missed targets, overall the numbers indicate a natural movement of users between different institutions as they pursue optimal services. Uneven Implementation of the Protocol for Persons Involved as Witnesses in Criminal Trials: 4 Femicide Case Studies As was mentioned previously, the prevalence of femicide in Guatemala has been rising in recent years due to a number of factors, with public security and justice sector institutions struggling to address the issue. As a result, the number of femicide cases brought to trial and receiving guilty verdicts is significantly lower than crimes against life in both the departments of Guatemala and Petén. The Protocol was designed with particular attention paid to the challenge of protecting victims and witnesses while prosecuting crimes under the Law Against Femicide, a law passed in 2008 that was the first to codify into law femicide and other forms of violence against women. In light of this, PAVI selected four cases to study the implementation of the Protocol. These are presented in greater detail in Annex B. Generally speaking, the case studies reveal a significant amount of work that remains to be done in fully implementing the Protocol in cases of femicide. Even in those cases that have successfully advanced through the investigation and trial preparation phases—a significant achievement given the developing nature of the judicial organism’s capacities in this area—various steps of the Protocol are not fully implemented. The connection between implementation of the Protocol and effective prosecution of femicides is most acutely demonstrated in Case Study 2, in which two family members of the victim and presumed witnesses to the crime have refused to cooperate with the investigation, highlighting the issue of witness fear and intimidation. The other cases reported in the case studies, however, demonstrated better progress: as of July 2012, Case Study 1 had concluded with a guilty verdict for attempted femicide; Case Study 3 resulted in Petén’s first guilty verdict for Femicide; and the trial for Case Study 4 was 12 scheduled to begin in early August, with the prosecution in possession of strong, scientific evidence. Approach to Institutional Coordination and Culture Monitoring and Evaluation of PAVI’s work under Sub-IR3, ―Increase Internal Accountability and Oversight within the Justice Sector,‖ focused on PAVI’s promotion of the Institutional Integrity Model (IIM), a transversal strategy meant to reinforce human capital, develop identity among justice sector personnel, and develop professional capacities that contribute to increased efficacy, efficiency, and morale. PAVI exceeded performance indicator targets in this respect, holding a number of trainings addressing components of the IIM that involved hundreds of justice sector personnel and met with high levels of participant satisfaction. Participants particularly appreciated the unique nature of IIM trainings, addressing such issues as emotional resilience and mental health that are usually neglected in justice sector training programs. Furthermore, they found that these less technical, more participatory trainings provided a valuable opportunity for staff bonding and teamwork development, which directly translated into improved workplace communication and interpersonal dynamics following the trainings. PAVI met all impact indicator targets, demonstrating high degrees of approval from beneficiary institutions and participants in PAVI’s IIM promotion activities. Focus groups held with the Evaluation Unit of the Judiciary and personnel from the Public Ministry further detailed the results of IIM training activities, with participants noting that they were better equipped to manage stress, worked together more effectively as a team, had developed techniques for emotional resilience and psychological well-being and were better able to perform their jobs and serve members of the public. In another focus group session with prosecutors and Public Ministry personnel, participants discussed the extent to which the IIM trainings had empowered them to work effectively within challenging office environments defined by hierarchy and unbalanced power dynamics. Participants felt that the mental health and inter-personal office dynamics trainings had helped them to take ownership of their duty to provide necessary services to beneficiaries of the justice system, even when adverse conditions made it more difficult. Supporting High Impact Courts: Re-Building the Physical Infrastructure of Guatemala’s Justice System In addition to its broader work under the aforementioned Sub-IRs, PAVI focused on Special Activities of strategic value to its overall mission. Specifically, PAVI’s work to support high impact courts in Guatemala consisted of contributing to the design and remodeling of the 14th floor of Guatemala’s Judiciary headquarters, allowing for the integration of needed security and technological features that ensure the safety of all participants in high impact trials and help protect witnesses testifying in dangerous cases. PAVI met its targets in this indicator, supporting and monitoring the remodeling effort over the course of the project. It is worth noting that this upgrade in justice sector facilities and infrastructure runs parallel to PAVI’s work to advance prosecutorial capacity in complex litigation and high impact case development. As a result of PAVI’s efforts in this area, numbers of high impact cases increased throughout the project and made the remodeling effort even more necessary than at the beginning of the project. Furthermore, the technological and security features installed in the 13 remodeled court represent essential components of the Protocol for Persons Involved as Witnesses in Criminal Trials, as they help protect witnesses’ identities through the provision of remote, video-conferencing testimony and other security measures. Combating Illegal Activities in Petén: The Unique Challenge of the Mayan Biosphere Reserve PAVI’s other Special Activity focus related to strengthening justice capacity to combat illegal activities in Petén. As mentioned previously, PAVI activities in Petén struggled to overcome high levels of illicit criminal group activity, an extended state of siege, reduced levels of justice sector capacity, and an administrative transition in a key prosecutor’s office. Nonetheless, PAVI was able to make a certain degree of progress in its efforts to increase prosecution of crimes against the environment and crimes against cultural patrimony in the Mayan Biosphere Reserve. These successes reflect PAVI’s effective leveraging of the Sub-Grants Program to engage local civil society in these efforts, as well as its pro-active engagement of local justice sector officials, as detailed in the Balam Association success story included in Annex A. Engaging Guatemalan Civil Society through the Sub-Grants Program PAVI’s grants program administered over $300,000 in sub-grants to 12 Guatemalan civil society organizations working in a variety of fields related to PAVI’s work. Due to its engagement of civil society organizations, these sub-grants most directly fell under Sub-IR2, ―Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence,‖ though the diverse nature of grantee activities supported PAVI objectives and activities across all sub-IRs and special activities. Furthermore, the sub-grants allowed PAVI to extend its programmatic reach to regions such as the departments of Izabal and Quiche, where it would not have been cost effective or efficient to station PAVI personnel and infrastructure. M&E efforts in this area focused on a selection of sub-grants with particularly strategic relevance to other PAVI monitored activities. Ongoing focus groups and interviews with representatives of the Balam Association, a Petén-based sub-grantee working to reduce criminal activity in the Mayan Biosphere Reserve (MBR), provided feedback on the collaboration between PAVI and Association Balam personnel. The most relevant inputs were the following:  Over the course of this partnership, the Balam Association provided an effective, local conduit for PAVI’s interventions in the MBR  PAVI’s legal expertise and technical assistance allowed the Balam Association to make significant strides in its organizational development  Leveraging PAVI’s support and training assistance were pivotal to establish the Forum for Environmental Justice, a multi-sectorial network of stakeholders working to promote greater rule of law in the MBR. Through this Forum, the Balam Association and PAVI were able to organize a series of orientation sessions and guided aerial tours of the MBR for local and national justice system officials, leading to significantly improved results in crimes against the environment and crime against cultural patrimony brought to trial. In Izabal, another sub-grantee, the Guatemala Foundation, approached crime prevention, violence reduction, and advocacy from a different perspective: It engaged local women’s leaders throughout the department in the development of mobile human rights units and 14 implementation of citizen safety audits within their own municipalities. This training program and advocacy method developed local women’s capacity in citizen security analysis and advocacy, helping to address structural social issues that contribute to rising rates of femicide throughout the country. Guatemalan Foundation’s institutional strength and political capital helped to cultivate local governmental support for grant activities, securing meetings with local officials and generating letters of support from the Governor of Izabal. Annex A provides more information, in a case-study format, about this intervention. PAVI’s use of sub-grants also allowed it to extend its programmatic reach into the areas of at￾risk youth outreach, a crucial area of intervention given Guatemala’s increasing struggles with youth gangs. The Extraordinary Guatemalans Foundation (EGF) is a social service and youth outreach organization working with the marginalized communities in Guatemala City’s Zone 3. EGF’s sub-grant supported over 30 educational and artistic projects with 15 at-risk youth. Focus groups with EGF beneficiaries and case studies of particular interventions proved the potential for radical change in at-risk youths’ behavior and outlook, while also demonstrating the need for sustained, local commitment to the provision of social services and youth engagement. B. RESULTS BY SUB-IR AND INDICATORS The following analysis presents quantitative data concerning the 18 performance and impact indicators by which the evaluation team has monitored and evaluated PAVI. This data has been collected from official sources and databases, interviews with justice officials, surveys of training participants, and focus groups with project stakeholders. The evaluation team has supplemented a basic statistical analysis of the data with qualitative analysis in order to explain trends and patterns. The precise nature of the data and its collection method is further detailed alongside its respective indicator. Sub-IR 1: Improve Justice System Capacity to Prosecute and Try Serious Crimes This Sub-IR addressed: (i) poor treatment and attention to victims and witnesses; (ii) inadequate management of information resources; (iii) inadequate inter-institutional coordination; (iv) lack of institutional planning and development; (v) lack of compliance with key legislation, such as the Law Against Organized Crime; and (vi) inadequate management causing undue delays in case processing. Sub-IR 1 Performance Indicators INDICATOR 2. Number of trainings/workshops/coordination meetings with justice sector professionals that are related to the prosecution and charge of guilty verdicts of high impact cases (F)8 BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3: OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of trainings/workshops/coordination meetings 0 30 35 50 111 20 44 8 The reference ―(F)‖ indicates that the reported information feeds into the USAID Guatemala Mission’s monitoring and evaluation indicators. The F indicator includes judges, magistrates, prosecutors, advocates, inspectors and court staff. Training refers to all training or education events whether short-term or long- term. 15 Indicator two was measured by the number of trainings, workshops, and coordination meetings with justice sector professionals working on prosecution and conviction of crimes against life. The numbers of trainings/workshops/coordination meetings targeted for each of the project years were 30, 50, and 20 respectively. PAVI consistently met annual targets in indicator 2, surpassing the number of trainings/workshops/coordination meetings by 5, 61, and 24 in project years 1, 2, and 3, respectively. Strong demand from justice sector institutions such as the Public Ministry, Judiciary, and National Institute for Forensic Sciences (INACIF) drove the increase in workshops. According to PAVI staff, these institutions requested more trainings and workshops over the course of the project to improve investigative capacity and coordination between prosecutors, and in the case of the INACIF, between prosecutors and forensic experts. The trainings for the Public Ministry were also held in Petén and Cobán, which could serve to explain the high number of trainings. INDICATOR 2(a). Number of participants in the trainings/workshops/coordina tion meetings (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of participants 0 80 321 250 497 100 240 Men None set None set 238 None set 276 None set 145 Women None set None set 83 None set 221 None set 95 The 2011 revision of the M&E plan set targets for project years 1, 2, and 3 at 80, 250, and 100 respectively. PAVI consistently met its annual targets in indicator 2(a), surpassing the number of participants by 241, 247, and 140 in project years 1, 2, and 3, respectively. Training participants were disaggregated by gender for illustrative purposes, but no targets were set. Overall, men comprised 75%, 56%, and 60% of the participants in project years 1, 2, and 3. These strong male participation rates may reflect the fact that there are more male prosecutors working in the area of crimes against life, which involves a high level of exposure to violence. INDICATOR 2(b). Level of participant satisfaction in trainings and workshops (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Participant Satisfaction (%) 0 80% 89% 80% 95.73% 80% 98.11% This indicator tracks participants’ positive evaluation of the trainer, the content and length of the workshop, and knowledge acquired by the participants. The 2011 revisions to the M&E Plan 16 clarified that this percentage represents a cumulative of all evaluations conducted during the project’s years. The target for level of participants’ satisfaction was established at 80% for each project year. PAVI consistently met its annual targets in indicator 2(b), surpassing the targeted level of participant satisfaction by 9%, 15.73%, and 18.11%. Participants lauded the trainings in the satisfaction surveys, noting that the themes were relevant to their work, that they were dynamic, and that subject areas were well-presented. Participants also requested longer trainings that could cover more themes and that the trainings be conducted on a regular basis. Lic. Hugo Rosales, prosecutor for the Litigation Unit of the Prosecutor’s Office for Crimes against Life, who participated in a training concerning litigation, accusations, and evidence management, confirmed that the skills he learned in PAVI trainings continue to impact his work litigating cases of increasing complexity. He pointed to two cases in particular that involved organized criminal structures—one of which included 40 defendants. Prior to this, the largest case he had prosecuted involved six people. The technical skills he learned in the trainings now facilitate his work on cases of complex criminal structures that are brought to the high impact courts. Lic. Rosales commented that, following the PAVI trainings, numerous judges have complimented him on the improved and organized way his cases are being developed and litigated. Sub-IR 1 Impact Indicators INDICATOR 3. Increase in Crimes against Life and Femicide brought to trial BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘129 TARGET TOTAL (# of Cases) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) CRIMES AGAINST LIFE Dept. of Guatemala 131 None set 320 5% 330 (3%) 7% 311 (-6%) 10% 204 (-35%) Dept. of Petén 0 None set 9 5% 11 (22%) 7% 9 (-19%) 10% 2 (-73%) FEMICIDES Dept. of Guatemala 0 None set 3 5% 10 (233%) 7% 21 (110%) 10% 22 (3%) Dept. of Petén 0 None set 1 5% 0 7% 1 10% 0 (0%) 9 The number of cases for 2012 is an estimate based on the actual figures for the period January–May. The estimate was determined by calculating the monthly average of reported cases and then multiplying that average number by 12 months. In cases where the estimate provided fractions, the number was rounded up or down as appropriate. The actual number of reported cases of Crimes against Life for the January–May period was 85 in the Department of Guatemala and one in the Department of Petén. For Femicides the actual number was nine in Guatemala and zero in Petén. 17 This indicator measures the change in Crimes against Life and Femicide cases brought to trial over the life of the project, based on data from the SICOMP system. In the 2011 M&E Plan revision, the target for calendar year 2012 was reduced from 12% to 10% as part of a broader decision to standardize targets and reporting across all indicators relating to Crimes against Life and Femicides in the Departments of Guatemala and Petén. The 2008 baseline for Crimes against Life in the department of Guatemala was 131 cases and in Petén was 0. Baseline data for femicides in both departments is zero, as to reflect the impact of the Law against Femicides, approved in 2008. Data from the Department of Guatemala includes the municipalities of Villanueva and Mixco. In Crimes against Life brought to trial in the Department of Guatemala, though no target was set for the January-December ’09 period, the increase of 189 over the baseline set in 2008 was noteworthy. As a result, of this strong initial progress, PAVI found it more difficult to maintain the same rate of improvement over subsequent reporting periods. The 2010 reporting period achieved 330 cases, or a 3% increase over the previous year. The 2011 reporting period witnessed a 6% decrease in the overall cases brought to trial, while early 2012 data is on pace for a 35% decrease over the entire year. PAVI staff noted that this decrease could in fact be the result of PAVI’s interventions, as improvements in prosecutors’ investigative capacity has led to the development of more complex cases that involve/link a greater number of discrete cases, thereby increasing their efficiency and impact without an accompanying increase in the total number of cases brought to trial. During a final interview, Lic. Jazmin Gálvez Barrios, President of the High Impact Court ―A‖, reinforced PAVI’s assertion that measuring project impact by number of cases brought to trial does not necessarily reflect changes in justice sector capacity to prosecute crimes. Many cases tried in Lic. Gálvez Barrios’ high impact court involve multiple defendants and inter-related crimes, implying that the impact of the project may be better measured by the quality and complexity of the cases brought to trial, in addition to the number. In light of this fact, a slight reductions in cases brought to trial should not necessarily be regarded as a lack of progress in improving justice system capacity to prosecute and try serious crimes. In the Department of Petén, cases of Crimes against Life brought to trial met the target increase of 5% for the January-December ’10 period after a significant increase from the baseline. In 2011 and the beginning of 2012, Crimes against Life brought to trial failed to meet project targets. Difficulties in Petén can be partially attributed to a persistent lack of capacity. Beginning in 2011, this challenge was further exacerbated by the Los Cocos Massacre of 27 people and the resulting State of Siege that was declared in May, 2011 and continued as a State of Alarm until January, 2012. While this heavy police and military presence reduced overt illicit criminal group activity in the area, lingering perceptions of insecurity and rampant criminal activity combined with the declaration of martial law and suspension of civil contributed to a climate that reduced citizens’ reporting of crimes and interaction with justice system officials. In the Department of Guatemala, PAVI successfully met targets for femicide cases brought to trial. In 2010, femicides brought to trial surpassed the target 5% increase, with the total of 10 cases brought to trial representing a 233% increase over the previous year. In 2011, PAVI surpassed the 7% target increase for the reporting period, with its 21 cases representing a 18 110% increase over the previous year. Preliminary 2012 data is on pace for a 3% increase over the course of the entire year. In the Department of Petén, cases of femicide brought to trial over the course of the project never exceeded one case in a reporting period, making the statistical significance of tracking this small sample size in terms of 5, 7 and 10 percent increases unhelpful. This low number is not only a result of the justice sector’s limited capacity. Interviews with PAVI staff also suggested high levels of fear associated with filing claims because there are little to no means of protecting victims and witnesses from retaliation. This issue is further complicated by rampant levels of organized criminal activity and impunity in the region. Observers have noted the sharp rise in rates of femicides in recent years has run parallel to the rise of drug trafficking activities in the region. With Petén having been particularly affected by the destabilizing and violent influence of these criminal groups, progress in any one indicator is likely to be difficult without broader advances in rule of law. INDICATOR 4. Increase in the number of guilty verdicts for Crimes against Life BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘1210 TARGET TOTAL (# of Verdicts) TARGET TOTAL (# of Verdicts) (% Change) TARGET TOTAL (# of Verdicts) (% Change) TARGET TOTAL (# of Verdicts) (% Change) CRIMES AGAINST LIFE Dept. of Guatemala 100 None set 157 5% 195 (24%) 7% 208 (6.6%) 10% 256 (23%) Dept. of Petén 0 None set 0 5% 0 7% 1 10% 2 (100%) This indicator measures the number of guilty verdicts in the departments of Guatemala and Petén. The 2008 baseline values are based on SICOMP data. In the 2011 M&E Plan revision, the target for calendar years 2010, 2011, and 2012 were set at 5%, 7%, and 10% as part of the aforementioned decision to standardize targets and reporting across all indicators relating to Crimes against Life and Femicides in the departments of Guatemala and Petén. Following a strong increase in 2009, guilty verdicts for Crimes against Life exceeded the target 5% increase in 2010 by achieving 195 guilty verdicts, or an increase of 24%. In 2011, PAVI virtually achieved the 7% target increase, with an increase of 6.6%. Estimates for 2012, based on January – May actual figures, show that the number of guilty verdicts could be on track for a 23% increase. However, given the reduction in cases brought to trial during the same period (see indicator 3), it is likely that the rate of convictions will slow down as well. 10 The number of guilty verdicts for 2012 is an estimate based on the actual figures for the period January–May. The estimate was determined by calculating the monthly average of reported cases and then multiplying that average number by 12 months. In cases where the estimate provided fractions, the number was rounded up or down as appropriate. The actual number of reported guilty verdicts of Crimes against Life for the January–May period was 107 in the Department of Guatemala and one in the Department of Petén. 19 Further cross-referencing of indicators 3 (crimes against life cases brought to trial) and 4 (guilty verdicts in crimes against life) provides further insight into the impact of PAVI’s prosecutorial trainings. While there is not a direct, one-to-one relationship between cases brought to trial and guilty verdicts issued in a given calendar year—variable trial durations and circumstances can lead to cases being brought to trial and concluded in different years—dividing the numbers of guilty verdicts by the numbers of cases brought to trial in each reporting period demonstrates a promising upward trajectory in these estimated conviction rates throughout the project life. For crimes against life in Guatemala in 2009, 2010, and 2011, the conviction rates were 49%, 59%, and 67%. In the first five months of 2012, guilty verdicts outpaced cases brought to trial, with 107 guilty verdicts issued and only 85 cases brought to trial. This reflects the somewhat counterintuitive effect of improved prosecutorial efficiency: As prosecutors are more effectively investigating, consolidating, and strategically litigating cases, they are presenting a fewer number of cases, usually involving a greater number of defendants, but with a greater chance of obtaining convictions. Interviews with two PAVI prosecutorial training participants, Lic. Rosales and Lic. Ingris Xuc, head of the Public Ministry’s Analysis Unit, reported that the cases they have tried since receiving their trainings have resulted in guilty verdicts and maximum sentences. By contrast, Lic. Gálvez Barrios, president of the High Impact Court ―A‖, argued that there is still a need for greater prosecutorial training. Lic. Gálvez Barrios illustrated this point by showing the evaluation team the records of a 2011 case in which the quality of evidence and charges presented by the prosecution precluded the issuing of a guilty verdict, despite otherwise strong indications of the defendant’s guilt. Guilty verdicts in the Department of Petén never exceeded one case in any reporting period, making statistical analysis of the percentage increase insignificant. The PAVI team suggested that the low capacity of the justice and security sectors in Petén makes building a case and bringing it to trial, and therefore obtaining a guilty verdict, extremely difficult. While Petén showed a measure of progress in cases brought to trial prior to the declaration of a state of siege, Indicator 4 demonstrates the particularly low capacity of Petén’s justice system, as even in 2010 and 2011, when Petén registered relatively high numbers of cases brought to trial (11 and 9, respectively), conviction numbers remained exceptionally low, at zero and one. PAVI staff attributed this to continuing challenges to rule of law, as investigation and witness participation are hampered by intimidation and fear of reprisals in a justice system that cannot provide adequate protection to citizens. Sub-IR 2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence This Sub-IR seeks to improve: (i) inadequate or insufficient governmental and non-governmental legal services for victims of violent crime, including domestic violence; (ii) insufficient coordination amongst government and non-governmental institutions that provide attention to victims of violent crime: and (iii) insufficient violence prevention measures for the most vulnerable groups, such as women and children. 20 Sub-IR 2 Performance Indicators INDICATOR 5. Number of Workshops/Coordination Meetings/Trainings for institutions offering victims assistance that are participating in the prevention and reduction of violence (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of trainings/workshops/coordination meetings 0 0 9 10 19 5 11 This indicator corresponds to the number of workshops, trainings, and coordination meetings with justice sector professionals and civil society advocates that participate in the prevention and reduction of violence. The 2011 M&E plan revision set Project year 1, 2, and 3 targets at 0, 10, and 5, respectively. PAVI consistently met targets in this indicator. It exceeded annual targets for number of workshops/coordination meetings/trainings by nine in Project years 1 and 2, and six in Project year 3. It should be noted that since October 2011, the trainings carried out under this indicator were complemented by PAVI’s sub-grants to the Barbara Ford Foundation and the Asociación de Justicia Alta Verapaz (ASOJAV). In total, these organizations either coordinated or facilitated training for 66 and 57 participants, respectively. This was an important example of PAVI strategically collaborating with sub-grant recipients’ in support of the overall project goals. INDICATOR 5(a). Number of participants from the justice sector and civil society organizations that provide assistance to victims attending Coordination meetings, workshops and trainings (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of participants 0 80 132 40 330 20 123 Men None set None set 34 None set 104 None set 30 Women None set None set 98 None set 226 None set 93 This indicator corresponds to the number of participants from the justice sector, civil society, and non-governmental organizations offering assistance to victims that are attending workshops, trainings, and coordination meetings. PAVI consistently met its annual targets in indicator 5(a), with participant numbers surpassing targets by 52, 290, and 103 participants in Project years 1, 2, and 3, respectively. Over the course of the project, 168 men and 417 women participated in workshops/coordination meetings/trainings, yielding a male/female ratio of 29/71. The large 21 number of female participants is due in part to the fact that many of trainings were aimed specifically at female victims and women leaders. INDICATOR 5(b). Level of participant satisfaction in coordination meetings, workshops and trainings (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Participant Satisfaction (%) 0 80% 89% 80% 97.74% 80% 98.38% The percentage listed in indicator 5(b) corresponds to the participants’ level of satisfaction of the trainer, the content and length of the workshop, and knowledge acquired by the participants. This percentage is a cumulative of all evaluations conducted during the Project’s fiscal year. PAVI exceeded the 80% participants’ satisfaction target by 9%, 17.74%, and 18.38% in Project years 1, 2, and 3, respectively. Participants noted that they gained new knowledge from the trainings, and expressed a strong desire for the activities to be replicated in all justice sector institutions. Female participants, in particular, indicated strong levels of satisfaction with the content of the trainings. While not directly involved in the trainings listed above, PAVI sub-grantee the Guatemalan Foundation contributed to its impact by organizing 150 women leaders from five municipalities in Izabal into informal ―Mobile Human Rights Units.‖ These units served as a vehicle for women in each municipality to learn about their rights from trained women leaders, some of whom were also victims of violence. These mobile human rights units subsequently taught women how to advocate for greater security within their communities through the use of ―citizen safety audits‖ which were then presented to local government officials. Sub-IR 2 Impact Indicators INDICATOR 6. Improved coordination between organizations offering victim’s assistance through implementation of Protocol for Persons Involved as Witnesses in Criminal Trials BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Protocol Implementation (%) 0 11 0% 0% 30% 58% 70% 95% 11 The baseline for this indicator is zero as the Nation-wide Diagnostic Assessment of Institutional Coordination and Services for Victims’ Assistance (August 2010) showed that no formal mechanisms were in place (or had not been effectively used) to improve coordination among the public and non-governmental service providers. The Nation-wide Diagnostic identified the services and legal assistance offered to crime victims in 15 departments of the country. Regarding inter and intra-institutional coordination, the Diagnostic revealed that although service providers participated in social networks, their levels of organization, communication, and coordination were based on daily needs and practices, rather than on articulated and standardized practices. 22 The percentage in this indicator refers to the progress in implementing the Protocol for Persons Involved as Witnesses in Criminal Trials, approved in March 2011. It is based on a survey that was given to prosecutors at the end of 2011. The 2011 M&E Plan revision set targets for Project years 1, 2, and 3 at 0%, 30%, and 70%, respectively. PAVI consistently met its targets in indicator 6. After conducting a nation-wide diagnostic assessment of institutional coordination and services for victim’s assistance in Project year 1, PAVI successfully implemented an integrated and inter-agency coordination strategy to provide assistance to victims and exceeded annual targets for Protocol implementation by 28% and 15% in Project years 2 and 3. In collaboration with PAVI, the evaluation team developed four case studies of femicide to illustrate the application of the Protocol and the types of coordination that take place between institutions (Annex B). INDICATOR 7. Increase in the number of victims who receive free legal and other assistance BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘1212 TARGET TOTAL (# of Assisted Victims) TARGET TOTAL (# of Assisted Victims) (% Change) TARGET TOTAL (# of Assisted Victims) (% Change) TARGET TOTAL (# of Assisted Victims) (% Change) Police Victims Attention Division 7,577 None set 10,678 10% 11,832 (11%) 15% 11,778 (-0.5%) 20% 12,477 (5%) Public Ministry Office of Victims Attention 12,335 None set 15,355 10% 13,331 (-13%) 15% 12,517 (-6%) 20% 10,056 (-20%) Public Defender’s Office of Free Legal Aid 11,422 None set 13,375 10% 12,641 (-4%) 15% 19,723 (56%) 20% 30,885 (56%) Fundación Sobreviventes 2,207 None set 3,041 10% 3,155 (4%) 15% 4,496 (43%) 20% 3,453 (-23%) This indicator corresponds to the number of victims who receive assistance services from the four institutions monitored by PAVI. The 2011 revisions to the M&E Plan changed the 2010, 2011, and 2012 targets from 10%, 20%, and 30% to 10%, 15%, and 20%, respectively. Based on the nationwide diagnostic, PAVI developed the Protocol for Persons Involved as Witnesses in Criminal Trials. This protocol outlines the protection measures for victims who act as witnesses in criminal proceedings. The protocol establishes mechanisms for coordination between the public ministry, its constituent offices like the Office of Victims Assistance and Office of Witness Protection, and independent government institutions like the National Civil 12 The number of victims receiving free legal and other assistance for 2012 is an estimate based on the actual figures for the January–May period. The estimate was determined by calculating the monthly average of reported cases and then multiplying that average number by 12 months. The actual reported number of victims assisted by the Police Victims Attention Division (Jan-April period only) was 4,159; by the Public Ministry Office of Victims Attention was 4,190; by the Public Defender’s Office of Free Legal Aid (Jan-April period only) was 10,295; and by Fundacion Sobreviventes was 1,439. 23 Police. It also guides external coordination with organizations like the Fundacion Sobreviventes and other local networks. The specific structure, sequencing, and content of the Protocol are illustrated in the four annexed case studies included in Annex B. The PAVI project successfully developed a database of victim information from each of the above institutions which is used to report on this indicator. Results under indicator 7 varied across different institutions, though all four institutions providing assistance to victims initially demonstrated substantial progress in 2009 over the 2008 baseline figures. In 2010, the Police Victims Attention Division (PVAD) surpassed its annual 10% increase target by assisting 11,832 victims, which represented an increase of 11%. In 2011, the PVAD served 11,778 victims, a decrease of 0.5%. While data for May 2012 was not yet available, estimates based on the numbers for the period January – April indicated that PVAD’s number of victims assisted in 2012 could experience an increase of 5%, which would still fall short of the 20% increase targeted for 2012. After a large increase in the number of victims receiving assistance from 2008 to 2009 (from 12,335 to 15,355), the Public Ministry’s Office of Victims Attention failed to meet all annual targets, demonstrating diminishing numbers of victims assisted throughout the Project, with decreases of 13% and 6% in 2010 and 2011, and figures on pace for a decrease of 20% in 2012. The PAVI team suggested that the OAV’s difficulties in meeting project targets may be due to the slower response times victims confront within the Public Ministry. Further, because of the large jump in 2008 and 2009, sustaining high rates of improvement over this greater base number proved more challenging. The Public Defender’s Office of Free Legal Aid experienced strong increases in numbers of victims assisted throughout most of the Project life. The number of victims receiving assistance increased from 11,422 in 2008 to 13,395 in 2009. Following a slight decrease of 4% (12,641 victims) in 2010, the Public Defender’s Office experienced an increase of 56% (19,723 victims seeking assistance) in 2011 and is on its pace for another 56% increase in 2012. The PAVI team attributed this rate of increase to victims finding less bureaucratic and more stream-lined services at the Public Defender’s Office, leading many to prefer it to the other institutions providing victims assistance. Fundacion Sobrevivientes increased the overall number of victims assisted throughout the life of the Project, though it did not meet all annual targets, falling short of the 10% target increase in 2010: It assisted 3,155 victims, which only represented a 4% increase. In 2011, it successfully surpassed the 15% target annual increase with an increase of 43%. Based on the actual numbers for the period January – May, it was estimated that the number of victims receiving assistance is likely to suffer a 23% decrease. The results across institutions providing assistance to victims demonstrate varying levels of success in implementing PAVI’s recommendations and strategy within different institutional contexts. As has already been mentioned, the PAVI team noted that diminishing numbers for certain institutions and increasing numbers for others could be understood in terms of victims’ pursuit of the most efficient and effective assistance among the available options. While this reveals the effect of institutional conditions on the implementation of the protocol, it also 24 indicates the value of a holistic analysis of numbers of victims assisted by all institutions monitored by PAVI. Aggregating these numbers of victims assisted, the overall trend demonstrates a strong increase in the number of victims assisted from 2008 to 2009, followed by a slight decrease of 4% in 2010, and an increase of 18% in 2011. Estimates for 2012 show that the institutions on the whole are on pace for a further 10% increase. These numbers indicate that while the overall trend is toward increasing numbers of victims assisted, results could be optimized by organization-specific interventions to improve efficiency of service provision in under-performing institutions or by directing resources and victims to increasing those institutions already performing at a high level. INDICATOR 8. Increase in the number of guilty verdicts under the Law Against Femicide BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘1213 TARGET TOTAL (# of Verdicts) TARGET TOTAL (# of Verdicts) TARGET TOTAL (# of Verdicts) (% Change) TARGET TOTAL (# of Verdicts) (% Change) Dept. of Guatemala 0 None set 3 0% 10 10% 7 (-30%) 10% 10 (42%) Dept. of Petén 0 None set 0 0% 0 10% 0 (0%) 10% 0 (0%) Indicator 8 tracks increases in the number of guilty verdicts for femicides in the departments of Guatemala and Petén over the life of the project. The 2011 revisions to the M&E Plan adjusted the targets to zero percent for 2010, 10% for 2011; and 10% for 2012. As the Law Against Femicide was passed in 2008, the baseline for this indicator was set at zero. In the Department of Guatemala, guilty verdicts made gradual progress over the course of the Project, advancing from three in 2009 to 10 in 2010, followed by a 30% decline in 2011. In 2012, guilty verdicts are on pace for a 37% increase by the conclusion of the year. This measured progress should be noted in light of the myriad challenges confronting the successful prosecution of femicides in Guatemala, as noted in the discussion of indicator 3. In the Department of Petén, there were no guilty verdicts throughout the Project, and none of the targets were met. This is not surprising, giving that over the course of the Project, only two femicide cases were actually brought to trial. The PAVI staff suggested that the challenge to getting guilty verdicts in Petén is related to a widespread culture of silence; witnesses believe they will not be protected. Sub-IR3: Increased Internal Accountability and Oversight within the Justice Sector This Sub-IR addresses: (i) lack of consolidation of financial management, lack of internal controls, teams that have little managerial capacity/experience; (ii) the inexistence, incomplete 13 The number of guilty verdicts for 2012 is an estimate based on the actual figures for the period January–May. The estimate was determined by calculating the monthly average of reported guilty verdicts and then multiplying that average number by 12 months. In cases where the estimate provided fractions, the number was rounded up or down as appropriate. The actual number of reported guilty verdicts of Femicides for the January–May period was four in the Department of Guatemala and zero in the Department of Petén. 25 implementation, or inadequate enforcement of job descriptions, hierarchies, and responsibilities; (iii) the lack of implementation of performance evaluation systems; and (iv) inadequate statistical reporting and analysis. Sub-IR 3 Performance Indicators INDICATOR 9. Number of units (judicial, technical, and administrative) from the different institutions that are currently implementing the Institutional Integrity Model (IIM) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of units 0 3 7 10 13 10 17 The IIM is a transversal strategy that PAVI is implementing throughout all Sub-IRs of the Project; therefore, there were not separate Institutional Plans or workshops specifically oriented toward implementing the IIM. As part of this strategy, PAVI worked with the Judiciary, Public Ministry, Institute of the Public Defenders, and the National Civil Police’s Office of Victim’s Assistance to implement elements of the IIM. This indicator tracks justice sector institutions attending trainings on institutional strengthening processes to implement the Institutional Integrity Model within their organizations. Targets are cumulative starting from Project year 1 and include the different institutions implementing the IIM. PAVI consistently met targets in this indicator. It exceeded annual targets for number of units implementing the IIM by 4, 3, and 7 in Project Years 1, 2, and 3, respectively. As this indicator represents cumulative progress, the total number of institutions implementing elements of the IIM at the conclusion of the project was 17. The PAVI approach to promoting the IIM changed in 2011 from a purely top-down approach to an approach that was both bottom-up and transversal. It began to target technical staff, as well as top-level officials, to create a leadership group that would champion the model within their institutions. According to PAVI staff, most of the training participants viewed the process as innovative because it was distinct from the technical trainings they were used to. These trainings, which focused primarily on mental health and leadership, were especially necessary because many of the prosecutors and judicial professionals had been provided with no training in these areas, despite the challenging nature of their work. To ensure sustainability for the IIM model, PAVI created the Positive Leadership Network, made up of 30 training participants specially selected because of their leadership qualities. These leaders were given additional training focusing on leading effective teams within their organizations. In focus group interviews, these leaders noted the importance of continuing to advocate for this methodology in their institutions and communicating with other members of 26 the network. In order to do so, they have committed to generating proposals for change in their own institutions. INDICATOR 9(a). Number of participants in the trainings and workshops; Prosecutors, judges, administrative personnel of justice sector institutions participating in PAVI training events on IIM (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of participants 0 80 374 100 148 100 351 Men 0 Not set 221 Not set 73 Not set 170 Women 0 Not set 153 Not set 75 Not set 181 This indicator corresponds to the number of prosecutors, judges, and administrative personnel of the justice sector institutions who attend PAVI training events on the Institutional Integrity Model (IIM). The 2011 revisions to the M&E Plan revision set the targets for project years 1, 2, and 3 at 80, 100, and 100 participants, respectively. PAVI consistently met its annual targets in indicator 9(a), with participant numbers exceeding targets by 294, 48, and 251 in Project years 1, 2, and 3, respectively. Over the course of the project, 464 men and 409 women participated, yielding a nearly equal male/female ratio of 53/47. INDICATOR 9(b). Level of participant satisfaction in trainings and workshops (F) BASELINE 2008 PROJECT YEAR 1 JUN ‘09 – SEPT ‘10 PROJECT YEAR 2 OCT ‘10 – SEPT ‘11 PROJECT YEAR 3 OCT '11- MAY '12 TARGET TOTAL TARGET TOTAL TARGET TOTAL Participant Satisfaction (%) 0 80% 95% 80% 98.05% 80% 95.81% Indicator 9(b) tracks participants’ evaluation of the trainer, the content and length of the workshop, and knowledge acquired by the participants on the IIM activities. The percentage presented is an average of all the evaluations conducted during the project year. PAVI consistently met its annual targets in indicator 9(b), surpassing the target 80% participant satisfaction by 15%, 18.05%, and 15.81% in Project years 1, 2, and 3. Training participants in addition pointed to the strength of the training facilitator, the participatory and active nature of the trainings, and the uniqueness of the training model in their satisfaction surveys. 27 Sub IR 3 Impact Indicators INDICATOR 11. The Institutional Integrity Model (IIM) has an impact on improving internal ethics and management within target justice system institutions BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - MAY ‘12 TARGET TOTAL TARGET TOTAL TARGET TOTAL TARGET TOTAL Participants’ Positive Perception (%) 0 None set 0 None set 0 70% 79% 80% 81.33% The institutional integrity model contributes to the development of leadership abilities of justice sector professionals in their everyday functions and fosters a better institutional climate. This indicator measured the impact of the IIM as perceived by the justice sector professionals participating in the project activities. The percentage corresponds to the number of justice sector professionals attending the leadership development trainings, and is further informed by pre- and post-perception surveys, personal interviews with project participants, and focus groups interviewing. The 2011 M&E Plan revision set the targets for 2011 and 2012 at 70% and 80% respectively. PAVI met its annual targets in indicator 11, exceeding the 2011 annual target by 9% and the 2012 annual target by 1.33%. Participant perception surveys and focus groups discussions with the staff of the Judiciary Performance Evaluation Unit, the Villa Nueva, Escuintla, and Mixco 24- hour Courts, the Crimes against Life Prosecutor’s Office, and the Public Ministry’s Investigation, Analysis, and Litigation Units, showed that while the participants in the IIM trainings were initially resistant or lacked knowledge of the nature of the trainings, they were extremely pleased following the trainings. They were particularly pleased with the facilitation of the trainings and suggested that these types of active and participatory trainings were necessary to disrupt the monotony of their daily work lives. Over the course of the final year of the project, PAVI interviewed prosecutors and judicial professionals who received trainings under the implementation of the IIM. A key element to promoting the IIM’s sustainability was the creation of the Positive Leadership Network, described in the discussion of indicator 9. During the final evaluation, the evaluation team interviewed two of the Positive Leadership Network members to evaluate whether their training helped to improve internal management within their institutions. Lic. Xuc, head of the Analysis Unit of the MP, was selected to be part of the network. According to Lic. Xuc, the positive leadership training helped her to form a highly integrated team in which all staff members were involved in the development of cases and in which each individual’s strengths were employed in order to achieve excellent results. She reported employing these skills and strategies in her day-to-day work and in her role as a leader. When complex cases are assigned to the unit, she is better able to organize her team to work 28 efficiently and provide support throughout the process. According to Lic. Xuc, the improved quality of the Analysis Unit’s work has been affirmed by the judges, especially in high risk/high impact cases. Lic. Saul Pensamiento, an administrator at the 24 court of Villanueva, was also selected as a member of the Positive Leadership Network. He first participated in a training on mental health with some of his colleagues. The training was significant for the group because it represented the first time the court staff had met outside of their workplace. Lic. Pensamiento noted that this gave staff members the opportunity to communicate in different ways and generated improved daily communication and collaboration among those staff members who participated in the training. However, he noted that his colleagues who did not participate in the trainings were somewhat marginalized by this smaller group’s improved dynamic. Lic. Pensamiento suggested that his court would benefit from further workshops on teamwork, empathy, and tolerance. In accordance with the 2011 Revised M&E Plan, PAVI developed two additional success stories on the implementation of the IIM model. They can be found in Annex A. INDICATOR 13. Number of High Impact Courts established BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - MAY ‘12 TOTAL TARGET TOTAL TARGET TOTAL TARGET TOTAL Number of High Impact Courts that is functionally and operating in the Dept. of Guatemala 0 0 1 1 1 1 1 1 The 2011 revisions to the M&E Plan set targets for 2010, 2011, and 2012 at one High Impact Court that is functionally and operating over the life of the project. PAVI met its targets in the establishment of a high impact court under indicator 13. The high impact court located on the 14th floor of the Torre de Tribunales in Guatemala City, together with the high impact court on the 15th floor, now constitutes the judicial headquarters for high risk cases, housing first instance courts, sentencing courts, and administrative staff. This effort entailed remodeling the courtrooms on the 14th floor to better reflect the security and operational needs of these types of courts, particularly the need to protect victims and witnesses testifying in front of the court. The remodeled court rooms include a secure and concealed room for protected witnesses in which they are able to give their testimony via video-conference. PAVI visited the new courtrooms and spoke with both a judge of first instance court and a sentencing judge about the impact of the newly remodeled spaces. Lic. Gálvez Barrios, President of the High Impact Court ―A‖, who has presided over numerous high profile and high risk cases, 29 noted that the secure, enclosed space designed for defendants was crucial to ensuring the safety of judicial personnel, particularly judges. Lic. Galvez, President of the High Impact Court ―B‖, currently presides over cases in the remodeled 14th floor courtrooms. He noted that the remodeling and added courtroom capacity has allowed them to use the space more efficiently and that the additional courtrooms increases judges’ efficiency in processing cases and has made the schedule more predictable. Prior to the remodeling, trials would often start late while personnel waited for a court room to become available. Further, he noted that the new courtrooms reflect the need for larger facilities as a result of increasingly complicated cases involving that involve complex criminal structures and numerous defendants under one case. Special Activities Special Activity 1: Supporting High Impact Courts INDICATOR 14. Number of cases referred to the High Impact Courts BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘1214 TARGET TOTAL (# of Cases) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) Number of cases 0 None set 0 5% 31 (N/A, previous value 0) 7% 36 (16%) 10% 60 (66%) This indicator corresponds to the number of cases referred to and processed in the High Impact Courts. Following two years, 2008 and 2009, with no cases referred to the High Impact Courts, 2010 witnessed a significant increase to 31 cases. In 2011, PAVI surpassed the 7% target, increasing cases referred to the High Impact Courts by 16%. Based on the 25 cases referred to the High Impact Courts during the period January-May 2012, PAVI projects a 66% increase on the number of cases referred to a High Impact Court by the end of the year. The increased number of cases referred to the High Impact Courts may be the result of prosecutors’ improved investigative capacity following PAVI’s trainings in complex case development and strategic litigation. According to Lic. Galvez, Chief Justice of the High Impact Court ―B‖, prosecutors’ cases following PAVI trainings demonstrate better investigative skills and use of more formal, scientific evidence. Lic. Galvez also reported that prosecutors have benefitted from the use of new technologies, such as phone-tapping, that have allowed prosecutors to uncover complex criminal structures. This has resulted in a particular increase in the number of extortion cases brought to trial. With the current administration launching a special force of high impact case investigators, increasing numbers of cases are likely to be referred to these courts. However, Lic. Gálvez 14 The number of cases for 2012 is an estimate based on the actual figures for the period January–May. The estimate was determined by calculating the monthly average of reported cases and then multiplying that average number by 12 months. The actual number of reported cases referred to the High Impact Courts during the January–May reporting period was 25. 30 Barrios cautioned against measuring success by sheer number of cases referred to High Impact courts, due to the risk of overwhelming the system. Lic. Gálvez Barrios asserted that judicial authorities should be selective in deciding which cases truly require the additional security and capacity of the High Impact Courts. Special Activity 2: Strengthen Justices Capacity to Combat Illegal Activities in Petén INDICATOR 15. Increase in Serious Crimes brought to trial in Petén (See indicators for Sub-IR1and Sub￾IR2) BASELINE 2008 PERIOD JAN - DEC ‘09 PERIOD JAN - DEC ‘10 PERIOD JAN - DEC ‘11 PERIOD JAN - DEC ‘1215 TARGET TOTAL (# of Cases) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) TARGET TOTAL (# of Cases) (% Change) Crimes against the Environment 56 None set 19 5% 25 (31%) 7% 68 (172%) 10% 43 (-37%) Crimes against Cultural Patrimony 25 None set 91 5% 187 (105%) 7% 20 (-90%) 10% 14 (-28%) Crimes against Life 10 None set 9 5% 11 (22%) 7% 9 (-19%) 10% 2 (-78%) This indicator tracks the number of serious crimes brought to trial in the Department of Petén. Success in increasing cases of Serious Crimes brought to trial varied, though the overall trends matched those of other Petén indicators in which the impact of the Los Cocos massacre and State of Siege were apparent. Crimes against the Environment surpassed annual targets, achieving increases of 31%, and 172% in 2010 and 2011, respectively, while only surpassing the 2008 baseline in 2011. Early results of 2012 progress suggest the possibility of a 38% decrease in Crimes against the Environment brought to trial over the year. Following a sharp rise in cases brought to trial in 2009, Crimes against the Cultural Patrimony surpassed the 2010 target with a 105% increase before experiencing a significant decrease of 90% in 2011. Early 2012 data point in the direction of a continued decrease of 28% for 2012. In Crimes against Life, PAVI surpassed the annual target in 2010, achieving a 22% increase. In 2011, it failed to meet the annual target increase of 7% and experienced a 19% decrease over the previous year. Early data from 2012 points in the direction of a further decrease in 2012. As was previously mentioned, the overall prosecutorial capacity in Petén is notably less than in the Department of Guatemala while the conditions it confronted over the course of the project were more challenging. However, project success in improving the numbers of Crimes against the Environment brought to trial should be highlighted, as it is closely connected with one of PAVI’s sub-grant organizations, the Asociacion Balam. PAVI’s support allowed Asociacion Balam to establish the inter-institutional Environmental Justice Forum, providing government officials, international donors, and local civil society representatives with a mechanism for coordination 15 The number of cases for 2012 is an estimate based on the actual figures for the period January–May. The estimate was determined by calculating the monthly average of reported cases and then multiplying that average number by 12 months. The actual number of reported cases brought to trial in Petén for the January–May period was 18 Crimes against the Environment, six Crimes against Cultural Patrimony, and one Crime against Life. 31 and advocacy in issues relating to environmental crime and management in the Mayan Biosphere Reserve. This initiative was integral to improvements in the local government’s capacity and political will to take aggressive action in opposition to environmental crime in the region. It is also important to note that PAVI supported the transfer of the SICOMP information system from the Department of Guatemala to Petén in 2011. By August 2011, all jurisdictions in Petén had been trained in the new technology and SICOMP users reported satisfaction with the ongoing support they received from PAVI following their initial training. Nonetheless, deficiencies in hardware capacity made use of the SICOMP difficult in some circumstances, as users reported delays in uploading and downloading SICOMP data. 32 III. LESSONS LEARNED This section presents lessons learned around PAVI’s interventions in Petén and those emerging from the Sub-Grants Program. A. ACTIVITIES IN PETEN PAVI’s approach to the activities in Petén heavily relied on strategic coordination and local capacity development aiming at changing traditional cultural patterns in the region. 1. Coordination with local actors creates synergies that multiply efforts and helps achieve desired results: For over two decades, the Mayan Biosphere Reserve has benefited from substantial investment from USAID and other U.S. agencies and international support. To ensure optimal program impact, PAVI promoted interventions between the U.S. agencies and local institutions. For example, it promoted coordination between the US Office of Economic Development and its local partners in Petén (Wildlife Conservation Society, Rainforest Alliance, and Balam Association), the US Department of the Interior of the United States, and the Governance Office of USAID. Ultimately, this coordination effort took the form of the Environmental Justice Forum, a space for dialogue, discussion and analysis aimed at promoting a common agenda to strengthen law enforcement and compliance in issues relating to protected areas, environment, and natural and cultural resources. The Forum successfully facilitated discussion of proposals to restore governance and improve implementation of environmental and cultural laws, the signing of institutional agreements between the Public Ministry and the National Council on Protected Areas for the prosecution of high profile environmental criminal cases in the MBR, and the development of an outreach strategy on legal protection of the environment in Petén. Future interventions should take advantage of the Environmental Justice Forum established by the PAVI project before, during, and after committing funds to activities in Petén. Doing so will prevent program replication and facilitate leveraging of previous and ongoing projects. 2. Interventions in Petén must account for particularly low levels of capacity and the need for targeted awareness-raising activities among judicial and law enforcement officials: For years it has been a policy of the justice sector and public security institutions to station judges, prosecutors, and police in Petén as a disciplinary measure. Due to its remote location, difficult living conditions, and high levels of violence and corruption, judicial officials have come to regard the posting as a punishment, as opposed to a call to action. This pattern has led to a tendency toward "foreign officials" in Petén who are disdainful in their attitude, negligent in their administration of justice, and indifferent to the needs of the population and the local social reality in Petén. Following initially weak results in its work with officials in Petén, the PAVI project’s targeted awareness-raising interventions met with marked success and helped facilitate 33 local officials’ buy-in and support for project activities. Future interventions should be aware of the problematic local culture of civil service and design project interventions accordingly, allocating additional resources to counteract low capacity and the challenges of collaborating with counterparts in local government. B. SUB-GRANTS PAVI’s sub-grants program was designed to advance project objectives while developing the capacity of local organizations. The PAVI team has identified the following lessons learned: 1. Grants manuals need to be flexible and consider the CSOs’ different capacity levels: Administering a number of small grants to a range of organizations with varying capacities and structures presented challenges to PAVI staff. Projects administering a sub-grants program should be equipped with a simple and functional grants manual providing administrative guidance that can be adapted to fit the type of organization and activities supported. 2. Sub-grants programs provide a valuable opportunity to develop local capacity: The sub-grants program was designed to advance PAVI goals by complementing project activities in the field. In order to meet acceptable standards of program and financial management, PAVI provided ongoing audits and technical assistance to these organizations. In the process, PAVI observed increased capacity within grantee organizations. In light of USAID FORWARD’s objective of reform in the areas of procurement and implementation, future interventions stand to benefit from increased focus on training grantees in financial management and data collection in order to more fully integrate them into the monitoring process on a quarterly and annual basis. This assistance should take into account the varying levels of grantees’ institutional development, providing differentiated technical assistance tailored to the size and history of the grantee organization. PAVI found that support in this area for small or recently established grantees demonstrated the greatest returns, as they were able to absorb and integrate this guidance into their own institutional development processes. ANNEX A SUCCESS STORIES/CASE STUDIES - Developing Teamwork in the Justice System’s Performance Assessment Unit - Fostering Emotional Resilience in Petén’s Office of Victims Attention - Bringing Environmental Justice to the Mayan Biosphere Reserve - Helping At-Risk Youth Create a Better Life - Providing 24-Hour Judicial Services to Mixco, Guatemala - Training Guatemala’s Criminal Prosecutors - Reducing Gender-based Violence in Izabal With USAID support, the Evaluation Unit of the Guatemalan Justice System has developed new capacities, improved its efficiency, and is now better able to manage its tremendous workload. Developing Teamwork in the Justice System’s Performance Assessment Unit The 16-person staff of the Justice System’s Performance Assessment Unit is responsible for monitoring and evaluating all judges and justices of the peace throughout Guatemala Photo by: PAVI Team Challenge The Judicial System’s Performance Assessment Unit (PAU) is responsible for the design and implementation of monitoring and evaluation (M&E) systems for judges and magistrates’ technical performance and professional conduct. The PAU is also responsible for submitting evaluation reports to authorities as prescribed by law. Despite its extensive responsibilities encompassing all judges and justices of the peace in Guatemala, the PAU has only a limited budget providing for 16 staff members to design and implement the nationwide M&E systems, presenting a serious obstacle to fulfilling its duties as one of the most important technical units of the Guatemalan Judiciary. The strain of limited resources and overwhelming workloads had contributed to an office environment in which staff members were unable to optimize performance, and team leaders felt unprepared to proactively address these problems. Initiative Given the demonstrated need of the PAU, the USAID Program Against Violence and Impunity provided targeted intervention through the Institutional Integrity Model and offered training and consultation to staff members on issues of interpersonal relationships and power dynamics. By addressing areas for improvement such as management style, communication, teamwork, and opportunities for innovation, the training and consultation helped strengthen workplace relations in the PAU. Results PAU managers are now cultivating a better working environment for staff members, improving confidence levels and promoting positive interpersonal relationships. Staff members reflect enhanced teamwork and communication skills, allowing the team to more easily address areas for improvement and manage staff workloads. The PAU team has successfully demonstrated the value of teamwork in achieving workplace efficiency and effectiveness. With USAID support, the Office of Victims’ Attention in Petén has improved morale and implemented new programs to address the needs of its particularly vulnerable beneficiaries. Fostering Emotional Resilience in Petén’s Office of Victims Attention The Office of Victims’ Attention provides legal and social support services to victims of crime in Guatemala’s most remote region Photo by: PAVI Team Challenge The Office of Victims’ Attention (OVA) in Petén is responsible for providing victims of crime with urgently needed information and assistance, including access to psychological, medical, social, and legal services. These services are meant to provide immediate solutions to crime-related problems and prevent long-term trauma-related issues. In addition to working with limited resources, OVA staff members in Petén are subject to significant emotional burdens resulting from the traumatic nature of the cases they address, particularly those relating to violence against women. Initiative The Institutional Integrity Model (IIM) implemented by the USAID Program Against Vionelnce and Impunity (PAVI) in Petén was the first human resources development program at the OVA to focus on improving the staff’s emotional resilience, mental health, and interpersonal relationships. PAVI identified these issues as essential to the personal motivation and effective performance of the OVA staff, including in technical areas more commonly targeted for external assistance. Results Any doubts that Lucia, the head of the OVA, initially had about the value of the IIM, are now gone after she participated in the trainings. The skills that Lucia gained at the training sessions have helped her deal with the many challenges presented by her work at the OVA in Petén. Notably, Lucia has initiated an awareness campaign with users of the OVA aimed at challenging the acceptance of abusive and co-dependent domestic relationships and providing support for victims to end unhealthy relationships in their lives. Lucia states that she would not have been able to proactively develop a campaign to confront such challenging issues without PAVI’s training and recognizes that human resources development is a fundamental aspect in fostering active and successful public service institutions. With USAID support, local community members organized a guided tour and fly-over of the Mayan Biosphere Reserve for justice system officials from Guatemala City. Bringing Environmental Justice to the Mayan Biosphere Reserve Civil society and justice sector officials coordinate efforts to promote the rule of law in the Mayan Biosphere Reserve Photo by: Asociación Balam Challenge The problem of environmental justice in Guatemala’s Mayan Biosphere Reserve (MBR) has long been a focus of national and international attention. Recently, as Guatemala’s remote department of Petén has become a focal point for international drug, weapons, and contraband trafficking, issues of governability in the MBR have spilled over from the environmental realm to broadly challenge government capacities and the rule of law in Petén. The increasing presence and activities of illicit criminal groups have increased the risk of conflicts over the environment, including crimes against cultural patrimony and plundering of natural resources. As a result, conflicts in the MBR frequently turn violent, with clashes between criminals and government authorities, intimidation of public officials and civil society representatives, and threats to the legitimate inhabitants of the MBR. Initiative The USAID Program Against Violence and Impunity worked with the Balam Association, a local environmental conservation organization, to organize a coordinated response to the increasingly complex and serious challenges to justice in the MBR and its surrounding areas. The result of the collaboration was the creation of the Forum for Environmental Justice, a multi-stakeholder initiative that convenes environmental, cultural, and ecological organizations to coordinate environmental, justice, and governance promotion projects in the region. Through the Forum for Environmental Justice, PAVI, the Balam Association, and other allied organizations sponsored a series of guided visits and helicopter flights over the MBR to raise awareness among public prosecutors and judges within the Public Ministry and Supreme Court. Results Following these guided visits to the MBR, the Forum for Environmental Justice was able to sign cooperation agreements and develop close relationships with judges and prosecutors to facilitate joint work on criminal investigations, training, and the allocation of technical resources to respond to the challenges facing the MBR. With USAID support, Gutemaltecos Extraordinarios has been working with at-risk youth in one of Guatemala City’s most impoverished areas and providing social and emotional support services to keep them in school away from a life of crime. Helping At-Risk Youth Create a Better Life Guatemaltecos Extraordinarios empowers at-risk youth to choose school and community instead of crime and gangs Photo by: Asociación Balam Challenge Drugs, alcohol, gangs, graffiti, police chases, juvenile courts, robbery, arrogance, indifference to others, domestic violence, and the deaths of close friends and family describe only a little of the suffering that Josué has experienced in his short life. Josué was referred to Guatemaltecos Extraordinarios, a non-profit funded by the USAID Program Against Violence and Impunity, as a lost cause following a series of fights with his mother and school principal. Two months later, he was expelled from school for failing grades, poor behavior, and smoking marihuana. Initiative The first months that Josué spent working with Guatemaltecos Extraordinarios, were challenging. He attracted negative attention, laughed at his peers, and dismissed the program as meaningless. He arrived to trainings under the influence of drugs before leaving early to paint graffiti. He was nearly expelled from the program three times. After three months of a variety of treatments, one-on-one conversations, home visits, and interventions, he seemed to be an impossible, untreatable case. But the Gutemaltecos Extraordinarios team continued to care, express genuine interest in his life, celebrate his birthday, take him on community service trips, and develop a friendship with him, gradually creating a breakthrough opportunity. When he was invited to a spiritual support group where he heard the testimony of an ex-drug addict and felt the support of a caring father-figure, Josué confided that it was the first time he felt at peace. Results Following his breakthrough, Josué has committed to the program and enrolled in a new school. His mother recently reported that, for the first time in her life, she has not received a complaint about her son, saying, "My heart is where it belongs... I'm happy." Josué is giving back to his community by providing free tutoring to new participants in the Gutemaltecos Extraordinarios program, attending a church youth group, and giving talks to at-risk youth about his experiences. He has focused on his academic studies, winning 2nd place in his school’s science fair, and is a more thoughtful son, helping his mother cook corn for sale and taking his mother to church every Sunday. Carlos Mazariegos has played an instrumental role in overseeing the implementation of a new 24-hour court in Guatemala, allowing minor offenders immediate access to judges for hearings Providing 24-Hour Judicial Services to Mixco, Guatemala New leadership provides Guatemalan citizens immediate access to an efficient and effective court system For years, Carlos Ernesto Mazariegos, better known as “Neto” in his home town of Mixco, served as one of five secretaries in Mixco’s 24-hour court. The round-the-clock court is a recent innovation in Guatemala’s justice system that streamlines arraignments, first declarations, and pre-trial motions by providing 24-hour access to a judge, prosecutor, public defender, and police personnel. While the model has helped to reduce congestion and caseloads in Guatemala’s justice system, a USAID funded diagnostic and needs assessment found that the Mixco court had grown inefficient and expensive to administer. When Neto attended his first meeting concerning a newly approved reform plan for his own 24-hour court, he was immediately skeptical. Nonetheless, he familiarized himself with the changes to judicial procedures and administration that he would be responsible for implementing as the newly appointed chief administrator of the reformed court. The first several weeks in this new position were a struggle as Neto faced strong opposition to the reform plan from other staff members. With dedication and the technical assistance of the USAID Program Against Violence and Impunity, over time he was able to gain his staff’s buy-in and support as they acclimated to the procedures and workloads. Since then, Neto has exceeded expectations in his competent and efficient administration of the busy Mixco court. He has assisted in training new court staff members, monitored daily proceedings and legal decisions, and led regularly scheduled coordination meetings for the complex 24-hour operation involving five judges and 26 assistants. Neto’s strong resolve and leadership throughout the entire process have provided the people of Mixco with access to reliable and efficient legal services, making their community safer. With USAID support, international specialists trained Guatemalan prosecutors on development and linking of cases, strategic litigation techniques, and other international best practices to analyze and prosecute complex criminal activities. and organizations Training Guatemala’s Criminal Prosecutors International specialists trained Guatemala’s prosecutors to more effectively combat crime Photo by: PAVI Team Challenge In recent years, the organized crime has pushed the Guatemalan justice system to its limits. While all aspects of the justice system have been strained, many of its newer and less experienced prosecutors, like Hugo Rosales, were unprepared for the increasing complexity and seriousness of the cases being presented to them. Mr. Rosales’ traditional legal training had taught him to analyze and address each case and suspect as an isolated event or individual. The rapid emergence of organized crime and illicit criminal groups required a new set of skills and techniques that Mr. Rosales and his colleagues never learned. Initiative The USAID Program Against Violence and Impunity (PAVI) organized specialized trainings for junior prosecutors from the Guatemalan Public Ministry and various units of the Attorney General’s Office. These trainings, led by international specialists in the prosecution of organized crime, introduced international best practices in issues such as the filing of charges and presentation of evidence in complex, multi-defendant cases. Additionally, PAVI helped coordinate communication and collaboration between key justice sector institutions, bringing together the National Civil Police, the National Institute of Forensic Sciences, the Public Ministry’s Analysis Unit, and the Public Prosecutor’s offices. Results Guatemalan prosecutors are better equipped to address the high rates of crime throughout the country, using more complex methods of analysis that identify connections between criminal activities. Since his training in September, Mr. Rosales has consistently received praise for his prosecutorial skills, presented cases to Guatemala’s high impact courts dedicated to organized crime, and successfully litigated an organized crime case against 48 people, receiving convictions for illicit association, money laundering, illegal possession of firearms, and international drug trafficking. The defendants in this organized crime case were sentenced to a total of 434 years for their crimes. The Fundacion Guatemala team trains Izabal’s women leaders to create mobile human rights units and develop citizen safety audits strategies for women in their own communities. Reducing Gender-based Violence in Izabal Municipalities reduce gender-based violence through training and advocacy Photo by PAVI Staff Challenge Communities in Guatemala’s northeastern department of Izabal suffer from exceptionally high rates of violence against women. While gender-based violence and discrimination have increasingly impacted women in the region in recent years, women’s rights and gender equality have not been adequately addressed, leaving communities in search of a strategy to resolve the issue. Initiative With support from the USAID Program Against Violence and Impunity (PAVI), Fundación Guatemala (FUNGUA), a civil society organization founded by women, initiated a project called “Women’s Participation in the Development of Safe Cities and Citizenship Development.” This project increased awareness of women’s rights and gender equality in Izabal by training local women leaders to conduct human rights and citizen safety audits in their communities. These audits were then used to help communities develop strategies to create safe spaces for women, advocate for local government actions to address violence against women, and develop local land-use plans with a focus on gender issues and equality. Results Thanks to PAVI’s support, the FUNGUA-led project established three mobile human rights monitoring units in Izabal, conducted three citizen safety audits in municipalities throughout this department, and trained a cadre of 30 women in each municipality. With these tools, the women of Izabal are better prepared to identify human rights violations and safety needs, develop strategies to address them, and engage their local governments to work towards the goal of having safe and violence-free communities for women. ANNEX B FEMICIDE CASE STUDIES METHODOLOGY OF FEMICIDE CASE STUDIES The PAVI team worked directly with the prosecutors from the Office of the Prosecutor for Crimes against Life. PAVI identified 4 cases falling under jurisdiction of Guatemala’s recently passed Law Against Femicide and Other Forms of Violence Against Women and provided ongoing consultation and support to prosecutors. PAVI offered recommendations for the development of case theories and the formulation of investigation plans, reviewed the processing of crime scenes, helped with the incorporation of gender and human rights considerations into the process, and contributed to the development of evidence for presentation at trial. PAVI’s accompaniment and case study methodology was based on the Protocol for Persons Involved as Witnesses in Criminal Trials, which sets out the actions to be performed by personnel from the prosecutor’s office in the investigation and development of the case as well as the development of the plan for attention to the victim. In these cases studies, PAVI supported the application of this methodology in the development of the charges brought, placing particular emphasis on the participation and rights of the surviving victim/witness in one case and the secondary victims/witnesses in the others. The case studies often refer interchangeably to victims and witnesses, due to the fact that the protocol can be applied to both types of individuals. While PAVI initially planned to present case studies in a story format, the complex nature of the process and the numerous actions entailed in the Protocol necessitated a more highly structured presentation. For the sake of clarity, each case study has been presented in a matrix format. The four case studies have progressed to varying stages of the process and involve different aspects of its procedures and steps. As a result, matrices include different parts of the protocol and conclude at the point in the process at which each respective case remained on June 30, 2012. Case Study 1: Attempted Femicide Description: The case was brought by the Metropolitan Public Prosecutor’s office. The crime occurred on the night of June 17th, 2010, in a bedroom located in the interior of a bar in Zone 4 of Guatemala City. The victim is Mrs. Elvira Chinchilla Godoy, 45. The accused is Mr. Oscar de Jesús Nájera. On July 17th, 2010, Oscar Jesús Nájera entered the bar in which Elvira worked as a waitress and where she was dining with a friend in a bedroom. After threatening the owner with a machete, he was directed to the room in which Elvira could be found. Oscar de Jesus attacked Elvira with the machete, completely severing her right hand and wounding her in the skull, face, and arms. He then attacked Elvira’s friend, before fleeing from the scene. The investigation found that the accused had an extensive criminal history of domestic violence and had attacked Elvira seven times since 2004. Following the investigation and issuance of an arrest warrant for Oscar de Jesus, he was captured on February 17th, 2011, seven months after the attack. The testimony of the witnesses was the primary evidence against Oscar de Jesús Nájera. The intermediate hearing was held on June 28th, 2011 in the 5th Criminal Court. The judge admitted the charge brought against Oscar de Jesús Nájera by the MP. The court date was scheduled for February 8, 2012 and a final guilty verdict was rendered. PROGRESS IN APPLICATION OF THE COORDINATED “ASSISTANCE PROTOCOL FOR WITNESSES IN THE CRIMINAL PROCESS” SELECTED CASE: PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Case No. 1– Attempted Femicide Municipality: Guatemala Elvira Godoy Chinchilla 5th Criminal Court, Narcoactivity and Crimes KNOWLEDGE OF FACTS Report– Direct, from victim 1. OAP receives report/complaint in written or verbal form (if the victim’s state requires it) with the support of a professional from the OAV. NO 2. Victim is referred to a forensic and medical examination. NO 3. OAV personnel provide crisis intervention support before and during the presentation of the report/complaint. NO Crime Scene (direct victim or collateral or witnesses) 1. Security and transfer to the appropriate prosecutor’s office is coordinated with the National Civil Police. NO SELECTED CASE: PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO against the Environment (Guatemala) 9 th Criminal Sentencing Court Complied with 2 of 2 applicable actions. Compliance: 100% 2. Medical attention and transfer to hospital is coordinated. NO 3. Awaited physician’s authorization to conduct first interview. YES 50 % YES 4. In the case of children or adolescent victims without legal representation, coordinate presence of the Procuraduria General de la Nacion for the interview and protection. NO 5. If the prosecutor determines that there is a high risk of danger to the victim or witness, coordinate support with the Office of Witness Protection. YES 50 % YES Police Complaint or Prevention 1. Locate the victim. NO 2. Develop Investigation and Victim’s Attention Plan. NO RECORDING OF STATEMENTS In Prosecutor’s headquarters Complied with 2 of 3 applicable actions. Compliance: 66% 1. Protect identity. YES 33 % NO 2. When working with a protected witness, coordinate with the Office of Witness Protection. YES 33 % YES 3. Arrange transfer of victim from their home or workplace to the judicial headquarters either through PNC or referral network. YES 33 % YES Before Judicial Authority 1. Request for protective, security, or emergency measures, as appropriate. YES 0 % NO 2. Use of video-conferencing, victim or witness proxies, screens, voice distortion, Gesell Dome, or other measures that prevent direct visual and auditory contact with the suspect. NO 3. Inform the judge or court comptroller if there are protection measures for the victim that must be taken into account in the judicial headquarters. NO 4. When there are safety reasons (art. 217 CPP), request that the judge receives witness testimony in a place other than the court. NO Procedure for hearings relating to especially vulnerable victims (FEMALE VICTIMS) 1. Provide immediate emotional support and kindness; avoid sexist judgments against the victim or placing blame on them for what occurred. YES 25 % YES 2. Provide a private space in which the victim can be heard. YES 25 % YES 3. Allow the victim access to the evidence file. NO 4. Inform the victim of their right to serve as a concurring plaintiff or as a plaintiff in a civil action. YES 25 % NO SELECTED CASE: PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Complied with 3 of 4 applicable actions. Compliance: 75% 5. Facilitate the conduct of psychological and psychiatric assessments to establish the moral, psychological, and psychiatric harm caused. YES 25 % YES Evidence Produced Before Trial 1. Request the statement of the victim/witness as pre-trial evidence once appropriate laws are verified. YES 0 % NO 2. When the judicial authority does not approve submission of victim/witness statements as pre-trial evidence and the prosecutor is not in agreement with the judicial authority’s judgment, appropriate legal remedies are used. NO PREPARATION OF VICTIMS SERVING AS WITNESSES Strengthening Actions for the victim or witness Complied with 4 of 4 applicable actions. Compliance: 100% 1. Coordinate with Office of Victims’ Attention, welfare office, or administrative units of the Public Ministry, as appropriate. YES 25% YES 2. Prepare and empower the victim by explain the dynamics and scope of the hearing, develop interpersonal confidence with the victim, explain the content of potential questions and answers, and teach the victim anxiety management techniques when under interrogation. YES 25% YES 3. Following hearings explain their results and verify comprehension. YES 25% YES 4. Provide emotional support, address bereavement, and set an appointment for follow up YES 25 % YES Accompaniment in the hearing (design) Complied with 2 of 2 applicable actions. Compliance: 100% 1. Explain the procedure for the progression of the hearing and the location of the parties in the court room. YES 50 % YES 2. Explain how the interrogation will be supported with visual aids. NO 3. Presence of an accompanying family member or professional providing psycho-emotional support. YES 50 % YES Logistical Aspects Complied with 2 of 2 applicable actions. 1. Coordinate transportation, accompaniment, lodging, and food for victims or witnesses with five days of advance notice. YES 50 % YES 2. Provide letters necessary to being excused from work, school, NO SELECTED CASE: PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Compliance: 100% or any other responsibility affected by appearance in court. 3. Maintain constant communication with the victim or witness to ensure their appearance in court. YES 50 % YES Action in the debate Complied with 2 of 2 applicable actions. Compliance: 100% 1. Initiate action for compensation of damages as a result of crime. YES 50 % YES 2. Use simple and accessible language that is appropriate for the age and particular conditions of the victim or witness. YES 50 % YES 3. Use, when appropriate, the Gesell Dome, anatomical models, or other tools that help facilitate the trial statement. NO 4. Request from the judges the support of specialized consultants or advice in providing emotional support during the trial process in order to reduce the risk of secondary victimization of the victim or witness. NO Case Study 2: Femicide Description: The case was brought by the Metropolitan Public Prosecutor’s office. The crime occurred the morning of November 27th, 2010 in the living room of a condominium located in Zone 13. The victim was Diana Michelle Bueso Recinos. The investigation identified Marco Vinicio Barrundia, an ex-boyfriend of the victim, as the primary suspect, though it also called for further investigation of Carlos Marroquín, the victim’s fiancé at the time of the crime. On November 27th, 2010, around 7:30 am, the perpetrator arrived at the victim’s home, where she was with her mother and a friend. Diana's mother allowed the perpetrator to enter and then went to the second floor of the house to tell her daughter that he was waiting in the foyer. As soon as the victim arrived to greet the perpetrator he shot and killed her, then immediately fled the scene. During the course of the investigation, no evidence was found linking Marco to the killing of Diana. Although both the victim’s mother and friend have valuable information relating to the case, neither has been willing to collaborate with authorities. Analysis of the record has given investigators reason to believe that the boyfriend of the victim at the time of her death, Carlos, could also be involved in the incident. However, the prosecution has not explored this possibility, nor has it practiced key, recommended investigation steps. Currently, there is insufficient evidence for a conviction of femicide. PROGRESS IN APPLICATION OF THE COORDINATED “ASSISTANCE PROTOCOL FOR WITNESSES IN THE CRIMINAL PROCESS” SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Case No. 2 – Femicide Municipality: Guatemala Diana Bueso Crimes Against Life Prosecutor KNOWLEDGE OF FACTS Report– Direct, from victim 1. OAP receives report/complaint in written or verbal form (if the victim’s state requires it) with the support of a professional from the OAV. NO 2. Victim is referred to a forensic and medical examination. NO 3. OAV personnel provide crisis intervention support before and during the presentation of the report/complaint. NO Crime Scene (direct victim or collateral or witnesses) 1. Security and transfer to the appropriate prosecutor’s office is coordinated with the National Civil Police. YES 100 % NO SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO (Guatemala) Complied with 0 of 1 applicable actions. Compliance: 0 % 2. Medical attention and transfer to hospital is coordinated. NO 3. Awaited physician’s authorization to conduct first interview. NO 4. In the case of children or adolescent victims without legal representation, coordinate presence of the Procuraduria General de la Nacion for the interview and protection. NO 5. If the prosecutor determines that there is a high risk of danger to the victim or witness, coordinate support with the Office of Witness Protection. NO Police Complaint or Prevention 1. Locate the victim. NO 2. Develop Investigation and Victim’s Attention Plan. NO RECORDING OF STATEMENTS In Prosecutor’s headquarters Complied with 2 of 3 applicable actions. Compliance: 66% 1. Protect identity. YES 33 % NO 2. When working with a protected witness, coordinate with the Office of Witness Protection. YES 33 % YES 3. Arrange transfer of victim from their home or workplace to the judicial headquarters either through PNC or referral network. YES 33 % YES Before Judicial Authority Complied with 0 of 1 applicable actions. Compliance: 0 % 1. Request for protective, security, or emergency measures, as appropriate. YES 100 % NO 2. Use of video-conferencing, victim or witness proxies, screens, voice distortion, Gesell Dome, or other measures that prevent direct visual and auditory contact with the suspect. NO 3. Inform the judge or court comptroller if there are protection measures for the victim that must be taken into account in the judicial headquarters. NO 4. When there are safety reasons (art. 217 CPP), request that the judge receives witness testimony in a place other than the court. NO SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Procedure for hearings relating to especially vulnerable victims (FEMALE VICTIMS) 1. Provide immediate emotional support and kindness; avoid sexist judgments against the victim or placing blame on them for what occurred. 2. Provide a private space in which the victim can be heard. 3. Allow the victim access to the evidence file. 4. Inform the victim of their right to serve as a concurring plaintiff or as a plaintiff in a civil action. 5. Facilitate the conduct of psychological and psychiatric assessments to establish the moral, psychological, and psychiatric harm caused. Evidence Produced Before Trial 1. Request the statement of the victim/witness as pre-trial evidence once appropriate laws are verified 2. When the judicial authority does not approve submission of victim/witness statements as pre-trial evidence and the prosecutor is not in agreement with the judicial authority’s judgment, appropriate legal remedies are used. NOTE: PAVI measured the knowledge of facts and investigation stages. Current Stage: Investigation continues. Case Study 3: Femicide Description: The case was brought by the District Attorney’s Office of Petén. The crime occurred on July 13th, 2011 in the house the victim shared with her brother in San Benito, a few blocks from the headquarters of the District Attorney. The victim was Tanya Chacón Góngora, 20. The accused is her uncle, Maynor Góngora Cruz, 21. At 2:30 pm on July 13th, Maynor entered Tanya's house. After an argument and physical struggle, Maynor suffocated and beat to death. He then fled the scene but later returned, at which point the police apprehended him. The investigation discovered that prior to the crime Maynor had lived with Tanya and her brother, but was kicked out of the residence after trying to film her with his cell phone when she was alone in her room. The investigation also found that Maynor used alcohol and drugs and had emotional problems. The investigation also conducted a series of scientific tests, including of the tissue sample found underneath Tanya's finger nails. The preliminary hearing was held on November 3rd, 2011 in the Petén Magistrate’s Court of Criminal, Drug Trafficking, and Environmental Crimes. The judge admitted the charge and began the trial process for the crime of femicide, scheduling the trial for June 19, 2012 in the Sentencing Court for Drug Trafficking and Environmental Crimes in Petén. PROGRESS IN APPLICATION OF THE COORDINATED “ASSISTANCE PROTOCOL FOR WITNESSES IN THE CRIMINAL PROCESS” SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Case No. 3 – Femicide Municipality: San Benito, Petén Tanya Sofía Yolanda Chacón Court of Criminal, Narco￾activity, and Environmental Crimes in Petén KNOWLEDGE OF FACTS Report– Direct, from victim 1. OAP receives report/complaint in written or verbal form (if the victim’s state requires it) with the support of a professional from the OAV. NO 2. Victim is referred to a forensic and medical examination. NO 3. OAV personnel provide crisis intervention support before and during the presentation of the report/complaint. NO Crime Scene (direct victim or collateral or witnesses) 1. Security and transfer to the appropriate prosecutor’s office is coordinated with the National Civil Police. YES 100% NO SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Complied with 0 of 1 applicable actions. Compliance: 0% 2. Medical attention and transfer to hospital is coordinated. NO 3. Awaited physician’s authorization to conduct first interview. NO 4. In the case of children or adolescent victims without legal representation, coordinate presence of the Procuraduria General de la Nacion for the interview and protection. NO 5. If the prosecutor determines that there is a high risk of danger to the victim or witness, coordinate support with the Office of Witness Protection. NO Police Complaint or Prevention 1. Locate the victim. NO 2. Develop Investigation and Victim’s Attention Plan. NO RECORDING OF STATEMENTS In Prosecutor’s headquarters Complied with 0 of 1 applicable actions. Compliance: 0% 1. Protect identity. YES 100 % NO 2. When working with a protected witness, coordinate with the Office of Witness Protection. NO 3. Arrange transfer of victim from their home or workplace to the judicial headquarters either through PNC or referral network. NO Before Judicial Authority 1. Request for protective, security, or emergency measures, as appropriate. 2. Use of video-conferencing, victim or witness proxies, screens, voice distortion, Gesell Dome, or other measures that prevent direct visual and auditory contact with the suspect. 3. Inform the judge or court comptroller if there are protection measures for the victim that must be taken into account in the judicial headquarters. SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO 4. When there are safety reasons (art. 217 CPP), request that the judge receives witness testimony in a place other than the court. Procedure for hearings relating to especially vulnerable victims (FEMALE VICTIMS) 1. Provide immediate emotional support and kindness; avoid sexist judgments against the victim or placing blame on them for what occurred. 2. Provide a private space in which the victim can be heard. 3. Allow the victim access to the evidence file. 4. Inform the victim of their right to serve as a concurring plaintiff or as a plaintiff in a civil action. 5. Facilitate the conduct of psychological and psychiatric assessments to establish the moral, psychological, and psychiatric harm caused. Evidence Produced Before Trial 1. Request the statement of the victim/witness as pre-trial evidence once appropriate laws are verified. 2. When the judicial authority does not approve submission of victim/witness statements as pre-trial evidence and the prosecutor is not in agreement with the judicial authority’s judgment, appropriate legal remedies are used. NOTE: PAVI measured from the Knowledge of Facts stage up to the first statement of the suspect. Current stage: Debate begins 6-19-12. Case Study 4: Femicide Description: The case was brought by the Mixco Municipality public prosecutor’s office. The crime occurred during the early morning of July 19th, 2011 in Zone 7 of Mixco. The victim was Mrs. Evlin Tellería, 27. The suspect is Levin García, 23. At 4:45 am on July 19, following a heated argument, the suspect stabbed Evlin approximately 20 times, killing her. The cohabitants had gone to a nightclub that night with a friend of Evlin, who observed the two arguing. When leaving the nightclub, the same friend reported witnessing Levin striking Evlin in the face and then left. Shortly after the victim’s murder, police captured Levin at the scene of the crime. During the course of the investigation, it was discovered that prior to the femicide, Levin had beaten Evlin on multiple occasions and had even been imprisoned on charges of violence against women a few months earlier. The negligence of the MP in failing to provide protection to a victim whose life was in danger was compounded by the fact that the court had applied for the provisional dismissal of the violence against women charges against Levin. Two months after regaining his freedom, he killed Evlin. The investigation employed important methods of scientific evidence collection (DNA testing and fingerprints) that are not usually used in this type of process. On November 2nd, 2011, a preliminary hearing was held in the Special Court for Femicides in the capital city. The judge admitted the charge and began the trial process for the crime of femicide, scheduling the trial for August 3, 2012 in the Special Sentencing Court for Femicide in Guatemala City. PROGRESS IN APPLICATION OF THE COORDINATED “ASSISTANCE PROTOCOL FOR WITNESSES IN THE CRIMINAL PROCESS” SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO Case No. 4 – Femicide Municipality: Mixco Evlin Tellería Crimes Against Life Prosecutor (Mixco) KNOWLEDGE OF FACTS Report– Direct, from victim 1. OAP receives report/complaint in written or verbal form (if the victim’s state requires it) with the support of a professional from the OAV. 2. Victim is referred to a forensic and medical examination. 3. OAV personnel provide crisis intervention support before and during the presentation of the report/complaint. Crime Scene (direct victim or collateral or witnesses) Complied with 2 of 2 applicable 1. Security and transfer to the appropriate prosecutor’s office is coordinated with the National Civil Police. YES 50% YES 2. Medical attention and transfer to hospital is coordinated. NO SELECTED CASE PROCEDURAL STEP STEP COMPLIANCE PROCEEDINGS ACTIONS APPLICATION OF MEASURE YES/NO % YES/NO actions. Compliance: 100% 3. Awaited physician’s authorization to conduct first interview. NO 4. In the case of children or adolescent victims without legal representation, coordinate presence of the Procuraduria General de la Nacion for the interview and protection. YES 50% YES 5. If the prosecutor determines that there is a high risk of danger to the victim or witness, coordinate support with the Office of Witness Protection. NO Police Complaint or Prevention 3. Locate the victim. 4. Develop Investigation and Victim’s Attention Plan. NOTE: PAVI measured the knowledge of facts stage, investigation procedures performed, and preparation for the trial. Current Status: Debate, scheduled for August 3rd, 2012. ANNEX C M&E PLAN (REVISED AND FINAL VERSION) PROGRAM AGAINST VIOLENCE AND IMPUNITY IN GUATEMALA REVISED MONITORING AND EVALUATION PLAN SEPTEMBER 2011 September 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Partners for Democratic Change, a subcontractor of Tetra Tech DPK. PROGRAM AGAINST VIOLENCE AND IMPUNITY IN GUATEMALA REVISED MONITORING AND EVALUATION PLAN SEPTEMBER 2011 PROGRAM: PROGRAMA DE USAID JUSTICIA CONTRA LA VIOLENCIA USAID OFFICE: USAID/GUATEMALA TASK ORDER: DFD-I-08-04-00173-00 CONTRACTOR: TETRA TECH DPK DATE: SEPTEMBER 26, 2011 The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Program Against Violence and Impunity Page 2 Revised Monitoring and Evaluation Plan September 26, 2011 TABLE OF CONTENTS LIST OF ACRONYMS ....................................................................................................... 2 I. INTRODUCTION .......................................................................................................... 3 II. APPROACH .................................................................................................................. 4 III. METHODOLOGY ........................................................................................................ 4 IV. CRITERIA FOR SELECTING QUALITY INDICATORS ........................................ 5 V. DATA COLLECTION .................................................................................................. 6 VI. MONITORING AND EVALUATION MATRIX ....................................................... 7 ANNEXES ........................................................................................................................ 18 Program Against Violence and Impunity Page 3 Revised Monitoring and Evaluation Plan September 26, 2011 LIST OF ACRONYMS CENADOJ: Centro Nacional de Documentación del Organismo Judicial. Judiciary’s National Archive Center. DAV: División de atención a la víctima de la PNC. Division of assistance to victims (PNC) ETI: Equipo técnico interinstitucional (MP, INACIF, PNC). Inter-institutional Technical Team IDPP: Instituto de Defensa Púbica Penal. Institute of Criminal Public Defense INACIF: Instituto Nacional Forense. National Forensic Institute JAI: Juzgados de Alto Impacto. High Impact Courts LDO: Ley Delincuencia Organizada .Law Against Organized Crime MCC: Corporación Retos del Milenio. Millennium Challenge Corporation MP: Ministerio Público. Public Ministry OAV: Oficina Atención a la Víctima del MP. Public Minstry’s Office of Assistance to Victims OJ: Organismo Judicial. Judiciary PNC: Policía Nacional Civil. National Civil Police SICOMP: Sistema Informático de Casos del Ministerio Público. Public Ministry’s Integrated Computer System Program Against Violence and Impunity Page 4 Revised Monitoring and Evaluation Plan September 26, 2011 I. INTRODUCTION The Program Against Violence and Impunity (PAVI) is a USAID-funded initiative in Guatemala, implemented by Contractor Tetra Tech DPK (Tt DPK) which aims to strengthen the rule of law by improving the delivery of judicial and prosecutorial services, heightening coordination among justice sector institutions, and building civil society’s capacity to monitor and report on the performance of the justice operators.Subcontractor Partners for Democratic Change (PDC) is currently executing a subcontract from Tt DPK to provide Monitoring and Evaluation (M&E) technical support to the PAVI and act as an external evaluator of the Program.Under this subcontract, PDC developed the M&E Plan, designed the data collection methodology and instruments, and reports on the progress of PAVI’s performance and impact indicators on a quarterly and annual basis. Tt DPK has offered this unique evaluation strategy to bring in an external team of evaluation specialists early in the execution of the Project and thus ensure an external yet collaborative approach to M&E that involves PAVI’s technical staff and counterparts in all phases of monitoring performance and evaluation of this Project.PAVI has two dedicated staff personnelthat work on M&E: the Deputy Project Director and the Planning and Supervision Assistant. They are in charge of implementing all M&E activities (management, data collection, and information processing) under each Project Sub Intermediate Result. PDC works directly with these two staffers and the rest of the PAVI team to gather all the required information to conduct the quarterly and annual M&E reports. Data Evaluation Responsible parties:  External = PDC  Internal = PAVI  Participatory = PDC-PAVI-Counterparts Coordination and Technical Support Responsible party: Tt DPK Home Office Data Collection Responsible party: PAVI Data Reporting and Analysis Responsible parties: PDC & PAVI Report Evaluation Responsible party: USAID Program Against Violence and Impunity Page 5 Revised Monitoring and Evaluation Plan September 26, 2011 II. APPROACH Many different types of evaluation exist, including internal or self-evaluations (those conducted by the implementing project team), external evaluations (conducted by an outside expert or organization not directly associated with the project activities), and participatory evaluations (conducted with participation from multiple stakeholders.) The following M&E Plan is a combination of all three types of evaluation methods, due to the innovative strategy of bringing in an outside evaluation team to work closely with the Project team and local counterparts throughout the Project implementation. During each evaluation visit the evaluator will apply them together or independently based on each indicator. The PDC team has crafted an M&E Plan that will involve the PAVI team, key Guatemalan government institutions and CSOs, as well as the external evaluators (PDC) to collaborate during all phases of the evaluation, including planning, data collection, analysis, reporting, dissemination, and any follow-up activities. The advantages of this combined approach are that the Project staff and Guatemalan institutions are the most suitable to analyze information and to come up with new ideas for solving problems and improving performance because of their deep understanding of the organizations in which they work and the reality “on the ground.” This allows managers in the counterpart institutions to use the resulting information and recommendations and assess the effect of their reform strategies. A recognized disadvantage of a participatory evaluation, however, is that it can be less objective because participants may have a vested interest or may defend current strategies, and the credibility of data collection can be questioned. The Partners team, as a subcontractor that is not directly involved in the day-to-day implementation of the Project, will assume key aspects of the M&E Plan methodology to assuage these disadvantages of a more participatory approach. III. METHODOLOGY Although the terms “Monitoring” and “Evaluation” are used together when determining a “M&E Plan,” these are two different functions, with different conceptual and operational implications.Performance monitoring is an on-going process intended to provide information to managers about the progress of specific project outputs according to fixed objectives and targets.Monitoring is intended to tell what is happening or how it is happening in a project, in order to assess if the project is on track to meet its objectives.The M&E Plan presents a series of indicators to monitor the performance of project activities against agreed targets. The performance indicators will be measured on a project-year base1 and will be reported on a quarterly basis. Since the PAVI Project is scheduled to finish on June 28, 2012, the data for 2012 to complete the Final M&E Report will be collected through the month of May of that same year as a cut date. Evaluation is a more comprehensive concept. It looks at the effects and impacts of the project’s activities, intended to help establish lessons learned and prioritize development strategiesin the 1 The Project Year 1 corresponds to the period from July 29, 2000 to September 30, 2010. The Project Year 2 corresponds to the period from October 1, 2010 to September 30, 2011. The Project Year 3 corresponds to the period from October 1, 2011 to June 28, 2012 Program Against Violence and Impunity Page 6 Revised Monitoring and Evaluation Plan September 26, 2011 future.Evaluation takes place at a specific moment in time, usually mid-term or at the end of a project life-cycle, using a range of quantitative and qualitative measures. The M&E Plan also sets a series of indicators and targets to measure the impact of project activities. The data to report on these indicators will be collected mostly from selected public sources such as the Public Ministry’s SICOMP. Since these sources issue their reports at the end of the calendar year, the complete progress on the impact indicators will be reported on a calendar-year basis2 . The evaluation of the PAVI Project will take place at the end of the project life-cycle and will be included in PAVI’s M&E Final Report. The final monitoring and evaluation will be comprised of an analysis of the overall progress of the impact indicators (through May 2012) and the development of case studies that look for changes in outcome that are directly attributable to the PAVI Project. The goal in providing the PAVI Project with a complete M&E Planis to generate credible and useful information that contributes to improved performance, accountability, and learning from experiencein order to inform decision-making. The baseline data for the M&E of the Project will be collected according to the 2008 calendar year. IV. CRITERIA FOR SELECTING QUALITY INDICATORS The following M&E Plan includes a set of indicators to accomplish both the Monitoring and Evaluation functions mentioned above. PAVI is a three year project (June 2009-June 2012) operating under complex development environments where the Project’s interventions andinstitutional-level changes are often not apparent for many years. It is expected that over the life-cycle of the Project, different types of indicators will be more useful at different times.Over the first six months of the Project, process (output) indicators will be the primary source of performance information, while during the second and third years the impact indicators will become more revealing.The intention of the following M&E Plan is to specify a full set of indicators from the outset, acknowledging that some might need to be added as the project activities unfold, or further revised or dropped because they are unsatisfactory or no longer needed because their targets have been met. Determining adequate and appropriate indicators is not an exact science and the precision of a laboratory experiment is not required, especially when operating in a data-poor development setting such as Guatemala. Notwithstanding these shortcomings, the indicators were carefully selected to reflect USAID’s criteria of integrity, accuracy, reliability, and timeliness in its M&E exercises. The Tt DPK approach to determining appropriate indicators is to acknowledge the trade-offs between what might be considered the “best” indicators (but ones that are prohibitive because they are too costly to implement, or require too much lag time to be able to measure) and what are “reasonable” 2 The complete progress of the impact indicators will be reported in PAVI’s M&E Quarterly Report for the period October – December of each year. Program Against Violence and Impunity Page 7 Revised Monitoring and Evaluation Plan September 26, 2011 indicators that reflect what is realistic to measure with a focus on how the information gathered will be used.A healthy dose of common sense helps to streamline the process, so that the Project is only collecting and reporting on information that is most directly useful to measure performance and manage for results.More information is not necessarily better, especially if it increases the burden on the Project staff and involves excessive costs to collect and analyze.The indicators that have been recommended within the M&E Plan below have been crafted to reflect objectivity, practicality, validity, and attribution to the Project. V. DATA COLLECTION Many of the quantitative indicators will require the Project staff to obtain official data from the government institutions with whom they are working.We know that the data collection methods within the judicial branch and legal system in Guatemala are less than perfect, and in fact one of the priority areas of the Project is to offer technical assistance to improve their internal information systems.As the Project implements its activities, therefore, one of the effects will be that the quality of the official data will surely improve.The external evaluator will assist PAVI staff in conducting brief spot checks to verify data reporting within a sampling of the Public Ministry (PM) units whose data is being used for the Project’s indicators (and other institutions offering victim’s assistance as well).On an annual basis,the external evaluator will verify the cases reported during a particular month and compare to the data officially used by the institution.This external verification will not only serve to report on the level of error within the PM and others’ statistics, but will also assist the Project in identifying areas within the record keeping system that needtechnical assistance for improvement. Other data will be collected directly through internal Project documentation and records, including participant surveys applied by the PAVI staff, personal interviews conducted by the external evaluator and information to build case studies and success stories (that will be reported as part of the Final M&E Report). The following articles of the Guatemalan Criminal Code define the crimes that will be tracked by the PAVI staff under the different indicators:  Crimes against life: (a) Homicide: Articles 123 and 124, (b) Infanticide: Article 129, (c) Parricide: Article 131, (d) Murder: Article 132; and Femicides: Article 6, Law Against Femicides and Other Forms of Violence Against Women (Decree 22.2008)  Crimes against cultural patrimony: Articles 332 A, B, and C.  Crimes against the environment: Articles 346 and 347. VI. MONITORING AND EVALUATION MATRIX The PAVI Project fits within USAID’s broad goal of Ruling Justly: More Responsive, Transparent Governance, and will seek to address USAID’s IR 1: Greater State and Society Capacity to Prevent and Combat Violence and Serious Crime.The following indicator matrix includes performance indicators and impact indicators. It also includes a breakdown of both quantitative measurements and supporting qualitative analysis depending on each indicator. Following the matrix for each Sub-Intermediate Result (Sub-IR), there is a descriptionin narrative form of the methodology that will be used to collect, analyze, and report on the data. Critical Assumption: It is assumed that there will be sufficient political will and institutional cooperation in support of the Project and in delivery of information (at both the technical and institutional level), and that the current transition in institutional leadership within key counterparts does not hinder the Project’s ability to continue with planned activities. Sub-IR1: Improve Justice System Capacity to Prosecute andTry Serious Crime. Disaggregate Baseline value Data source Timing (Data collection and reporting period) Responsible persons Targets This Sub-IR will address:(i) poor treatment and attention to victims and witnesses; (ii) inadequate management of information resources; (iii) inadequate inter-institutional coordination; (iv) lack of institutional planning and development; (v) lack of compliance with key legislation, such as the Law Against Organized Crime; and (vi) inadequate management causing undue delays in case processing. PERFORMANCE INDICATORS: 2. Number of Trainings/Workshops/Coordination Meetings for justice sector professionals (F) Explanation: This indicator applies to number of trainings, workshops and coordination meetings with justice sector professionals working on prosecution and conviction of crimes against life. - Per institution or unit - Per jurisdiction 2008 0 Project reports Quarterly PAVI technical Staff PY13 30 PY24 50 PY35 20 (a) Number of participants (justice sector professionals) in the trainings and workshops (F) Explanation: Prosecutors, judges, and administrative personnel - Gender of participant 2008 0 Participant sign-in sheets Quarterly PAVI Staff PY1 80 PY2 250 PY3 100 3 The Project Year 1 comprises the period from July 29, 2009 – September 30, 2010 4 The Project Year 2 comprises the period from October 1, 2010 – September 30, 2011 5 The Project Year 3 comprises the period from October 1, 2011 – June 28, 2012 Program Against Violence and Impunity Page 9 Revised Monitoring and Evaluation Plan September 26, 2011 from Justice Sector institutions participating in PAVI’s training events. (b) Level of participant satisfaction (justice sector professionals) in trainings and workshops (usefulness of information and quality of presentation/facilitation) (F) Explanation: The percentage corresponds to the participants’ evaluation of the trainer, the content and length of the workshop, and knowledge acquired by the participants. The percentage is an average of all the evaluations conducted during the project year - N/A 2008 0 Post-event survey Quarterly PAVI Staff 80% satisfaction in each project year Sub-IR1: Improve Justice System Capacity to Prosecute andTry Serious Crime (cont.) Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets IMPACT INDICATORS: 3. Increase in Crimes Against Life and Femicide brought to trial Explanation: Increase in the number of crimes against life and femicides that are brought to trial over the life of the project. This indicator will be measured in incremental percentages over the life of the project. Targets are cumulative in accordance to the baseline. - Per crime - Per target jurisdiction (Guatemala Department and Petén) Guat. 2008 Petén 2008 SICOMP Annual PAVI Staff 2010 5% 2011 7% 2012 10% Cal 131 Fe 0 Cal 0 Fe 0 4. Increase in the number of guilty verdicts for Crimes Against Life Explanation: Increase in the number of guilty verdicts for Crimes Against Life to be measured over the life of the project. - Per target jurisdictions - (Guatemala Department and Petén) Guat. 2008 Petén 2008 SICOMP Annual PAVI Staff 2010 5% 2011 7% 2012 10% 100 0 Program Against Violence and Impunity Page 10 Revised Monitoring and Evaluation Plan September 26, 2011 NARRATIVE EXPLANATION OF METHODOLOGY FOR IMPACT INDICATORS FOR SUB-IR1: Improved Justice System Capacity to Prosecute and Try Serious Crime 1. Increase in number of Crimes Against Life and Femicide brought to trial This impact indicator will be measured by looking at the official statistics from the PM for criminal charges filed for crimes against life and crimes under the Law against Femicide within the target jurisdictions: Guatemala Department (that includes Villanueva and Mixco) and Petén. The PAVI M&E staff person will be responsible for gathering the statistics from the SICOMP on an annual basis.The Project will actively workwith SICOMP to improve their information systems and the validity of the data.Acknowledging that the official data reported by the PM is less than perfect, a comparative model will be used to look at the increase in the number of criminal charges annually, making the level of error acceptable to determine whether an increase has been achieved. 2. Increase in the number of guilty verdicts for Crimes against Life Baseline information from 2008 will be collected by the PAVI M&E staff personfrom SICOMP6 . This information will be compared with the official data on an annual basis. A comparison between the baseline and 2012 data will enable us to determinewhether the Project has had an impact on the rise in guilty verdicts. On an annual basis, the external evaluator will conduct interviews with criminal court judges to discuss with them the quality of prosecutors’ charges, improvements that have been made in the target courts to handle this increase in charges, and their perception of this impact on an increase in convictions. 6 The information will not be collected from CENADOJ to avoid discrepancies in the form of data reporting between these sources Program Against Violence and Impunity Page 11 Revised Monitoring and Evaluation Plan September 26, 2011 Sub-IR2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence. Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets This sub-IR seeks to improve:(i) inadequate or insufficient governmental and non-governmental legal services for victims of violent crime, including domestic violence; (ii) insufficient coordination amongst government and non-governmental institutions that provide attention to victims of violent crime; and (iii) insufficient violence prevention measures for the most vulnerable groups, such as women and children. PERFORMANCE INDICATORS: 5. Number of /Workshops/Coordination Meetings/Trainings for institutions offering victims assistance (F) Explanation: This indicator corresponds to the number of workshops, trainings, and coordination meetings with justice sector professionals and civil society advocates that participate in the prevention and reduction of violence. - Per institution orunit - Per jurisdiction 2008 0 Project reports Quarterly PAVI technical staff PY1 0 PY2 10 PY3 5 (a) Number of participants in the Coordination Meetings, Workshops and Trainings (F) Explanation: This indicator corresponds to the number of participants from the justice sector, civil society, and non￾governmental organizations that provide assistance attending coordination meetings, workshops, and trainings - Gender of participant 2008 0 Participant sign-in sheets Quarterly PAVI technical staff PY1 80 PY2 40 PY3 20 (b) Level of participant satisfaction in Coordination Meetings ,workshops and trainings (utility of information and quality of facilitation) (F) Explanation: The percentage corresponds to the participants’ evaluation of the trainer, the content and length of the workshop, and knowledge acquired by the participants. The percentage is an average of all the evaluations conducted during the project year) - N/A 2008 0 Post-event survey Quarterly PAVI technical staff 80% satisfaction in each project year 6. Improved coordination between organizations offering victims’ assistance Explanation: Organizations using the Protocol will coordinate better the provision of integral assistance to victims acting as - Phases of the criminal procedure National level diagnostic on the provision of services to victims administered in 16 Surveys (Guatemala) and analysis of 4 selected cases (Guatemala, Quarterly PAVI technical staff PY1 0% PY2 30% PY3 70% Program Against Violence and Impunity Page 12 Revised Monitoring and Evaluation Plan September 26, 2011 witnessed during criminal proceedings. The use of the Protocol by the Prosecutor’s Office for Crimes Against Life will help this unit improve in-house and external coordination. This indicator will be measured based on surveys applied to prosecutors on the use of the Protocol by the Prosecutor’s Office for Crimes Against Life and information drawn from 4 cases that are using the Protocol departments) completed by August 2010. Mixco and Petén). IMPACT INDICATORS: 7. Increase in the number of victims who receive free legal and other assistance (medical, psychological, etc)(see methodology section below for quality measurements of victims’ assistance.) Explanation: Percentage of victims who receive free legal counsel and other assistance. This indicator will be measured incrementally over the life of the project. - Per type of crime - Gender of victim - Services offered - Per institution selected 2008 Data for number of victims that receive integral assistance services from the Office of Free Legal Assistance of the Institute of the Public Defender (IDPP), Office of Victim’s Assistance of the Public Ministry (OAV), Division of Assistance to Victims of the National Police, FundacionSobrevi vientes.(See Annex A) Counterpart institutions: DAV-PNC, OAV/MP, , Sobrevivientes, Public Defense Institute and PAVI’s sub￾grants recipients Annual PAVI technical staff 2010 10% 2011 15% 2012 20% Program Against Violence and Impunity Page 13 Revised Monitoring and Evaluation Plan September 26, 2011 Sub-IR2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence (cont.) Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets 8. Increase in the number of guilty verdicts for Crimes Against Femicides Explanation: Increase in the number of guilty verdicts for Femicides over the life of the project. - Per type of crime - Per target jurisdictions (Departments of Guatemala and Petén) Guat. 2008 0 Petén 2008 0 SICOMP Annual PAVI technical staff 2010 0% 2011 10% 2012 10% NARRATIVE EXPLANATION OF METHODOLOGY FOR IMPACT INDICATORS FOR SUB-IR2: Mobilize Justice Sector and Civil Society to Reduce and Prevent Violence 1. Increase in the number of victims who receive free legal and other integral assistance. The goal of this Sub IR is to improve the attention to victims.The Project technical staff and the M&E staff person will work with the various publicinstitutions and CSOs to collect their statistics on the number of victims that receive assistance.This includes the Police Victim’s Attention Division, the Office of Victim’s Attention at the PM, the Public Defender’s Office of Free Legal Aid, and Fundacion Sobrevivientes. A standard format will be developed by the PAVI staff so these institutions can submit their statistical information. These statistics will be gathered every year and captured in a database developed by PAVI for this purpose: - number of victims receiving assistance (disaggregated by gender) - type of case - service offered - per selected institution The Project will provide technical assistance to public institutions and non-government service providers (recipients of the sub-grants program) to develop protocols of victim’s assistance.In addition to the overall analysis of this impact indicator, the Final M&E Report will evaluate the project by presenting four case studies from the Guatemala Department and Peten that demonstrate the use of the protocols as a vehicle to improve the assistance to victims. The case studies will be presented in story form. Program Against Violence and Impunity Page 14 Revised Monitoring and Evaluation Plan September 26, 2011 2. Increase in the number of guilty verdicts under the Law Against Femicides Because the Law Against Femicides is a new Law, many counterparts are not sure how to determine which crimes fall under it and how to report their cases.This indicator will also be used at the end of the Project to look at the official statistics that SICOMP reports on the number of criminal convictions under the Law Against Femicides.Baseline information for 2008 (year in which the law was passed) will be collected by the PAVI M&E staff from the SICOMPand compared with the official statistics on an annual basis.A comparison between the baseline and 2012 data will demonstrate whether the work of the Project has resulted in a higher number of criminal convictions. On an annual basis, criminal court judges will be interviewed to discuss their perception of the Law and its application, and their perception of any impact on an increase in convictions. The Final M&E Report should also include final interviews and document any changes in the judges’ perception. Program Against Violence and Impunity Page 15 Revised Monitoring and Evaluation Plan September 26, 2011 Sub-IR3: Increased Internal Accountability and Oversight within the Justice Sector. Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets This Sub-IR addresses: (i) lack of consolidation of financial management, lack of internal controls, teams that have little managerial capacity/experience; (ii) the inexistence, incomplete implementation, or inadequate enforcement of job descriptions, hierarchies, and responsibilities; (iii) the lack of implementation of performance evaluation systems; and (iv) inadequate statistical reporting and analysis. PERFORMANCE INDICATORS: 9. Number of units (judicial, technical, and administrative) from the different institutions that are currently implementing the Institutional Integrity Model (IIM) (F) Explanation: Justice sector institutions attending trainings on institutional strengthening processes to implement the Institutional Training Model within their organizations.Targets are cumulative starting from Year 1 of the project and include the different institutions implementing the IIM. - N/A 2008 0 Project reports Quarterly PAVI technical staff PY1 3 PY2 10 PY3 10 (a) Number of participants in the trainings and workshop Explanation: Number of Prosecutors, judges, administrative personnel of the justice sector institutions that attend PAVI training events on the IIM. - Gender of participant 2008 0 Participant sign-in sheets Quarterly PAVI technical staff PY1 80 PY2 100 PY3 100 (b) Level of participant satisfaction in trainings and workshops (utility of information and quality of presentation) Explanation: Percentage of people who give a positive feedback regarding the quality and implementation of the training event. The percentage corresponds to the participants’ evaluation of the trainer, the content and length of the workshop, and knowledge acquired by the participants. The percentage is an average of all the evaluations conducted during the project year - N/A 2008 0 Post-event survey Quarterly PAVI technical staff 80% satisfaction in each project year Program Against Violence and Impunity Page 16 Revised Monitoring and Evaluation Plan September 26, 2011 IMPACT INDICATORS: 11. The Institutional Integrity Model has an impact on improving internal ethics and management within target justice system institutions Explanation: The institutional integrity strategy contributes to the development of leadership abilities of justice sector professionals in their everyday functions and fosters a better institutional climate. This indicator measured the impact of the IIM as perceived by the justice sector professionals participating in the project activities. The percentage corresponds to the number of justice sector professionals attending the leadership development trainings. - Per institution - Per gender 2008 0 -Pre- and post￾perception surveys. -Personal interviews with project participants, -Focus groups interviewing. Annual PAVI technical staff 2011 70% 2012 80% NARRATIVE EXPLANATION OF METHODOLOGY FOR IMPACT INDICATORS FOR SUB-IR3:Increased Internal Accountability and Oversight within the Justice Sector 1. The Institutional Integrity Model has an impact on improving internal ethics and management within target justice system institutions. The Institutional Integrity Model (IIM) is a transversal strategy that PAVI is implementing throughout all Sub-IRs of the Project; therefore, it is not contemplated that there will be separate Institutional Plans or workshops specifically to implement the IIM. As part of this strategy, PAVI is working with the judiciary, the Public Ministry, the Institute of the Public Defenders and the National Civil Police’s Office of Victim’s AssistanceTheIIM is at its heart a process that capitalizes on individual leaders working within their institutions to effectuate change.This positive approach fosters greater ownership of participants and supports true change agents and champions of reform who want to bring more accountability and oversight into the judicial institutions in Guatemala.The impact indicator for this Sub-IR, therefore, is more focused on the process and its perceived impact on the individuals participating in the PAVI activities. This indicator will be measured through a perception survey (pre and post) and personal interviews during year II and III of the Project. The Final M&E Reportwill include focus groups interviews to capture the process and results. As a result, PAVI expects to be able to develop two success stories that will tell the story of two participants and their perception on the impact of the IIM activities. Program Against Violence and Impunity Page 17 Revised Monitoring and Evaluation Plan September 26, 2011 SPECIAL ACTIVITY 1: Supporting High Impact Courts Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets The goal of these courts is to focus on combating serious crimes such as organized crime, kidnapping, and drug and human trafficking cases, in a secure environment for judges, prosecutors, lawyers, witnesses, forensic experts, victims, defendants, evidence, etc. IMPACT INDICATORS: 13. Number of High-Impact Courts established Explanation: Number of High-Impact Courts that are functional and operating. - Guatemala department 2008 0 - Criminal Tribunal ofthe Supreme Court Annual PAVI technical staff 2010 1 2011 1 2012 1 14. Number of Cases referred to the High Impact Courts Explanation: Number of cases referred to and processed at the High Impact Courts. - N/A 2008 0 - Administrativ e Center of Criminal Courts Annual PAVI technical staff 2010 5% 2011 7% 2012 10% SPECIAL ACTIVITY 2: Strengthen Justice Capacity to Combat Illegal Activities in Petén *. Disaggregate Baseline value Data source Timing (data collection and reporting) Responsible persons Targets Petén is one of the main organized crime trafficking routes for drugs, money, arms, and persons. The activities under this special activity are intended to improve the ability of local prosecutors and criminal courts to process serious crimesin the region. IMPACT INDICATORS: 15. Increase in Serious Crimes brought to trial in Petén (See indicators for Sub-IR1and Sub-IR2) Explanation: Number of serious crimes brought to trial. This indicator will be measured incrementally over the project’s life - Per crime as defined by the Project’s target laws CAL 2008 EC 2008 CP 2008 SICOMP Annual PAVI technical staff 2010 5% 2011 7% 2012 10% 10 56 25 Program Against Violence and Impunity Page 18 Revised Monitoring and Evaluation Plan September 26, 2011 The indicator for special activity 2 is closely tied with the data that will be reported within Sub-IR1 as Petén will be one of the jurisdictions included in the data collection for the increase in criminal prosecutions and sentences of Crimes Against Life and Femicide.To measure the additional results under this special activity, the PAVI technical staff and M&E staff person will also work with the SICOMP to attain additional information on specific crimes that are more pertinent to the Petén region (e.g., environmental crimes, crimes against cultural patrimony).The Project will provide technical assistance for the implementation of SICOMP II in Petén. To measure the results of this activity, the external evaluator will conduct interviews with users of the new platform on an annual basis. ANNEXES A. 2008 Baseline Indicator # 7 Program Against Violence and Impunity Page 20 Revised Monitoring and Evaluation Plan September 26, 2011 ANNEX A 2008 Baseline for Indicator # 7 Institución OAV-Ministerio Público IDPP Fundación Sobrevivientes OAV-PNC Número de víctimas que reciben servicios de Asistencia Legal gratuita y otra asistencia (médica, psicológica, etc.) Actividades y tipos de asistencia Asistencia Legal Gratuita Servicios ofrecidos (Social, Psicológica y Legal) Tipo de delito (Violencia Sexual e Intrafamiliar) Casos nuevos 12,335 11,422 2,207 7,557 Re-consultas 25,114 Referencias y acompañamientos 10,060 Referencias 7,334 Acompañamientos 2,736 Visitas 744 Oficina de Atención a la Víctima, Ministerio Público: Actividades y Tipo de Asistencia Total I. CASOS NUEVOS 12,335 Atención psicológica 10,448 Atención médica 217 Atención social 1,494 Atención jurídica 176 II. ACTIVIDADES DE SEGUIMIENTO Y APOYO (RECONSULTAS) 25,114 Psicológica 20,744 Médica 54 Social 4,199 Jurídica 117 Referencias y acompañamientos 10,060 Referencias 7,334 PNC 231 Juzgados 427 Bufetes 3,169 Red de Derivación 3,507 Acompañamiento 2,726 AOP 582 PNC 26 Juzgados 188 Program Against Violence and Impunity Page 21 Revised Monitoring and Evaluation Plan September 26, 2011 Bufetes 110 Otros 1,820 Visitas 744 Visitas domiciliares 565 Visitas institucionales 179 Instituto de la Defensa Pública Penal (Asistencia Legal Gratuita): Oficina de Atención a la Víctima-Policía Nacional Civil: Tipo de asistencia Total: 7,557 Violencia Sexual 74 Violencia Intrafamiliar 7,483 Fundación Sobrevivientes: Tipo de asistencia Total: 2,207 Área Social 293 Área Psicológica 1,705 Área Legal 502 Tipo de asistencia Total: 11,422 Asesorías verbales 6,351 Pensión alimenticia 1,350 Medidas de seguridad 1,337 Asesorías documentadas 757 Ejecutivo 380 Ejecutivo en la vía de apremio 135 Modificación de pensión alimenticia 133 Oposición a medidas 99 Casos penales 88 Juicios ordinarios de afiliación de paternidad 79 Otros casos 713 ANNEX D 2010 M&E ANNUAL REPORT (Attached Separately) ANNEX E 2011 M&E ANNUAL REPORT (Attached Separately)