1 April 2015 This evaluation was made possible by the support of the American People through the United States Agency for International Development (USAID).The contents of this evaluation are the sole responsibility of Linn Hammergren, Diego Bolivar, Diana Maureen Moreno and Management Systems International, and do not necessarily reflect the views of USAID or the United States Government. Mid-term Performance Evaluation of the Access to Justice Activity ii MID-TERM PERFORMANCE EVALUATION OF THE ACCESS TO JUSTICE ACTIVITY Linn Hammergren, Diego Bolivar, Diana Maureen Moreno and Management Systems International Management Systems International 200 12th Street South Arlington, Virginia www.msiworldwide.com April, 2015 Contracted under AID-514-C-13-00003 USAID/Colombia EVAL (Evaluation and Analysis for Learning) DISCLAIMER This evaluation was made possible by the support of the American People through the United States Agency for International Development (USAID).The contents of this evaluation are the sole responsibility of Linn Hammergren, Diego Bolivar, Diana Maureen Moreno and Management Systems International, and do not necessarily reflect the views of USAID or the United States Government. iii CONTENTS Acronyms ...................................................................................................................................... iv Executive Summary ...................................................................................................................... v Introduction ................................................................................................................................... 1 Background ................................................................................................................................... 1 Evaluation Design and Limitations ............................................................................................. 3 Findings, Conclusions and Recommendations ........................................................................... 4 Component 1: Assist the MJL in implementing access programs ........................................... 5 Question 1-A: Improving the equity conciliators’ program...................................................................... 8 Question 1-B To what extent has the AJA augmented the NPEC’s sustainability? .................................. 9 Question 1-C: How successfully has gender been incorporated in the NPEC?.......................................10 Component 2: Gender ................................................................................................................ 10 Question 2-A: Increased gender sensitivity and knowledge, and impacted handling of cases..............13 Question 2-B: Engaging institutions in a coordinated response to GBV cases.......................................14 Question 2-C: Increasing citizen awareness of the legal framework for gender issues.........................15 Question 2-D: Engaging civil society partners in advancing gender aims?.............................................15 Component 3: Judicial Reform.................................................................................................. 16 Questions 3-A and 3-B: Improving LRC case management.....................................................................19 Question 3-C: To what extent has AJA included gender in the component?.........................................22 Question 3-D: Improving performance and sustainability of local justice committees.........................22 ISSUES: Technical and/or Administrative Concerns ............................................................. 23 Documents Reviewed .................................................................................................................. 26 Annex 1: Getting to Answers (G2A).......................................................................................... 31 Annex 2: Scope of Work (SOW) ................................................................................................ 34 Annex 3: List of Interviewees..................................................................................................... 42 Annex 4: EVAL Framework for Findings, Conclusions and Recommendations ................. 47 Annex 5: Notes, permissions, data collection instruments (electronic submission) .............. 59 iv Acronyms Note: most acronyms follow the English translation of their names, but in a few cases, the Spanish acronym is used because it is so well known (e.g. ANSPE) ADR Alternative Dispute Resolution ANSPE National Agency for Overcoming Extreme Poverty AJA Access to Justice Activity (the Project) CECAR University Corporation of the Caribbean CEJ Corporation for Excellence in Justice CISPA Intersectoral Commission for Monitoring of the Accusatory Criminal System COMPOS Municipal Committee for Social Policy (Tumaco) COP Chief of party CSO Civil society organization CZ Conflict zone/s EVAL Evaluation and Analysis for Learning G2A Getting to Answers – evaluation planning format GBV Gender-based violence GoC Government of Colombia EC Equity conciliator FIP Ideas for Peace Foundation FJJD Formal Justice and Jurisdiction Division (of the MJL) IACA International Association of Court Administration ICT Information and communication technologies IP Implementing partner ISO International Organization for Standardization IT Information technology LJC Local Justice Committee LJS Local Justice System LRPD Land & Rural Development Program LRU Land Restitution Unit Mag. Magistrate (in Colombia, a high court judge) MICE Framework for the Implementation of Equity Conciliation MJL Ministry of Justice and Law MSI Management Systems International NGC National Gender Commission NGO Non-governmental organization NPEC National Program for Equity Conciliation NPJH National Program for Justice Houses PACE Point of Attention for Equity Conciliation RECOMPAS Network of Community Councils of the Southern Pacific SGC Sector Gender Commission SJC Superior Judicial Council TCU Territorial Consolidation Unit U.S. United States USAID United States Agency for International Development v Executive Summary This document reports the results of a mid-term evaluation of USAID/Colombia’s Access to Justice Activity, implemented by Checchi and Company Consulting (Checchi). The evaluation is organized around a series of questions about the impact and sustainability of the activities of most interest to USAID. The evaluation process was participatory; both USAID and the implementing partner (IP) were invited to help in the formulation of the work plan and research instruments and to join interviews and field trips taken by the three-person research team. The purpose of the AJA is to improve access to justice at the local level, in 29 municipios in Colombia’s conflict zones, with particular attention to the poorest rural citizens. The project has three components: • Access -- assistance to the Ministry of Justice and Law (MJL) in strengthening its access to justice (or alternative dispute resolution, ADR) programs and making them more effective and sustainable at the local level; • Gender -- assistance to the Superior Council of the Judiciary (SJC), other GoC agencies, local authorities and civil society organizations in improving state response to gender￾based violence and other legal issues affecting women as well as making women aware of their rights; and • Judicial Reform -- support to the SJC in furthering judicial reforms, especially in the areas of the new land restitution jurisdiction, the criminal procedures code, and local justice systems. AJA operates through its seven regional offices; it has two national counterparts (the MJL and SCJ), four national and seven regional strategic partners, and additional contracted firms, consultants and grantees. The evaluation team visited three of these offices and one target municipio in each region. A first limitation on its findings is thus the restricted geographic coverage of the data collection. A second is the lack of statistics on justice operations in the regions (although the project is tasked with improving certain databases). Third, like the project, the evaluation lacks data on the attitudes, behaviors and experience of the ultimate beneficiaries – the poorest inhabitants of the targeted regions. While not stressed in the evaluation questions, improvements to beneficiary experiences are a principal project goal. In Component 1 (Access) the principal finding is that only a small portion of trained and certified equity conciliators (ECs) remain active. As the ECs work on a volunteer basis, the high attrition is not surprising. The situation is unlikely to change as providing salaries is not feasible. Additional factors contributing to the attrition are a shortage of places to work and lack of provisions for reimbursing expenses. While AJA uses the MJL’s new training methodology, no one had evaluated its impact on work quality or retention. AJA’s efforts to explore additional incentives for ECs, strengthen the MJL’s oversight through creation of a database on conciliations, and encourage local support for the conciliators have been marginally successful. The database in particular has not received sufficient cooperation from the MJL, the conciliators or local authorities. vi The MJL’s Director for Access programs reported a new plan for the conciliation program that recognized the inevitability of attrition, proposing to sustain the program through continuous training of new conciliators. He said the MJL has sufficient funds to do this. The evaluation’s principal recommendations are to investigate the status of this plan, encourage the MJL to seek national support for whatever plan it now endorses, and if the Ministry truly can finance all training, shift some project support to 1) a series of studies on conciliators’ activities and impacts; and 2) training other local officials who also do conciliation. In Component 2 (Gender), AJA works with a series of national, regional and local actors. Their different perspectives and methods have produced a fragmented program. AJA’s counterpart, the SJC-supported National Gender Commission, works with judges and focuses on the development and use of jurisprudence on gender. AJA’s national strategic partner, an NGO alliance, mobilizes women victims and promotes recognition of their rights, at both the national and regional levels, while regional strategic partners and grantees do much the same at the local levels. Partners working on other components are still less systematic in how they incorporate gender, although the regional advisors have introduced the concepts, if not the practices, into local justice committees (LJCs). In its gender work, AJA shares the stage with several other donors, making it difficult to attribute project influence to outcomes. One exception is in AJA’s emphasis on rights-based jurisprudence, which has influenced higher-level magistrates, if not first-instance judges in the 29 municipios. The evaluation team found three key issues requiring attention. First is the activity’s ill-advised focus on judges and their use of jurisprudence in their sentences, rather than the many actors (local authorities, prosecutors, police, etc.) who first receive and thus often discourage complaints. Second, the project lacks information on changes in women’s attitudes and behavior, despite attempting to address these issues as part of programming. Third, AJA’s focus on rights and demand does not help officials understand how to act differently. Some of these issues are dealt with under the project’s recent contract amendment. It is recommended as well that AJA develop an overall gender strategy, to bring more unity among its implementers and to emphasize activities and groups that differ from what the mass of donors support. Under Component 3, evaluators were asked to review only the land restitution and local justice elements. Work with the land restitution jurisdiction includes developing a model for better case management and supporting the work of judicial committees. The committees are exploring ways to improve the land courts’ procedures, security, training, technology and inter-institutional cooperation. As with the Gender Component, the exclusive focus on judges seems misplaced as most problems with the land restitution process occur elsewhere, including in the interface between judges and the Land Restitution Unit (LRU). Moreover, work on case management appears less relevant as judges currently have little work. The ISO-based model that is being implemented focuses on standardization rather than efficiency, and it is unclear how it will meld with the Superior Judicial Council (SJC)’s simultaneous “zero paper” project. The evaluation team recommends that the AJA turn its focus to the judge-LRU interface, ideally in coordination with USAID’s Land & Rural Development Program (LRPD). The team also recommends that AJA replace the ISO approach with more proven instruments for improving court efficiency in the event that the judicial caseload expands significantly. vii The local justice sub-component aims at creating local justice committees and encouraging them to formulate policies and articulate local services to improve quality and access. Although both the Territorial Consolidation Unit and the MJL’s Formal Justice Division appear as counterparts in the project contract, neither has been influential in this work, which is largely conducted by AJA’s regional advisors. The committees have been formed and meet periodically in the sites visited; however, work on coordinating local services or on gender remains highly abstract, possibly undercut by the emphasis on designing small projects as a means of retaining participation. Nonetheless, AJA activities have revealed a good deal about obstacles to LJC performance and if there are successful examples they should be shared within and across regions. Recommendations also include discussing any MJL plan with the ministry in recognition that the MJL will design Colombia's “10-year justice program” and that local justice will be an essential part of that plan. Furthermore, studies on barriers to local justice should be more widely disseminated within the MJL and other interested GoC actors. 1 Introduction The purpose of the exercise was to evaluate USAID/Colombia’s Access to Justice Activity (AJA) at mid-term, two years into its four-year lifetime. The project seeks to increase access to justice for inhabitants of Colombia’s conflict zones, especially among the poorest rural citizens. It has three components, as discussed further in later sections: Access, Gender, and Judicial Reform. The evaluation’s target audience is USAID and its implementing partner (IP), Checchi and Company Consulting. USAID intends to use the evaluation to strengthen project performance and enhance the likelihood that the desired results are reached in the time remaining. Selected findings may inform future discussions with GoC counterparts, strategic partners and other project implementers. Discussions with the GoC and especially the MJL will be especially critical as their plans for both access and local justice have evolved considerably since the project was designed. Conclusions and recommendations may also affect decisions to deprioritize certain areas or activities deemed less likely to contribute to project goals. Evaluation team members structured their work around a series of questions relating to impact and sustainability. (See Annex 1, G2A) Evaluators did not review activities with which USAID was satisfied (e.g. support to the National Program for Justice Houses under the “Access” Component) or for other reasons did not wish to continue (e.g. further support to the implementation of the criminal procedures code under the Judicial Reform Component). Finally, although a recent amendment to the project modified the Gender Component to permit concerted work with the Attorney General’s Office, this addition was not evaluated given the short time it had been in effect. The evaluation is unusual in its participatory nature. Both Checchi and USAID were invited to collaborate in drafting the work plan and to accompany the team in interviews and field trips, as possible. This participatory method was considered especially important here given a series of events leading to the departure of three of Checchi’s key personnel and the simultaneous amendments to the scope of work. By encouraging ample participation, the evaluation aimed to bring the parties to mutual understanding and agreements even before the findings, conclusions and recommendations are presented. The goal was to allow the IP to move into future activities with more confidence of USAID support. Background The project (Access to Justice Activity or AJA) being evaluated is the latest in a series of USAID projects supporting Colombia’s justice sector. It has three principal components: • Access -- assistance to the Ministry of Justice and Law (MJL) in strengthening its access to justice (or alternative dispute resolution, ADR) programs and making them more effective and sustainable at the local level; 2 • Gender -- assistance to the Superior Judicial Council (SJC), other GoC agencies, local authorities and civil society organizations in improving treatment of gender-based violence and other justice issues affecting women as well as making women aware of their rights; and • Judicial Reform -- support to the SJC in furthering judicial reforms, especially in the areas of the new land restitution jurisdiction, the criminal procedures code, and local justice systems. While AJA’s two counterpart organizations and four principal strategic partners1 are located in Bogota, the project’s focus is on improving justice service delivery at the local level and especially in 29 of 58 GoC-identified municipios in conflict zones (CZs).2 To this end, AJA has offices in seven regions or sub-regions, each staffed by two technical advisors and an assistant.3 Within each region, AJA has a local strategic partner, typically a university, to implement activities. In some instances it developed “strategic allies” (public, private and not for profit organizations with which it coordinates). It also has additional contractors and grantees. This simple explanation belies a series of organizational and substantive complexities. First, the three components are neatly separated only on paper. Gender is cross-cutting, while the local justice systems activities also incorporate the ADR mechanisms and the equity conciliators. Territorially, Checchi personnel working in the seven regions coordinate and help implement the three components in the municipios they oversee, working closely with their regional strategic partners. The four national strategic partners operate at both the national and regional levels, and there are also a series of contracts and grants with other institutes, civil society organizations and individuals. Second, over time the two GoC counterparts – the MJL and the SJC -- have made requests that have modified some of the activities, while certain strategic allies (Territorial Consolidation Unit or TCU, ANSPE4 , the departmental governments, and in Antioquia three additional private or public-private sector allies) have by their presence, policies and institutional offerings shaped implementation. In short, the context within which the project operates is constantly changing, requiring continual adjustments within each region. Mid-term modifications to the project were aimed at the larger national context, but there are some region-specific issues that continue to shape how each set of activities develops. The evaluation is intended to provide insights to USAID, Checchi, and the GoC to guide the 1 These partners are: Corporación Excelencia en Justicia (CEJ) a think-tank involved in justice reform programs and known for its empirically-based research on justice issues; the School for Community Justice of the National University, which has supported access programs for years and especially equity conciliation and local justice systems; the Gender Alliance, a group of gender-focused NGOs led by SISMA-Mujer; and FIP (Fundación Ideas para la Paz) another widely recognized research institute and think-tank. Each has conducted studies, developed models, and directly implemented activities in its area of expertise. 2 Municipio is used without translation as the term comes closer to county than municipality and in conflict zones is typically a large area with several small population centers and one “urban” center. The project initially called for work in 33 municipios, but for security reasons only 29 are currently included. 3 Staffing problems mean that some regions (e.g. Tumaco) currently have only one advisor. 4 ANSPE is the Spanish acronym for the National Agency for Overcoming Extreme Poverty. Both it and the TCU belong to the Department for Social Prosperity, located in the Presidential Office. 3 remaining two years of project life. Evaluation and Analysis for Learning (EVAL) hired a three￾person team for this purpose: an international expert in justice reform, a Colombian expert in justice operations, and a Colombian expert in gender (the latter was only added in the fifth week after two predecessors resigned for health reasons). Fortunately, the initial expert played a formative role in designing the work plan and methods and participated in the first interviews, making it feasible for the other team members to fill in during the early field work. Despite the eight weeks allowed for in-country work and its focus on selected questions and activities, the team had to limit its field observation to only a part of the project’s operative area. Team members visited one municipio within each of three regions, as well as Cartagena and Medellin5 . The sites visited were chosen according to several criteria: likelihood that all three project components could be observed in operation there, relative security, and location of the regional office. The team also interviewed all national counterparts and strategic partners, as well as Checchi staff, in Bogota. Evaluation Design and Limitations The methods used in the evaluation include document review, informant interviews, group discussions, analysis of statistics and other quantitative data drawn from studies and databases developed with AJA financing or by other sources, and field observation and site visits. Documents and interviews were the main sources of information. Annexes to this report list documents consulted and interviewees. Over 120 men and women were interviewed, individually or in group discussions. As the design and content of the evaluation are highly qualitative, the instruments for the interviews were constructed to allow adaptation to discussion group formats as necessary. The principal data analysis methods are comparison, content analysis, event reconstruction and change theory identification, trend/pattern identification, and self-assessment. How the methods were applied and to what ends varied by question. For example, comparative analysis was used to contrast actual implementation versus planned timeline, reported progress versus views of other actors and observation, implementers’ views of changes produced versus views of those within affected organizations, and evaluations of activities provided by different interviewees. Content analysis was used to evaluate various training and outreach materials, especially for gender content, but also for inferred effectiveness. It was applied to AJA reporting mechanisms (where comparative analysis was used to test what was reported against real events and also paradigmatic examples that might have been mentioned). The team used event reconstruction and identified theories of change, applying these theories to much of the interview material to draw out details of what was done. However, this was only the first stage. Once the “story” was amassed from the various (and sometimes contradictory) 5 Visits to Medellin (Bajo Cauca) and Cartagena (Montes de María) allowed interviews with land courts and tribunals, sector gender commissions, and departmental and national agencies with links to the project. In retrospect our visit to Tumaco might have included a side-trip to Pasto, where some of these agencies are located. 4 sources, further analysis required the team’s expertise in specific areas (as to “better practices,” success of approaches in other settings, etc.); comparisons with project plans; and identification of different developments and trends in each region. Trend/pattern analysis was conducted in two fashions: using developments reported in interviews to identify and compare changes across regions (or within a single one over time) and using available statistics to track predicted changes in outcomes and impacts. Capacity self-assessment analysis was applied to detect changes in actors’ orientations, whether self-reported or simply inferred from statements in interviews and observed behaviors. This was particularly relevant for the project’s objective of increasing gender sensitivity among judges and creating “an elite class of land restitution judges.” (Project Agreement, Section C, p. 37) Aside from the partial geographic coverage, the two principal limitations or gaps in data sources are the scarcity of quantitative data and the impossibility of capturing the views and experience of the ultimate beneficiaries – the citizens for whom access is being expanded. The team had initially planned to use existing databases and statistics to track changes in impact variables like numbers of complaints filed by women, processing times for land cases, and numbers of conciliations, absolutely and per equity conciliators. Unfortunately, most of these data were not available despite the project’s emphasis on improving record keeping, a goal that could still be met by the project end date. Apart from the inventories of equity conciliators and some material on land cases available on line and from studies (including CEJ’s Land Observatory, contracted under the project), the best that could be done was to rely on estimates made by interviewees. Significantly, the project indicators include few quantitative results (as opposed to outputs), possibly in recognition of the difficulty of documenting them. Access to quantitative data might have lessened the second limitation – the nearly exclusive reliance on the views and experience of project participants6 as opposed to beneficiaries. For example, changing rates of complaints filed by women or of conciliations done and agreements reached could indicate greater access as well as new attitudes among citizens, two ultimate project goals. Instead we had to focus on reported changes on the supply side – number of conciliators, mobile clinics, rights campaigns, and so on – without being able to tap into their effect on citizen users, potential or actual. ANSPE’s household survey does include some questions about access, but the initial results suggest a lack of fine-tuning in their phrasing. The evaluators had neither the time nor the resources to explore the project’s impact on the demand side. Findings, Conclusions and Recommendations The following discussion is organized by component, and further divided by evaluation questions (See Annex, G2A matrix for framework). 6 This catch-all term refers to all actors involved in project implementation as well as the “justice operators” (a Colombian term for actors providing justice services) receiving training and other support. 5 Component 1: Assist the MJL in implementing access programs Context and Findings AJA activities in this component support the MJL to implement access programs, particularly the National Program for Justice Houses (NPJH) and the National Program for Equity Conciliators (NPEC). The evaluation only covers the NPEC, as USAID determined that the NPJH is now ready for graduation from Mission support. Although the aim of this component is to expand access to justice at the local level, a secondary goal entails enhancing the MJL’s ability to implement its access programs on its own. USAID has been supporting the MJL’s Access Division (under a series of different names but with considerable continuity in staffing) for over 20 years, with particular emphasis on the Justice Houses and Equity Conciliation. The two programs are closely related as one of the principal services of the Justice Houses is equity conciliation and many, but certainly not all, equity conciliators do their work there. Despite its interaction with the NPJH, the NPEC faces more impact and sustainability problems. This is especially true of the equity conciliators (ECs) working outside the Justice Houses and the services provided to areas where Houses do not exist. Although the training of ECs has been ongoing since at least the early 1990s, the MJL had not maintained contact with those it had trained (largely with the support of USAID and other donors). Inventories done by AJA indicate that most ECs are no longer active. Attrition rates vary by region and municipio. In Nechi (Bajo Cauca), 15 of the initial 17 ECs were said to remain active while in Tumaco of the 74 trained in 2009, only 10 were still conciliating, only three of them on a regular basis. In the regions surveyed most municipios had attrition levels between 50 and 70 percent. The MJL and to an extent the GoC have nonetheless portrayed the conciliators as key actors in expanding access to remote populations and in creating an environment of peaceful co-existence (“convivencia”) in the conflict zones. The diminished number of active conciliators raises questions as to their ability to play that role (as well as the value of training more), and thus poses an immediate challenge of how to support, expand, and sustain the program. AJA’s approach to these challenges has been to address the issues listed in evaluation question 1- A below, in the hopes that resolving them would increase the NPEC’s sustainability. The strategy operates at three levels – introducing improvements to MJL oversight (especially in data management and possibly in funding potential), improving training and incentives for ECs, and developing local support, especially through the Local Justice Committees. AJA funds training of new conciliators only in municipios where none exist. Elsewhere regional partners are to focus on strengthening the existing group. The NPEC has experienced some improvements, in part because of changes introduced by the MJL, and in part from project activities. Beginning in 2009, the MJL introduced a more systematic approach to the selection and training of conciliators. This is called MICE or Framework for the Implementation of Equity Conciliation. 7 Historically, the Ministry has lacked funds to carry out either activity, depending on USAID, departmental governments or other 7 MICE has four stages: a preliminary diagnostic, selection of candidates, a 120-hour training, and a final “accompaniment”. It is controversial even among those charged with applying it. The National University’s School for Community Justice, for example, has added aspects not included in the MJL framework although claiming these are simply details. Observers (and conciliators) also question the use of a “psychotechnical” exam to weed out the unsuitable after they have been trained. The exam is applied by consultants hired by the Procuraduría under an agreement with the MJL. 6 donors for implementation. 8 The MJL reports it now is financing most training; the evaluation could not verify this claim. If true, it means USAID might drop this component or eliminate training and reinforcement activities. AJA has adopted MICE and instructed its implementing partners to use it in their training programs. AJA’s national strategic partner (the National University’s School for Community Justice) is applying it in Bajo Cauca as are regional partners in other regions. A few departmental governments have funded universities to train conciliators using the MICE approach.9 Unfortunately, MICE does not include a component for “strengthening” existing conciliators meaning that where this is done with AJA funds, implementing partners are on their own as to how to proceed. In the three regions visited approaches varied. In Tumaco, the University of Nariño provided a diploma course in administration of justice (including a section on conciliation) for ECs and other local actors. In Montes de Maria and Bajo Cauca, “strengthening” involved some short seminars on conciliation, aimed not only at ECs but also local authorities, and a vaguely defined “accompaniment” that seemed limited to sending one or two trainers to visit and work with ECs within each region. The MJL reports it is developing a program for EC strengthening, but so far has provided no indication of what this will involve. Another MICE shortcoming is the absence of any material on gender or the differential focus. Although Checchi’s contract notes the necessary inclusion of both in training efforts, we found only two examples in our interviews and fieldwork. The University of Nariño has a section on gender in its diploma course, but it is very academic and not aimed at conciliators in particular. The School for Community Justice reports that it has developed a gender component it is piloting in Santander (not included in the project). The school, however, also believes intra-familial violence can be conciliated, despite this being illegal in Colombia and contrary to the views of gender activists. The project has partially compensated for what the MJL methodology does not include through the work of the regional advisors. The advisors meet with conciliators in all the municipios in their regions, encourage their inclusion in the local justice committees (LJCs) and in mobile clinics and brigades, and keep the gender focus alive at the local level in their work with the LJCs and support to national partners and local grantees promoting gender rights. Aside from its support to the MJL’s selection and training methodology and the advisors’ various contributions, potentially the most significant part of AJA financing has gone into diagnostic work and other activities to strengthen the program as a whole – the inventories, assessments of the situation of conciliators in each region, studies on incentives, promotion of alternatives for placing ECs outside the Justice Houses, and efforts to develop databases on conciliations done within each municipio. AJA also attempts to assess the impact of its overall Access Program (including Justice Houses) by including questions in ANSPE’s10 Red Unidos surveys. 8 For example, in 2009 the European Union financed the training of 74 ECs in Tumaco.. 9 The University of Antioquia is now training ECs in Bajo Cauca with departmental funds. In Bajo Cauca, AJA uses the School for Community Justice for this purpose. The AJA uses the University of Antioquia as a regional partner in Bajo Cauca to provide outreach programs of psychological and social work assistance to outlying communities. These activities are sometimes coordinated with the local ECs. 10 ANSPE, a strategic ally, is the National Agency for Eliminating Extreme Poverty. See below for further details. 7 Interviews with ANSPE in Medellin indicated that the answers provided are derived by an ANSPE caseworker evaluating the responses of Red members to a series of questions and then calculating a single “score” for each one regarding his/her ability to identify ADR mechanisms. The score is not very useful and of suspect reliability as we don’t know how the different responses have been aggregated (or whether different caseworkers use the same method). All this work is shared with the MJL in periodic meetings with its Access staff, but the ministry’s support for or use of the results has been sporadic at best. The database in particular has received little cooperation from the ministry, local authorities or the conciliators; interviewees reported that it functions only in areas where there is a Justice House and with conciliators who work within it. AJA efforts to encourage the introduction of PACEs (Attention Points for Equity Conciliation) outside the Justice Houses have been frustrated by the lack of local financing for their creation and equipment. 11 The MJL’s limited concern for high attrition rates and thus the importance of continued training is reflected in the comment by one ministry representative: “if 70 percent of the equity conciliators abandon the program, that is not a problem.” The comment is consistent with what the Director of MJL’s Access Division describes as the ministry’s new plan: MICE training (largely financed by the MJL), funding to municipios to finance EC expenses, and constant formation of new ECs in the expectation that unsalaried volunteers will not remain active for long. The plan, the official status of which could not be determined, shifts the issue of sustainability from retention of individual conciliators to the program as a whole. Despite AJA’s Access Coordinator’s regular meetings with MJL Access staff, the “new plan,” which has significant implications for AJA’s on-going work, had not been communicated to her. Although not included in AJA, conciliation is also in theory practiced by many other officials at the local level – community mediators, comisarios de familia, inspectores de policía, justices of the peace and prosecutors12) – many of whom receive little if any training in the methods. The mediators belong to the Juntas de Acción Comunal, organizations developed under Ministry of Interior auspices (as further specified in a 2012 agreement between the two ministries). Some AJA advisors (and at least one department, Bolivar) have begun to work with mediators as a more practical means of expending ADR access. 11 This is one area where the Territorial Consolidation Unit (discussed in Component 3) has been important in equipping some PACEs with computers. 12 Because the comisarios, inspectores and the related personeros will be referenced repeatedly, an explanation of their roles is offered here. The names are untranslatable. In theory all are lawyers. The Inspector oversees compliance with local and national law and is often the first person contacted when there is a problem. The comisario deals, as the name implies with family problems ranging from child support to sexual abuse. The personero often helps drafting legal documents for individual citizens and is responsible (through a link with the Procuraduría de la Nación) for ensuring compliance with human rights. They are all political appointees and what each actually does depends on local demand and politics as well as their own inclinations. 8 Question 1-A: To what degree has the IP improved the NPEC in the dimensions established in the agreement? (Selection, training, retention, data management, coordination with local justice bodies, incentives, local financing, service provision, outreach) Conclusions Selection and training are more systematic under MICE, but neither Checchi nor the MJL has assessed its impact on the quality of services (or the attrition rate). Any assessment will be difficult without the IT database that is to be created by AJA and additional impact indicators. MICE’s lack of a program for strengthening existing ECs is a concern, but Checchi’s ability to develop its own methodology is s constrained by its partnership with the ministry and the latter’s contention that it is developing an approach. If Checchi can systematize the results of experimental efforts by its strategic partners, these could be used in discussions with the ministry. The use of a volunteer (i.e. unpaid) labor force for the conciliation program is not likely to change, and attrition will likely be high as long as they are unpaid. Also, volunteers are less likely to provide information on their services, keep regular hours, or travel distances to serve communities.13 To pay them would create an employment relationship with longer-term legal and budgetary consequences, something the GoC wishes to avoid. Another alternative explored by AJA (in a study by the National University) – the use of enhanced access to benefits provided by other ministries – would be hard to negotiate and is a less effective incentive than direct pay. The mediators (also unpaid) are another possibility, but we know very little about who they are, what they do and with what impact. Since in Colombian terms, mediation differs from conciliation in the absence of a written agreement for the former, it would be difficult to determine use and efficacy. The AJA inventories (essentially lists of conciliators operating in each community) are a good start, but understanding the program, its impacts, and its potential value would require information on use, either as part of a systematic and (eventually) centralized record, or as a consequence of more in-depth community surveys. We also lack information on citizen attitudes in the communities where ECs operate. The working hypothesis supporting expansion is that use will grow with a greater supply of conciliators and knowledge about their services. This might be true, but experience elsewhere suggests there may be other sources of resistance to their use.14 13 This conclusion is based on observation and comparison of “paid” and unpaid conciliators in one regional visit Although payment is illegal, one municipio had found a way around the prohibition for several but not all its conciliators. We were told by Checchi staff that this is a practice in other municipios as well. 14 See, for example, Sally Engle Merry (1993) on the ADR movement in San Francisco and its eclipse. 9 Recommendations If it is true that the MJL can now finance all training, AJA should devote more resources to developing information on what conciliators are achieving, who is using them, and whether agreements are being enforced. Use of the Red Unidos survey to tap into citizens’ experience of and attitudes about conciliation would work with better questions and a different methodology.15 If the National Planning Department is able to implement a proposed survey on unmet justice needs in rural areas, AJA should try to use it as a still better instrument. In coordination with the MJL and the various strategic partners, Checchi should develop a more uniform approach to strengthening existing conciliators. We understand the desire to avoid “cookie-cutter” methodologies, but the current methodological free-for-all among strategic partners is not good practice, either. Despite the disappointing results of efforts to create a system for registering ECs and their activities, AJA should work with the MJL to resolve the numerous obstacles, and utilize both the Justice Houses and the LJCs to encourage ECs to provide the required information. If the 2012 agreement between the MJL and the Ministry of Interior is really in effect, then any work with community mediators should be coordinated not with the MJL but with Interior. Question 1-B To what extent has the AJA augmented the NPEC’s sustainability? Conclusions The NPEC will not be sustainable until 1) the MJL or some other national or sub-national entity takes it to heart and assigns it significant funding, and 2) enough conciliators are active to constitute a critical mass as a local and regional pressure group. The MJL’s new plan seeks to meet these conditions by redefining the issues – not as keeping trained conciliators in place, but as guaranteeing a constant supply of newly trained ECs. However, if the plan is to succeed the MJL will have to ensure it is made national policy in order to overcome any resistance and ensure the plan survives a change of Minister. Although its proponents see the plan as more feasible and no more costly than trying to retain ECs, it may well require a constant stream of funds in excess of their calculations AJA efforts to build local capacity (within regions or departments) to train conciliators could help ensure NPEC survival so long as the MJL or departmental governments will sustain the effort with ongoing funding. 15 Rather than asking whether respondents are familiar with ADR, it would be preferable to have answers to a series of questions -- e.g. whether respondents have heard of ECs, whether they have ever used them or know someone who has, and what they think about the results. Similar questions might be asked about the justice houses, legal clinics and mobile exercises. 10 Recommendations The MJL’s plan for equity conciliation should be discussed and explored by Checchi and USAID to determine whether MJL priorities have changed. If the MJL can finance all training, AJA might focus, in coordination with the Ministry, on studies (see Q 1-A), implantation in the local justice committees in line with the Ministry’s plan, and developing improved programs to publicize conciliators’ services among ordinary citizens. Depending on conversations with the MJL, unneeded funds for training conciliators might also be redirected to training other local or locally-based authorities who, in theory, can conciliate. Question 1-C: How successfully has gender been incorporated in the NPEC? Conclusions MICE’s apparent inattention to gender is a concern, but with the MJL as a partner or counterpart, Checchi again cannot afford to go it alone. It bears mentioning that the conciliation ideology in some sense runs counter to the “rights approach” (although “legal” rights, but not gender rights in particular, are mentioned in passing in MICE). The difference lies in conciliation’s emphasis on “convivencia” and reaching peaceful, negotiated solutions to conflicts. For activists, rights are not negotiable. More information is needed on how gender is being treated by the various training programs. It would also be useful to know how other local actors doing conciliation (or mediation) with or without training are handling gender in general, and gender-based violence in particular. There may be a legitimate debate about the conciliability of gender violence, but in Colombia the law makes it illegal. Recommendations In coordination with its own Gender leader and the MJL, Checchi should establish basic guidelines for how gender will be treated in EC trainings (whoever does them). Unless Colombian law changes, conciliation of gender violence should be explicitly precluded. Assuming it will continue EC training or strengthening, Checchi (and its regional coordinators) should monitor the treatment of gender by its strategic partners engaged in these activities. If AJA reduces or eliminates EC training, it might include in its studies a focus on the gender implications of conciliation. However, caution should be exercised here as the focus will place it between two conflicting, possibly irreconcilable ideologies – convivencia versus gender rights. Component 2: Gender Context and Findings Gender is another area in which USAID and the GoC have invested efforts over years, if not decades. USAID is not the only fish in the sea, however; virtually all donors support gender 11 work, often financing the same CSOs to advance it. This was especially evident in Tumaco where nearly half the members of the municipio’s “gender table” were donors. Colombia presents an odd combination of very progressive legislation on gender rights (including the differential focus, a unique aspect of national policy16), an active NGO community promoting them, and at the same time, high levels of gender violence and traditional cultural biases against women (as well as other protected groups). The negative characteristics are especially evident in the types of rural, conflict zone municipios where the project operates. AJA’s Gender Component incorporates two goals: raising women’s awareness of their basic rights and improving their treatment by operators in the formal and informal justice systems. To accomplish this, it works with a variety of agencies at the national, regional, and local levels. AJA’s chief counterpart for its gender work is the SJC and its National Gender Commission (NGC). The NGC aims at influencing judges and jurisprudence through a series of workshops, conferences, and publications (in hard copy and on its website). While working on the supply side, the commission restricts its aims to improving the quality and use of high court jurisprudence in judicial sentences. AJA’s national strategic partner – the Gender Alliance of CSOs organized by Sisma Mujer – works more on the demand side, devising campaigns to spread knowledge about gender rights and mobilize women to claim them. Alliance members occasionally conduct regional workshops and seminars. The project’s regional strategic partners are expected to include a gender focus in all their work, as are the regional advisors in their contacts and operations with local justice operators. As noted in the section on Access, this provision is sometimes honored in the breach. AJA’s CSO partners, grantees and subcontractors hired under the Gender Component have invited justice sector personnel to workshops and seminars on rights and rights violations. The Gender Alliance admits the approach is ineffectual in changing behavior. There is also the issue of whether partners’ work with “justice operators”17 is based on a sound understanding of their roles, weaknesses and capacities. Except for a partially successful effort at building networks of local lawyers and psychologists to support women victims, local work with supply has been limited to informing local actors about gender rights or sensitizing (“sensibilizando”) them to women’s issues rather than showing them how to treat victims in a more gender-sensitive, and thus effective, fashion. These choices, whether made by Checchi or by their partners, illustrate a theory of change that holds that demand forces modifications in supply (different attitudes, different receptivity to complaints, more willingness to deal with women victims). An alternative theory argues that 16 This has been likened to affirmative action, but goes much further, requiring special treatment for vulnerable populations – including not only women, but LGBT groups, children and the aged, ethnic minorities, the handicapped, and victims of the internal conflict. 17 This is a state-of-the-art term in judicial reform and refers to professionals, administrators and staff working in justice services. They were formally called “judicial operators,” but the new term is preferred because it incorporates agencies beyond the courts. 12 supply must change first, if increased demand is to find a satisfactory response and thus an incentive to continue registering complaints. Most probably, the ideal formula balances the two sides, never letting one get too far ahead of the other. The recent contract amendment (and the MJL’s stated resolve to work more closely with municipal actors like the comisario de familia) appears to be an effort to address the imbalance by improving the supply of gender-sensitive access to justice. These disagreements over method may also have affected AJA’s efforts to build a national network of gender NGOs. Interviewees reported that a major impediment has been conflict or competition (possibly over donor support) among potential members. Another factor, applicable to virtually all gender work, involves differences about focus. Although AJA mentions other vulnerable groups, gender is taken to mean women and gender issues are increasingly defined as gender-based violence (GBV). Even here there is the question as to whether the project focus includes GBV victims of the internal conflict (in which case additional GoC entities would be involved, including the Transitional Justice Office of the MJL) or is limited to current and future victims. AJA supports a Gender Observatory (a model developed by CEJ and applied by local partners). However, the trial runs in the analysis of sentences for a “gender focus” were not very successful and thus the model is being readjusted.18 This is the only activity we could identify that aims at measuring impacts on the ultimate goal – the better treatment of women in local justice systems. Otherwise, project and partner indicators count numbers of participants in workshops and trainings or publications; these are input not impact indicators. The former Gender Component coordinator did mention that the massive denunciation exercise (no longer funded by the project in part because of the conflicts it generated with GoC agencies) increased victims’ complaints ten-fold. Considering the broader target universe (and lack of information on impacts in the 29 municipios), the significance of this change is hard to assess. While this multi-actor, multi-level strategy should cover all the bases, it does so with limited coordination among the parts. AJA’s gender component is the least well integrated and to some extent (according to several interviewees) the least effective in having a local impact. The explanation for both developments is that it not only works at two (sometimes three) levels of government, but that at each level the strategic approach and goals differ. This has not precluded impact but tends to fragment results rather than encouraging mutual reinforcement. Comments by interviewees at the local level (AJA advisors, judges, members of the LJCs) suggest they are aware of the fragmentation. However, with the exception of members of a few Gender Sector Commissions, this seems to be missed at higher levels, possibly because the higher levels (including nationally based CSOs) do not communicate constructively with each other. 18 One impediment was accessing sentences featuring the issues selected by CEJ – the way cases are categorized and archived made this difficult. However, the sentence only reveals what the judge thought and wrote, not what the complainant or victim experienced. Identifying gender-correct language, “contextualization,” and the citation of jurisprudence does not tell us whether justice was done and rights recognized. 13 Question 2-A: To what extent has the AJA increased first instance judges’ gender sensitivity and knowledge, and its impact on their handling of cases? Conclusions Gender and the differential focus are embedded in Colombian political discourse. The project has helped expand the audience attentive to these principles. Notably, though, were AJA’s gender component to disappear tomorrow the campaign would continue unabated, given GoC and donor interest in its persistence. Despite initial skepticism from some quarters, the NGC campaign with high level judges brought these issues to a new audience, making it clear to participants (and especially magistrates in the high courts) that overt disregard for gender rights will be personally and professionally costly. The impact on the behavior of first-instance judges in sixth category municipios is doubtful, 19 but over time, if not within the next two years, the message will trickle down to this level. Nonetheless, the project’s emphasis on inclusion of jurisprudence and a gender focus in first instance sentences is misplaced (or at least less relevant). In terms of where women victims suffer most it was the wrong location. Current efforts to work with prosecutors (under the project amendment) and on-going work by the project and others with comisarios, inspectores de policía, and forensic medicine, are on a better track. Recommendations Continue to finance the NGC’s activities, but try to cut back on workshops and seminars. The NGC and its local equivalents, Sector Gender Commissions (SGCs), are important allies but the NGC approach is overly academic and formalistic and even if successfully implanted, it will not matter much to women victims. The SGCs might be more open to introducing other types of action, and if, pursuant to an agreement with the NGC, AJA could devolve more funding to them, the impact on local judges could be accelerated. In coordination with the Judicial School, include judges or create special programs for them in the training envisioned under the project amendment: hands-on and focused on techniques and case studies, as opposed to the current emphasis on drafting gender-sensitive sentences. 19 We cannot offer much direct evidence here, but a review of the content of the workshops (“magisterial” speeches by high level judges) certainly does not suggest an impact on judges’ day-to-day activities. Moreover, two established principles of effective behavioral change are honored in the breach -- frequent repetition and a focus on examples of positive practices (rather than negative ones). 14 Question 2-B: Has the AJA successfully and neutrally engaged GoC institutions (nationally and regionally) in a coordinated response to improving handling of GBV cases? Conclusions The AJA’s capacity to engage GoC institutions in a coordinated response to gender issues has never been high. It partners with institutions but does not drive their programs, which they typically pick according to their own sense of what is appropriate and effective. Moreover, judges often resist coordination as a threat to their “independence.” The project further limited its ability to encourage a coordinated response by working largely with judges at the national level. This should change under the amendment and the decision to work with prosecutors and other actors lower in the criminal justice chain. Within the LJCs, observation and interviews suggest that training and advisory services have not induced in-depth reflection on gender issues. While the language may be given a nod, LJC members have avoided coming to terms with their own weaknesses in this (and other) access areas. Encouraging LJCs to develop a bank of projects (see Component 3) has been useful in attracting members, but it diverts attention from discussion of existing service weaknesses. The gender project in one municipio was to invite a speaker from an NGC workshop to deliver a lecture for Women’s Day. Recommendations Focus on actors involved in the pre-trial stages and provide hands-on training in better ways to deal with victims. Here the new Checchi approach seems on the right track, but it will be important to tailor the trainings to the specific roles of each type of local operator. It would be useful for the project to collect data on the types of cases first registered with each “input” actor (from inspector de policía through prosecutor) as well as whether and to whom they are passed on. This information can be used both to illustrate problems and to measure advances. The “Obstacles to Access” (barreras) studies produced under a contract with FIP contain important information on treatment of women victims that should be used in discussions with national actors to expand their understanding of local-level problems. Such discussions should precede the proposed publication of the studies to avoid provoking resistance to the findings. 15 Question 2-C: To what degree have AJA actions increased citizens’ awareness of the legal framework for gender issues (including rights)? Which have been the most and least effective methods implemented? Conclusions While we know NGO campaigns (in general) have increased attention to gender rights, we do not know how this has affected women in the target municipios and their willingness to make civil and criminal claims. The Gender Observatory does not measure this. Information from the various entry points (inspectores de policía, comisarios, police, local prosecutors) tracking claims over time could be used to assess effectiveness. Unfortunately, the team did not have access to this information (and AJA is not collecting it). Considering the quantity of donors engaged in gender activities, even with data it will be difficult to identify AJA’s value added. Recommendations To advance its goals of influencing citizens’ awareness and measure its own efficacy in so doing, AJA needs to tap citizen attitudes directly, as discussed above in the recommendations on survey questions and themes. Taking into account the quantity of donors supporting gender work and their reported tendency to finance the same CSOs, AJA might assess its marginal utility in this area. If it is doing more of the same, in the same areas, and with the same actors, then doing something less redundant could be in order. Question 2-D: To what extent has the AJA engaged civil society partners in making an effective contribution to advancing gender aims? Conclusions Variety is often an asset, but civil society partners have worked under differing definitions of gender, focus areas, and methods. This has created conflicts in the project and also explains difficulties in forming a national network among the partners (as well as consolidating a GoC response). There may be too little variety, however, in AJA’s selection of strategic partners and its support to their activities as compared to what other donors do. Donor support to pro-gender groups tends to considerable redundancy both in who (which CSOs) and what (largely workshops and campaigns to create awareness of rights and egregious violations) is financed. AJA’s value added would increase to the extent it can break away from what every donor does and focus on a unique contribution –possibly by working with supply rather than demand, or by introducing better methods of driving the rights message home and inspiring the audience to 16 action. Of course the denuncias masivas (mass denouncements) campaign did just this with AJA support, but USAID assessed it as being too conflictual; the Organization of International Migration is now providing funding. Building demand for gender rights is a necessary step but has little immediate impact on how women are treated by justice sector institutions. Recommendations Especially in gender, but also in the other components, there is a need to coordinate activities across regions and strategic partners. Checchi might set up its own internal committee to do this or organize meetings to discuss the formulation of an overall gender strategy for its work. Funding to national- and local-level gender rights groups should be continued as cutting them off would send the wrong signals. However, unless these groups can be encouraged to find a way to work productively with justice sector officials, there is little need to expand the activity. AJA should focus partners on measuring their own impact (not just women attending workshops but measures of conversion of the message to action) to encourage more effective approaches. Despite the difficulties in establishing local victims’ assistance networks of lawyers and psychologists, the effort is worth continued effort. In line with a suggestion from AJA, building regional (rather than a national) networks of gender-focused CSOs would be closer to the aim of improving local access. Component 3: Judicial Reform Context and Findings This component incorporates three very different subcomponents (land restitution courts, criminal justice, and local justice systems), joined only by their connection to judicial reform. The SJC is the AJA counterpart for the land restitution jurisdiction and the criminal procedures reforms (as well as for the entire gender component). AJA lacks a government counterpart for local justice work, a situation that, as discussed below, is increasingly problematic. As regards the SJC’s counterpart role, this means that the project coordinates closely with one member of the SJC’s Sala Administrativa. For the Land Courts and Gender activities that member is currently Mag. Nestor Raul Correa who heads the SJC’s Gender Commission and represents the Sala in the Inter-institutional Committee formed to oversee implementation of Law 1448 of 2011 (the Victims Law that also established land restitution procedures). A change in Sala Member or the elimination of the SJC could radically affect judicial policy; both of these are part of the reforms currently under discussion. 17 There are currently 54 land restitution judges and magistrates, divided into five judicial regions (each with a Tribunal to hear appeals from the first instance courts and to handle the estimated 30 percent of claims to which there is opposition). The process is complex. During the largely administrative pre-trial stage, the Land Restitution Unit (LRU) investigates the claim and assembles a dossier of pertinent evidence, facts and appreciations for consideration by the judge. The trial stage can either be conducted in a one-judge first instance court, or, when a claim is opposed, in two steps with the first instance judge reviewing the LRU’s submission and doing any additional work needed before passing the case to the respective Tribunal. There is also a post-trial stage in which the judge must follow up on the many orders included in the sentence – not only the return of the land to the claimant (assuming s/he prevails) but also a series of social services and even infrastructure (e.g. roads, water) detailed in Law 1448 to guarantee the claimant can effectively use the land. The land restitution process has been remarkably slow and has affected only a small portion of the universe of potential claimants. As of March 28, 2015, only 74,161 requests had been presented to the LRU and only 7,801 of these had been sent to the judges, resulting in 2,040 sentences. As the statistics indicate and interviews and observation confirmed, neither the judges nor the 1,500 LRU staff members are overwhelmed with work; judges have so few caseloads that some have been transferred to ordinary civil courts while others receive tutelas and civil cases to fill up their time. In the land jurisdiction, AJA’s work is done under contracts with specialized consultants. AJA’s regional offices have little involvement as most municipios in the project do not have a land judge or LRU. 20 Nonetheless, the contracts of regional strategic partners typically include their implementation of one or more workshops or seminars on the land restitution process. We were told that these events are likely to focus on partners’ studies of the enforcement of sentences, but for lack of time were unable to review what else partners have done, with whom, or how (if at all) it is coordinated with the rest of AJA’s work with land judges. In line with its goal of producing an elite class of land judges, AJA has supported national meetings of land judges and those of the five substantive committees set up by judges to suggest better practices in procedures, security, training, IT, and inter-institutional coordination. The committees were suggested by a U.S. federal judge brought in by Checchi, as a means of involving the judges in jurisdiction governance. Each committee meets at least once annually. The national meetings formerly included members from other institutions involved in the land restitution process as a means of encouraging coordination, but Checchi reported USAID now prefers to finance substantive committee meetings. In theory each substantive committee has members from all five regions; our interviews suggest membership has dropped off and that most judges are not familiar with the committees’ recommendations. Most AJA work with land judges has focused on developing a more efficient organization for case management (the so-called modelo de gestión). This parallels and theoretically should complement a project organized and financed by the SJC – the development of an IT system 20 Even where they do (e.g. Carmen de Bolivar and Caucasia), the regional offices do not deal with them. 18 (“zero paper”) that will upload in-coming documents as scanned PDFs as well as allowing judges and staff to draft their own work directly into the system. Apparently because of judicial preferences, the modelo de gestión is being developed by a firm specializing in ISO 9001 techniques and will be linked to an accreditation process. The modelo de gestión and zero paper are currently being piloted in Montes de Maria (Cartagena(and Carmen de Bolivar) and Bogota respectively as a prelude to wider replication. At the SJC’s request the ISO vendor is conducting (reportedly at no extra cost) similar exercises for administrative and ordinary civil courts in Montes de Maria.21 It is worth noting that Colombia’s judiciary has been developing (often with donor financing) ISO-based courtroom management models for 20 years without any of them being adopted more widely. One major impediment is the reluctance of “independent” judges to adopt a model not tailored to their own preferences. Also, while possibly not a disincentive, ISO (re)accreditation adds further work and costs for the courts, which must keep records of “deviations” from the standards defined in the ISO process and pay for reaccreditation. Significantly although judges outside Bogota were eager to be included in “zero paper,” the modelo de gestión was not widely known. Moreover, while participants in the ISO project were positive about the experience, their enthusiasm was focused on the lengthy ISO process; neither they nor the vender representatives could describe the model itself. Related work under this subcomponent includes contracts with an expert to recommend security measures for land courts, with CEJ to develop a Land Observatory, and with an independent consultant (hired at the request of the SCJ and NGC) to develop a means of applying a gender approach to land cases. This approach will presumably be added to information collected by CEJ’s Observatory, which in its first application documented in great detail the land restitution process’ overwhelming inefficiency. However, CEJ’s 400-page report had less immediate impact than another USAID- financed study (contracted by the Human Rights Project) that made the same points in about three pages of text and graphs. Modifications to land court staffing – the addition of systems engineers in every courtroom – and some apparent oversights (such as lack of experts to assist judges in interpreting cadastres) were, so far as we could determine, not attributable to the AJA (or at least not to ISO). Work with the criminal procedures code (which USAID has also been supporting for years, and through three different codes) will largely be terminated and so is not evaluated here. In the remaining two years AJA will focus on legal clinics, not on improving university teaching or training judges and prosecutors in generic criminal code principles. The training remains on￾going although the Component coordinator says it is shifting to an emphasis on GBV investigations. AJA work in the third subcomponent aims at the creation of local justice committees (LJCs) as a means of improving local justice services. LJC members in theory include both local authorities 21 The University of Ibague (the strategic partner for Tolima) has also been contracted for an ISO exercise in that region, but not related to land courts. 19 and local staff of national organizations. AJA’s regional advisors are responsible for convincing local officials to establish committees, giving them guidance in making annual plans, and in the beginning convoking committee meetings. It is at this level that the three project components should converge and to the extent that they do so successfully, the project advisors play a key role. For example, advisors may encourage the introduction of a gender focus in LJC work, promote better coordination among the various local actors, suggest that committees publicize the role of equity conciliators, or promote the use of local “brigades” or “mobile justice houses” to reach outlying communities. The project has defined certain initial steps for LJC organization – the development and discussion of a “barriers to justice” study (done by FIP), an inventory and map of justice actors, and, over time, efforts to produce local projects and budgetary line items for justice. Aside from these tasks, membership in and activities of the LJCs depend on the AJA staff advisory skills and the local political context. Although the SJC is the counterpart for the land restitution courts and the criminal procedures activities, the local justice systems or committees (LJCs) have no sponsor, or rather a plethora of candidates. In the project contract, both the Territorial Consolidation Unit (TCU) and the MJL’s Formal Justice and Jurisdiction Division (FJJD) are mentioned, but the latter has no local presence while despite their presence, TCU representatives have not played an active role in LJC formation. The MJL’s Access Division, the counterpart for the Access Component, has, however, been working on a model and methodology for years, and now claims to have local justice system (LJS) pilots in Chia and some CZ and frontier municipios. We do not know how similar these are to AJA’s LCJs, but it is also evident that some aspects of Checchi’s approach (i.e. the inclusion of ethnic communities and community mediators) “belong” to the Ministry of Interior not Justice. Finally, most members of the LJCs (or LJSs) are administrative officials in the local governments, some with ties to other national actors.22 While judges and prosecutors are invited to participate neither of their institutions has responsibility forthe committees. There were in fact indications that higher level judges or prosecutors may not know their underlings are committee members. To add more complications, several departmental governments have shown an interest in the LJCs, usually through their Secretaríias de Gobernación. Like the TCU their interest is rarely backed by resources, but AJA’s regional offices cannot afford to ignore their ideas as to LJC structure and responsibilities vis-a-vis the Department. Questions 3-A and 3-B: Is the AJA’s approach to improving LRC case management effective and sustainable? What are judges’ and magistrates’ perceptions of AJA, and has change management been successful so far?23 Conclusions The land restitution process’ slow response and limited coverage cannot be blamed on judicial delay or inefficiency. The most important causes are the macro and micro focalization processes (managed by the defense services verifying that the areas are “secure” enough for claims to be 22 The personero for example is also a representative of the national Procuraduria in its human rights protection role. The comisario de familia is linked to the national family welfare agency (ICBF). 23 These two questions have been combined to avoid redundant entries. 20 considered) and the subsequent need for police assurance that the LRU and judges can work in the affected areas. Ignorance of many claimants as to how to proceed and fear of reprisals help to account for the low number of claims placed to date. At all stages of the process – pre-judicial, judicial and post-sentencing – less than complete cooperation by the other entities involved (for cadastre, titling, ensuring land does not have protected status, provision of additional services including security) has added further delays. In both the LRU and the land courts, lesser delays originate in uncertainty about individual responsibilities and practices. Both agencies spend an inordinate amount of time repeating the same investigations (e.g. establishing the presence of violent events that are already well documented, confirming that the claimant was affected by these or other incidents, and even revisiting the same plots under claim). Despite its 208 articles, neither Law 1448 nor the related Decree Laws 4633, 4634 and 4635 (dealing respectively with indigenous, Rrom and Afro￾Colombian communities) establish a clear guide to LRU and judicial practices, leaving far too much room for idiosyncratic interpretation. 24 Delay has implicit benefits for larger landholders, little of whose land has yet been affected, but also for the government. Given the many services provided to winning claimants, a vast increase in resolved cases will place a high burden on national and local budgets. If anyone has calculated the likely costs, their results have not been made public. It is doubtful that “better case management” for judges with few cases would make much difference, especially since the ISO project focuses on standardization, not greater efficiency.25 Moreover judges used to taking their time with a few cases will have difficulty adjusting should the flow from the LRU increase substantially. As one LRU director noted, the result would be system collapse. With regard to the zero paper effort, less (or zero) paper does not mean greater efficiency (and with a poorly organized electronic archive could mean far less). A few aspects of the model that may not be applicable more broadly – for example email notifications – may speed processing to some degree, but neither this nor the modelo de gestión’s better organized physical archives (containing very few cases) will make much difference in time to resolution or in judges’ ability to handle vastly larger caseloads. 24 Knowledgeable team members noted that Law 1448 lacked “reglamentacion,” or an expanded secondary law that would define the procedures in detail. As a consequence they and others report that judges often use the more complex ordinary civil proceedings, again adding delays. 25 As ISO’s name indicates, standardization is its goal. Admittedly, getting everyone to follow the same procedures is a step ahead, but the next step would be to vet those procedures for redundancy, unnecessary details, and so on. As for the judges’ repetition of the LRU investigation, the ISO team seemingly ignored this. 21 The case tracking software introduced in the two uncoordinated models raise the question of which will be used when the models are rolled out and combined. There may be similar issues with use of templates and archiving techniques. On-site observation of security measures suggests that whatever the security expert had recommended was not being used. Metal detecting doorways were often turned off and the one judge who had backed up her electronic files said she did this at the recommendation of her systems engineer. The work of the courtroom systems engineers is typically limited to repairing computers, setting up simple Excel spreadsheets, or even clerical tasks. The engineers are unlikely to be needed for much more unless the piloted projects develop truly complicated case management systems and databases. Even then, assigning one to each courtroom may be excessive. According to interviews, the outputs of the substantive judicial committees were either non￾existent or unknown. The one exception (significant but not widely known) is the development of protocols intended to standardize treatment of types of cases and develop guidelines for LRU submissions to the courts (thereby reducing the problem of judges’ varying preferences as to the form and content of LRU submissions). So far the protocols have not been adopted. Recommendations To improve the land restitution process, USAID should focus more on the LRU and its interface with the judges. Closer coordination is recommended with the Mission’s Land and Rural Development Project, which works with the LRU and other agencies whose cooperation is needed. Judges, who rarely attend the project’s coordination meetings, should be encouraged to participate. AJA should explore the potential for increasing process-wide coordination through reinstitution of the national meetings with attendance by other institutions or more work with the inter￾institutional commission on the implementation of Law 1448. Checchi (and USAID) experience with the comparable committee for the implementation of the criminal procedures code (CISPA) might provide ideas as to what support would be useful or whether this is indeed feasible. A reconsideration of the AJA’s use of ISO is merited. To put it bluntly, we recommend that AJA and USAID not finance further ISO activities and end ongoing projects as quickly as possible. If AJA insists that the model produced for one courtroom be adopted by others on the basis of a three-day workshop, replication might be attempted, and the results (adoption and improvements to outputs) carefully tracked. To replace ISO, the project should introduce some more useful (and proven) instruments for improving court efficiency – for example the “International Framework for Court Excellence” or the work of members of the International Association of Court Administration (IACA). Courts in 22 more developed countries that have increased their efficiency and improved services have not relied on ISO for this purpose. Judges’ insistence on redoing the LRU investigations is perverse. If a way could be found to encourage a reconsideration of this practice, it would be useful. The protocols produced by the procedural subcommittee may provide one solution, although there is no guarantee that individual judges will adopt them (because of judicial independence). Further work on a management model for land courts would benefit from a review of current staffing and consideration of how to handle much larger workloads (on the assumption that they eventually will emerge). It would also be useful to conduct a study on the probable costs of the additional awards (e.g. roads, police protection, training) to winning claimants and their impacts on national and municipal budgets. However, based on comments by judges, this would not affect their decisions. Question 3-C: To what extent has AJA included gender in the component? Conclusions As with gender violence, the test is not in the sentence (as implied by the contract “transversalizing” gender in land court sentences) but how the case is handled before the judgment is made. On the basis of available data we do not know if or how gender is an issue in land cases. Recommendations Monitoring for a gender focus should extend to variables like differences in time to resolution or additional benefits awarded by gender. CEJ’s Land Observatory includes these variables but seemingly could not develop data on many of them. Given its potential significance, efforts should be made by CEJ or others to collect the missing data. Question 3-D: Has the AJA improved the performance and sustainability of the LCJs (local justice committees)? Conclusions The LJCs’ creation and chances of continuity hinge on the proactivity of AJA advisors and members’ sense that participation will give access to more resources. While much lip service is paid to the benefits of coordination of local agencies, we heard few concrete examples of what was being attempted or accomplished. 23 Since many LJC members are local officials who are appointed by each new mayor, one acid test will be what happens after the 2015 local elections. We know from experience that a mayor opposed to the process can quash it; we don’t yet know what it takes to survive the transition. While all LJCs spoke of an interest in a gender focus, this produced few concrete actions among the LJCs interviewed. In Tumaco, despite the presence of a municipal gender subcommittee, references to products largely involved the development of plans, policies and workshops. Recommendations Now that the MJL (and possibly the Ministry of Interior or that of Post-Conflict) are taking more interest in local justice systems, Checchi’s former ability to largely go it alone, could become risky. Checchi, in cooperation with USAID, needs to establish a governmental counterpart (beyond the TCU, which despite its local presence has not been active here) if its actions are to have an impact. While AJA may work more closely with Justice over the short run, it should be attentive to the possibility that one of the other likely candidates will become the lead actor. The project also should explore in discussions with the MJL the reality of the “new” MJL model (as well as the reasons for its sudden presentation to the evaluation team). Studies done by FIP (including the one on obstacles to access and the 2013 basic approach to forming LJCs) should be more widely disseminated and discussed, including with Justice and Interior. If the LJCs are to produce enhanced access, less attention should go to drawing up projects for funding by others and more to enhancing cooperation and developing a better division of labor among local level actors. If there are good examples already in existence, they should be disseminated to other LJCs. Discussion among regional teams could illuminate the obstacles and possibly produce more solutions. ISSUES: Technical and/or Administrative Concerns Although most are not major, the team did observe some practices meriting further attention. First, the one major issue is the late and still insufficiently internalized recognition of the persisting politico-cultural impediments to constructing effective local justice systems (or for that matter local governments). As documented in the AJA-funded studies on obstacles to access, municipios in all seven regions still suffer the effects of decades if not centuries of exploitative, violent, and usually illegal political control, whether exercised by traditional strongmen, guerrillas, paramilitary bands or common criminals. In some cases, the “parallel” or illegal systems remain; in others, their practices carry over to “legitimate” authorities or simply shape 24 citizen relations with whoever is in charge. The evaluation team saw little of this but evidence provided by those who have done the research (including project studies) is indisputable. Old habits die hard, especially when backed by armed force, meaning that substantial change will come slowly and possibly not within the project lifetime. Project advisors and others working locally recognize the situation, but it is not evident that national counterparts and strategic partners are sufficiently attuned to the difficulties or that the project change theories have adjusted to the realities. Adjustment could mean lowering short-run expectations, but it also implies recognizing that apparent change may be superficial and thus easily reversed once the project ends. To ensure it does not, strategies may need revision. Second, there are several less dramatic contextual impediments to implementation, many of which converged in Tumaco. Regionalism can provoke resistance to consultants from Bogota, but in Tumaco it also limits the efficacy of advisors and strategic partners from Pasto (the departmental capital). This combined with Tumaco’s unpopularity as a career posting made it hard for the project and national agencies to retain staff. Interviewees reported frequent out￾rotation of judges and prosecutors, while the University of Nariño had replaced its one local representative (the lawyer managing the clinic) three times. Tumaco proved a difficult environment for the ICT innovations used in the Access Component. Their efficacy was diminished by user limitations (ordinary and computer illiteracy, concerns about anonymity) and numerous maintenance problems (energy supply, protection of equipment from theft or damage, etc.). Finally, Tumaco has an abundance of donors competing for the attention of local organizations and visibly diverting the latter from project aims. The ICT issues and the impermanence of judicial personnel were referenced elsewhere. Other issues may be unique to Tumaco, but they are worth exploring more broadly. Third, the advisors in Bajo Cauca have been inventive in adding “strategic allies” – ANSPE, Empresas Públicas de Medellin, the Fundación Minera, and Oleoductos (the only one not interviewed) -- but less attentive to possible repercussions. ANSPE’s role is not unique; it is a useful ally in other regions. However, alliances with the other three are troubling. Their parent companies, essentially extractive industries, face considerable criticism and legal attacks for their impacts on indigenous and other populations as well as the environment. The alliances are doubtless favored by the Departmental Government. However, should the legal issues expand, they could reflect unfavorably on a project aiming at augmenting access to justice for the poor. Fourth, the content of much gender (and some other) training offered by universities and NGOs is overly academic and abstract, as is much of the treatment of the differential focus (enfoque diferencial). This approach may work for academic jurists, but its impact on local officials, even if trained in law, is questionable. Something more concrete, contrasting specific examples of good and bad handling of cases would have more influence on real behavior. It bears mentioning that throughout our interviews we found virtually everyone explaining what they were doing in 25 abstract terms. When asked what they meant by “articulation,” “accompaniment,” “contextualization”, or “strengthening the social fabric,” implementers and participants had difficulty pointing to specific actions (methods) advancing these intermediate goals. In short, the campaigns have everyone talking the talk – our doubts are as to whether they also walk the walk. Fifth, given our own difficulties in tracking progress, we understand why the project chose the indicators it uses. Still, not a single one addresses changes in the behavior, attitudes and, most importantly, the experience of the citizens the project is to help or the impact on rights violations and conflict. In four years’ time, with the amount of funds involved, the project will not change the world (another point worth stressing). Nonetheless, if only as a means of pointing participants toward the future, it should develop indicators tapping the larger changes toward which it aims. The project should be able to increase the number of complaints about GBV (or other issues affecting women) as well as the percentage of these cases investigated, transferred to the judiciary, and resulting in sentences favoring the female victim/complainant. If not available nationally, these statistics should be accessible from the local authorities, prosecutors and judges within each municipio. Sixth, like most USAID (and other donor) efforts, AJA has produced a large quantity of “paper,” a significant portion of which is underutilized. We are not sure all of it is filed in Checchi’s Bogota office, but our interviewees constantly referred to studies no one had mentioned before. Some of this work is quite valuable; what is not may serve as object lessons on how not to pose a question or do research. More for USAID, than for Checchi, this material should be captured in a form accessible to interested outsiders. While we know USAID staff is overworked, perhaps the agency can find a way to review the highlights for its own benefit. Finally, our interviews appear to have spurred the MJL to announce new models for equity conciliation and local justice. This is obviously a prelude to its involvement over the next two years in producing the “Ten-year Justice System Plan.” The MJL has historically been a weak partner, but USAID cannot afford to ignore the ten-year plan, the ministry’s role in producing it, or the probable importance of local justice systems within it. Moreover, now that LJSs are being prioritized (by the Ministry of Interior as well), Checchi and USAID need to link their actions to a national sponsor and strategy. Ideally, Checchi’s work will have an enormous impact, but only if it recognizes that bottom-up is fine, but bottom-up implemented by a foreign donor may elicit rejection. Go-it-alone is no longer advisable. 26 Documents Reviewed • Decrees and Agreements Consejo de Estado (2014) Sentencia 2005-01640 Consejo de Estado 9/10/2014. Tema abordado: Responsabilidad del Estado por violencia de Género. Bogotá. Comisión Nacional de Género de la Rama Judicial. República de Colombia. (2013) Violencia y Discriminación Contra las Mujeres. Bogotá. Comisión Nacional de Género de la Rama Judicial. República de Colombia. (2013) Análisis de las Evaluaciones de los Conversatorios Regionales de Equidad de Género en la Rama Judicial durante el Año 2013. Bogotá. Congreso de Colombia (2002). LEY 743 DE 2002. Por la cual se desarrolla el Artículo 38 de la Constitución Política de Colombia en lo referente a los Organismos de Acción Comunal. Bogotá. Congreso de Colombia (2014). Proyecto de Ley de 2015 Cámara. Por la cual se expide el Plan Nacional de Desarrollo 2014 -2018 “Todos por un Nuevo País”. Bogotá. Consejo Superior de la Judicatura. (2014). Proyecto Interinstitucional. 1 Documento Versión 3 Actualizado. Bogotá. Consejo Superior de la Judicatura. (2011). Informe al Congreso de la República 2009-2010. Bogotá D.C.: Rama Judicial del Poder Público. Bogotá. Consejo Superior De La Judicatura. (2012). Plan de Acción para la Implementación del Código General del Proceso. Colombia: CSJ. Departamento Nacional de Planeación. (2008). Garantizar una Justicia Eficiente. DNP, Dirección de Justicia y Seguridad. Colombia: Presidencia de la República, DNP. Escuela Judicial Rodrigo Lara Bonilla. (2012) Programa de Desconcentración de Servicios de Justicia. Retrieved 09 19, 2013, From Escuela Judicial Rodrigo Lara Bonilla: Http://Www.Ejrlb.Com/Contenido2/Programas/Desconcentracion. Bogotá. • USAID Documents Checchi and Company Consulting. (2015) Access To Justice Activity Quarterly Technical Report October 1 – December 31, 2014. Bogotá. Checchi and Company Consulting. (2014) Access To Justice Activity Fiscal Year 2014 Performance Management Plan Report. Bogotá. 27 Checchi and Company Consulting. (2014) Access To Justice Activity Second Annual Report October 1, 2013 — September 30, 2014. Bogotá. Checchi and Company Consulting. (2014) Access To Justice Activity Quarterly Technical Report January 1 - March 31, 2014. Bogotá. Checchi and Company Consulting. (2013) Access To Justice Activity Performance Management Plan. Bogotá. Management Systems Internacional-MSI (2015) USAID Colombia Eval Design and Workplan for the Mid-Term Evaluation of the Access to Justice Activity. Bogotá Management Systems Internacional-MSI (2014) USAID Colombia Eval Workplan and Methodology for Mid-Term Evaluation of the Access to Justice Activity. Bogotá Management Sciences for Development-MSD (2012) Access to Justice-USAID. Program Assessment and Impact Evaluation of Colombia Justice House Program. • Contracted Studies Arbeláez Tobón, L. y Flórez Mejía, M. (2014) Proyecto de Acceso a la Justicia-USAID. Entregable 3. Guía de Transversalización del Enfoque de Género en el Sistema de Gestión de la Calidad. Bogotá. Alina Tech Ltda (2014) Proyecto de Acceso a la Justicia-USAID. Despachos Civiles del Circuito Especializado en Restitución de Tierras. Plan de Seguridad de la Información del Proceso Restitución de Tierras. Bogotá. Corporación Excelencia en la Justicia (2014) Proyecto de Acceso a la Justicia-USAID. Observatorio al Proceso para la Restitución de Tierras (Oprt) Subcontrato No. Aja-Subk-2013- 005 Entregable 10. Documento de Línea de base del Observatorio de Restitución de Tierras con corte a Diciembre de 2012. Bogotá. Corporación Excelencia en la Justicia (2014) Proyecto de Acceso a la Justicia-USAID. Necesidades Jurídicas Insatisfechas. Subcontrato No. Aja-Subk-2013-005 Entregable 20. Metodología Ajustada de Necesidades Jurídicas Insatisfechas. Bogotá. Corporación Excelencia en la Justicia (2014) Proyecto de Acceso a la Justicia-USAID. Observatorio al Proceso para la Restitución de Tierras (OPRT) Subcontrato No. Aja-Subk-2013- 005 Entregable 11b: Documento de Monitoreo con corte al 31 de Diciembre de 2013 del Observatorio de Restitución de Tierras. Bogotá. Corporación Excelencia en la Justicia (2013) Proyecto de Acceso a la Justicia-USAID. Metodología de Seguimiento a la Jurisprudencia de Género. Subcontrato AJA-Subk-2013-005. Bogotá. 28 Corporación Excelencia en la Justicia (2013) Proyecto de Acceso a la Justicia-USAID. Observatorio al Proceso para la Restitución de Tierras. Documento Explicativo del Diseño con cuatro Territorios Objeto de Observación - Subcontrato AJA-subk-2013-005. Bogotá. Escuela de Justicia Comunitaria. (2014) Proyecto de Acceso a la Justicia-USAID. Producto No. 2 Documento de Discusión denominado Plan Piloto de Incentivos y de Información de la Conciliación en Equidad. Facultad de Derecho, Ciencias Políticas y Sociales Universidad Nacional de Colombia. Bogotá. Fundación Ideas Para La Paz. (2014).Proyecto de Acceso a la Justicia-USAID. Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en Municipios de Consolidación de Nudo de Paramillo Briceño-Anorí – Ituango y Valdivia. Bogotá. Fundación Ideas Para La Paz. (2014). Proyecto de Acceso a la Justicia –USAID. Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en los Municipios de Consolidación de la Región de la Macarena: San Juan de Arama, La Macarena, Vistahermosa, La Uribe, Mesetas y Puerto Rico. Bogotá. Fundación Ideas Para La Paz. (2014). Proyecto de Acceso a la Justicia-USAID. Sur de Córdoba. Diagnostico de Barreras de Acceso a la Justicia en los Municipios del Sur de Córdoba. Bogotá. Fundación Ideas Para La Paz. (2014). Proyecto de Acceso a la Justicia-USAID. Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en los Municipios de Consolidación de la Región de los Montes de María. Bogotá. Fundación Ideas Para La Paz. (2014). Proyecto de Acceso a la Justicia-USAID: Producto 20: Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en el Municipio de Consolidación de Tumaco. Bogotá. Fundación Ideas Para La Paz (2013) Pautas y Lineamientos para la Implementación del Diseño del Sistema Local de Coordinación de Justicia. Proyecto de Acceso a la Justicia – USAID. Bogotá. Fundación Ideas Para La Paz. (2013).Proyecto de Acceso a la Justicia-USAID. Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en Municipios de Consolidación del Bajo Cauca Antioqueño: Caucasia, Cáceres, Nechí, Tarazá, Zaragoza Y El Bagre. Bogotá. Fundación Ideas Para La Paz. (2013).Proyecto de Acceso a la Justicia-USAID. Diagnóstico de Barreras y Restricciones de Acceso a la Justicia en Municipios de Consolidación del Sur Del Tolima: Ataco, Chaparral, Planadas y Rioblanco. Bogotá. Universidad de Ibagué (2014) Proyecto de Acceso a la Justicia –USAID. Sur De Tolima Entregable 41. Documento de Diagnóstico sobre el Cumplimiento y Ejecución Material de las Sentencias de Restitución de Tierras, en los 4 Municipios del Sur del Tolima. Bogotá. 29 Universidad de Nariño (2015) Informe Semestral de la Implementación de la Clínica Jurídica en Tumaco. Proyecto de Acceso a la Justicia – USAID, Tumaco Producto 39. Bogotá. Universidad de Nariño (2014) Proyecto de Acceso a la Justicia – USAID, Tumaco Producto 29. Módulos para la formación en Género dentro del Diplomado para Operadores de Justicia. Pasto Universidad de Nariño (2014) Proyecto de Acceso a la Justicia – USAID, Tumaco Producto 31. Identificación de Acciones y Estrategias que Contribuyan al Acceso a la Justicia de Generó y Propuestas Alternativas que aseguren la Protección de los Derechos de la Mujer. Pasto Universidad Santo Tomas (2014) Proyecto de Acceso a la Justicia-USAID, Diagnóstico sobre el Cumplimiento y Ejecución Material de las Sentencias de Restitución de Tierras en el Departamento del Meta-Villavicencio. • GoC Publications, Academic Studies and those Contracted by Other Donors Ardila Amaya, É. (2006). ¿A Dónde Va La Justicia En Equidad En Colombia? Medellín: Corporación Región. Comisión Nacional de Género de la Rama Judicial. República de Colombia. (2013) Construcción de la Igualdad en la Justicia. Documentación del Cambio. Con el Apoyo del Fondo de Población de las Naciones Unidas-UNFPA y El Consejo Superior de la Judicatura. Bogotá. Corporación Razón Pública (2012). Sistema Local de Coordinación de Justicia de Zonas de Consolidación –SLCJ. Ministerio de Justicia y del Derecho. Bogotá. Corporación Excelencia en la Justicia (2014) Diseño de las Bases de un Modelo de Justicia para las Zonas Rurales de Colombia. Bogotá. Coorporación Nuevo Arco Iris (2015) El Derecho de Acceso a la Justicia otra Herramienta Para Construir El Postconflicto. Recuperadada de: Http://Www.Arcoiris.Com.Co/2015/03/El￾Derecho-De-Acceso-A-La Justicia-Otra-Herramienta-Para-Construir-El-Post-Conflicto/. Bogotá. Dejusticia (2013) Encuesta Nacional de Necesidades Jurídicas. Bogotá Engle Merry, Sally (1993) The Possibility of Popular Justice: A Case Study of American Community Mediation. Ann Arbor, MI: Univ. of Michigan Press. Fundación Ideas Para La Paz. (2011). Balance de la Política Nacional de Consolidación Territorial. Bogotá Fundación Forjando Futuros. (2014) Publicación 500 Sentencias. Así fallan los Jueces de Tierras. ¿Cumplirá el Gobierno sus Órdenes?. Bogotá. 30 García Villegas, M. (2008). Jueces Sin Estado. Bogotá: Siglo Del Hombre Editores. Mak, E. (2008). Balancing Territoriality And Functionality; Specialization As A Tool For Reforming Jurisdiction In The Nethernalds, France And Germany. International Journal For Court Administration. Ministerio de Interior y Justicia (2008). Manual para la formación de conciliadores y conciliadoras en equidad en Colombia. Ministerio de Interior y Justicia (2009). La Justicia entra por Casa. Bogotá Muñoz Labajos, N. (2009). Justicia Itinerante en Zonas con Limitada Cobertura de Operadores de Justicia en el Territorio Nacional . Bogotá Nilsson, M. (2013). El Concepto de Reconstrucción en el Marco de la Política Nacional de Consolidación y Reconstrucción Territorial en Colombia. Växjö, Suecia: Universidad Linnaeus. Red Nacional de Mujeres (2009) Campaña Sin mi puño con mi letra. Recuperado de: http://www.rednacionaldemujeres.org/index.php/galeria-fotografica Semana. (2010, 10 22). "El Gran Problema de la Justicia Colombiana se llama Presupuesto". Retrieved 07 22, 2013, From Semana.Com: Http://Www.Semana.Com/Nacion/Articulo/El-Gran￾Problema-Justicia-Colombiana-Llama-Presupuesto/123542-3 Unidad de Restitución de Tierras (2014) Informe Trimestral de Gestión – Plan de Acción a 30 de Septiembre de 2014. Bogotá. Universidad Externado de Colombia y Alcaldía Mayor de Bogotá (2002) Capacitación a Funcionarios y Formación de Ciudadanos de Bogotá como Mediadores Comunitarios para el Distrito Capital. Proceso de Mediación y Habilidades del Mediador. Bogotá. UNHCR & UNFPA (2012) Implementación del Sistema de Gestión de Datos sobre Violencias Basadas en el Género-GBVIMS en Colombia. Sistematización de La Experiencia Colombiana. Uprimny, R. 2001. “¿Justicia comunitaria en contextos violentos y antidemocráticos?” InB. Sousa Santos, El caleidoscopio de las justicias en Colombia; 309-324. Bogotá: Siglo del Hombre Editores. 31 Annex 1: Getting to Answers (G2A) Evaluation questions Data collection methods Selection Data analysis methods Source Method COMPONENT 1 (Equity conciliators) A. To what degree has the IP improved the NPEC in the dimensions established in the agreement? (selection, training, retention, data management, coordination with local justice bodies, incentives, local financing, service provision, outreach) B. To what extent has the AJA augmented the NPEC’s sustainability? C. How successfully has gender been incorporated in the NPEC? Project documents IP staff MJL staff (national level) IP partners Justice House staff and other members of LJCs Equity conciliators and their local/national organizations Red Unidos household surveys Agencies’ MIS and records (e.g. conciliation agreements registered with courts or personeros) Document review Interviews (KII) Interviews (KII) Observation Interviews (KIIs) KIIs Observation Focus groups Interviews (KII) Statistical analysis Review of contents and uses (Observation and interviews) Relevant portions of Agreement, PMP and IP progress reports; examples of CSO media campaigns from selected regions IP C1 and field staff http://www.msiworldwide.com/w pcontent/uploads/FeeInvPROJEC Tinstructions. MJL Access staff Partners working with ECs Staff at Justice Houses and LJC members in selected regions ECs in selected sites, selected randomly from the database;Asociación Colombiana de Conciliadores/mesa de trabajo (Bogotá) Extraction of relevant statistics from MJL and regions Comparison with plan Content analysis Event reconstruction Trend/pattern analysis 32 Evaluation questions Data collection methods Selection Data analysis methods Source Method COMPONENT 2 (Gender) A. To what extent has the AJA increased first instance judges’ gender sensitivity and knowledge, and its impact on their handling of cases? B. Has the AJA successfully and neutrally engaged GOC institutions (nationally and regionally) in a coordinated response to improving handling of GBV cases? C. To what degree have AJA actions increased citizens’ awareness of the legal framework for gender issues (including rights)? Which have been the most and least effective methods implemented? D. To what extent has the AJA engaged civil society partners in making an effective contribution to advancing gender aims? Project documents IP Staff Partners’ staff Judges SCJ members NGC members SGC members SAU staff (only if want ideas about future developments) CSO staff Campaign documents and media products Red Unidos survey Observatories’ databases Siglo XXI and other databases on court outputs e.g. from the AGO, Medicina Legal) Document review Interviews (KII) Observation Document and content review Statistical review Statistical review Statistical review Agreement, PMP, progress reports and other relevant material At national level from engaged organizations/individuals and among staff working on gender At local level from relevant organizations/groups in selected regions Examples from selected regions From relevant sections of survey Depending on size, universe or at national level or in selected regions Chosen from selected regions unless central statistics include general relevant cases Content, comparative analysis Self-assessment Content analysis, event reconstruction Event reconstruction, some content analysis of interviews, frequencies of mentions Content analysis Trends/pattern Comparison of scores over time Comparison of statistics, over time and across regions COMPONENT 3 (LRCs and LJCs) A. Is the AJA’s approach to improving LRC case management effective and sustainable? B. What are judges’ and magistrates’ Project documents Judges and courtroom Document review KIIs Those related to C3 In regions selected Event reconstruction, comparison with contract and workplan 33 Evaluation questions Data collection methods Selection Data analysis methods Source Method perceptions of AJA, and has change management been successful so far? C. To what extent has the AJA included gender in this component? D. Has the AJA improved the performance and sustainability of the LCJs (local justice committees)? staff LJC members LR Unit SCJ CMS26 Focus groups Observation KIIs KIIs Statistical analysis In selected regions In selected regions Magistrate and committees working on themes SCJ data base and that from selected regions Trend/ pattern analysis Event reconstruction, comparative pattern/trend Event reconstruction Comparisons across time and among regions Self-analysis CROSS-CUTTING (4) To what extent is the AJA successful in capturing and reporting on impact? Project documents Document review Quarterly, annual reports, other material on programs Content analysis against EVAL findings 26 CMS is Case tracking system 34 Annex 2: Scope of Work (SOW) Note: the initial SOW evaluation questions were modified in the approved work plan. USAID COLOMBIA EVAL PROGRAM SCOPE OF WORK Mid-term Management Performance Evaluation of the Access to Justice Activity (AJA) I. BACKGROUND INFORMATION Table 1. Access to Justice Activity Identification Data Program title The Access to Justice Activity (AJA) Award number AID-514-C-13-00001 Award dates February 1, 2013 to September 30, 2016 Funding US$ 23,094,666 Implementer Checchi and Company Consulting, Inc. COR Nate Tenny The Government of the United States of America through its Agency for International Development (USAID) has contracted with Checchi and Company Consulting, Inc., to carry out the Access to Justice Activity (AJA) in Colombia. The objective of AJA is to provide assistance to strengthen access to justice in Colombia. Access to justice allows citizens to enforce their rights against infringement and access the state’s justice system. The basic premise of this project is that greater access to justice is a cornerstone for the rule of law which, in turn, is a key element for the establishment of democracy.27 The overall objective of the AJA is to enhance and broaden access to justice, particularly for women and disadvantaged groups. Three components were foreseen for the activity in the initial contract: 1. Strengthening, improving and sustaining the Justice Houses implemented by the Ministry of Justice and Law, and supporting the creation of eleven new Houses in consolidation areas. 2. Strengthening and supporting organizations and institutions to improve women’s access to justice at the local level. This includes strengthening the National Gender Commission and its regional committees, and ensuring lower court compliance on gender￾differentiated treatment.28 27 See “Conducting a DG Assessment: A Framework for Strategy Development, USAID,” November 2000; citation taken from USAID’s contract for the AJA with Checchi, page 7. 28Significant attention in the evaluation will be paid to this component in particular. 35 3. Supporting the implementation of reforms to increase productivity particularly in consolidation regions, such as the land restitution first and second instance courts. A contract modification is in process at this writing, to adjust the terms of the IP’s contract, particularly around interventions in law schools. As this mod is shared with the evaluation team, its impact on the evaluation design and strategy will be included in this document. At the same time, the activity is undergoing key personnel changes, including the arrival of a new Chief of Party and a new technical lead for the second component. These characteristics, and the opportunity to evaluate at mid-term, make the evaluation propitious. The new leadership can take the initiative forward with the lessons learned and recommendations provided by the evaluation team. By the end of the activity, expected outcomes for AJA were contracted to include the following:29 • The Government of Colombia (GOC) will increase access to justice in the consolidation regions, especially for traditionally marginalized populations, including, women, AfroColombians, LGBTI and indigenous groups, through the improvement and expansion of the provision of alternative justice services; • GOC justice sector actors in targeted regions will be in better compliance with international and national norms for treatment of gender-based issues in the criminal justice process; • Populations in targeted regions will have a greater understanding of gender issues and the criminal justice process, including how to protect and enforce their rights within the process; • New land restitution courts will adjudicate land restitution cases more effectively and efficiently than traditional courts; and With the contract modification, this list may change. II. EVALUATION RATIONALE 1. Evaluation purpose USAID/Colombia requests an external mid-term management performance evaluation of the progress of AJA efforts after two years of implementation. The evaluation is designed to capture progress, identify management strengths and challenges, involve the IP in the evaluation process, and make course corrections in management and technical content for the remaining two years of the contract. 2. Audiences and intended uses The key intended audiences of the evaluation report will be: • USAID/Colombia Mission Front Office and Democracy and Governance (D&G) Office • Implementing partners, at both prime and sub levels 29 From page 4 of Checchi’s Performance Monitoring Plan. 36 3. Evaluation questions The themes selected for the evaluation were identified in consultation with USAID’s D&G and Program Office teams, including the activity’s COR. The questions derive from specific management and technical challenges experienced in the first two program years.30 1. Component 1: To what extent is the system of equity conciliators sustainable, and to what extent has the IP contributed to this sustainability? 2. Component 2: To what extent has the component catalyzed an array of interagency state services for women victims of sexual violence? a. Has the AJA made acceptable progress toward coordinating state response and successful case management (at investigation, indictment and prosecution stages)? b. Has the AJA successfully and neutrally engaged GoC institutions (nationally and regionally) for the coordinated state response to sexual violence cases? What evidence is there of results from this engagement? c. What findings can be cited about AJA’s engagement of civil society partners and their effectiveness in carrying out their roles? 3. Component 3: Is the IP’s approach to improving case management effective and sustainable in its work with the Land Restitution Courts? a. What have been the judges’ and magistrates’ perceptions of the AJA, and has change management in the Land Restitution Courts been successful so far? b. What, if any, changes in case management are seen in the functioning of the Land Restitution Courts? c. What is the sustainability of local justice committees implemented by AJA? What has been their effectiveness to date? 4. Cross-cutting question: To what extent is AJA successful in capturing and reporting on impact? The lettered sub-questions in the above list should be understood as lower-level probes that will be part of the research but that the numbered questions will be the priority, to avoid overstretching the evaluation design. As part of developing the evaluation work plan, the evaluation team will create a Getting to Answers matrix including all of the final evaluation questions and the methods by which the evaluation will attempt to answer them. This process will include USAID and the IP team, with the goal of a reasonable final list of questions that can credibly be answered given the time allotted for the evaluation, the budget for field work and analysis, and the availability and quality of data. III. EVALUATION DESIGN 1. Design This mid-term will evaluate AJA’s progress on significant and sustainable changes in access to justice for target populations and in targeted institutions. The design will include both quantitative and qualitative elements, and an analysis plan to examine and weigh the various data streams. Qualitative methods will comprise a large portion of the evaluation design and the 30 The list will be circulated with the IP team for its comments and so that it may propose questions for the list. 37 findings will be primarily based on these. Where possible, quantitative methods will also be used. An important feature of this mid-term evaluation design is a focus on participatory processes, in order to gain greater buy-in and collaboration from the key parties – USAID and the IP. To this end the evaluation team will include deep input and participation from these two actors with the goal of effecting greater dialogue about the management model, ongoing challenges, and the responses to these challenges during the remaining two years of the activity. As active participants, both USAID and the IP will be encouraged to discuss critically the model and its challenges. Qualitative methods Document review. There are a wide range of documents of import to the evaluation. These include the original contract, quarterly and annual reports and work plans, the Performance Monitoring Plan, and thematic or component reports. When available, the team will also review the contract modification as well as documentation from prior access to justice programming. In order to look at the cross-cutting evaluation question on reporting, this document review will include each report with a rubric based on the activity’s PMP and a set of higher-order outcomes from the IR and sub-IR levels of the results framework. Key informant interviews. Key informant interviews will comprise a major data collection category in the evaluation. GoC Justice and interagency actors will be essential interlocutors to understand the extent of the implementation to date, its institutional effects, any unintended consequences, and potential paths forward, including any course corrections. To look closely at Component 2 processes and outcomes, the evaluation will interview the National Gender Commission, GoC counterparts, and NGO and civil society partners. The IP’s team will also be interviewed for its understanding and perspectives of process and progress to date. These interviews will be focused on the component and evaluation questions most pertinent to each interviewee. Focus groups. The focus group method will be used to examine perceptions among groups of beneficiaries, for whom the group modality will allow for interactive, holistic responses to the questions. Groups will be used with three key types of respondents: equity conciliators, local justice committees, and the judges and magistrates in the Land Restitution courts. Where a sufficient number of individuals from any of these groups is unavailable for a focus group, semi￾structured interviews will be substituted for the group. Quantitative methods Data review. The improvement expected from the intervention – as embodied in the PMP indicators – has important quantitative elements. Rather than repeat the data collection on these elements, the evaluation team will review data quality and effect sizes for changes to date for the following AJA PMP indicators: 1. Percentage of families that identify proper dispute-resolution channels. These data are accessed through a survey conducted with the Red Unidos program. 2. Compliance with High Court standards and jurisprudence regarding the perspective of gender by lower courts. Percentage of sentences (in gender-related cases) meeting 38 minimum standards of use of High Court jurisprudence and gender perspective principles. These data are accessed annually through gender observatories and the AJA itself, using a common instrument. 3. Number of AJA-supported Land Restitution Courts and Tribunals that improve their Balanced Scorecard on administrative efficiency. An index built by the AJA to measure movement on court administration. 4. Case disposition efficiency in a random sample of cases in AJA-supported criminal courts in CSDI zones. These data are collected through the automated case tracking system supported by the AJA. In addition, the evaluation team site visits will review case data in the Justice Houses to get a detailed picture of the use of project inputs to achieve outcomes. As the Justice House data management system is more fully understood, the team will create a brief rubric for the types of data to answer the evaluation questions. Sample sizes and information collecting tools The following table describes the units of analysis, tools and sample sizes in which information will be collected, relative to the evaluation questions. The abbreviations used are IP (Implementing partner), GoC (Government of Colombia), JH (Justice House), KII (Key informant interview), FG (focus group), N (national level) and R (regional/local level). The evaluation team proposes to visit six sites, with overlapping interventions, as described in the last column of the table. Table 2. Sample sizes and information collecting tools Evaluation question Units/ respondent types Sample Size Tools Suggested sites Component 1: Justice Houses and Equity Conciliators Equity conciliators 2 FG Three sites with JHs IP staff (N, R) 3 KII GoC (R) 2 KII JH record review 3 Rubric Red Unidos family data NA Data rvw Component 2: Catalyst for interagency state services for victims of sexual violence. IP staff (N, R) 4 KII Three sites with C2 activities GoC interagency actors 4 KII Civil society partners and potential partners 3 KII Review of case data NA Data rvw Component 3: Improving Land Restitution Courts IP staff (N, R) 3 KII Three sites with Land Restitution Courts Judges and magistrates 4 KII Case data review 4 Rubric Local justice committees 3 FG Scorecard review NA Data rvw Cross-cutting: Reporting on outcomes Desk review and triangulation with field sources 1 Rubric and PMP Desk review This evaluation design provides multiple perspectives on each of the evaluation questions to capture varying perspectives. Analysis will consider the relationships between data designed to answer a given question, and the relative weight of varying sources. For example, a source that is 39 self-reported would be considered of somewhat lesser weight or strength than a source that reports on a phenomenon from a more disinterested perspective. Workshop A key design strategy for this evaluation is a one- to two-day workshop, as necessary, at the culmination of data collection. This workshop will include the full evaluation team, including the USAID and IP representatives, who will have participated in each of the data collection methods in at least some of the sites. The workshop will bring findings from all teams to the table for an in-depth discussion on the conclusions that can be drawn from these, and the recommendations that emerge. The workshop will be led by an expert facilitator who is particularly adept at bringing out areas of difference or disagreement (as well as consensus themes.) These will be carefully documented. In this way the IP and USAID will have the opportunity to examine the data together with the goal of developing recommendations that both actors can support. The findings, conclusions and recommendations that emerge from the workshop will form the backbone of the draft report. IV. EVALUATION PRODUCTS A. Deliverables The Start Date, generally marked with a team planning meeting (TPM), will be agreed upon once a final SOW is approved by USAID and once all contracting is in place. Team member profiles and/or CVs will be provided to USAID prior to the Start Date. Table 5. Evaluation deliverables DELIVERABL E DESCRIPTION DATE Design and work plan Detailed design and work plan that indicates evaluation activities. The work plan will be submitted to the EVAL COR at USAID/Colombia for review. 15 days after TPM Data collection tools Draft instruments will be prepared and submitted to USAID for review prior to fieldwork. 20 days after TPM Field data collection Weekly electronic progress reports will be submitted to USAID. Throughout field work and analysis First draft report Draft of the report submitted to the USAID COR, who will provide formal comments – one unified document from whatever sources are required within USAID – to facilitate preparation of the debriefing. 60 work days after TPM (national holidays are excluded from this count) Debriefing with USAID EVAL will present the evaluation findings to USAID through a presentation and discussion of findings, conclusions and recommendations. The team will consider USAID comments and revise the draft report, as appropriate. Ten work days after submission of draft Final report Final report incorporating issues identified by USAID during the debriefing as well as written comments. As part of the final submission, quantitative data will be delivered in Excel, and qualitative data will be shared in secure hard copies (encoded disks) to maintain respondents’ confidentiality. Ten work days after receipt of final comments from USAID B. Reporting Guidelines 40 The format for the Evaluation Report is as follows: • Executive Summary—most salient findings and recommendations, concisely stated (2-3 pages) • Introduction—purpose, audience, and synopsis of task (1 page) • Background—brief activity overview, and purpose of the evaluation (2 pages) • Design—data collection methods, including limitations and gaps (2 pages) • Findings/Conclusions/Recommendations— (15–17 pages) • Issues—list of key technical and/or administrative concerns, if any (1–2 pages) • References---including bibliography and other references as appropriate (as needed) • Annexes— detailed discussion of methods, schedules, interview lists and tables will be succinct, pertinent and readable. The evaluation SOW and instruments will be in the annexes, per the 2011 Evaluation Policy. The report will not exceed 30 pages, excluding table of contents, acronyms list, executive summary, issues, references and annexes. This format is consistent with the 2011 USAID Evaluation Policy. V. EVALUATION MANAGEMENT A. Team composition The evaluation will be carried out by a high-level team of social science researchers, with sector￾specific experience in the evaluation themes. They will work together in Bogotá at the start of the project to ensure inter-rater reliability and shared criteria, to interview USAID and the implementing partner, and to review the relevant documentation. A Project Director (senior) will direct the evaluation, including fieldwork, with support from Bogotá-based field researchers. Field teams will include expertise from among the key themes and sectors included in the evaluation. Logistics will be covered by the supervisory stratum of the field team and back-stopped from Bogotá. A native English-speaking editor will support the technical writing. The following table shows the detail for the team member roles expected: POSITION QUALIFICATION Project Director (Senior) • Very strong evaluation and process skills, in data collection and in the analysis of data in a workshop format. The director must have a completely neutral and disinterested stance toward the actors and stakeholders in this evaluation. • Education and experience in relevant development sector (rule of law and access to justice programming, particularly in conflict environments) • Minimum M.A. + 12 years’ experience; significant evaluation experience. • Understanding of the legal context, particularly as it pertains to vulnerable populations (victims, women, etc.) • Experience in qualitative analyses and in valuing quantitative and qualitative data sources Content Specialists (2-4 depending on coverage) • Team members with specialization in relevant content knowledge and experience, specifically in the Colombian context. • Areas of specialization to include: legal support for victims; women’s access to justice; the Colombian legal response to gender-based violence, informal and alternative dispute resolution; land restitution law and case management. • Minimum M.A. +10 years’ experience in the relevant content area 41 • Thorough understanding of Colombian law and practice for the relevant content area, particularly with vulnerable populations • Experience in field research in Colombia Mid-level specialist • Education and experience in relevant sectors (rule of law, women’s and victims’ access to justice, land restitution, alternative dispute resolution) • Minimum B.A. + 12 years experience, or M.A. +5 years • Experience in evaluations and qualitative field data collection for international development assistance in Colombia • Fieldwork experience for research or evaluation Qualitative data entry and analysis • Qualitative data analysis to meet the methodological needs of the evaluation design – combining in-depth interviews, focus groups and other data to form a coherent qualitative impact analysis. • 5+ years’ experience with data editing, cleaning, entry, coding and initial analysis Logistical support • To arrange site visits efficiently and with sufficient attendance for interviews, focus groups, etc. B. Logistics EVAL will hold a Team Planning Meeting at the outset, to guide the multi-faceted tasks of the first phase of the evaluation, to build team consistency, and to plan the evaluation. Initial interviews with USAID and implementing partner key personnel will be held. The evaluation team will work with USAID to arrange these and any other initial meetings in Bogotá, as well as the hand-over of implementer databases and reports. The first steps will include document and data review as described above. EVAL will direct the evaluation’s logistics in Bogotá and the field sites. The EVAL Team will be responsible for arranging all the transportation for fieldwork, and will monitor security issues and threats closely before and during field work. Each field team will each be led by an experienced field researcher. Communications will be maintained daily with each of the teams in the field, to ensure that challenges or difficulties are quickly resolved. C. Analysis Management of analyses will be based on the detailed analysis plan set forth in the final evaluation design, and led by the evaluation Team Leader. 42 Annex 3: List of Interviewees NATIONAL AUTHORITIES DATE NAME INSTITUTION 16-feb Anneth Person Coordinadora Componente 1 Checchi Consulting Cesar Reyes Coordinadora Componente 3 Checchi Consulting Rosario Montoya Asistente Componente 3 Checchi Consulting 17-feb Gloria María Borrero Directora Corporación Exelencia en la Justica 19-feb Guerthy Acevedo Coordinadora Componente 2 Checchi Consulting 20-feb Néstor Raúl Correa Magistrado Sala Administrativa Consejo Superior de la Judicatura 20-feb Mónica Cruz Subdirectora Escuela de Justicia Comunitaria 25-feb Olga Cecilia Pinilla Coordinadora equipos regionales Checchi Consulting Luis Enrique Sánchez Asociación Colombiana de Conciliadores en Equidad 26-feb Dora Elena Gallego Juez de Restitución de Tierras. Ana María Ramos Viceministra de Promoción a la Justicia Ministerio de Justicia Adela Parra Asesora Ministerio de Justicia Sonia Cardona Asesora Ministerio de Justicia 28-feb Mario Córdoba Director MASC Ministerio de Justicia 2-mar Edgar Ardila Académico. Ex director MASC Ministerio de Justicia 24-mar Mario Córdoba Director MASC Ministerio de Justicia 25-mar Carmenza Vélez FIP 26-mar Carolina Morales SISMA 26-mar Camila Jaramillo Strauss Programa de Tierras y Desarrollo Rural USAID 26-mar Christian Kolar Programa de Tierras y Desarrollo Rural USAID REGION MONTES DE MARIA CARTAGENA - CARMEN DE BOLIVAR DATE NAME INSTITUTION 3-mar Martha Castañeda Tribunal Superior de Cartagena- Sala Civil Especializada en Restitución de Tierras Ada Lallemand Abramuk Tribunal Superior de Cartagena- Sala Civil Especializada en Restitución de Tierras Laura Cantillo Tribunal Superior de Cartagena- Sala Civil Especializada en Restitución de Tierras 43 Betsy Romero Fuenmayor Corporación Visión Región Carolina Pérez Payares Corporación Visión Región Susana González Arroyo Secretaria Tribunal Superior de Tribunal Superior de Cartagena- Sala Civil Especializada en Restitución de Tierras 4-mar Mabeth Gamarra Vargas Checchi Consulting Ciria del Carmen Marrugo García Checchi Consulting Cira Velásquez Herazo Gobernación de Bolívar Liliana Gómez Benítez CECAR - SE Lérida Romero Fuenmayor CECAR - SE Keyla Andrea Arrieta CECAR - SE José Francisco Restrepo CECAR - SE Viviana Primera Vergara Gobernación de Bolívar 5-mar Sandra Rodelo CECAR Cesar Torres Guzmán CECAR Mary Luz Barrios Juzgado de Familia de Carmen de Bolívar Lina María Hoyos Juzgado Promiscuo del Circuito de Carmen de Bolívar Irina Conea Fiscalía Yair Cumplido Rodríguez CECAR Nidia Rosa Macea Organizaciones de mujeres Oscar Sarmiento Juez de Restitución de Tierras de Carmen de Bolívar Marquesa González Hernández Organización de mujeres Carmen de Bolívar Martina Cuesta Restitución de Tierras de Carmen de Bolívar Liliana Serrano Comisaría de Familia Ever Méndez Comisaría de Familia Jorge Varela Personería municipal de Carmen de Bolívar Daiuduam Herrera Yepes Inspección de policía Gladys Eugenia Galofre Méndez Juzgado Civil del Circuito de Cartagena Maribel Romero Unidad de Consolidación Territorial Felipe García González Personería 6-mar Ivan Latorre Sala Administrativa-Consejo Seccional de la Judicatura de Bolívar Gaudencio López Astaiza Sala Administrativa-Consejo Seccional de la Judicatura de Bolívar Hernando Sierra Porto Director Seccional de Administración Judicial Johannys Hernández Castillo Unidad de Restitución de Tierras – Bolívar Elisabeth Estrada Sala Especializada de Restitución de Tierras de Cartagena 44 REGION BAJO CAUCA - MUNICIPIOS DE CAUCASIA - MEDELLIN DATE NAME INSTITUTION 9-mar Gloria Stella López Magistrada Sala Administrativa Comisión Seccional de Género de Antioquia 9-mar Vicente Landinez Magistrado Tribunal Superior Sala de Restitución de Tierras. Paola Alejandra Martínez Gobernación de Antioquia Directora de la Unidad de Justicia Yuber Antonio Ordoñez Gobernación de Antioquia. Unidad de Justicia Mónica Marín Isaza Gobernación de Antioquia. Unidad de Justicia 10-mar Juan Carlos Vásquez Rivera Universidad de Medellín María Victoria Rendón Universidad de Medellín Juan Carlos Hoyos Universidad de Medellín Zellez Álvarez Orrego Universidad de Medellín Paula Andrea Polo Departamento para la Prosperidad Social Carlos Jaime Taborda Director Seccional de Fiscalías de Antioquia Cesar Quiroz Subdirector Seccional de Fiscalías de Antioquia América Restrepo Fiscal-Coordinadora componente de Victimas Francisco Franco Departamento para la Prosperidad Social Antonio José Salazar Unidad Administrativa de Consolidación Territorial-UACT Yanneth Arango Inspección de Policía Briceño Maryury Castro Ortiz Cacica Mayor Cáceres 11-mar Carlos Mario Mesa Comisaria de familia de Tarazá Wilmer Molina Universidad de Antioquia-brigadas y consultorios psicosociales. Julieta Gil Gutiérrez Casa de Justicia de Cáceres Jaime Higuita Guzmán Fiscalía Seccional de Cáceres Oscar Hernández Terán Comité de convivencia Ciudadana del Bagre Luis Alberto Mora Rendón Enlace UACT-Caucasia Edwin Madera Casa de justicia municipio de Nechí Adriana Moreno Enlace UACT-El Bagre Lina Pérez Comité Local de Justicia Luis Fernando Macías Casa de justicia Anorí Martha Valderrama Universidad de Antioquia brigadas y consultorios psicosociales Divis Milena Bohórquez Universidad de Antioquia brigadas y consultorios psicosociales Alejandro Agudelo Checchi Consulting Luis Fernando Pineda Checchi Consulting Natalia Gámez Juzgado de Restitución de Tierras Caucasia Liliana Arango Inspección de policía de Briceño Glenis Núñez Rivero Casa de justicia del Bagre 45 Maruja Castro Cacique Mayor Cáceres Jorge Raúl Machado Enlace UACT-Ituango Lina María Escobar Comité local de justicia Samara Comité local de justicia Erly González Casa de justicia de Caucasia Hernán Alvarado Comisario de familia de Caucasia Itsmenia Vásquez Casa de Justicia de Tarazá Jorge Ángel Gallo Comité local de justicia 12-mar Olga Isabel García Consultorio Psicosocial-Universidad de Antioquia Ana Cristina Rendón Arango Jueza 2 Promiscuo Municipal de Caucasia Yainer González Rodríguez Trabajador social-Universidad de Antioquia Diego Acosta Conciliador en Equidad Matha Romero Conciliador en Equidad Darwin Fabián Sucerquia Conciliador en Equidad Nora Margarita Hernández Conciliador en Equidad Antonio José Areiza Conciliador en Equidad Elkin Rocha Noriega UTR de Caucasia Benito Santero Suarez Ex Cacique Mayor Etnia Senu Ludís Romero Etnia Senu Leonardo Fabio Guerrero Consultorio Psicosocial Angélica Natalia Ocampo Consultorio Psicosocial Faysulys Yanes Segura Consultorio Psicosocial Martha Cecilia Mejía Gómez Líder Comunitaria Matha Escobar Suarez Secretaria de Gobierno 13-mar Mario Andrés Daza Vargas Empresas públicas de Medellín EPM Ramón Gutiérrez Agencia para la Superación de la Pobreza Extrema. Luis Carlos Quintero Fundación Mineros Maria Dulce Gómez Fundación Mineros REGION TUMACO DATE NAME INSTITUTION 17-mar Karen Angulo COMPOS Duber Jácome COMPOS Luis Alfredo Vázquez Conciliadores en Equidad Tumaco Flor María Barrero Pérez Conciliadores en Equidad Tumaco Gieseken Alegría Checchi Consulting Equipo regional Ovidio Urbano Juzgado Especializado de Tumaco Mariela Meza Coordinadoras de Equidad Ligia Ospina Coordinadoras de Equidad 46 Diana Quiñones COMPOS Edward Paz Juzgado del Circuito de Tumaco 18-mar Amarantha Gallego Universidad de Nariño. Coordinadora subcontrato AJA Luis Carlos Cabezas Universidad de Nariño Juan Carlos Angulo RECOMPAS Dora Elena Guerrero RECOMPAS Hader Chillambo Alcaldía de Tumaco Johnny Romero Alcaldía de Tumaco Víctor Salvador Down RECOMPAS Carlos Edmundo González Benavides Universidad de Nariño Célimo Cortez RECOMPAS Fernando Preciado Alcaldía Municipal Katalina Ortiz Fiscalía Yesenia Burbano Procuraduría Alex Enrique Castillo Personería municipal Jhon Ángel Romero No especificada 19-mar Walter Segura ANSPE Nancy Ortiz Defensoría del Pueblo de Tumaco 47 Annex 4: EVAL Framework for Findings, Conclusions and Recommendations Question 1-A: To what degree has the IP improved the NPEC in the dimensions established in the agreement (selection, training, retention, data management, coordination with local justice bodies, incentives, local financing, service provision, and outreach)? Findings Conclusions Recommendations AJA adopted MJL’s MICE to systematize training and selection, although not all strategic partners use it exactly. The MJL says it is simultaneously financing its own MICE training using another group of implementers. MICE does not include a section on techniques for “strengthening” existing equity conciliators (ECs). Approaches adopted by strategic partners charged with the task vary considerably. Checchi’s inventories show that most conciliators trained and “certified” since 2009 are no longer active, or cannot be found. Interviews in 3 regions indicate the truly active are still fewer. Explanations for the attrition vary by locale – from lack of support or active resistance by “unfriendly mayors” to work/family demands getting in the way. The MJL’s roster of conciliators lists about 7,800 but it admits many are not active. Updating is sporadic and the ministry says it has checked only 1000 names (by phone). According to interviews, the presence of a Justice House and room for the conciliators within it was a positive factor MICE’s impact on the quality of conciliation services (or the attrition rate) has not been assessed (by Checchi or by the MJL). However any assessment will be difficult without the IT database to be created by AJA and additional impact indicators. Attrition rates will continue high so long as conciliators are not paid (as studies of volunteerism in other countries demonstrate). Volunteer conciliators are less likely to provide information on their services, keep regular hours, or travel distances to serve communities. Based on observation and comparison of “paid” and unpaid conciliators in one regional visit.31) Compensation for expenses may help, but will not resolve all problems. The use of a volunteer (i.e. unpaid) labor force for the conciliation program will not change. To pay them would create an employment relationship with longer term legal and budgetary If it is true that the MJL can now finance all training (see Q 1-B), AJA might devote more resources to developing information on what conciliators are achieving, who is using them, and whether agreements are being enforced. Use of the Red Unidos survey to tap into some of these questions would work with better questions and a different methodology.32 If the National Planning Department is able to implement a proposed survey on unmet justice needs in rural areas, AJA should try to use it as a still better instrument. In coordination with the MJL and the various strategic partners, Checchi should develop a more uniform approach to “strengthening” existing conciliators. We understand the desire to avoid “cookie￾cutter” methodologies, but the current methodological free-for-all is not good 31 Although payment is illegal, one municipio visited by the team had found a way around the prohibition for several but not all its conciliators. We were told by Checchi staff that this is a practice in other municipios as well. 32 Interviews with ANSPE (which does the “survey”) in Medellin indicated that the answers provided are derived by an ANSPE caseworker evaluating the responses of Red members to a series of questions and then calculating a single “score” for each one as to his/her ability to identify ADR mechanisms. The scored answer is not very useful and of suspect reliability; as we don’t know how the different responses have been aggregated (or whether different caseworkers use the same system). It would be preferable to have answers to a series of questions -- e.g. whether respondents have heard of equity conciliators, whether they have ever used them or know someone who has and what they think about the results. Similar questions might be aimed at the justice houses, legal clinics and mobile exercises. 48 in keeping them active. This does not however resolve the problem in municipios without Houses or in rural areas. The only effort by AJA to tap into the impact of its overall Access efforts (including Justice Houses) is the use of a survey conducted by ANSPE. However, the methodology is suspect and the “scores” not sufficiently fine-tuned to be very useful. Where AJA innovates is in exploring alternatives for enhancing sustainability -- improving incentives, increasing retention, securing local financing and so on -- that the MJL until very recently seemed to ignore. AJA work developing IT databases for the NPJH and NPEC has advanced slowly. Some municipalities collect information, especially in the Justice Houses, on successful conciliations – by registering the agreements. However, these data are incomplete and not consolidated at the regional, departmental or national level. We could find no information on the level of compliance with conciliator agreements reached or party satisfaction with their contents. Some AJA regional offices are exploring ways to coordinate the conciliation program with community mediators (part of the Juntas de Acción Communal). However, according to the MJL, a little known 2012 agreement with the Ministry of the Interior established a division between the jurisdictions of the mediators and equity conciliators, further clarifying that the mediators’ program was part of Interior’s responsibility. consequences. Another alternative explored by AJA (in a study by the National University) – the use of enhanced access to benefits provided by other ministries -- will be hard to negotiate and moreover is a less effective incentive. . The mediators are another possibility, but we know very little about who they are, what they do and with what impact. Since in Colombian terms, mediation differs from conciliation in the absence of a written agreement for the former, it will be difficult to determine use and efficacy The AJA inventories (essentially lists of conciliators operating in each community) are a good start, but to understand the program, its impacts, and its potential value, we need information on use, either as part of a systematic and (eventually) centralized record, or as a consequence of more in-depth community surveys. We also lack information on citizen attitudes in the communities where ECs operate. The working hypothesis supporting expansion is that use will grow with a greater supply of conciliators and knowledge about their services. This might be true, but experience elsewhere suggests there may be other sources of resistance to their use. practice either. Despite the disappointing results of efforts to create a system for registering ECs and their activities, AJA should work with the MJL to resolve the numerous obstacles, and utilize both the Justice Houses and the LJCs to encourage ECs to provide the required information. If the 2012 agreement between the MJL and the Ministry of Interior is really in effect, then any work with community mediators should be coordinated not with the MJL but with Interior. Question 1-B: To what extent has the AJA augmented the NPEC’s sustainability? Findings Conclusions Recommendations MJL interest in a conciliation program independent of the Justice Houses appeared to be waning, but as The NPEC will not be sustainable until 1) the MJL or some other national or sub-national entity takes The MJL’s plan for equity conciliation should be discussed and explored by Checchi and USAID to 49 of late March, the Ministry’s Access Division said it was implementing its own three-stage plan based on MICE, legal change (to facilitate financial transfers to municipalities33) and a consolidation program featuring training of new conciliators every three years. This plan had not been communicated to USAID or AJA when it was presented to the evaluation team. USAID has traditionally been the most constant source of support (and funding) for the NPEC. Except for a few departmental governments most EC proponents depend on others for their funding. The MJL says it now has sufficient funds (US $400,000 annually) to make donor support unnecessary. USAID and AJA have tended to define sustainability as retaining ECs. Program sustainability could be defined differently, as the “new plan” appears to do. Although AJA’s Access Component leader communicates regularly with the MJL’s Access Division, what she has been told by staff is not consistent with what the Access director told the evaluation team. Admittedly, given the frequent changes of Minister, keeping track of policy is not easy. Moreover although the Division is small, staff members are often in disagreement. it to heart and assigns it significant funding and 2) enough conciliators are actively present to constitute a critical mass so far as a local and regional pressure group. The MJL’s new plan seeks to meet these conditions by redefining the issues – not as keeping trained conciliators in place, but as guaranteeing a constant supply of new trainees. However, if the plan is to succeed the MJL will have to take it upward, ensuring it is a national policy and not just that of the current minister. This is not only to overcome any resistance but also to ensure the plan survives a change of Minister of Justice. Although its proponents see the plan as more feasible and no more costly than trying to retain ECs, it may well require a constant stream of funds in excess of the MJL’s calculations AJA efforts to build local capacity (within regions or departments) to train conciliators could help ensure survival so long as the MJL or departmental governments will fund these services. determine whether MJL priorities have changed. If the MJL really can finance all training, AJA might focus, in coordination with the Ministry, on studies (see Q 1-A), implantation in the local justice committees (for which the Ministry also says it now has a plan) and developing improved programs to publicize conciliators’ services among ordinary citizens. Depending on conversations with the MJL, unneeded funds for training conciliators might also be redirected to training of the many other local or locally based authorities who in theory can conciliate. 1-C: Question: How successfully has gender been incorporated in the NPEC? Findings Conclusions Recommendations The MJL’s training guide (MICE) includes no gender component. We found no evidence of a gender focus in More information is needed on how gender is being treated by the various training programs. The issue of the inclusion of gender in the In coordination with its own Gender leader and the MJL, Checchi should establish basic guidelines for how gender will be treated in EC trainings. 33 This involves modifications to Law 1743, but moneys directed to the municipios would only cover EC expenses, not salaries. 50 conciliation work being monitored by AJA for content or impact. We found only two examples of “Strategic Partners” incorporating a gender approach in their project work, and one, the School for Community Justice, disturbingly reported that family violence could be conciliated. conciliation programs is complex because to some extent the conciliation ideology (negotiation and “convivencia”) conflicts with the gender activists’ notion that rights are not negotiable. It would also be useful to know how other local actors doing conciliation (or mediation) with or without training are handling gender in general and gender violence in particular. There may be a legitimate debate about the conciliability of gender violence, but in Colombia the law makes it illegal. Unless Colombian law changes, conciliation of gender violence should be explicitly precluded in training programs. Assuming Checchi will continue EC training, it and its regional coordinators should monitor the treatment of gender in the training or “strengthening” programs it funds. Question 2-A: To what extend has the AJA increased first instance judges’ gender sensitivity and knowledge, and its impact on their handling of cases? Findings Conclusions Recommendations Interviewees reported that the NGC’s program to familiarize judges with high court jurisprudence on gender does not affect first-instance judges in rural municipios. However, the reactions of higher court judges and even first instance judges in larger urban areas do suggest it is expanding recognition of the principles within that audience. CEJ’s Gender Observatory (and its replication in some regions) did not find much evidence of jurisprudence being cited. However, as the CEJ says, judges tend to decide in favor of women (make the right choice) and that is what most matters to the parties. Aside from CEJ’s work, AJA does not monitor impacts at the level of citizen or official behavior. As reported in interviews (and as team members know from experience), the judge’s role comes late in a criminal (or even civil) proceedings. Poor treatment of victims or claimants is most likely to occur (and so discourage persistence or lead to revictimization) among local level officials, the Gender and the differential focus are pretty much embedded in Colombian political discourse. The project has helped expand the audience attentive to these principles, but were it to disappear tomorrow the campaign would continue unabated. For all the USAID and Checchi skepticism, the NGC campaign with high level judges brought these issues to a new audience and on that count cannot be faulted. The audience was not the low level judges in sixth category municipios, but over time changes at the top could impact them. The emphasis on inclusion of jurisprudence and a gender focus in first instance sentences was misplaced (or at least less relevant). In terms of where women victims suffer most it was the wrong location. Current efforts to work with prosecutors (under the project amendment) and on-going work by the project and others with comisarios, inspectores de policía, and forensic medicine, are on a better track. Continue to finance the NGC’s workshops and seminars but so far as possibly at lower funding levels. The NGC) is an important high-level ally, but its approach is overly academic and formalistic and even if successfully implanted will not matter much to woman victims. Explore the potential for devolving some funding to the Sector Gender Commissions as they are closer to the first level judges and moreover may be more accepting of different approaches. In coordination with the Judicial School, try to include judges, or create special programs for them, in the training envisioned under the project amendment – hands on, and focused on techniques and case studies, as opposed to the current emphasis on drafting gender-sensitive sentences. 51 police, forensic medicine or the prosecutors. Lower level judges in the target municipios are hardly unaware of the gender focus, even if they do not attend the NGC workshops. There are other project-sponsored programs as well as activities outside the project they cannot ignore. Whether this changes the likelihood of their sentencing in favor of a woman is unknown; it visibly does not induce them to use the gender-based sentencing techniques outlined in NGC publications. Question 2-B: Has the AJA successfully and neutrally engaged GOC institutions (nationally and regionally) in a coordinated response to improving handling of GBV cases? Findings Conclusions Recommendations As reported in several interviews, including with other projects, judges often resist “coordination” as an infringement on their independence. The NGC’s emphasis on meetings and workshops might not have been a first choice of USAID or Checchi, but neither is in a position to tell high court judges that they need another approach. AJA’s change theory here, and in other aspects of its gender work, has prioritized increasing demand over improving supply. It is time to correct the imbalance, as AJA appears to do with the project amendment. Despite efforts to encourage a coordinated response through the LJCs (where AJA’s resources allow more leverage and its advice may bear more weight), the LJC members interviewed provided few concrete examples of measures taken to introduce a gender focus in their activities. Most did acknowledge that gender is important (although by self or advisors’ selection we believe we were sent the most positive members). The AJA’s capacity to engage GoC institutions in a coordinated response to gender issues has never been high. It partners with institutions, but does not drive their programs, which they typically pick according to their own sense of what is appropriate and effective. The project further limited its ability to encourage a coordinated response by working largely with judges at the national level. This should change under the amendment and the decision to work with prosecutors and other actors lower in the criminal justice chain. Within the LJCs, observation and interviews suggest that training and advisory services have not induced in-depth reflection on gender issues. While the language may be given a nod, LJC members have avoided coming to terms with their own weaknesses in this (and other) access areas. Encouraging LJCs to develop a bank of projects has been useful in attracting members, but it diverts attention from discussion of existing service Much more could be accomplished by focusing on actors involved in the pre-trial stages and especially with hands-on training in better approaches to dealing with victims. Here the new Checchi approach seems on the right track, but it will be important to tailor the trainings to the specific roles of each type of local operator. It would be useful for the project to collect data on the types of cases first registered with each “input” actor (from inspector de policía through prosecutor) as well as whether and to whom they are passed on. This information can be used both to illustrate problems and to measure advances. The “Obstacles to Access” studies (barreras) produced under a contract with FIP contain important information on treatment of women victims that should be used in discussions with national actors to expand their understanding of local-level problems. Such discussions should precede the proposed publication of the studies to avoid provoking resistance to the findings. 52 weaknesses. The gender project in one municipio was to invite a speaker from an NGC workshop to deliver another lecture for Women’s Day. Question 2-C: To what degree have AJA actions increased citizens’ awareness of the legal framework for gender issues (including rights)? Which have been the most and least effective methods implemented? Findings Conclusions Recommendations The project has not collected data on citizens’ awareness and given the evaluation’s time constraints we were not able to search for other sources of this information. The former Gender Component coordinator did point to a 10-fold increase in victims’ complaints as a result of the denuncias masivas, but considering the broader universe targeted, the significance of the increase is hard to assess. As noted in interviews, and especially evident in Tumaco (and its “Gender Table”) gender is an area where most donors work, often with the same agencies and CSOs and with similar activities and programs. While we know NGO campaigns (in general) have increased attention to gender rights, we do not know how this has affected women in the target municipios and their willingness to make civil and criminal claims. CEJ’s Gender Observatory did not measure this. Information from the various entry points (inspectores de policía, comisarios, police, local prosecutors) tracking claims over time could be used to assess effectiveness. Unfortunately, the team did not have access to this information (and AJA is not collecting it). Considering the quantity of donors engaged in gender activities, even with data it will be difficult to identify AJA’s value added. To advance its goals of influencing citizens’ awareness and to measure its own efficacy in so doing, AJA needs to tap citizen attitudes directly and to evaluate the impact of the various national, regional, and local events. Taking into account a comment by a cause sympathetic government official, that donors are doing “too much” gender work, AJA might evaluate its marginal utility in this area. If it is doing more of the same, in the same areas, and with the same actors, then doing something less redundant could be in order. Question 2-D: To what extent has the AJA engaged civil society partners in making an effective contribution to advancing gender aims? Findings Conclusions Recommendations Noted here, but relevant to this entire component, gender issues are defined differently by AJA’s various partners. Although AJA mentions other vulnerable groups, gender is taken to mean women and gender issues are increasingly defined as GBV. Even here there is the question as to whether the project focus Variety is often an asset, but the differing definitions of the topic, the issues, and the methods have created conflicts in the project and also explain difficulties in forming a national network (as well as consolidating a GoC response). Especially in gender, but also in the other components, there is a need to coordinate activities across regions and strategic partners. Checchi might set up its own internal committee to do this or organize meetings to discuss the overall gender strategy (which does not seem to exist.) 53 includes GBV victims of the internal conflict (in which case additional GoC entities would be involved, including the Transitional Justice Office of the MJL) or is limited to current and future victims. Additionally, as noted in Q 2-B, GoC agencies differ as to the preferred methods for addressing the problems (however defined). Most NGO campaigns, whether sponsored by the project or not, focus on informing women of their rights, mobilizing them to claim justice, and detailing egregious examples of miscarriages of justice. This was also true of the “jornadas de denuncias” sponsored by the project under the first Gender leader. Interviews with SISMA Mujer indicated they did not believe their quick visits to the regions to discuss rights were having much impact, even in terms of simply raising awareness. According to interviews both the attempts to build a network of lawyers and psychologists to accompany meetings and to build a national network of gender￾focused organizations have been less than fully successful. There may be too little variety, however, in AJA’s selection of strategic partners and its support to their activities as compared to what other donors do. Donor support to pro-gender groups tends to considerable redundancy. AJA’s value added would increase to the extent it can break away from what every donor does and focus on a unique or less usual contribution –possibly by working with supply rather than demand, or by introducing better methods of driving the rights message home and inspiring the audience to action. Building demand for gender rights is a necessary step but has little immediate impact on how women are treated by justice sector institutions. Funding to national and local level gender rights groups should be continued as a cut would send the wrong signals. However, unless they can find a way to work productively with justice sector officials, there is little need to expand it. AJA should focus partners on measuring their own impact (not as women attending workshops but as conversion of the message to action) as a means of getting them to devise more effective approaches. Despite the difficulties in establishing local victims’ assistance networks of lawyers and psychologists, the effort is worth several more tries. As per a suggestion by AJA’s COP, building regional (rather than a national) networks of gender focused CSOs would be closer to the aim of improving local justice. Question 3-A and 3-B: Is the AJA’s approach to improving LRC case management effective and sustainable? What are judges’ and magistrates’ perceptions of AJA, and has change management been successful so far?34 Findings Conclusions Recommendations The most striking aspect of the land restitution process is how few cases enter the system with fewer still reaching resolutions. As of March 28, 2015, only 74,161 requests had been presented to the LRU and only 7,801 of these had been sent to the judges, resulting in 2,040 sentences. When one considers that there are 7 million desplazados, The land restitution process has been remarkably slow, but not because of judicial delay or inefficiency. The most important causes are the macro and micro focalization processes and the additional need for police assurance that the LRU and judges can work in the affected areas. Ignorance of many claimants as to how to proceed To improve the land restitution process, USAID should focus more on the LRU and its interface with the judges. Closer coordination is recommended with the Land Restitution and Rural Development Project, which also works with the other agencies whose cooperation is needed. 34 The questions have been combined to avoid redundant entries. 54 even taking into account that many are members of the same family, it is evident that this is only the tip of the iceberg As a consequence, judges are so under-burdened that some have been transferred to ordinary civil courts and those remaining are given tutelas and ordinary civil cases to take up their time. The roughly 1,500 LRU staff members also have little work, judging both by average caseloads and the lack of activity in the offices visited. Goals set for case settlement over the next four years (20,000) are not ambitious and as one interviewee noted, LRU personnel are under no pressure to produce more. Most of the cases reaching the courts do not involve large holdings but instead one of three situations: one smallholder pitted against another; a smallholder returning to unoccupied land s/he had once farmed; a smallholder who never left his/her land (and thus is legally not eligible for the land restitution process) requesting a formal title (and possible some of the additional benefits accruing to the forcibly dispossessed). Opposed claims are only 30 percent of the total reaching the courts. The rest are described by many observers as essentially administrative in nature (so much so that De Justicia questions the need for judicial involvement). Judges’ dissatisfaction with case preparation by the LRU (which also takes its time in processing) and fear of reprisals also account for the low number of claims placed to date. At all stages of the process – pre-judicial; judicial, and post-sentencing – less than complete cooperation by the other entities involved (for cadaster, titling, ensuring land is not protected, provision of additional services including security) has added further delays. In both the LRU and the land courts, additional delays originate in uncertainty about individual responsibilities and practices. Both agencies spend an inordinate amount of time repeating the same investigations and establishing the presence of violent events that are already well documented. Despite its 208 articles neither Law 1448 nor the related Decree Laws 4633, 4634 and 4635 (dealing respectively with indigenous communities, Roma and Afro Colombian communities) establish a clear guide to LRU and judicial practices, leaving far too much room for inventive imagination.36 Delay may have some implicit benefits. Given the many other services provided to winning claimants, a vast increase in sentences will place a high financial burden on the national and local governments. If anyone has calculated the likely costs, their results have not been made public. It is doubtful that “better case management” for judges with few cases would make much difference, especially since the ISO team seemingly focused on standardization, not greater efficiency37. Moreover judges used to taking their AJA should explore the potential for increasing process-wide coordination through more work with the inter-institutional commission (with the CEJ acting as technical secretary). Judges’ insistence on redoing the LRU investigation is perverse and if a way could be found to encourage a reconsideration of this practice, it would be useful. The protocols produced by the procedural and inter-institutional subcommittees may be a way into the solution, although there is no guarantee that individual judges will adopt them (because of judicial independence). AJA should reconsider its use of ISO. We recommend not financing more ISO exercises and terminating existing work as quickly as possible. If AJA believes quick replication is feasible, contrary to 20-years of Colombian experience of non-replication, it should try this In a few courts to test the results. Further work on a management model for land courts would benefit from a review of current staffing (e.g. all those systems engineers; absence of experts who could help interpret cadasters) and consideration of how to handle much larger workloads (on the assumption that they eventually will emerge). It would also be useful (although based on comments by judges would not affect their decisions) to conduct a study on the probable costs of the additional measures awarded to winning claimants and their impacts on national and municipal budgets. 36 Knowledgeable team members noted that Law 1448 lacked “reglamentación,” or an expanded secondary law that would define the procedures in detail. As a consequence they and others report that judges often use the more complex ordinary civil proceedings, again adding delays. 37 Admittedly, getting everyone to follow the same procedures is a step ahead, but the next step would be to vet those procedures for redundancy, unnecessary details, and so on. As for the judges’ repetition of the URT investigation, the ISO team seemingly ignored this. 55 has led to their either returning the cases for more work or repeating the LRU’s investigations. This does increase times, but not productively. AJA’s answer to the land restitution challenges has been the modelo de gestión and the zero paper project (the latter funded by the SCJ). Both originate in judicial preferences, and in the case of the modelo de gestión a longstanding belief that ISO 9001 improves services and increases efficiency. We were unable to get anyone associated with the case management model (modelo to gestión) to describe it to us. References to the experience instead focused on the ISO process – the production of flowcharts, creation of a quality management committee and the development of detailed manuals to standardize steps in case processing and template documents for easier drafting. From observation of the tribunals and courts, we could find little different from “business as usual” in other civil jurisdictions, except for the land courts’ radically smaller workload and the placement of a systems engineer in each courtroom (we do not know whether this was recommended by AJA). Both the modelo de gestión and the zero paper effort are introducing simple electronic registries to allowing tracking of individual cases and give the judge and staff a view of the overall caseload. The zero paper model uses the outdated Siglo XXI system, adding a capacity to generate basic statistics.35 The modelo de gestion uses a simple spreadsheet. “Zero paper” has raised great expectations, and in time with a few cases will have difficulty adjusting should the flow from the LRU increase substantially. As one LRU director noted, the result would be system collapse. Less (or zero) paper does not mean greater efficiency (and with a poorly organized electronic archive could mean far less). A few aspects of the model that may not be applicable more broadly – for example email notifications – may speed processing to some degree, but neither this nor the modelo de gestión’s better organized archives (containing very few cases) will make much difference in time to resolution or in judges’ ability to handle vastly larger caseloads. The case tracking software introduced in the two models are not coordinated efforts and thus raise the question of which will be used when the models are rolled out and combined. There may be similar issues with use of templates and archiving techniques. A quick review of security measures suggests that whatever the expert hired for this purpose had recommended, it was not being used. Metal detecting doorways were often turned off and the one judge who had backed up her electronic files said she did this at the recommendation of her “systems engineer”. The work of the courtroom systems engineers is typically limited to repairing computers, setting up simple Excel spreadsheets, or even clerical tasks. The engineers are unlikely to be needed for much more unless the piloted projects develop truly complicated case management systems and databases. Even assigning one to each courtroom The project should introduce some more useful (and proven) instruments for improving court efficiency – for example the “International Framework for Court Excellence” or the work of the International Association of Court Administrators (IACA) -- in place of ISO. Courts in more developed countries that have increased their efficiency and improved services have not relied on ISO for this purpose. 35 The SCJ has protected this system for over 20 years, but while once state-of-the-art it is now far behind the times and arguably should be replaced. 56 Bogota it has reduced to a minimum the amount of paper in the office of the judge we visited. Judges piloting the modelo de gestión are pleased with it, but outside the piloted regions it attracts no attention. AJA activities related to the models include a contract to recommend security measures for the courts Other AJA tasks include support to and funding of the judicial committees established to identify and find solutions to court problems, and support to the inter-institutional committee reviewing Law 1448 implementation. AJA was asked to do a study of land court corruption, but with USAID’s agreement this has been modified to cover the entire land restitution process. may be excessive. The most significant product of the judicial committees has been the development of protocols intended to standardize treatment of types of cases and develop guidelines for LRU submissions to the courts (thereby reducing one source of problems – judges’ varying preferences as to the form and content of LRU submissions). So far the protocols have not been approved; furthermore there is no guarantee “independent” judges will use them. Question 3-C: To what extent has the AJA included gender in this component? Findings Conclusions Recommendations Although the project has financed, through the NGC, a document demonstrating how to apply a gender focus to land cases, its appearance is too recent and the decided cases too few cases to say whether it is being applied and with what if any impact on case processing. CEJ’s Land Observatory includes little gender data, and what there is does not indicate specific gender issues in these cases. Except for the magistrates in Cartagena, land judges interviewed did not appear especially attuned to a gender focus, but neither were there indications they were against it. As with gender violence, the test is not in the sentence but in factors like the priority given to cases involving women, application of protective measures and possibly the types of additional benefits awarded to female complaints. CEJ’s land observatory has collected data on more than gender-correct sentences. This additional material, while not limited to gender cases, is more illustrative of the land restitution process issues. To monitor the gender focus, AJA will need to do more than evaluate sentences. Suggested variables include differences in time to resolution or additional benefits awarded by gender. Also important, but as applied to all complainants would be a tracking of delays in processing cases from the LRU onward, and an examination of the politics of the focalization and other security constraints. The “corruption study” may cover the latter theme, but considering rumors circulating about the reasons for slow focalization, either version will expose the researcher to considerable risks. 57 Question 3-D: Has the AJA improved the performance and sustainability of the LCJs (local justice committees)? Findings Conclusions In addition to the six Local Justice Committees already functioning in Meta, the project says it has established another 22 (a claim we do not disbelieve but cannot verify). According to representatives interviewed, membership is broad including both local government officials and judges and prosecutors. Family Welfare (ICBF) was conspicuously absent from most (as it is from the Justice Houses). According to the AJA SOW, local justice work will be done in coordination with the Territorial Consolidation Unit (TCU) and the MJL’s Formal Justice and Jurisdiction Division (FJJD). Our interviews with and observation of TCU representatives demonstrated their presence but no more active contribution to LJC formation or outputs. No FJJD representatives were identified in the regions. LJC members were enthusiastic and cited some achievements, especially in building links (articulación) among local actors (e.g. comisarios de familia, personeros, judges and prosecutors). Checchi reports cite instances of committees having encouraged local governments to include a justice component in their budgets and the formation of a bank of justice projects for financing from AJA, departmental or national funds. There are visible differences in the continuing dependence of the LJCs on Checchi staff. In Carmen de Bolivar but not in Caucasia, Checchi advisors still convened meetings. In Tumaco, it appears that the recently departed local advisor was active in shaping the multiple plans and project proposals produced, but that local politics accounted for some strange variations in membership. Although the MJL on various occasions expressed disagreement with the “Checchi model,” the reasons seemed largely academic (use of “committee” instead of “system”). However, what the MJL describes as its own model does differ in some structural aspects, and there is no indication of Checchi and MJL having discussed them. The LJCs’ creation and chances of subsistence so far hinge on the proactivity of AJA advisors and members’ sense that participation will give access to more resources. While much lip service is paid to the benefits of “articulation” of local agencies, we heard few concrete examples of this being accomplished. Since many LJC members are local officials who are appointed by each new mayor, one acid test will be what happens after the 2015 local elections. We know from experience that a mayor opposed to the process can quash it; we don’t yet know what it takes to survive the transition. While all LJCs spoke of an interest in a gender focus, this produced few concrete actions among the LJCs interviewed. In Tumaco, despite the presence of a municipal gender subcommittee or table, references to products largely involved the development of plans, policies and workshops. Now that the MJL (and possibly the Ministry of Interior or that of Post￾Conflict) are taking more interest in the local justice systems, Checchi’s former ability to largely go it alone, could become risky. Checchi, in cooperation with USAID, needs to establish a governmental counterpart i(beyond the TCU) if its actions are to have an impact. The project also should explore in discussions with the MJL the reality of the “new” MJL model (as well as the reasons for its sudden presentation to the evaluation team). Studies done with FIP (including the one on obstacles to access and the 2013 basic approach to forming LJCs) should be more widely disseminated and discussed, including with Justice and Interior. If the LJCs are to produce enhanced access, less attention might go to drawing up projects for funding by others and more to enhancing cooperation and developing a better division of labor among local level actors. If there are good examples already in existence, they should be disseminated to other LJCs 58 Question 4: Cross Cutting: To what extent is the AJA successful in capturing and reporting on impact? Findings Conclusions Recommendations Justice project achievements are typically challenging to report because both the context and the goals are difficult for non-experts to appreciate. While the evaluation team believes the project should focus on quantitative indicators, it recognizes that most of these are meaningless to someone not imbued in justice sector dynamics and also in the specifics of the country under consideration. Even in two years, the project has generated an enormous amount of information on the situation of and obstacles to local access to justice. Unfortunately, much of it is highly sensitive and potentially offensive to the GoC if widely published – for example material about local justice actors’ collusion with or intentional inattention to armed groups operating in the same region, or their prejudicial treatment of women victims of violence. However, this same material is important to understanding what the project is attempting and what obstacles it faces. Some of the positive reports for public consumption have been undercut by AJA’s own studies and those of other organizations (and USAID projects) – for example, see the July-August 2014 Justicia al Día report on the land restitution process versus studies by CEJ (AJA financed) and Forjando Futuros (funded by USAID’s Human Rights project). The AJA treatment does mention problems but is overly optimistic about a government sponsored bill that would cut restitution times to 60 days. The project has published much literature on its activities, but most of it (e.g. the periodic 6 page bulletins) is probably interesting to and only read by project participants. The reports are very informative (if somewhat dismissive of real obstacles to and caveats about progress), but even in translation would not be useful to anyone not connected to the project. In working with relatively uninformed publics (Colombian or US), the “why something is needed” is as important as what was done. This means more explanation of context, the status quo ante, and the problem, tailored to a thorough understanding of the target audience’s knowledge of the subject. Whereas, following good journalistic format, the AJA Bulletins start with the accomplishments, they might do better to begin with the problem. Prior knowledge also includes misunderstandings. There are many of these circulating even among the informed public in Colombia (e.g. that judges are overworked, that budgets are too low, that any ICT innovation will increase efficiency), and there may be more as regards Colombian justice among the US public (especially as what is most likely to reach them are examples of egregious miscarriages of justice and violations of human rights, rather than the day-to-day inefficiencies of the system). Information should be designed for the audience one wants to impact. If USAID’s interest in posing this question is how to register better with the US Congress, State Department or public, then the prior knowledge of those audiences should be considered first. The same considerations, but different answers apply if the intent is to influence the GoC, the Colombian judges or the general public. The bulletins and project reporting documents are fine for in-house audiences, but if they are to be the source of information for other publics they should be rethought and represented. The examples are not bad – it is the presentation that lessens their impact, and there should be different presentations for each public. 59 Annex 5: Interview Notes and permissions, and data collection instruments (electronic submission) Please see protected disk submission for these confidential materials.