i Colombia-US Human Rights Law School Partnership Program Evaluation Report Presented by Fernando Rojas, Catherine Romero and Françoise Roth July 2015 US-Colombia Partnership Program – Evaluation report Page ii Table of content ABBREVIATIONS IV 1. EXECUTIVE SUMMARY V FINDINGS V RELEVANCE VI EFFICIENCY VII EFFECTIVENESS VIII RESULTS IX SUSTAINABILITY IX MAIN RECOMMENDATIONS X 2. INTRODUCTION 1 OBJECTIVE OF THE EVALUATION 1 EVALUATION METHODOLOGY 2 APPROACHES OF THE EVALUATION 2 Comprehensive and systematic approach 2 Analysis to construct learning 3 Principles included in the analysis 3 DATA COLLECTION METHODS 3 Desk review of material 3 Semi-structured interviews and field visits 3 LIMITATIONS ENCOUNTERED DURING THE EVALUATION 5 PRESENTATION OF THE REPORT 6 3. OVERVIEW OF THE PROGRAM AND THE PARTNERSHIPS 6 OVERVIEW OF THE PROGRAM 6 OVERVIEW OF EACH PARTNERSHIP 9 ANTIOQUIA PARTNERSHIP 9 VALLE DEL CAUCA PARTNERSHIP 10 CARIBBEAN COAST PARTNERSHIP 11 4. EVALUATION FINDINGS 13 RELEVANCE OF THE PROGRAM 15 EFFICIENCY OF THE PROGRAM 20 EFFECTIVENESS OF THE PROGRAM 27 RESULTS OF THE PROGRAM 32 SUSTAINABILITY OF THE PROGRAM 34 5. CONCLUSIONS 35 6. RECOMMENDATIONS 38 7. ANNEXES 7-1 ANNEX 1. EVALUATION QUESTIONS GRID 7-1 ANNEX 2. LIST OF THE PERSONS INTERVIEWED 7-6 US-Colombia Partnership Program – Evaluation report Page iii ANNEX 3. PARTNERSHIP LEVEL FINDINGS 7-13 ANTIOQUIA PARTNERSHIP 7-13 Relevance 7-13 Efficiency 7-14 Effectiveness 7-16 Results 7-18 Sustainability 7-21 VALLE DEL CAUCA PARTNERSHIP 7-21 Relevance 7-21 Efficiency 7-23 Effectiveness 7-25 Results 7-27 Sustainability 7-30 THE CARIBBEAN COAST PARTNERSHIP 7-31 Relevance 7-31 Efficiency 7-32 Effectiveness 7-34 Results 7-37 Sustainability 7-39 US-Colombia Partnership Program – Evaluation report Page iv Abbreviations ACE American Council on Education AU American University CGR Center for Governmental Responsibility - UF CLAS Center for Latin American Studies - UF HED Higher Education for Development HEI Higher education institution LGBTI Lesbians, Gays, Bisexuals, Trans, Interex M&E Monitoring and Evaluation NGO Non governmental organization PUJC Pontificia Universidad Javeriana de Cali RBM Results-based management RFA Request for Application SOW Scope of Work UCO Universidad Católica del Oriente UDEA Universidad de Antioquia UDEM Universidad de Medellín UF University of Florida UM Universidad del Magdalena UMN University of Minnesota-Twin Cities UN Universidad del Norte UPB Universidad Pontificia Bolivariana US United States USAID US Agency for International Development USC Universidad Santiago de Cali WCL Washington College of Law US-Colombia Partnership Program – Evaluation report Page v 1. Executive summary This report presents the findings of the external final evaluation of the Colombia-US Human Rights Law School Partnership Program (the Program), a three-and-a-half year initiative implemented by Higher Education for Development (HED) in Colombia. The Program was aimed at improving human rights education and training in regional law schools, and promoting a culture of acceptance and respect for human rights. Its specific objectives were to: 1. Strengthen the institutional capacity of Colombian schools of law to train future legal practitioners in human rights by introducing or strengthening curriculum in human rights as well as experiential models of legal education, such as clinics and externships; 2. Enhance the outreach capacity of regional law schools to better serve vulnerable populations with limited access to or knowledge of the legal system through the creation of cross-regional law school human rights networks; and 3. Equip future legal professionals with an understanding of national and international standards of human rights and the skills to support human rights reform in Colombia. Funded by the US Agency for International Development Mission in Colombia (USAID/Colombia), it supported three partnerships between United States (US) and Colombia universities:  Antioquia Partnership: University of Minnesota (UMN) with Universidad de Antioquia (UDEA), Universidad de Medellín (UDEM), Universidad Pontificia Bolivariana (UPB), and Universidad Católica de Oriente (UCO);  Valle del Cauca Partnership: American University (AU) with Universidad Santiago de Cali (USC) and Pontificia Universidad Javeriana Cali (PUJC);  Caribbean Coast Partnership: University of Florida (UF) with Universidad del Norte (UN) and Universidad del Magdalena (UM). The evaluation was conducted between March and June 2015. It considered the Program from a comprehensive and systemic performance programming perspective, as well as a prospective approach. The findings present the analysis according to the evaluation criteria: relevance, efficiency, effectiveness, results and sustainability (the findings for each of the three partnerships are separately analyzed in Annex 3 of the report). They derived from a detailed desk review of Program materials as well as the realization of more than 125 interviews in the three regions of implementation and in the US. FINDINGS The US-Colombia Partnership Program was a well-structured initiative, carefully managed and monitored, which has produced some noticeable results. As with most other educational programs, its final impact can only be judged in the medium term. For the time being, the US-Colombia Partnership Program – Evaluation report Page vi assessment focused on the extent to which the Program achieved the products and outcomes expected from its three-year implementation. The Program had a rather complex structure and demanding operation:  Three regional US-Colombia partnerships, involving directly eleven partners that had little or no interaction prior to the Program, which implied huge logistical challenges in dealing with different management stability and capabilities, each institution having its own procedures, restrictions and opportunities;  Asymmetry of context, mandates, vision, specialization, interests or opportunity costs among the parties involved;  Application of a results-based management for program design and program management to/by some organizations that were not familiar with performance programming or performance management;  Different baseline per partner/partnerships, demanding different goals, strategies, support or resource per partner/partnerships. Relevance Relevance of the objectives. Undoubtedly, the Colombian context makes human rights relevant –even priority-- for Colombia’s higher education institutions (HEIs). Improving the teaching of human rights is relevant to train generations of lawyers to be more knowledgeable on the issues involved, and more motivated to pursue human rights-related tasks. Furthermore, after years of relative silence, Colombian universities have an important role to play in human rights debates.1 This is particularly true in regions other than the capital city. Design of the Program. The Program design reflected an overall strategy to deal with its complex structure, demanding operation and risk factors through:  flexible specific objectives per partnership within a wide-open menu of areas of intervention grouped under three general categories. This scheme allowed each partner to find its own interests and define its own roles within the partnership;  common overarching goals and expected impact for a shared, unifying concept and purpose of the Program;  monitoring and mid-term evaluation tools that used a common set of indicators, yet allowed for customized indicators tailored to each individual partnership (or even partners within each partnership). However, the Program operated as if it had been formulated from the activities rather than focusing first on the outcomes to consider all necessary preconditions to achieve them. To strengthen the capacity of law school to train in human rights, a project needs to start from this goal to consider its intermediate objectives, define its activities and define the essential inputs required to attain its goal, such as ensuring sufficient and adequate resource and processes (including faculty and administrative staff, budget, financial management, decision-making 1 During the last four decades, human rights issues have been intensively debated in Colombia among NGOs, the Executive, the Judiciary, Congress and independent monitoring and control organizations. US-Colombia Partnership Program – Evaluation report Page vii process, implementation capacity). A thorough analysis should have also identified the necessary preconditions for as well as the risk factors that may hinder the achievements of the Program’s outcomes. Otherwise, and it is what the development of the Program showed, a disconnect will appear between the activities implemented and their impact. HED Program management team, including its monitoring system, should have made every possible effort to bridge detected gaps for the success of the Program. Additionally, the Program was clearly conceived to pursue its overarching goals through three different partnerships, not from a comprehensive perspective. Its results were reported accordingly, i.e. according to each partnership’s results, rendering the analysis of the Program as a whole a complex endeavor. Institutional arrangement. Institutional arrangements, such as partnerships, need to fit the perceived needs or opportunities of participating institutions as well as their awareness of the costs and benefits of engagement in partnerships. In Colombia, the choice of working through partnerships of universities was set by USAID/Colombia as a precondition for the execution of the Program; it did not result from a feasibility analysis or the request of the partners. Some conditions favored positive results in Antioquia, not so in the other two regions. Gender perspective in the Program design. Isolated references were made to women and LGBTI persons as targeted populations, and to differential approaches, mentions that did not suffice to give a gender perspective to the initiative. Efficiency The choice of the law schools as critical resource for the Program. The most critical resource for the success of the Program was the participating law schools. The selection of Colombian law schools appeared to have been made on the basis of the assessment of each school’s commitment to the specific objectives of the Program: (i) curriculum development; (ii) community outreach and service; (iii) faculty development; and (iv) institutional strengthening. According to these criteria, the choice of the targeted education centers was sound. It encompasses HEIs that have a great potential for improvement, and have the capacity to impact in their region. However, other aspects, critical for the implementation of the Program, appear not to have been assessed nor given significant weight, such as the institutional capacity and willingness to implement the above commitments; demonstrated achievements and commitment regarding the overarching goal of the project; commitment to combine actions and tools in order to create synergies among the three specific program objectives; or demonstrated experience and/or commitment to work together in partnership or otherwise capacity to work in a partnership. Consideration of these elements would have certainly allowed assessing some of the universities differently, appraise and foresee potential risks, and develop a mitigation strategy. The choice of working in partnerships to ensure major efficiency. The Program chose to have the universities in each region work in partnership. Partnerships were perceived as coverage and quality leverage to secure the expected regional impact of the Program. However, since neither Colombian law schools nor, more broadly, HEIs have much of an associative tradition to strengthen capacity and reach higher goals, the inclusion of the partnership as a US-Colombia Partnership Program – Evaluation report Page viii necessary element of the Program design had, at its onset, no clear relevance for most of the Colombian parties. The willingness of the Colombian universities to work in partnerships, the operative conditions of the consortium, and the criteria to choose them are not explicit in the design documents. Moreover, the choice of working through partnership introduced high risks to project efficiency and effectiveness together. The monitoring and evaluation system (M&E). HED M&E system is a very sophisticated and solid mechanism. Planning and reporting on activities and according to higher-level objectives was new for most of the Colombian universities. Virtues and benefits were recognized to the M&E system: it forced universities to plan and organize their activities, respond for them accordingly, make their results visible, and gather relevant information on and for their work. However, almost-unanimously, the M&E system (from the establishment of the baseline and indicators to the reporting requirements) was perceived as burdensome, time￾consuming and not the most adequate to reflect the outcomes or impact of the Program in terms of the changes the universities were realizing. The financial resource. Regarding the use of the Program financial resource, three issues are worth noting. First, the fact that, upon a USAID decision, the Program finally applied to eight Colombian universities, rather than five as initially planned, necessarily affected the distribution of financial resources. The second issue deals with the staffing of the partnerships. In Antioquia, part of the budget was allocated to pay the salary of three persons especially in charge of coordinating the partnership. This option contributed significantly to the efficiency of the implementation of the activities there. The last point concerns the contract with the security company Gardaworld: its approach appeared not to be necessarily the most adequate for human rights work; the company lost credibility with the Colombian partners, and was considered of very little use to them. Effectiveness Undoubtedly, the Colombia-US Human Rights Law School Partnership Program proved that higher-level education partnerships could encourage and foster concrete results: a change of paradigm in faculty and students; joint work between universities on strategic issues; the inclusion of new methods for higher-education in human rights; the strengthening of professors’ knowledge and capacity; modifications in some university curriculum. Taking together, these achievements should have led to the completion of the Program objectives. However, not all of them were attained in each region –or at every law school -, and not all of them were reached at the same level of execution. Several factors that contributed to positive results within the partnerships can be identified: (i) the approach of the US universities and their relationship with their Colombian partners; (ii) their knowledge of the Colombian context, their capacity to adapt; (iii) the management scheme of partnership. However, the Program lost opportunities to foster its results. Most of the partners focused their work towards the implementation of activities, losing the sight of the outcome result. Furthermore, the Program did not encourage law schools to develop each one of its three areas of intervention in a balanced, much less a combined way. US-Colombia Partnership Program – Evaluation report Page ix Effectiveness of the Program depended largely on effectiveness of the monitoring exercise. In order to be effective, HED management and implementation teams (two separate teams) should have been guided by a prefigured implementation plan to guide the Program monitoring, taking into account its possible implementation risks and foreseeing risk-mitigation strategies. It would have allowed the partners to re-focus their work toward the outcomes. Results Some concrete and very promising results were achieved through the Program. However, the delay incurred at the onset of the Program to establish the baseline and the indicators for each partnership, if necessary and perhaps unavoidable, led to the perception that results are just beginning to be seen –or are expected for some time later. This perception further jeopardized expected outcomes during the life of the Program. On the positive side, it protected the credibility of the Program by making it look more like the pilot-exploratory phase of a more ambitious and long-term initiative. When the magnitude of the Program’s risks is taken into account, the Program can at best be characterized as a high risk-high impact venture. However, this characterization for the entire Program veils the fact that results and impact were very different by regional partnership and – above all—by law school. All in all, it was a risky Program with highly differentiated actors and results. In view of this overall finding, the most positive characterization that can be made of the Program is that of the first phase of an envisioned strategy towards strengthening engagement of law schools in and enhancing their impact upon human rights in Colombia. This first phase concluded with some unequal results. Sustainability Did the Program successfully institutionalize enough capacity to secure continuation after USAID resources are gone? Are law schools committed to continue supporting (or expanding) that precise capacity that has explained achievements leading to the higher-level goal of the Program? Some participating law schools have shown their interest and commitment at pursuing the efforts developed, for instance by securing the institutionalization of legal clinics and/or financing faculty to manage the clinics or train on human rights. However, institutionalizing human rights training in universities implies not only financial resources. It entails foremost the commitment and involvement of the HEIs leadership with human rights, as a fundamental piece to ensure the development and sustainability of an initiative of that sort. This involvement is clear in some universities, uncertain in others, absent in some others. Did the Program produced outcomes whose benefits are guaranteed to extend beyond the closure of the Program? The Program has produced some interesting results, some of which might extend beyond the closure of the Program. However, it is too premature to assess their sustainability, since the Program is scarcely showing its first results. US-Colombia Partnership Program – Evaluation report Page x Is the relationship and collaboration between the partners sustainable? Sustainability of the core concept of the Program is heavily dependent on the initial selection of participating law schools. If partnership arrangements are seen as an essential component of future institutional arrangements, the process of selection should have essentially required experience and/or evidence of commitment of selected law schools to regularly or permanently work in partnerships. If sustainability of the Program objectives includes further institutionalization of partnerships, partnership-formation should have been seen as a goal by itself from the early design of the Program. MAIN RECOMMENDATIONS 1. Bearing in mind that the present report is aimed at learning from the experience, the evaluation team strongly recommends that a validation process be conducted with the Program partners, in Colombia and in the US, in order to share the conclusions contained in the present report, and receive and discuss their comments.2 The evaluation team considers that the following recommendations could guide the implementation of future initiatives similar to the Partnership under consideration: 2. Projects this complex can be flexible indeed. Flexibility needs to be balanced with explicit, ex-ante (during planning phase) agreements on those arrangements that will be decisive for the outcomes and impact of the program. This entails: a. Criteria for school selection need to be structured in such a way that they help minimize predictable risks for reaching program outcomes and impact. Accountability for quality of program design and management is enhanced when one agent is fully responsible for selection of participating schools; b. Engagement of the partners during program planning needs to explicitly include: a) criteria for selection of domestic law schools, b) alignment between the school’s mission and the Program’s expected outcomes; c) matching of interests and capabilities with other members of the partnership, d) selection criteria, commitments and expectation from participation of the US partners; c. If a partnership is either expected or required, the US partners need to know in advance the strengths and weaknesses, opportunities and restrictions of their domestic partners and vice versa; d. Should participants lack direct experience in partnership formation and implementation, prior training is needed before the parties enter into the negotiation phase of each partnership; e. The joint plan of action should consist of fully structured individual projects that mirror the interests of participating parties and produce benefits no individual party can attain by itself; f. Planning time should not be confused with implementation time; 2 The focus of the proposed partners’ validation is this report’s global analysis, lessons learned and recommendations. Since no attempt has been made to prepare case studies --law school by law school, validation is not about the extent to which the report did/did not take into account individual interviews or every piece of evidence supplied to the evaluation team. US-Colombia Partnership Program – Evaluation report Page xi g. Sustainability of the initiative is crucial and should be envisaged from the onset of the Program. 3. Regarding the M&E system: a. Monitoring indicators should be discussed and agreed during the planning phase; b. Familiarity and acceptance of the costs and benefits of results-based management framework, including M&E, should be secured before the Program starts; c. Program goals can be adjusted over time –particularly in risky programs -, as long as new goals are negotiated and their monitoring and evaluation is agreed upon; d. Monitoring should be based upon pre-defined implementation trajectory –not just result matrices. e. Domestic partners need to be actively engaged in analytical monitoring, conclusions and subsequent commitments; f. Simple tasks need to be reported and analyzed in terms of their contribution to Program outcomes. 4. Regarding roles and responsibilities: a. Comparative advantages of domestic partners should be identified at the beginning and be included as an essential component of the partnership. b. Flexibility and attention to the needs of the partners by US HEIs are key factors to favor a successful relationship; c. Incentives for each participant need to be explicitly listed and accounted for. This is an essential part of any successful partnership; d. External support by an expert partnership formation and facilitator is desirable. Partnerships are neither processes nor results that can be left to initial commitment or Program inertia. 5. Regarding universities’ strategy to strengthen human rights training: a. The work on the curriculum is essential but might require time. Alternative strategies should be encountered to pave the way to the required curricular modification. b. Before launching any outreach program (for example, in communities, in schools, in prisons), universities should ensure that: (i) the activities respond to the needs of the targeted population, and are prepared with them; (ii) professors and students have the sufficient knowledge to conduct the proposed activities (especially knowledge of popular education); (iii) the universities are aware of the work that might have been initiated by other entities in the same areas (such as NGOs or public institutions); (iv) the universities do not enter to compete with NGOs but complement their work. c. On the same token, it is essential that universities maintain a constructive dialog with the NGOs and civil society organizations in general in order to learn from their experiences, coordinate their initiatives, when possible work together, and avoid, to the extent possible, to enter into competition with one another. US-Colombia Partnership Program – Evaluation report Page 1 2. Introduction Higher Education for Development (HED) is a Program of the American Council on Education (ACE) aimed at managing higher education partnerships that address global development challenges. HED is funded by a cooperative agreement with USAID, which provides the mechanisms through which USAID missions can use HED Program to manage the grants of higher education partnerships. Through the partnerships, United States (US) higher education institutions (HEIs) enter in partnership with peer entities in developing countries to improve their capacity to contribute to local and national development goals. The Colombia-US Human Rights Law School Partnership Program (hereinafter “the Program”) is a three-year initiative implemented in Colombia. It is aimed at improving human rights education and training in law schools, and promoting a culture of acceptance and respect for human rights. Funded by the US Agency for International Development Mission in Colombia (USAID/Colombia), it supports three partnerships: in Antioquia, Valle de Cauca and in the Caribbean Coast. The Program has been managed, closely supported and monitored by HED. HED foresaw the realization of a final evaluation during the last semester of the Program execution, which concludes in the field on June 30, 2015. HED appointed three external consultants for this purpose. The evaluation was conducted between March and June 2015. Objective of the evaluation According to its Scope of Work (SOW), the evaluation is to generate knowledge about how higher education partnership could contribute to addressing the needs of groups that are disproportionally affected by human rights violations in Colombia. To this end, the Program was analyzed according to the following criteria: (i) the relevance of its design; (ii) the efficiency of its management processes and systems; (iii) the effectiveness of its implementation; (iv) its results; and (v) the potential for its sustainability. At the request of HED, specific attention was drawn on the results achieved under Program Objective 2 (To enhance the outreach capacity of regional law schools to better serve vulnerable populations with limited access to or knowledge of the legal system through the creation of cross-regional law school human rights networks). Regarding the management efficiency, the evaluation team focused on the quality of performance management process, including management efficiency where there was enough US-Colombia Partnership Program – Evaluation report Page 2 information for cost/effectiveness analysis, and the use of monitoring progress and intermediate evaluation to secure achievement of development goals through better utilization of available resources. By primarily addressing management assessment through the lens of results￾framework (as proposed in the SOW), the evaluation team overcame the lack of information for cost-benefit analysis strictu sensu3 while expanding the efficiency analysis to more relevant Program management issues. Evaluation methodology Approaches of the evaluation The evaluation is approached from a comprehensive and systemic performance programming perspective and a prospective stance. Comprehensive and systematic approach As outlined in the Internal Mid-Term Evaluation SOW, the Program is “specifically designed and is being implemented around the Program-level results framework”. This evaluation thus considers the Program from a comprehensive and systemic performance programming perspective, as having one overarching development goal, several components and intermediate results as well as synergies among components, and lower level products, processes and activities. This evaluation approach thus gives more weight to those Program features that have contributed to its overarching objective. The comprehensiveness of the approach entails (i) considering the Program as a whole, as opposed to considering the three partnerships in isolation; (ii) whenever possible, doing intra￾and inter-partnership analysis; and (iii) examining how the Program strategically took into account the context where it took place. The latter has to do with the way the Program capitalized on strengths and opportunities while minimizing the impact of context-restrictions and mitigating the risks. This analysis includes the particular review of each partnership in terms of design, management and results, taking into account the institutional and contextual differences existing between regions and universities. This analysis is developed extensively in annex 3. It also entails comparison of the way each partnership capitalized on its strengths to contribute to the Program’s development goal while securing relevance to and appropriation by each participating school. The main challenge of the evaluation is to account for the major structural achievements of the Program in terms of its development goal, i.e. how the Program contributes to the strengthening of the human rights culture and social changes in Colombia to create a democracy more inclusive and respectful of the rights of its citizens. Therefore, the present analysis keeps in mind at all times how leadership, activities, processes and intermediate results play a part in this 3 Which would have required specialized activity-product-result-benefits costing and comparative cost￾benchmarking. US-Colombia Partnership Program – Evaluation report Page 3 overarching goal. In assessing the results achieved, consideration is also made of (i) the changes in individual learners; (ii) the changes in the work of learners’ organizations; and (iii) the changes in the state or condition of beneficiaries of the work, and in the broader community. Analysis to construct learning Evaluations ask simple questions: What are the results of the work? What is the contribution of the institution to that result? What does work and what does not? Why? What can be changed? It offers an institution an opportunity for improvement through the identification of innovations, lessons learned and good practices. Although the Program is ending in June 2015, the evaluation had a prospective goal with the view of gathering knowledge for future programs of similar nature. It is oriented at (i) identifying key results, best practices and lessons learned; and (ii) formulating general and specific recommendations should there be, in the future, supplementary or similar programs, either in Colombia or elsewhere. Principles included in the analysis The evaluation placed special emphasis on two main perspectives: i) the human rights-based approach, with particular emphasis on the principle of participation of stakeholders; ii) the gender perspective. Data collection methods Data and information were collected through two main methods: (1) a desk review of materials, which provided valuable information resulting from meticulous design recording and monitoring process; (2) semi-structured interviews carried out in the regions where the Program was implemented as well as in the US, with the persons in charge of the Program and Program Quality and Impact Team at HED, and with US participating universities. The evaluation team also discussed preliminary findings with USAID/Colombia officials. Desk review of material The Program has been meticulously recorded and reported on during planning and implementation phases. From the beginning of the evaluation exercise, HED put at the disposal of the evaluation team detailed and organized material on the Program’s design, management, monitoring and results. These documents provided helpful project concept background and quantitative data (resulting from the reporting of results as well as from the survey carried out during HED mid-term assessment). Initial assessment reports, regular monitoring reports, and meeting notes from the three partnerships completed them from a qualitative perspective. The review of these documents allowed the evaluation team to develop preliminary hypothesis from which to prepare a detailed work plan, with emphasis on information gathering and hypothesis testing while in the field (see “Inception report”, April 2015). Semi-structured interviews and field visits The field visits to the three regions where the Program was implemented took place between April 23 and May 19, 2015. The evaluation team was divided into two sub-groups of two US-Colombia Partnership Program – Evaluation report Page 4 persons: one sub-team conducted the visit in Antioquia (April 23-30), while the other went to Cali (May 4 to 7), Santa Marta (May 8 to 11) and Barranquilla (May 11 to 14). The main objective of these visits was to collect first-hand information to complement the data obtained through the desk review. Individual and group conversations were conducted according to a semi-structured interview method, based on the questions previously identified (see question grid in Annex 1). The triangulation of the information allowed perceiving similarities and/or discrepancies of the information. More than 125 interviews were carried out, in the three regions, in Bogotá and in the US (see list of the persons interviewed in Annex 2). Information was obtained from a wide range of stakeholders, directly involved in the Program or external to it:  partnerships’ coordinators;  deans of the law schools; 4  faculty involved in the partnerships;  students that benefitted from the partnerships;  individuals from the communities where the partnerships carried out outreach activities;  administrators of the universities part of the partnerships;  academics, external to the partnerships;  NGOs and other civil society members;  leaders of law faculty networks focused on impact-driven law school community services;  experts on human rights legal teaching, action-oriented legal research and outreach services. Additionally, the two sub-groups of the evaluation team met separately with USAID/Colombia (Office of Democracy and Human Rights) and HED coordinator in Bogotá. One focal group was organized in Antioquia5 with the coordination team (which consists of the coordinators of the Antioquia partnership and the universities’ focal points), following the method defined in the inception report. The discussion enabled to observe the group dynamic (what is said, what is silenced), and to take into consideration the priority given by the group to some issues. Positive feedbacks were obtained from some of the participants in the focal group: According to them, it gave them the opportunity to discuss issues they have not had the opportunity to reflect on collectively. In Antioquia, the evaluation team accompanied the Universidad Católica del Oriente (UCO) faculty and students to a prison visit to the municipal detention center of the city of Rionegro. It gave the evaluation team the opportunities to grasp the challenges students face while doing this sort of outreach activities. The team also visited Universidad de Antioquia (UDEA) section in Sonsón, where it met with students recipient of the courses provided through the partnership.6 They described their work in the Victims’ Attention Center (Centro de Atención a Víctimas), 4 Seven (out of eight) law school deans were interviewed. 5 Complicated logistical arrangements and limited time in the field prevented the evaluation team from having focus groups in Valle del Cauca and in the Caribbean Coast. 6 This visit was particularly relevant since it was the first time the students were interviewed in the framework of the Program’s monitoring. US-Colombia Partnership Program – Evaluation report Page 5 established in the municipality, and with the resettled community of Arrayán, in the neighboring municipality of Nariño. In Valle del Cauca, the team met with members of the Cali archdiocese to obtain information with regards to the regional law schools outreach projects, in particular of the Pontificia Universidad Javeriana de Cali (PUJC) and Universidad Santiago de Cali (USC). In the Atlantic Coast, Universidad del Norte (UN or Uninorte) organized meetings with different stakeholders, NGOs, and community leaders to discuss the role the Center of Humans Rights and of the partnership play in the Caribbean Coast. In addition, the evaluation team met with community leaders, students, the law school dean and UN staff. In Universidad del Magdalena (UM), the team met with the Dean of the law school, UM Program Coordinator, its Department of Anthropology Director and many students associated with the Program. Finally, in the US, the partnerships’ directors in the three American universities involved (University of Minnesota – UMN-, American University –AU-, and the University of Florida – UF-) were also asked to provide their analysis. Valuable inputs were also obtained from two separate meetings organized with HED, one with the Program Quality and Impact team the other with the Colombia Program coordinator, and meetings with HED representative based in Colombia. Despite the busy end-of-the-semester agenda and general workload, partners and other interviewees showed great disposition for sometimes-long interviews. These interviews gave rise to very honest and constructive reflections and dialog. The evaluation team greatly appreciated the disposition of all the interlocutors and the quality of their contributions. Without those, the analysis would have been partial. It also very much valued the work that HED and universities’ staff realized to allow for a smooth development of the evaluation activities. Limitations encountered during the evaluation In general, the evaluation team benefitted from a cordial reception and collaboration from partners in Colombia and in the US. It did not encounter very worrisome limitations, more so some minor inconveniences –more related to logistical coordination under severe time constraints than anything else. 7 Certain tiredness was perceivable among the partners with regards to the Program’s monitoring and evaluation processes,8 reinforced by the sensation that reporting requirements had not necessarily benefited the Program’s higher-level goals or the school’s vision. As a result, it was sensed that some interlocutors were reluctant to apart the time for the interview. Yet, interviewees recognized that the external evaluation was tackling issues they had not reflected on before. Linked to the prior fact is certainly the insufficient knowledge and understanding by the partners in the field of project planning, monitoring and evaluation techniques. This situation might have 7 For example, competing agenda or absence of the main interlocutors obliged the evaluation team to last￾minute meetings reorganization 8 In fact, the field visit followed HED internal final evaluation by only few weeks. US-Colombia Partnership Program – Evaluation report Page 6 entailed certain incomprehension of the evaluation reiterative interview practices, data collection methods and exceptional last minute requests to locate new interlocutors. A translation into Spanish of the evaluation methodology might have helped to overcome this inconvenience.9 Presentation of the report The report is divided into four parts. After a brief description of the Program and of the three partnerships, the analysis of the findings is presented according to the five evaluation criteria defined in the SOW: relevance, efficiency, effectiveness, results and sustainability. The Program is reviewed from a comprehensive perspective. The evaluation is based on a four￾step analysis: (i) the analysis of the Program design; (ii) the analysis of the intervention logic, including the examination of the strategy logic model, the thematic areas of intervention, the channels chosen for the intervention, the choice of the partners and of the supported projects; (iii) the analysis of the contribution of the intervention: It focuses on the results, achievements and quality of results-information; (iv) lessons learned and best practices. The characteristics and results of the Program cannot be understood without a look at each of the three partnerships, which ended to be very distinctive one from the other. For this reason, the three partnerships are analyzed separately. For space limitation, this analysis is presented in Annex 3. In the third part, drawing from the lessons learned provided by the implementation of the Program, the conclusions intend to provide reflective elements to answer the central question of the evaluation: how higher education partnership could contribute to addressing the needs of groups that are disproportionally affected by human rights violations in Colombia. The final part offers key recommendations that Program leaders at different levels could take into consideration for the development of similar initiatives. 3. Overview of the Program and the partnerships Overview of the Program The Program is a three-and-an-half-year initiative that started in December 2011 with the signing of an Associate Award agreement between HED and USAID/Colombia. It will conclude in September 30, 2015 (in June 30, 2015 for the sub-awards). 9 As discussed later, the Colombian partners were broadly unfamiliar with results or performance driven programs. In the absence of special training and demonstration, with emphasis on clear selection of higher level goals and corresponding indicators as well as the value of measuring, reporting and analytical monitoring, the Colombian schools could not appreciate effort to objectively register and report large amounts of information –often too detailed. US-Colombia Partnership Program – Evaluation report Page 7 It is framed in the 2011-2015 USAID’s Education Strategy and contributes to its goal 2: improved ability of tertiary and workforce development Programs to generate workforce skills relevant to a country’s development goals. It also plays a part in USAID/Colombia Assistance Objective framework (in particular its objective 3: strengthened democratic and economic governance and respect for human rights), in its Human Rights Program, which focuses on enhancing the efforts of civil society and the government of Colombia to improve respect for human rights, and in its rule of law programming, which aims at promoting greater access to justice.10 The overarching purpose of the Program is to support national and local initiatives to improve respect for and protection of human rights. Its intermediate outcome is aimed at strengthening the capacities of Colombian regional law schools to provide formal and non￾formal training in human rights. The specific objectives of the Program are to: 1. Strengthen the institutional capacity of Colombian schools of law to train future legal practitioners in human rights by introducing or strengthening curriculum in human rights as well as experiential models of legal education, such as clinics and externships; 2. Enhance the outreach capacity of regional law schools to better serve vulnerable populations with limited access to or knowledge of the legal system through the creation of cross-regional law school human rights networks; and 3. Equip future legal professionals with an understanding of national and international standards of human rights and the skills to support human rights reform in Colombia. The Program places special emphasis in law schools located in conflict-prone regions of the country, and in addressing the rights of the groups disproportionally affected by human rights violations. The following diagram illustrates the Program’s overarching logic. 10 HED, Request for Application (RFA): Colombia-US Human Rights Law School Partnership Program, p. 4. US-Colombia Partnership Program – Evaluation report Page 8 The identification of the Program’s partners followed a multi-phased process. During the first semester of 2012, a “design team” of four human rights education experts from US and Colombian law schools conducted an initial institutional assessment of 17 Colombian universities in five regions of the country.11 The assessment pointed out four programmatic areas to be prioritized through the Program: (i) curriculum development; (ii) community outreach and service; (iii) faculty development; and (iv) institutional strengthening.12 On the basis of the assessment, three partnerships were formed involving eight law schools.13 A Request for Application (RFA) was then launched on May 21, 2012, to which five US universities presented their proposal. Based on the recommendations of a Peer Review, the following partnerships were chosen:14  Antioquia Partnership: University of Minnesota (UMN) with Universidad de Antioquia (UDEA), Universidad de Medellín (UDEM), Universidad Pontificia Bolivariana (UPB), and Universidad Católica de Oriente (UCO);  Valle del Cauca Partnership: American University (AU) with Universidad Santiago de Cali (USC) and Pontificia Universidad Javeriana Cali (PUJC);  Caribbean Coast Partnership: University of Florida (UF) with Universidad del Norte (UN) and Universidad del Magdalena (UM). The sub-award agreements between HED and the three US universities were signed in October 19, 2012, initiating the Program implementation at regional partnership level. During the first 11 USAID/Colombia pre-selected the 17 universities that would be reviewed. 12 HED/USAID, Colombia-US Human Rights Law Partnership, Mid-term evaluation report, June 17, 2014, pp. 1 and 2. 13 Initially, the Program was considered for five Colombian universities. 14 It is important to note that USAID/Colombia made the final selection of the universities that would be involved. HIGHER LEVEL OBJECTIVE /FINAL OUTCOME Prevent and remedy human rights violations OUTCOME 1 Strengthen institutional capacity for human rights legal educacion OUTCOME 2 Increase law school outreach capacity OUTCOME 3 Equip future legal professionals with human rights training INTERMEDIATE OBJECTIVE Strengthen institutional capacity to provide formal and non-formal training in HR US-Colombia Partnership Program – Evaluation report Page 9 months, each of the US universities reviewed with their corresponding partners in Colombia their planning, and established the required baseline and indicators. Within the framework of the sub-award agreements, each partnership had the autonomy to decide on the priorities and implementation mechanisms, and to orientate its work. HED/Washington and its representative in Colombia closely monitored the implementation of the Program,15 ensured the administrative support the three partnerships required, and organized inter-partnerships meetings. An internal mid-term assessment was conducted in May 2014 that pinpointed key results and challenges of the three partnerships development. It was followed in 2015 by a final internal evaluation. Overview of each partnership Antioquia partnership The Antioquia partnership is a complex partnership, which associates the UMN with four Antioquia universities (three in Medellin and one in the nearby sub-region of Oriente), different in size, reputation, capacity and visible impact on public policy issues. The sub-award agreement was signed between ACE/HED and the UMN for a total sum of US $ 1,250,000 with a cost share of US $670, 196. 16 After the 90-day initial phase, the UMN and its Antioquia partners agreed on defining their partnership’s objective as: to strengthen the capacities of the Antioquia law schools to teach, research, and provide legal representation toward the promotion of international human rights and the rule of law in Colombia. Specifically, the partnership is aimed at: (1) Strengthening the institutional capacity of the Antioquia law schools to train future legal practitioners in human rights by expanding the curriculum and developing faculty expertise in human rights and the rule of law; (2) Strengthening the capacities of the Antioquia law schools to better serve vulnerable populations in the areas of legal services and human rights litigation; (3) Enabling students in Antioquia law school consortium to be better prepared to protect human rights in Colombia. Many activities have been carried out under the partnership. Under objective 1, they have included short courses on specific human rights issues, faculty externships and visits to the UMN, the enrollment of faculty in UDEA Master in Law Program, the establishment of a curriculum committee focused on human rights to review and improve curriculum. Amongst objective 2 main achievements are the creation of a law clinic in UCO, and the institutional strengthening of UDEM legal clinic. Additionally, joint litigation cases, the creation of a clinical network between the universities, and outreach activities in prisons and marginalized communities are also reported. Under objective 3, short courses, externships for students in the 15 The Program followed the same M&E system HED has developed for all its projects and programs. 16 Sub-award agreement between American Council on Education and Regents of the University of Minnesota for the Higher Education for Development Program, 19 October 2012, Attachment B. US-Colombia Partnership Program – Evaluation report Page 10 UMN, and internship participation in the Inter-American Commission on Human Rights count amongst the experiential learning opportunities provided to the students by the partnership. HED mid-term evaluation highlighted the relevance of the Antioquia partnership, with a special emphasis on its objective 2, and underscored its progress towards output level results. It remarked though that the relatively limited engagement with external stakeholders reflected that the partnership had concentrated its efforts towards strengthening the relationships among the partner institutions. The progress made in the articulation between very distinctive institutions was featured as a key element to gain visibility in the region, and ensure the sustainability of the partnership objective. The mid-term assessment especially recommended the partnership to: (1) Increase the number and diversity of experiential learning opportunities for students and faculty; (2) Advance work on curricular development and revisions at the four Antioquia partner law schools; (3) Promote human rights as a cross-cutting theme to broaden the partnership’s scope within and across the partner institutions and engage more faculty and students; (4) Further engage with external stakeholders and developing more formal partnerships with local, regional, and international universities and organizations in support of human rights initiatives; (5) Introduce alternative ways of communicating human rights principles to the community, such as through the development of informational materials and more diverse outreach activities; (6) Develop a sustainability action plan. Valle del Cauca partnership The Valle del Cauca partnership comprises two private universities,17 with funding distributed in equal terms between the two of them. The sub-award agreement was signed between ACE/HED and American University for a total of US $1,000,000 with a cost share of US $153, 848. The partnership in Valle del Cauca is focused on empowering the institutions and capacity of those communities most affected by conflict, and most vulnerable to human rights violations. 17 Universidad Javeriana Cali’s law program was established in 2001. While many students from high-income families attend the university, it also provides few scholarships for students from marginalized populations. It also offered a graduate program in Human Rights and the Culture of Peace. La Universidad Santiago de Cali (USC) is a non-profit private academic institution founded in 1958. The law school primarily serves middle to low income students in Cali. Approximately 50 percent of the student body is Afro-Colombian. The Santiago de Cali Law School has been known throughout the years for its special emphasis on criminal law. As highlighted during the field visit, the university has been on and off in financial crisis during the last 10 years. Financial management is centralized (even for earmarked funds for special programs such as this); decision￾making is cumbersome as it involves different authorities or departments at the university level. Active lobbying is required from deans, professors or administrative staff to get decisions across. US-Colombia Partnership Program – Evaluation report Page 11 AU, PUJC and USC agreed to work collaboratively to promote and protect human rights in Colombia on a core project designed to address the expressed needs and aspirations of the partnering institutions. PUJC indicated the following specific needs and program focus: • the development of a human rights legal clinic with expertise in submitting and litigating petitions through the Inter-American human rights system; • the establishment of a network with existing law clinics and teaching resources throughout Colombia and Latin America (especially the Red Latinoamericana de Clínicas Jurídicas); • the expansion of the impact and relevance of existing publications and the creation of an online human rights publication; • the strengthening of the Practice Law Center around LGBT issues and substantive work on displaced persons. For its part, USC indicated the following specific needs and program focus: • the development of a human rights legal clinic with expertise in submitting and litigating petitions through the Inter-American human rights system; • the enhancement of their substantive understanding and teaching focus on human rights issues pertaining to the displacement and migration due to the armed conflict context; • the strengthening of the legal English capacity of the professors and students; • the strengthening of the Practice Law Center around issues of Afro-Colombian descendants. Similarly to the consortium of Antioquia, the partners agreed to establish as general objective to strengthen the capacities of the universities law schools to teach, research, and provide legal representation towards the promotion of international human rights and the rule of law in Colombia. The mid-term evaluation highlighted the limited progress achieved in relation to each school’s specific program goals. It set out the importance of concluding the formalization of the human rights legal clinic at USC. It also recommended expanding work with external stakeholders, and develop more formal partnerships with local, regional and international universities and organizations in support for human rights initiatives. Finally, it stressed the importance of strengthening the relationship between PUJC and USC to include more opportunities for joint work and collaboration. The report recognized the institutional and strategic gaps that explain limited change in law school curriculums for the purpose of incorporating a human rights perspective across law school courses. Caribbean Coast partnership The Caribbean Coast sub-award agreement was signed between ACE/HED and the University of Florida (UF) for a total sum of US $760,000 with a cost share of US $304, 255. The primary partnership goal was the creation of the Colombian Caribbean Human Rights Center (CCHRC) jointly between UM and UN. The long-term goal for this collaboration is to expand its affiliations to other regional law schools, practitioners, and organizations. The vision for CCHRC is to become a sustainable and renowned center for building capacity in human rights among the Colombian Caribbean law schools through rigorous interdisciplinary research, excellence in formal and informal education, and effective community service with differential US-Colombia Partnership Program – Evaluation report Page 12 emphasis on serving vulnerable populations.18 During the implementation phase, the more challenging purpose of creating one common Caribbean Human Rights Center was downgraded to the simpler and more doable creation of two separate centers, one for each university. A special feature of this partnership has to do with the differences between the two selected Caribbean universities. On the one hand, the UM’s 19 mission proudly commits the university to working with the community, and has a high number of students. On the other, the UN20 is a private university with emphasis on commercial and other areas of corporate law. It does have high capacity and has incorporated performance management, monitoring and evaluation as essential parts of its management model. Another critical feature of the Caribbean partnership is the way the US partner, the UF, included most relevant units within its university in the support of Colombian law schools and partnerships: the Center for Governmental Responsibility (CGR) from the Levin College of Law (LCL) and the Center for Latin American Studies (CLAS). During the 90-day initial phase of the partnership, the UF and the Caribbean Coast partners conducted the baseline assessment (December 5-12, 2012), and agreed on defining the main objective for the partnership as follows: to strengthen the capacities of the Caribbean coast law schools toward the promotion of international human rights and the rule of law in Colombia. Many activities have been carried out under the partnership. Under objective 1, it is worth noting the commitment of the UM to open two faculty positions in the human rights area. Likewise, UN signed a cooperation agreement with the CLAS. Regarding objective 2, the UM structured, created and launched the Human Rights Center, and UM faculty defined a work plan to work for the rights of detainees in the area. Meanwhile, UN created different lines of action with the community and community leaders. Under objective 3, the broad participation of UN faculty and students at the UF programs and internships is highlighted. To a lesser extent, the UM has failed to benefit from short programs of training and education because of language difficulties. Finally, noteworthy is the visit by the deans of UM and UN to UF in 2015. HED mid-term evaluation recommended that UF and local universities conducted deliberate efforts to overcome the language barrier UM staff and students confronted, and offer incentives for collaboration among UM and UN. 18 University of Florida, Building Human Rights Capacity in the Colombian Caribbean, Gran Proposal, page 10. 19 The Universidad del Magdalena is a public higher education institution located in Santa Marta. It is one of the largest universities in the Caribbean Coast of Colombia. With six schools (Engineering, Humanities, Business, Health, Education, and Natural Sciences), the university offers approximately 21 undergraduate degrees, and has more than 9,000 students. 20 Located in Barranquilla, the Universidad del Norte was founded in 1966 by businessmen. The university encompasses eight academic divisions including Engineering, Administration, Humanities, Social Sciences, Health, Legal, Basic Sciences, and Education. In total, the university offers 22 undergraduate degrees, 69 professional specialization programs, 37 Master’s programs, and 6 doctoral programs. With 700 students, its law school emphasizes public service through mandatory courses on constitutional law, and public and private international law. The law school dean has run the program for more than 10 consecutive years with a management vision results. The faculty has mostly certified doctoral and benefits for being permanent employees of the University. US-Colombia Partnership Program – Evaluation report Page 13 4. Evaluation findings The findings of the evaluation are presented according to the criteria set forth in the SOW and the guiding questions annexed to the present report. Under each of the criteria, a systemic analysis of the Program as a whole is submitted. The findings regarding each partnership are submitted in a separate document joined in Annex 3. The report examines the main achievements of the Program to explore what were the key factors that participate, hinder or obstruct the attainment of its goal. The Program being a results-driven initiative, the same logic and the underlying premises guided the evaluation. It considers that:  Results are hierarchically organized. Lower-level results, such as activity, process or product results (outputs), have more value when connected to outcomes and higher￾level goals. Taken in isolation, lower-level results indicate formal compliance with each school/partnership contractual commitments under the program –but generally do not meet relevance, effectiveness, efficiency or sustainability criteria.  Sound results-based program design (planning relevant regional objectives, selection or partners, partnership formation) begins with optimizing goals to initially available (baseline) capacity as strengthened and improved by the program. Design also identifies risks along the program trajectory and proposes risk-mitigation action.  Setting unattainable goals is not sound program design; setting goals already achieved or achievable without program support minimizes or eliminates the expected value-added of the initiative.  Effective and efficient management monitors risks and critical steps, adjusts and secures at all times the equation between enhanced capacity (as a results of program interventions) and highest achievable goals within program restrictions. US-Colombia Partnership Program – Evaluation report Page 14 The test is to demonstrate that the Program has impacted higher education institutions (HEIs) to the point that law schools add value to the present state of respect for human rights in their region. Otherwise, the Program may be targeting the right issue but it may be considered irrelevant. When the criteria of relevance and effectiveness are combined with efficiency, the test is to demonstrate that the Program has impacted HEIs to a point proportionate to the resources used and spent in the Program. When sustainability is also added, the test is to demonstrate that some of the main contributions of the Program will remain after the Program has been closed and/or that the Program results are sufficient to commit other actors to either maintain this initiative or develop related initiatives improving human rights education and training in law schools have been improved and the regional culture of acceptance and respect for human rights has been enhanced. future legal professionals have been equipped with an understanding of national and international standards of human rights and the skills to support human rights reform in Colombia the outreach capacity of regional law schools to better serve vulnerable populations with limited access to or knowledge of the legal system has been enhanced curriculum in human rights as well as experiential models of legal education, such as clinics and externships, have been strengthened Evaluation: program design, management and evaluation supported actions, processes, products that efficiently and effectively contribute to and develop synergies between the three outcomes of the periphery so that the overarching goal of the program(at the center) is best served US-Colombia Partnership Program – Evaluation report Page 15 Relevance of the Program The concept of relevance is understood as “the extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies […] Retrospectively, the question of relevance often becomes a question as to whether the objectives of an intervention or its design are still appropriate given changed circumstances.”21 It thus includes an analysis of the adequacy of the initiative in relation to the targeted population, as well as of the design of the Program. Relevance of the objectives. The program is based on a double assumption: i) better training of lawyers in human rights will improve the respect for human rights in the country, and ii) strengthening law schools is either more result-efficient than –or at least is supplementary to— other potential uses of resources geared to enhance the human rights situation in the country. Both assumptions are taken for granted for the purposes of this evaluation. What is analyzed here is the value added by the particular approach and the channels and instruments utilized by the Program to the strengthening of human rights via law schools. It should be noticed that the Program concept restricted the geographical area of influence to the regional level –not the entire country. Undoubtedly, the Colombian context makes human rights relevant –even priority-- for Colombia’s HEI. Many of them have adopted curricular reforms to introduce human rights￾related study areas, instituted post-graduate degrees specialized in human rights, and/or created experiential opportunities for students. Improving the teaching of human rights is relevant to train generations of lawyers more knowledgeable on the issues involved and more motivated to pursue human rights related tasks. Furthermore, after years of relative silence, Colombian universities have an important role to play in human rights debates. This is particularly true in regions. Even in universities more advanced in human rights, such as UDEA, there is still room for improvement to strengthen the institutional capacity of the universities. As per the Program development concept, benefits for US Human Rights environment and/or strengthening human rights in participating US Law Schools are not actively pursued nor reported or measured. Strictly speaking, those benefits will be considered irrelevant for the Program unless the weight of the proposed benefits appear to impinge on the quality of each individual partnership. Design of the Program. As with any other program, the higher-level purpose of the Program (to support national and local initiatives to improve respect for and protection of human rights) is not entirely controlled nor can it be entirely achieved by it. Since many factors do contribute to either fostering or hindering the higher-level objective, this evaluation has made an effort to separate the net effect that can be attributed to the Program. It should also be noticed that the Program did not meet any change in the human rights context of such magnitude that the higher-level objective or the expected outcomes had to be adjusted during the life of the Program. The entire Program took place in the pro-human rights environment of the 1991 Constitution. Twenty-four years after the adoption of the 1991 21 OECD-Development Assistance Committee (DAC), Glossary of Key Terms in Evaluation and Results Based Management, 2002, http://www.oecd.org/dac/evaluation/2754804.pdf US-Colombia Partnership Program – Evaluation report Page 16 Constitution, the human rights culture has already permeated most dimensions of Colombian society to the point experts characterize the present Colombian legal system as “la constitucionalización del derecho.” This is not to say that no new human rights issues appeared between 2012 and 2015 in Colombia. In some cases, like “transitional justice” for example, some of the participating schools managed to take advantage of the Program to raise awareness and technically research those issues. But incorporating (some/most) new emerging issues is not considered essential to the success of the Program. Regarding its outcome (to strengthen the capacities of Colombian regional law schools in the outlying and frequently conflict-prone regions of the country to provide formal and informal training in human rights), the Program operated as if it had been formulated from the activities rather than focusing first on the outcome to consider all preconditions necessary to achieve it. In that sense, the activities might all contribute to the outcome, but might not be sufficient to attain it. As demonstrated by performance management experiences in different fields, the option to design the project from activities all the way to the expected result, rather than beginning from the expected result backwards until all necessary conditions, risks and risk mitigation strategies have been identified, is absolutely critical for the success of a project.22 The Program might not have been conceived as described, but the final planning process of the partnerships made it appear to be conceptualized from the activities to the outcome. One of the reasons that could explain this situation is that each of the three partnerships was developed separately according to the guidelines provided in the RFA. The US universities submitted their proposal on the basis of their know-how, usually defined in the RFA in terms of activities. So they built their respective proposal according to these activities, which corresponded to a determined result, without necessarily making sure that the results and the outcome will be reached. From expected impact to program design The Universidad del Norte is a notable exception to the Program tendency to focus on activities and products. Uninorte planned activities as a function of and with a strategic view to positioning human rights within the entire campus and across the region. Program areas and project initiatives were conceived as building blocks towards that end. Synergies between research, law courses, outreach activities and dissemination, both within and outside the university were secured to reach those higher level, overarching objectives. To strengthen the capacity of law school to train in human rights, a project needs to consider other types of essential results, activities and inputs, such as ensuring the sufficient and adequate resource and processes (in terms of faculty, budget, financial management, decision-making, 22 Cf. Boyle, Richard (2009) “Performance Reporting: Insights from International Practice” (by Head of Research, Institute of Public Administration, Dublin, Ireland). IBM Center for the Business of Government – Managing for Performance and Results Series, p 26. See in particular Appendix A. Also, Perrin, Burst (2006) Moving from Outputs to Outcomes: Practical Advice from Governments Around the World. Joint publication by the World Bank and the IBM Center for the Business of Government – Managing for Performance and Results Series (January) US-Colombia Partnership Program – Evaluation report Page 17 implementation capacity, other) to attain the goal. It requires also another time framework. The questions should have been: What is required to reach this outcome, i.e. to strengthen the capacity of HEIs to train in human rights in the regions targeted? How and why a change process will happen in a particular context? Will the chain of events lead to the expected outcome? Furthermore, a thorough analysis should have identified the necessary preconditions for the realization of the Program, as well as the risk factors that may hinder the achievement of its outcome. Otherwise, and it is what the current development of the Program shows, a disconnect will appear between the activities implemented and their impact. Many actions can be carried out without reaching the outcome, or doing it partially. Since it is human nature to focus on those issues each one of us is evaluated on, the law schools tend to focus on the activities, processes and products that are being required at the time of measuring and monitoring progress –not necessarily on the actions needed to accomplish the outcome. The problem is that the Program can only mobilize a certain amount of human, financial, management, technological of infrastructure resources. When resources are allocated to producing, registering and reporting on rather isolated individual activities, there will be no energy left for connecting and supplementing those key individual activities that guarantee the path towards expected outcomes. In that case, it becomes necessary to filter down those activities that are essentially required for the expected outcome while minimizing those that are not –regardless of the intrinsic merits of each individual activity. A review of the outcome and the activities is required to ensure their causal relationship. As explained by UF, they made sure that each of the initiatives presented by their Colombia partners fit into the result and the outcome. This explains the reason the partnership has been successful to maintain the course of the Program. In the field of human rights, a challenge remains to match the need for broad, long-term action with short-term, micro-focused funding cycles. One avenue is to consider the changes as both “products” and processes. But even then, activities need to be geared towards the realization of concrete results. In the end, performance evaluation requires evidence of robust indicators of progress towards broad or long-term goals. Strengthening the capacity of a law school implies moving it from the lower level capacity A to higher level capacity B. Since baselines showed high level of differentiation among participating law schools, the path towards and the expected level of achievement of the higher level purpose varies among participating law schools/regional partnerships. In some cases, sustainable institutional capacity strengthening of some sort was a significant value added by the Program – regardless of the immediate impact of the Program on regional human rights culture. Still, this implicit substitution of the higher-level development objective by an originally unintended lower￾level objective is generally graded with a lower mark in this assessment. Such problem was probably due to the selection of law schools and/or the selection of goals for a given partnership during the design phase of the Program. Nonetheless it remains an inconsistency between the Goals and Objectives of the Results Framework and the actual capabilities of the selected law school –even when enhanced capacity thanks to the Program’s contributions. Finally, the Program was evidently conceived to pursue its overarching goals through three distinct and separate partnerships, not as a single, unified set of actions towards a common goal - US-Colombia Partnership Program – Evaluation report Page 18 -though mention is made of the achievement of the overall Program as such.23 Based on the proposal submitted by the US universities and modified during the 90-day initial phase with their Colombian counterparts, each partnership drew its own sub-program, defining its objectives, outcomes and outputs. No document put all the three sub-programs together to analyze the coherence of their logic to the Program framework. So, each sub-program is considered in a separate manner. The results of the Program have been reported accordingly, i.e., according to each partnership’s results. HED’s coordination (in Washington and in Bogota) mostly focuses on ensuring the oversight of the Program’s implementation by each partnership without analyzing the overall, encompassing achievement of the Program. Duration of the Program. The duration of the Program raises similar considerations. Some Program interlocutors claimed that objectives needed to be adjusted to the timing of the Program. In their view, the higher-level objectives were unattainable in the short life of Program implementation and delivery of results (in practice reduced to a 20-month execution period between the end of 2013 and the first semester of 201524). It is the view of this assessment that unsurpassable gaps between the higher-level development objective and either the duration of the Program or the actual capacity of selected schools cannot be legitimately solved by reducing the higher-level objective to whatever other objectives were achievable under the circumstances. For the purpose of this assessment, the higher-level development objective should have been preserved as the truly relevant objective; the Program management (in HED and in US universities), including monitoring, should have made every possible effort to bridge detected gaps for the success of the Program. The duration of the Program added risks at some law schools At the end of the Program, the University of Magdalena appears to be requesting some additional time to reach some of the Program objectives. The public nature of the university slowed down some Program processes while internal management and administrative changes delayed its implementation. Towards the end of the Program, the renovated management of the law school, within a reinvigorated university, was mobilizing at high speed to make the most out of the Program. Yet such preparedness might have come a bit late vis-à-vis the timetable of the Program. As a matter of fact, as sketched above and further discussed below, at least one school kept in mind all along and largely demonstrated attainment of the higher-level development objective for the Program. It then raises questions about either selection of those schools / partnerships that did not reach the development objective or the Program management that did not bring other schools to a level of commitment and capacity similar to that of the high achiever. Indeed, some of those “unsurpassable” constraints could and should have been dealt with in the design phase of the Program, for instance at the time of school selection and strategic partnership formation. Adjustments to the Program may indeed be introduced at the time of project implementation –but there is only so much management can do to improve the design of a program through analytical monitoring and corrections of program deviations. 23 See for example, Colombia Kick-off meeting notes: “Emphasis on facilitating collaboration among the partners and partnerships to work toward overall Program objectives.” 24 Furthermore, internal constraints at some of the participating schools reduced such effective period to a year or less. US-Colombia Partnership Program – Evaluation report Page 19 Institutional arrangements. Institutional arrangements, such as partnerships, need to fit the perceived needs or opportunities of the participating institutions. Such perception can be reinforced at the time of program negotiations with sponsors and partners. Since partnerships are living creatures, those perceptions can also be strengthened during the program execution or even when planning sustainability post-program. In any case, it is generally considered essential that all parties in a partnership identify jointly produced benefits as something none of the individual parties can achieve by itself. In the absence of those perceptions, partnerships are usually perceived as irrelevant, and it becomes oftentimes necessary to devote program resources to actively foster understanding of potential benefits for every party involved. On the other hand, efforts to promote partnership during the life of a program run the risk of introducing an additional objective at a later point in the design –creation and consolidation of partnerships tend to become an end in itself, supplementary to yet different from other program objectives. The overall logic of the program is weakened by competing objectives between, for instance, fostering a partnership at the cost of reducing potential impact of the program at/by an individual law school. Antioquia offers a good example of this dynamic. The partnership of the four universities did not result from their own initiative. It came as a condition of the Program. Various factors have contributed to its overall success as a partnership: among them, the dynamism of the persons involved in each university, the coordination team put into place in Antioquia, the joint administration of resources. This success has its downside: the activities of the partnership in itself have overshadowed the results and the remaining challenges in each university taking individually. Imposed partnerships are contrary to the voluntary nature of these institutional arrangements and are generally seen as counterproductive for the purpose of reaching higher-level program goals. If a party to the partnership does not clearly perceive benefits other than financial resources for its participation, such an arrangement matches better the model of (traditional) hierarchical technical assistance than the partnership or the partnership model. Efforts to develop/maintain a mix-model (partnership, hierarchical) is usually confusing and unsustainable: the senior party is not truly accountable for creating an equal level or developing initiatives of shared responsibility; neither is it accountable for engagement and commitment towards the creation of a sustainable partnership with junior partners. At the same time, the expectation of the junior party is to have an equal partner in terms of individual benefits, contributions and responsibilities, accountability, etc. This dual perception can of course be amended by either developing joint initiatives of mutual interest and benefit. Reducing the hierarchical role of the senior party can also alleviate the confusion. In the US-Colombia partnership program, the choice of having Colombian universities work in partnership did not come from an analysis of its feasibility or a request from the interested parties. It was set as a pre-condition for the implementation of the Program. Some conditions favored positive results in Antioquia, not so in the other two regions. Gender perspective in the Program design. In the different design documents of the Program, mentions were made to women and LGBTI persons as target populations. Allusion was also made to the differential approaches that should be adopted, and some indicators were sex-disaggregated. However, these elements do not give a gender perspective to the initiative. Explicit analysis and programming would have ensured that gender equality be considered in the US-Colombia Partnership Program – Evaluation report Page 20 Program goals and its implementation steps, and would have avoided to be transformed into isolated actions. Efficiency of the Program Efficiency is understood as “the measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.”25 It encompasses the analysis of the means put into place to guarantee that the project be carried out with coherence, expertise and organization. The criterion of efficiency refers to a ratio between costs and results or expected benefits. There is efficiency in allocation when a particular use of resources is reasonably expected to contribute more to the development goal than alternative uses of the same resources. The guiding efficiency questions for the purpose of this report are: were Program resources used in an efficient way? Could have they been used in a more efficient way? Of particular relevance for this analysis is the choice of the law schools as critical resource, the partnership as the operational model, as well as the functioning and implications of the monitoring and evaluation (M&E) system. The choice of the law schools as critical resource for the Program. The most critical resource for the success of the Program was the participating law schools. Did the selection of law schools created the best potential for efficiency in allocation and production of the same amount of resources? The selection of Colombian law schools appeared to have been made on the basis of the assessment of each school’s commitment to the specific objectives of the Program: (i) curriculum development; (ii) community outreach and service; (iii) faculty development; and (iv) institutional strengthening.26 The selection encompassed the combination of public and private universities,27 big and small education centers,28 institutions with different political vision,29 mission, areas of emphasis,30 and diverse types of student bodies. 31 It was also clear from the beginning that the potential leading role of some HEIs was considered as a motor for the partnerships.32 The final selection of the grantee universities was based on the Initial Institutional Assessment Report and made by USAID/Colombia. According to these criteria, the choice of the targeted education centers was sound. It encompasses HEIs that do have a great potential for improvement, and have the capacity to 25 OECD-Development Assistance Committee (DAC), Glossary of Key Terms in Evaluation and Results Based Management, 2002, http://www.oecd.org/dac/evaluation/2754804.pdf 26 HED/USAID, Colombia-US Human Rights Law Partnership, Mid-term evaluation report, June 17, 2014, pp. 1 and 2. 27 Except in Cali where both universities are private. 28 For example, in Antioquia, the UDEA, established in 1804, has 1,400 law students in its branch in Medellin, while UCO has currently about 400 law students. 29 For example, the “widely different ideologies” between UDEA and UDEM was highlighted. 30 For instance, UN has a clear emphasis on business law. 31 For example, UPJC gathers an elite population while USC concentrates students coming from a more modest economic background. 32 For example, UPJC in Valle, and UN in the Caribbean Coast appear to have enough institutional capacity to take advantage of the Program. US-Colombia Partnership Program – Evaluation report Page 21 impact in their region. A more daring selection might have also included universities in more remote areas, like Chocó, but might not have been recommendable for a first attempt. However, other aspects, critical for the implementation of the Program, appear to not have been sufficiently assessed nor have given significant weight to:  the institutional capacity and willingness to implement the commitments;33  demonstrated achievements and commitment regarding the overarching goal of the Program, especially concerning structural changes, such as curriculum reforms;  commitment to combine actions and tools in order to create synergies among the three specific Program objectives;  demonstrated experience and/or commitment to work together in partnership or otherwise capacity to work in an partnership;  correspondence or harmony between each school’s distinctive mandate or niche and/or present strategic objectives, on the one hand, and the program specific objectives. This is particularly important for large universities with long planning and programming cycles. Some considerations of these elements were made in the Initial Institutional Assessment Report, but a more careful attention to them would have certainly allowed assessing some of the universities differently, appraise and foresee potential risks, and develop a mitigation strategy. The selection was made following the exploratory visit. However, the chosen Colombian universities did not participate in the early conception of the Program. It entails that, after the definition was made about the Program’s possible lines of action (as described in the Initial Institutional Assessment Report and reproduced in the RFA), these universities were not consulted about the concrete feasibility for them to be involved in such a Program. Their active participation would have certainly contributed to better evaluate their definite suitability (from an administrative, political and substantial points of view), and the possibility for sustained efforts (essential considering the overarching objectives of the Program). Finally, it does not seem that the US universities that eventually got involved in the Program were fully aware of the potential risk factors. Since they did not have the possibility to acquaint with their prospective partners before presenting their initial proposal, they could not include mitigation strategies in it. The 90-day initial phase of the Program did not suffice to palliate the situation: the selection was already made, and the essential action lines identified. Moreover, some of this time was consumed establishing the basic programming documents and indicators, and in some cases, already trying to mitigate some of the negative risks of the partnership.34 The process for the selection as well as for the drafting of the proposals did not enable a genuine participation of the partners or a meticulous analysis of the universities as the main Program resource. 33 For example, the personnel assigned by some universities to work in the Antioquia Partnership was insufficient to accomplish the partnership’s objectives. Possible reticence to work with a US university was not fully analyzed either. 34 It is clear that in Antioquia most of the 90-day period was spent defining the baseline and establishing trust between UMN and its Colombian partners. US-Colombia Partnership Program – Evaluation report Page 22 The choice of working in partnerships to ensure major efficiency. The Initial Institutional Assessment Report explored multiple scenarios of partnerships between the US and the Colombian universities, with their positive and negative implications: “one to consortium”, “one to several” or “one to one”. Finally, working in consortium/partnership was opted for in all the regions by USAID: in Antioquia, the US university would work with a partnership of four universities, administratively led by UDEM; in each, the Caribbean Coast and in Valle del Cauca, the two Colombian universities would work together with their corresponding US partners. Nonetheless, the universities in the Caribbean Coast were not ready for a partnership between themselves, and, in the case of Valle del Cauca, the Program did not demand a real partnership approach between the local universities and AU –nor did the US law school made the extra￾effort to stimulate such collective approach. Partnerships were perceived as coverage and quality leverage to secure its expected regional impact of the Program. There appears to be some –yet no sufficient-- international evidence to back the claim that partnership formation is a powerful tool to strengthen human rights institutional capacity, human resources or outreach services of law schools.35 Furthermore, it also responds to a practical concern: for the US universities, working with two to four universities at the same time required a certain level of coordination and collaboration, to avoid duplication and enhance the efficiency of the partnership. However, since neither Colombian law schools nor, more broadly, HEIs have much of an associative tradition to strengthen capacity and reach higher goals, the inclusion of the partnership as a necessary element of the Program design had, at its onset, no clear relevance for most of the Colombian partners. The basic requirements for partnership negotiation, impact and sustainability involve: i) common interest; ii) transparent accounting of costs and benefits; and iii) an external facilitator. They are represented in the following diagram:36 35 There is sufficient evidence of the value of partnership and networking for the protection of human rights among Latin American NGOs, either from the same country or among countries. There is also a positive experience with networking among Latin American legal clinics –which has taken roots and is dynamically operating in Colombia. 36 Cf. Castells, M. (2009). The rise of the network society, Wiley-Blackwell. Emirbayer, M. and J. Goodwin (1994). 'Network analysis, culture and the problem of agency', The American Journal of Sociology, 99, pp. 1411-1454. Entwistle, T., G. Bristow, F. Hines, S. Donaldson and S. Martin (2007). 'The dysfunctions of markets, hierarchies and networks in the meta-governance of partnership', Urban Studies, 44, pp. 63-79. Faulconbridge, J. (2008). 'Exploring the Role of Professional Associations in Collective Learning in London and New York's Advertising and Law Professional Service Firm Clusters', SSRN eLibrary. Granovetter, M. (1983). 'The strength of weak ties: A network theory revisited', Sociological Theory, 1, pp. 201-233.Mische, A. (2003). Cross-talk in movements: Reconceiving the culture-network link. In: M. Diani and D. McAdam (eds.), Social movements and networks: Relational approaches to collective action. pp. 258–80. Oxford (UK): Oxford Univesity Press Newman, M., A. Barabasi and D. Watts (2006). The structure and dynamics of networks, Princeton Univ Press.Nonaka, I. (1994). 'A dynamic theory of organizational knowledge creation', Organization Science, 5, pp. 14- 37.Nonaka, I., R. Toyama and N. Konno (2000). 'SECI, Ba and leadership: A unified model of dynamic knowledge creation', Long Range Planning, 33, pp. 5-34. US-Colombia Partnership Program – Evaluation report Page 23 The particular interests or goals of the partnerships, their operative conditions, choices and implications are not explicit in the design documents. For instance, given overwhelming ex-post evidence of the lack of conditions for partnership formation between AU and PUJC, on the one hand, and USC, on the other, it comes as a surprise that “the three agreed to work collaboratively to promote and protect human rights in Colombia on a core project designed to address the expressed needs and desires of the partnering institutions.” The Caribbean Coast partnership does not appear to have been carefully assessed to meet the necessary conditions for a regional partnership capable of creating “the Colombian Caribbean Human Rights Center (CCHRC)” jointly between UM and UN with the support of UF. In Antioquia, the establishment of the consortium of the four universities was considered a “daring” initiative, whose results owe more to the convergence of a series of factors rather than a sound planning process. It is compatible with the nature of the partnership to produce different benefits or charge differential costs to each participating party. What is essential for the performance and sustainability of the partnership is that every party involved has a perception a positive balance of net individual benefits (including financial support) less monetary and other transaction costs. Program management can, of course, help develop such perception. In any case, efficiency ratios from joint initiatives are expected to be different from one party to the other. Indeed, engagement in a partnership is greatly helped when the mandate and objectives of participating institutions either include or assume partnership formation –as it is usually the case, for instance, with centers for Latin American Studies based at US universities. Unless program management and participating US school heavily insisted on the partnership, as in the case of UMN with the Antioquia partnership, each participating law school felt that most Common interest in a goal that cannot be equally achieved by each individual partner. Benefits are likely to be different -- yet better if each participating school clearly lists expected benefits above the table Transparent accounting of costs and benefits and facilitating individual management systems Leaders or external facilitators - Incentives for each law school´s champions of the partnership (at least equal to incentives for leading actions within each individual school) US-Colombia Partnership Program – Evaluation report Page 24 of the Program specific objectives could be achieved by each individual university; at the end, there was little incentive for partnership formation. In any case, high commitment of the US partner to stimulating the domestic partnership was essentially voluntary –at least in the eyes of the Colombian schools participating in the partnership. The Colombian schools did not participate in the discussion of the RFA for the US school neither did they know of the weight that support for the partnership might have in the agreement between HED and the US partners. Moreover, the choice of working through partnership introduced high risks to the Program efficiency and effectiveness together. Those risks – and the corresponding mitigation strategies that should exist - can be summarized as follows:  the opportunity-cost risk, meaning valuable resources (financial, management, faculty, time, coordination, monitoring) had to be devoted towards partnership building rather than individually advancing the goals of the Program with each school;  the risk of adding resistance from administrations, faculty or students to the idea of working in partnership, for either ideological or practical reasons. Those two risks were partially mitigated through at least two strategies: i) common strengthening activities, which served both individual schools and partnership climate; ii) letting members of each partnership work individually as much as they chose. There is also a management, evaluation and accountability risk to the establishment of the partnership: for all practical purposes, negotiations, specific goals and actions, performance, registries and reporting were all issues pertaining primarily to the level of individual schools –not partnerships. However, the Program design often made (artificially) the partnership the key subject of action. For example, the monitoring matrix of indicators were filled in by individual schools, then added by the US law partner to construct a combined matrix at the partnership level. Such an addition entails the risk of combining in a single indicator two activities or products that appear to be the same –yet may have a very different meaning depending on which school performed the task. For instance, for the Valle partnership, the scope and meaning of apparently similar outreach community services were indeed very different between PUJC and USC. The monitoring and evaluation system. Following USAID directive, HED is guided by a strong emphasis on results-based management (RBM)37, and has developed a monitoring and evaluation system (M&E) to follow and report on all its programs worldwide, according to a RBM approach. To be efficient and contribute to the efficiency of a project or program, monitoring needs to be selective and monitoring information needs to be organized, analyzed and presented in such a way that it is relevant for the attainment of the project’s higher goals. Above all, the relevance of monitoring reports depends on the utilization of the monitoring reports to correct deviations, prevent further ones, or define new critical goals, and propose resource strengthening if needed. In the absence of a monitoring exercise with those features and capabilities, any multi-year and multi-layer program is likely to fail in reaching its higher purposes. 37 See, for example, Colombia Kick-off meeting notes. US-Colombia Partnership Program – Evaluation report Page 25 In the case of the Program, planning and reporting on activities and according to higher-level objectives was new for most of the Colombian universities, or at least for the persons involved in the Program. It introduced a different paradigm to the way they work. In hindsight, virtues and benefits were recognized to the M&E system: it forced universities to plan and organize their activities, respond for them accordingly, make their results visible, and gather relevant information on and for their work. The relevance of the monitoring exercise depends on the capacity to document design and implementation weaknesses that may have become evident soon after the school selection and the configuration of the partnerships. The first exercise consisted on the establishment of the Program’s baseline and indicators for each region. It resulted for some partners in a time￾consuming, confusing and laborious process, which required the mobilization of efforts and many human resources (HED/Washington and Colombia, US partners), delayed the launching of the Program, and created tensions between partners. In Antioquia, one of the partners defined it as “a horrible process.” Finally, figures were inserted into the system.38 This laborious process should have alerted HED on at least two different aspects. First, in view of the time consumed for the identification of the baseline, HED should have offered the partners alternative solutions to allow the activities to start, taking into account the ambition of the Program, its complexity and its short period of implementation. Second, it was a clear sign of the potential difficulties of the partners’ capacity to handle a rather complicating reporting system, which should have indicated a need to take actions about it. The usefulness of a monitoring system depends on the relevance and quality of its indicators. The partners interviewed were concerned that the indicators finally established were not able to relate to the Program achievements. Although each partnership was in charge of selecting some indicators, HED seems to have defined others (some of them seem to be more for worldwide across-project comparative purposes than for reporting results of the Colombia Program specifically) and others were USAID standards indicators. The indicators for the Program were mostly quantitative, and not considered to be necessarily appropriate to reflect the results obtained, especially when it came to non-tangible results. It is to the point that HED decided to use another approach, rather more qualitative, to carry out its last monitoring visit in April 2015. As indicated in the HED notes of the Santa Marta meeting, “the indicators established for the partnerships limit the view of the results obtained thus far.” Securing results is identifying those critical factors that account for most or all of the expected results, preferably outcomes and impact. Too many indicators end up being primarily low-level indicators –which are hard to connect with expected impact of a Program. Finally, the evaluation team could sense that some activities were carried out by the partners for the sake of showing results, “to fill in the box,” not necessarily because of their feasibility or relevance, which constitutes a perversion of the system. More information is not better when following a performance management approach to project concept and evaluation. First, producing, processing and analyzing information is expensive. Second, too many reporting requirements, to the point of micro-management, are usually a waste of valuable management time, leading to high opportunity costs and inefficiency. Furthermore, fatigue with filling in extended reports is usually one of the main explanatory factors for gaming and creaming, two of the worst enemies of performance management. 38 An interlocutor told the evaluation team that they finally ended trusting what the figures the universities provided without double-checking their validity. US-Colombia Partnership Program – Evaluation report Page 26 Almost unanimously, HED M&E system was perceived as burdensome, time-consuming and not the most adequate to reflect the changes the universities were realizing. As one of the interlocutors affirmed, “it bureaucratized the system.” Under-utilization of monitoring reports is another usual source of inefficiency when doing performance management. For example, HED internal mid-term evaluation, by pinpointing key challenges and making recommendations by partnership, provided the opportunity to reorient the Program. But, as stated by an interlocutor, it “did not put to dialog quantitative elements with the reality in the region,” and it was not sufficiently analyzed and used. It is noteworthy that HED teams in Washington and in Colombia were perceived to have contributed very positively to the management of the Program. They were attentive to the Partnership’s needs and requests (especially regarding the definition of the baseline), and eased the way of their partners in the administrative maze. The positive role of USAID/Colombia officers and their ability to understand that the M&E was a roadmap that should accommodate the needs of the universities were also highlighted. In sum, HED M&E system is a very sophisticated and solid mechanism aimed at reporting on a wide range of projects implemented worldwide, while allowing for a certain margin of flexibility according to the specificity of each of them. In Colombia, it certainly introduced organization into the work of the universities, requiring them to plan their actions and respond accordingly. Nonetheless, instead of being a useful tool for the Program implementation, the system, for its complexity and its level of exigency, was converted into a burdensome, time-consuming, and cost greedy formal demand. The system is certainly very advanced; it was simply not adapted either to the beneficiaries or to the characteristics of the Program. The financial resource. The sharing of the budget between the US universities and their Colombian partners did not result as being an issue between the partners. The cost of US universities might be a relevant question to explore but it did not appear be a subject of preoccupation. Regarding the use of the financial resource of the Program, three issues are worth noting. First, the fact that the Program finally applied to eight Colombian universities, rather than five as initially planned, necessarily affected the distribution of financial resource. It is not clear if that shift was based on a sound analysis of its consequences. The selection of fewer universities might have optimized the use of resource and contributed to a more centered implementation. Looking retrospectively, it is plausible to argue that efficiency in the Program’s resource allocation might have increased by: optimizing allocations as a function of the risks and capacity of each participating school; reducing the number of participating universities, and devoting more time and resources for planning on the basis of in-depth knowledge of each candidate/participant university. The second issue deals with the staffing of the partnerships. In Antioquia, part of the budget was allocated to pay the salary of three persons especially in charge of coordinating the partnership and supporting the legal clinic and the curricular modifications. Without any doubt, this option participated in the efficiency of the implementation of the activities there. It certainly had its downsides as well (see below) but responded to a clear vision that the partnership could not work without persons in the field especially dedicated to it. US-Colombia Partnership Program – Evaluation report Page 27 The last point concerns the contract with Gardaworld. At the beginning of the Program, USAID and HED were concerned about the security of the persons involved in it, assuming that working on human rights in Colombia might put staff, faculty and students at risk. HED contracted the Canadian-based security company Gardaworld to train Program staff, and regularly inform and assess them regarding the security situation. Interlocutors acknowledged that the training was useful to consider basic security tips (especially for foreigners, and persons not accustomed to work in conflict zones and in human rights). However, overall, this contract did not turn to be very useful for several reasons. First, being native for the regions and for their own experience, local faculty and staff usually know how to manage their security. It is true that some basic reminders and recommendations are always useful. But, second, the approach Gardaworld adopted was not adequate to deal with persons working on human rights. For example, one of its recommendations to lower the risks was “not to assume controversial positions” or “to avoid sensitive subjects.”39 To evaluate the risk of clinical work, its forms also included information on the contact persons in the community, information that cannot easily be divulged in a context like the Colombian one. 40 As a consequence, the company lost credibility with the Colombia partners, and was considered of very little use to them. Effectiveness of the Program The concept of “effectiveness” relates to the “extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.”41 It involves the analysis of the extent an initiative has attained (or is expected to attain) its objectives in an efficient and sustainable manner. The analysis of the achievement of the Program’s objectives is obviously influenced by its design. Effectiveness is usually the key criteria for program success under performance evaluation methods. When effectiveness is read in terms of the expected impact, not any result attributable to the program is proof of program impact. For instance, a program can deliver lots of activities and products, such as training seminars or growing number of students in human rights courses. Such program can, at best, be considered effective only for the money spent in courses and seminars, and for the number of those seminars or students attending courses. Such an indicator would still be far from the higher-level development objective. It is only when those seminars and courses do make a dent in the demonstrated capacity of the school/partnership to positively impact the regional human rights environment that such a result will meet the effectiveness criteria guided by the outcomes or the higher level indicators of the Program. To draw the analysis of the achievements of the Program, the interventions contributed to reach its objective, i.e. if the Colombia-US Human Rights Law School Partnership Program strengthened the capacities of regional law schools to provide formal and informal training in human rights, and through that to impact on regional human rights dynamic. What kind of activities allowed it to reach this goal? What conditions allowed for these results? 39 See for example, Guía personal de seguridad para trabajos en campo. 40 See Formato de información sobre clínicas. 41 OECD-Development Assistance Committee (DAC), Glossary of Key Terms in Evaluation and Results Based Management, 2002, http://www.oecd.org/dac/evaluation/2754804.pdf US-Colombia Partnership Program – Evaluation report Page 28 Undoubtedly, from a global analysis, the activities carried out by each of the partnerships have responded to the Program outcomes and find themselves in its logic. Each outcome can report concrete results.42 Outcome 1.  modifications in some university curriculum. Curriculum changes could be more easily attained in smaller education centers, like in UCO, than in bigger ones, where curriculum reforms respond to more complex processes. However, interesting initiative such as the constitution of inter-university curriculum committee, like in Antioquia, has proven to be useful to foster further changes. In other universities, human rights classes have been introduced as elective or as areas of concentration, such as in PUJC, UDEM, UM or USC.  institutional changes: Various initiatives can be mentioned: PUJC has guaranteed the coordinators of the human rights area more dedication and incentives to conduct research on human rights; PUJC has generated dialog with other program of the university to mainstream human rights; UM has opened two faculty positions in the human rights area. Noteworthy is the creation in UM of the human rights center as a space of debates and knowledge sharing.  the strengthening of faculty’s knowledge and capacity through short courses on specific human rights issues, faculty externships to US university, the participation of faculty in Master’s Degrees. These courses have opened them to the regional and universal human rights protection systems.  a change of paradigm in faculty and students: the inclusion of a human rights-based approach to outreach activities implemented by faculty and students in communities has brought substantial modifications, such as the inclusion of the beneficiaries or them being considered as rights-holders (this is clearly the case in UDEA and in UCO). Outcome 2.  strengthening of the legal clinics. The Program has allowed for the creation (in the case of UCO) or the strengthening of existing legal clinics,43 as well as the application of new methodologies to strategize their intervention.  more focused attention to persons in vulnerable conditions through the application of methodologies to serve victims, the publication of materials to disseminate knowledge on human rights,44 or agreements with private offices to position socio-legal work in the communities (USC).  creation of alliances with civil society actors, such as with the PUJC and UM, and to a lesser extent the Antioquia partnership (on children’s rights). Outcome 3. 42 The list provided is not exhaustive. Although the evaluation team intended to highlight the main achievements, some of them might have been omitted. 43 The Program allowed to create the UCO legal clinic, now part of the Consultorio Jurídico, and to strengthen the work of UDEM’s one. 44 For instance, UCO students have produced pamphlets destined to persons in detention on criminal procedure. US-Colombia Partnership Program – Evaluation report Page 29  the inclusion of new methods for higher-education in human rights, especially through the legal clinics. The exchange of experiences between the legal clinics in the US and in Colombia contributed to strengthening their practice. The use of other methods (such as radio stations, promotion of human rights through theater groups in UDEM or the establishment of an observatory and a web page in PUJC or in UM) is also to be noted.  the strengthening of students’ knowledge and capacity through, in particular, short courses on human rights issues, externships to US university, internships to the Inter￾American Commission on Human Rights, or the participation of students to moot court competitions. The inclusion of law students from semi-rural or rural population, such as in Sonsón and Rionegro-UCO, is also key to impact regions further from capitals. Many of the students who participated in the activities organized under the Program declared having experienced an internal transformation, and being motivated to dedicate their career to human rights or social issues.45 Additionally, the Program has showed that joint work of different HEIs on strategic issues is beneficial. For example, the partnership in Antioquia was able to present a shadow report to the Committee on the Rights of the Child and participate to its subsequent advocacy work with NGOs, as well as leading an audience on resettlement before the Inter-American Commission on Human Rights. The coordination between the universities has also benefitted the students, allowing them to participate in classes or activities organized by other HEIs and to share experience with their peers. However, although the Program foresaw to “encourage communication and substantive exchanges among the three US partners and the 8 Colombian partners,”46 very few opportunities were developed for inter-partnership relationships although identification of opportunities for communication between partnership. Some initiatives to coordinate and promote networking between the Universities have been carried out (such as the Santa Marta meeting, video conferences between the three US universities), and were received with interest. However, they have not sufficed to give the Program a global identity or to give partners ownership of the Program as a whole. Taking together, these achievements should lead to the completion of the objective of the Program. Nonetheless, not all of them were attained in each region, and not all of them were reached at the same level of execution. The Program did not force law schools to develop each one of the three areas of Program intervention in a balanced, much less a combined way.47 The relevance of the Program was almost guaranteed by a flexible Program concept where practically every school could potentially see itself in the mirror of its own interest. Some Colombian schools took advantage of the Program to reinforce the pre-existing law school focus / specialization and basis of comparative advantage. PUJC, for instance, differentiated itself based on a focus on corporate lawyer training –and was able to maintain this focus while taking advantage of the Program. The Program’s value added for PUJC did not 45 It should be noted that many of the students interviewed seemed unsure regarding what a career in human rights is, means and implies. 46 Colombia Kick-off meeting notes 47 See above discussion on combined indexes. The design and management of the partnerships were conceived individually, losing opportunities to generate alliances in both research and national incidence projects in common areas, such as on the rights of persons in detention, victims’ rights or LGTBI’s rights. US-Colombia Partnership Program – Evaluation report Page 30 consist on developing new areas of interest but consolidating their interest in competition training (including the moot court with American University) and experiential training. In turn, the Program recognized that the priority needs and the interests of each regional partnership were likely to differ from one another. Indeed, one of the Program’s strengths was its flexibility to accommodate particular interests within an overarching goal and broadly defined specific objectives. Several factors contributed to positive results within the partnerships. The approach of the US universities and their relationship with their Colombian partners played a critical role. The knowledge of the Colombian context, the capacity to adapt, their cultural sensitivity as well as their flexibility constituted key factors to adapt to the regional context and comprehend the needs of each education center and the population. The management scheme of the partnerships also contributed to better results, allowing for time and dedication that could be provided to the Program. As already underlined, Antioquia functioned with a local coordination team (integrated by professionals who know the universities very well). UF put into place a multi-disciplinary follow-up team that allowed for a closed collaboration. To the contrary, other factors played against the effectiveness of the Program in some regions or during some periods of time, such as administrative inconvenience linked to the public nature of the universities, administrative changes and staff turn-over, time required to adapt to the M&E system, and time-consuming trust-building between Colombian and US universities. In sum, the Program ended up summing up many activities, but in many instances, these activities, often scattered over sub-programs, did not necessarily entail the attainment of higher￾level objectives. In many instances, partners lost track of the latter. One exception can be mentioned. A particular success story –and the main explanatory factors of success The Universidad del Norte seems to shine above all other partners for its appreciation and capitalization of the Program as a major opportunity to consolidate its vision of university￾community interaction for learning and servicing on critical social and economic issues throughout the Caribbean Coast. Uninorte utilized every field of action to benefit from the Program: - It promoted and capitalized the willingness of UF to support the vision of Uninorte to make human rights a dominant theme across the entire campus. UF facilitated active involvement of its own Latin American Center for context analysis and human rights issues of particular interest to the Caribbean Coast and the entire country - It expanded research to some of the most visible and deeply felt cases of violations of human rights in the region - It understood and taught human rights as a comprehensive issue, including gender, One of the main success factors that increased the level of effectiveness of the Program was the level of knowledge and commitment of the US universities to adjust the activities to the logic of the Colombian universities, taking into account the local context, and thus generating strategies of institutional training and attention to the communities. US-Colombia Partnership Program – Evaluation report Page 31 The criterion of effectiveness is intimately linked with the quality of monitoring as well as management’s capacity to adjust implementation trajectory during the life of the program. As discussed below, the criteria of program effectiveness should not be lost during its implementation or conclusions. It cannot either be substituted by action or process deliverables, no matter how many products of this sort the program may have contributed to. A program’s analytical monitoring and intermediate evaluations need to differentiate between activity or process effectiveness and program effectiveness. The Program missed opportunities to foster its results. Most of the partners focused their work towards the implementation of activities, losing the sight of the outcome result. In order to guarantee the Program’s effectiveness, HED and US universities management and the implementation teams should have been guided by a prefigured implementation trajectory that oriented monitoring to the riskiest and most important steps of the Program. Such an implementation plan should also have pre-proposed risk-mitigation strategies and have allowed the partners to re-focus their work toward the outcomes. Risks would not be limited to technical risks or the availability of highly specialized skills. As demonstrated time and again in low and middle income countries, the single most important risk of an ambitious Program is management capacity, including governance, procurement, timely financing, hiring and finding the right technical skills. Those capabilities cannot be taken for granted; they should be secured from the beginning as part of the Program implementation design. 48 Leadership level commitment. ethnicity, and the environment. - It involved in the human rights program some of its most talented researchers, professors and managers - It was open to collaboration with other schools, particularly UM - It took the guiding principles of experiential training and legal clinics very seriously and developed capacity to further consolidate both - It anticipated ways to secure sustainability of the Program, particularly in partnership with UM And the factors that made all the above possible at Uninorte? Some of the key factors of success this team found: - Top level support48 - Good systems management and familiarity with results programming - Willingness to invest and commit resources of its own for the Program specific objectives –as defined by Uninorte—do correspond to the university’s vision and strategy for development - A prestigious school and high reputation in the region and elsewhere - Exposure to international experience, commitment to engaging international experts and learning from them - Level of English of students and faculty - Commitment of the law school to work with other schools, both in campus and at the UF or the UM - Full time teachers – Full time commitment of the Program’s administrative staff US-Colombia Partnership Program – Evaluation report Page 32 Results of the Program With “results” comes an analysis of the “output, outcome or impact (intended or unintended, positive and/or negative) of a development intervention.”49 Did the Program transform each partnership and/or law school to the point it is having an impact on the regional human rights environment? Some concrete and very promising results have been achieved through the Program. However, the delay incurred at the onset of the Program to establish the baseline and the indicators for each partnership, if necessary and perhaps unavoidable, led to the perception that results are just beginning to be seen –or are expected for some time later. This perception further jeopardized expected outcomes during the life of the Program. On the positive side, it protected the credibility of the Program by making it look more like the pilot-exploratory phase of a more ambitious and long-term initiative. Which assessment standard fits better this type of Program? When the magnitude of the Program’s risks is taken into account, the Program can at best be characterized as a high risk￾high impact venture. However, this characterization for the entire Program veils the fact that results and impact were very different by regional partnership and –above all—by law school. All in all, it was a risky Program with highly differentiated actors and results. In view of this overall finding, the most positive characterization that can be made of the Program is that of the first phase of an envisioned strategy towards strengthening engagement of law schools in and enhancing their impact upon human rights in Colombia. This first phase concluded with some unequal results. Such characterization of the Program design, implementation and results gives the Program an innovative/exploratory nature, the sort of risky programs created to either try new avenues or test or raise hypothesis. The key question for assessing this type of Program is “what did we learn from….”? rather than which are the Program major achievements and immediate impact. Instead of an impact evaluation of the most rigorous scientific type, assessment of this sort of Programs follows a case study like methodology. Evaluation teams are then called in not so much to assess results but to dig in through interviews and documents with a view to identifying lessons learned –even if originally unintended by the Program. As demonstrated in the previous sections of this report, this is in large part the prevailing approach followed by this evaluation team through this exercise. Under such an approach, the question of “How much/how many results is satisfactory to judge the Program a success worth extension or replication” is superseded by the selection of partial success stories (by law school, by partnership) we can draw lessons from. Even small progress, like learning to greet the first-time client at the point of entry of the legal clinic, might take precedence over ambitious attempts to reform the entire (and usually rigid) basic law curriculum. 49 OECD-Development Assistance Committee (DAC), Glossary of Key Terms in Evaluation and Results Based Management, 2002, http://www.oecd.org/dac/evaluation/2754804.pdf US-Colombia Partnership Program – Evaluation report Page 33 In consequence, this assessment proposes that one or two cases of demonstrated success by law schools/partnerships in a given criterion is sufficient to declare the Program successful. The standard adopted here is based on the consideration that its very ambitious higher development objective makes the Program highly risky. Given the state of the theory in terms of the connection between law school capacity and the culture of human rights in the region, this is indeed an exploratory Program. As it is usually the case, high risk is justified when there is a potential for high rewards. In this case, the demonstration effect of one or two successful cases provides sufficient basis for further research, adjustment and replication. This assessment already draws conclusions that can be considered lessons from this experience. Further analysis of success stories should expand the accumulated knowledge basis to reduce the risks of future projects. Universities Promising Practices Universidad Católica de Oriente Establishment and creation of a legal clinic with strategic lines of action. Universidad de Antioquia The human rights training students in conflict zones and rural impact of violence. Universidad de Medellín The multidisciplinary approach to investigations and prosecutions of the Legal Clinic. For example: electromagnetic engineering and human rights. Partnership Joint work on strategic litigation in human rights. Universities Promised Practices Universidad Santiago de Cali Strengthening partnerships with law firms that do pro bono human right practices Universidad Pontificia Javeriana Developing manuals and statutes of the Legal Clinic. Universities Promised Practices Universidad del Magdalena The inclusion of students from other faculties of human sciences in human rights training. Expanding the offer in superior studies to audiovisual programs and anthropology education. An internationally-supported and assisted human rights program that further stimulates reorganization of the law school and consolidation of a public institution. US-Colombia Partnership Program – Evaluation report Page 34 Sustainability of the Program The sustainability analysis examines the probability of the continuation of the results coming from the development of an initiative. Is the impact of this Program sustainable? Will the value added continue after the Program is closed? When monitoring is guided by a result-driven implementation trajectory, such trajectory usually includes provisions to secure sustainability. That is to say, sustainability is made part of the expected result of the Program –not a last minute addendum at the point of Program closure. The proposed type of monitoring leads to supplementary actions and propose utilization of resources with a view to secure sustainability. Sustainability is usually defined in terms of extended capacity to deliver outcomes beyond the life of the Program. In the case of partnerships, monitoring should include those essential factors that can make the partnership sustainable. In the case of the Program, sustainability was not foreseen at the onset of the Program. In fact, HED mid-term assessment requested to all the partners to establish a sustainability plan. Three elements can be sustainable: (1) the institutional systems developed to achieve the objectives; (2) the results obtained; (3) the relationships and collaboration between partners (1) Did the Program successfully institutionalize enough capacity to secure continuation after USAID resources are gone? Sustainability usually depends on relevance of project outcome and suitable institutional arrangements from the point of view of the recipient country. Are law schools committed to continue supporting (or expanding) that precise capacity that has explained achievements leading to the higher-level goal of the Program? Some participating law schools have shown their interest and compromise at pursuing the efforts developed, for instance by securing the institutionalization of legal clinics and/or financing professors to animate the clinics or train on human rights. However, institutionalizing human rights training in universities implies not only financial resources but foremost the compromise with human rights of university leaders. The involvement of the universities’ leadership is fundamental for the development and sustainability of an initiative of that sort. Working on human rights implies more than the simple opening of a study area. Reaching the goal of promoting the role of universities in the field of human rights necessarily requires the existence of an institutional political positioning as well as the involvement of all its components, in particular its faculty. The active involvement of the deans of the law school as well as the rectors of the universities is key to securing the sustainability of an institutionalization. Not all of the leaders of the universities involved have shown the same interest and motivation towards the project. (2) Did the Program produced outcomes whose benefits are guaranteed to extend beyond the closure of the Program? As we saw above, the Program has produced some interesting results, some of which might extend beyond the closure of the Program. However, it is too premature to assess their sustainability, since the Program is scarcely showing its first results. US-Colombia Partnership Program – Evaluation report Page 35 (1) Is the relationship and collaboration between the partners sustainable? Sustainability of the core concept of the Program is heavily dependent on the initial selection of participating law schools. If partnership arrangements are seen as an essential component of future institutional arrangements, the process of selection should have essentially required experience and/or evidence of commitment of selected law schools to regularly or permanently work in partnerships. If sustainability of the Program objectives includes further institutionalization of partnerships, partnership-formation should be seen as a goal by itself from the early design of the Program. Rather than taking them for granted, the Program should include strategies, resources, actions and performance measurement for the creation of the necessary pre-requisites for a partnership. In any case, the insistence on partnerships as a pre￾requisite for the Program sustainability adds substantial risks to sustainability strategies and initiatives. Likewise, if the sustainability of the Program includes further institutionalization of partnerships, the Program might have included working together with more consolidated networks, partnerships or partnerships of proven effectiveness. The existent network of legal clinics is an example of such collective action in Colombia. Sustainability of the Program would be more likely should managers let schools in Colombia and in the US find the best way to further pursue their relationship for the higher goals of the Program. Bilateral agreements between US universities and Colombian counterparts are likely to be developed in the future, which could also work to the benefit of the Program objectives. 5. Conclusions The US-Colombia Partnership Program is a well-structured initiative, carefully managed and monitored, which has produced some noticeable results. As with most other educational programs, final impact can only be judged in the medium term. For the time being, the assessment is focused on the extent to which the Program did produce the products and outcomes expected from its three-year implementation. The Program had a rather complex structure and demanding operation:  Three regional international and domestic partnerships, involving directly eleven partners that had had little or no interaction prior to the Program;  Asymmetry of context, mandates, vision, specialization, interests or opportunity costs among parties involved. Human rights are a matter of passion, debate and ongoing action in Colombia. Participant laws schools and universities, especially Colombian participants, grew up and live in an environment that calls for engagement through policy and action, participatory research, training, community services;  Huge logistical challenges in dealing with 11 institutions with different management stability and capabilities, each one with its procedures, restrictions and opportunities;  Application of a results framework for program design and program management to/by some organizations that are not familiar with performance programming or performance management;  Different baseline per partner/partnership, demanding different goals, strategies, support or resources per partner/partnership. US-Colombia Partnership Program – Evaluation report Page 36 Program strengths. The Program design reflects an overall strategy to deal with those challenges and risks through:  flexible specific objectives per partnership within a wide-open menu of areas of intervention grouped under three general categories. This scheme allowed each participant to find its own interests and define its own roles within the partnership;  common overarching goals and expected impact for a shared, unifying concept and purpose of the Program;  monitoring and intermediate evaluation tools that use a common set of indicators yet allow for customized indicators tailor-made to each individual partnership (or even partners in each partnership). The above factors taken together appear to guarantee a balance between individual creativity, on the one hand, and a unifying purpose and common format, on the other. In addition to its flexibility, the Program had some strengths that were evidently taken into account at the time of design and implementation:  it relied on HED’s demonstrated experience in fostering and operating development programs with partnerships among higher education institutions as well as HED management and monitoring capabilities;  it relied also on the demonstrated capacity to deliver of distinguished US academic institutions that applied to the Program and have selected their partnership-partner in Colombia;  the sensitivity and dynamics of the topic in Colombia practically guaranteed the relevance, commitment to effectiveness and efficiency as well as promising sustainability perspectives for the Program in Colombia. Program weaknesses. Those strengths notwithstanding, the Program did not appear to have sufficiently anticipated the potential damage of some critical risks to the quality of activities or products –not to speak of outcomes or expected impact. The design did not include either sufficient attention to risk management and risk mitigation strategies of the following factors that limited results of the Program:  The lack of a predefined linkage between specific objectives of each individual partnership and higher level goal allowed for inclusion of particular goals that had no certain substantive impact on the overarching goal of the Program. Leaving specific objectives to remain open for negotiations within each particular partnership threatened the consistency and the impact of the Program. Furthermore, there were no clear indicators for supporting the partnerships’ specific objectives. Neither was there an explicit elaboration on the causal relations between each partnership’s specific objectives and the Program overarching objectives (upward) or specific objectives and the actions leading to them (downward). Indeed, the quality of each partnership’s specific objectives differed between one partnership and the other. The Program did not include an explicit gender perspective. Isolated initiatives regarding women’s and LGBTI’s rights were implemented by some partners and gave rise to interesting results, such as those at Uninorte. In other opportunities, a gender US-Colombia Partnership Program – Evaluation report Page 37 analysis would have contributed to better outcomes or analysis. A gender perspective goes further than sex-based differentiation; it allows understanding of how a society is organized along gender roles, rules and stereotypes, and acts upon this reality. The integration of a gender perspective in a human rights Program should have been explicit from its onsets.  Given the weak connections between specific objectives and the common, unifying goals and purposes, practically any activity that pertains to the Program’s areas of intervention is justifiable. The selection of action-priorities was not necessarily done in terms of expected contribution to the Program’s impact. As evidenced in some cases, selection of actions rather followed each school’s tradition or more broadly, a position of comfort rather than creatively making the most out of the resources and guidance available for the Program. The value added of the Program to the regions or to Colombia has no longer a necessary reference for planning, reporting or monitoring.  Given those initial weaknesses in the Program design, M&E could not help much to re-focus the partnerships on the overarching goals of the Program. First, because there were neither clear criteria nor indicators to assess the contribution of processes and products to outcomes and overarching objective. Second, because the whole exercise of M&E was necessarily pre-committed to extended results-matrices, enriched by some analysis, as required by USAID. It is true that there was room for customization; but customization under those circumstances would have required a disproportionate amount of time and energy, either for formulating adequate indicators or for assessing the quality of such indicators and the causal connections between actions and impact. In practice, the focus and utilization of M&E was narrowly defined at HED and US subcontractors as having to do primarily with actions, processes and products. Such emphasis further fatigued and discouraged most Colombian partners –which did not clearly see the relevance of the exercise. The solution found was to give the US law schools the role and responsibility for organizing and assessing the data, filling in the forms and submitting to HED. Reports prepared in that manner are too distant from the concerns and interests of the Colombian schools.  The rather large number (8) of participating Colombian schools and –above all-- the selection of schools increased unnecessarily the risk of deviation from Program focus at the time of its implementation. This additional risk was compounded by the Program’s insistence not only on regional partnerships but also on making the partnership the unit of action and assessment at each region. In two countries where law schools have little precedent of and have shown no particular appetite for such partnerships, it can hardly be expected that law schools will mobilize with interest to suddenly adjust their mandates, business or management models or will quickly learn the intricacies of partnership formation, implementation, monitoring and control. Attention was deviated from bilateral path towards achieving impact at the regional level. Even achievements in terms of learning how to and developing interest in sustaining the partnership are in high risk of collapsing if you take into consideration that universities are accustomed to work individually or bilaterally. US-Colombia Partnership Program – Evaluation report Page 38  Some concrete and very promising results have been achieved through the Program. However, the delay incurred at the onset of the Program, if necessary and perhaps unavoidable, led to the perception that results are just beginning to be seen –or are expected for some time later. This perception further jeopardized expected outcomes during the life of the Program. On the positive side, it protected the credibility of the Program by making it look more like the pilot-exploratory phase of a more ambitious and long-term initiative. 6. Recommendations 1. Bearing in mind that the present report is aimed at learning from the experience, the evaluation team strongly recommends that a validation process be conducted with the Program partners, in Colombia and in the US, in order to share the conclusions contained in the present report, and receive and discuss their comments. 50 The evaluation team considers that the following recommendations could guide the implementation of future initiatives similar to the Partnership under consideration: 2. Projects this complex can be flexible indeed. Flexibility needs to be balanced with explicit, ex-ante (during planning phase) agreements on those arrangements that will be decisive for the outcomes and impact of the program. These agreements cannot be left open for each individual or partnership to shape the program at the time of partnership￾negotiations and decision on specific objectives, lest the Program is derailed or Program outcomes and impact can be minimized. This entails: a. Criteria for school selection need to be structured in such a way that they help minimize predictable risks for reaching program outcomes and impact. Criteria should include not just substantive but administrative and financial capabilities as well. Demonstrated implementation experience is desirable. Accountability for quality of program design and management is enhanced when one agent is fully responsible for selection of participating schools; b. Engagement of the partners during program planning needs to explicitly include: a) criteria for selection of domestic law schools, b) alignment between the school’s mission and the Program’s expected outcomes; c) matching of interests and capabilities with other members of the partnership, d) selection criteria, commitments and expectation from participation of the US partners; c. If a partnership is either expected or required, the US partners need to know in advance the strengths and weaknesses, opportunities and restrictions of the domestic partner and vice versa; d. Should participants lack direct experience in partnership formation and implementation, prior training is needed before the parties enter into the negotiation phase of each partnership; 50 The focus of the proposed partners’ validation is this report’s global analysis, lessons learned and recommendations. Since no attempt has been made to prepare case studies --law school by law school, validation is not about the extent to which the report did/did not take into account individual interviews or every piece of evidence supplied to the evaluation team. US-Colombia Partnership Program – Evaluation report Page 39 e. The joint plan of action should consist of fully structured individual projects that mirror the interests of participating parties and produce benefits no individual party can attain by itself. The joint plan of action should be discussed and approved before committing funds and other resources to either party; f. Planning time cannot be confused with implementation time. As a general rule, planning should include consensus building, verifying incentives for each key participant, negotiations, joint plans and overall implementation trajectories. If any of these planning-phase steps is left for later, delays can be expected and implementation will be at risk. Furthermore, delays during implementation usually bring about frustration among parties that feel ready to start going; g. Sustainability of the initiative is crucial and should be envisaged from the onset of the Program. Strengthening human rights teaching in universities could be a long process and require commitment from the head of the institution. Short￾term projects will not guarantee the sustainability of such an objective. 3. Regarding the M&E system: a. Monitoring indicators should be discussed and agreed upon during the planning phase. Discussion of validation criteria, performance indicators and expected ex￾post evaluation among all key parties should include not only individual school performance but also performance indicators for the entire Program and program management; b. Familiarity and acceptance of the costs and benefits of results-based management framework, including M&E, should be secured before the Program starts. Parties not familiar with results framework need to know its benefits and be willing to pay its costs; c. Program goals can be adjusted over time –particularly in risky programs-, as long as new goals are negotiated and their monitoring and evaluation is agreed upon; d. Monitoring should be based upon pre-defined implementation trajectory –not just result matrices. Implementation trajectory starts with the expected outcome and goes back to starting line, with identification of moments of risks and risk mitigation. Common administrative procedures, such as timely budgeting and disbursement should often be included among risks –especially in cases of low management capacity; e. The role of domestic partners in the monitoring exercise should not be limited to passively filling in forms. Domestic partners need to be actively engaged in analytical monitoring, conclusions and subsequent commitments; f. Simple tasks need to be reported and analyzed in terms of their contribution to Program outcomes. Routine exercises need not only substantial capacity from the senior partners but logistical and incentives planning as well as follow up and institutional commitment. This is true of activities such as training of professors or curriculum review or outreach programs. 4. Regarding the roles and responsibilities: a. Comparative advantages of domestic partners should be identified at the beginning and be included as an essential component of the partnership. It is also the basis to identify learning benefits to US partners, other than monetary basis. Such comparative advantage often includes context knowledge, better risk identification; US-Colombia Partnership Program – Evaluation report Page 40 b. Flexibility and attention to the needs of the partners by US HEI are key factors to favor a successful relationship; c. Incentives for each participant need to be explicitly listed and accounted for. This is an essential part of any successful partnership; d. External support by an expert partnership formation and facilitator is desirable. Partnerships are neither processes nor results that can be left to initial commitment or Program inertia. 5. Regarding universities’ strategy to strengthen human rights training: a. The work on the curriculum is essential but might require time. Alternative strategies should be encountered to pave the way to the required curricular modification. For example, an immediate step could be to carry out a substantial training on human rights for all the law school’s faculty and faculty from other schools, in order to promote a culture of human rights and to ease the way towards human rights mainstreaming. b. Outreach Program. Some interlocutors recognized several advantages in partnering with universities in community work: in a polarized country like Colombia, universities are considered neutral and open, and could easily gain the trust of the targeted population; they also offer a better guarantee of continuity, since they are not contingent on financial support. The validity of these arguments evidently depends on the context and on the institution. What is clear however is that, before launching any outreach Program (for example, in communities, in schools, in prisons), universities should ensure that: (i) the activities respond to the needs of the targeted population, and are prepared with them; (ii) professors and students have the sufficient knowledge to conduct the proposed activities (especially knowledge of popular education); (iii) the universities are aware of the work that might have been initiated by other entities in the same areas (such as NGOs or public institutions); (iv) the universities do not enter to compete with NGOs but complement their work. Thus, a thorough reflection is required before intervening to mitigate the risk to do harm. There is also a need to psychologically prepare the students for the work in communities. Not all of them have the knowledge and the preparation to be confronted to sensitive issues (example: prison work); c. On the same token, it is essential that universities maintain a constructive dialog with the NGOs and civil society organizations in general in order to learn from their experiences, coordinate their initiatives, when possible work together, and avoid, to the extent possible, to enter to compete to each other. 7-1 7. Annexes Annex 1. Evaluation questions grid Criteria Sub criteria Guiding Questions Specific questions applied to the US-Colombia partnership Program Efficiency Management Systems How did the partners make use of results-based management systems? • How was the Program designed at different level (Program, partnerships)? • How were the partners chosen? • Did the base-line identify policies, role of strategic actors and critical actions for the protection of human rights in Colombia? Or else, was the base-line essentially focused in activities and products internal to the law schools human rights Programs? • What were the main foreseeable risks of the Program? Was there a risk-mitigating strategy? Which? Were security risks addressed? How? • Do current result indicators capture the essence of the contribution of the Program? Are there supplementary indicators that could capture more of the richness of the Program and its impact? • Does the data disaggregation as established allow to adequately register the expected results in terms of targeted population? • How was the Program management system foreseen to operate? • Did the monitoring/evaluation system operated to the benefit of the Program? Why? Did the monitoring practices effectively detect deviations from expected Program trajectory? Did the monitoring practices alert to adjustments needed to keep the Program on track to final results? Which alerts were sent? US-Colombia Partnership Program – Evaluation report Page 7-2 Criteria Sub criteria Guiding Questions Specific questions applied to the US-Colombia partnership Program Decision Making processes How did the partnership make decisions regarding implementation? Did decision￾making processes contribute to efficient Program implementation? • How did the Program management system operate in reality? Did it contribute to the efficiency of the Program implementation? Why? • How did the cooperation between the universities operate in reality? • To what extent was the rights-holders (such as students, communities) part of the decision-making process? • Was the language a barrier in the management of the Program/partnership? • Did the resources affected to the Program implementation enough? Why? Were they efficiently used? Why? Relevance Contextualization To what extent were Program activities and interventions adapted for the local context? • Is there evidence of the causal link or the impact between strengthened institutional capacity and reduction of human rights violations? In Colombia? Elsewhere? • Did the Program respond to specific needs of the identified Colombian universities? • Did the Program respond to specific needs of human-rights holders? How was it established? • Policies towards reduction of violation of human rights in Colombia appear to engage primarily other actors (government, NGOs, communities, international organizations). How do university human rights Programs contribute to the effectiveness of such key players? Where does the training of effective human rights defendants take place and how much do universities HR Programs contribute to such training? • Was the country-specific knowledge of US partner schools sufficient to approach the specifics of violations of human rights in Colombia? Logic Did Program design consistently link activities and outputs logically • Were the expected contributions from the schools (in terms of activities and output) geared towards the overall success of the US-Colombia Partnership Program – Evaluation report Page 7-3 Criteria Sub criteria Guiding Questions Specific questions applied to the US-Colombia partnership Program to Program outcomes and objectives? Program? Effectiveness Results To what extent have the intended outputs and outcomes been achieved or are likely to be achieved? How were the results affected by Program fidelity? (Program fidelity may be defined as the extent to which delivery of an intervention adheres to the protocol or Program model originally developed.) • Did the schools carried out the foreseen activities and meet those expected results? Indicators/metrics? To what extent? Why? • How did the division and complementarity of expertise between Colombian and US schools work, in practice? Who did what? Who helped the most to reach xx key Program result? • Did the schools benefit from capacity from or lessons learned by other participating schools? In the region? In Colombia? • Are there alternative approaches for reaching the expected results of the Program (for example, individual work, partnerships among Colombian schools, partnerships between Colombian universities, communities, NGOs. or partnerships with courts or a ministry or the police, etc)? • Did security issues affect the development of the Program? Impact Outcome level results What were the outcome results of implementation? To what degree are the outcome results attributable to implementation? • What were the (intended or unintended, positive or negative) results of the Program? What results can be directly attributable to the Program? • What were the (expected, unexpected) benefits of partnering the schools with the US universities? What did US universities bring to their Colombian partners? • What are the benefits that could only be reached through this partnership (those that could not be reached by individual school alone nor by the regional consortium of Colombian schools)? • Would the schools invest (time, resources) again in the formation of a similar partnership? Would they advise other Colombian schools to invest in the formation of such US-Colombia Partnership Program – Evaluation report Page 7-4 Criteria Sub criteria Guiding Questions Specific questions applied to the US-Colombia partnership Program partnerships? Why (yes/no)? • Were the conditions for the success of the partnership approach effectively met? Was each party aware of and committed to reaching the additional benefits of the partnership approach (i.e., those benefits that cannot be reached individually by each partner)? • What were the benefits to be part of the Program? Expansive effects Is there evidence that outcome level results had a wider effect than anticipated? Consider strategic partnerships, i.e., host country higher education institution -private sector partnerships • Have the schools experienced any change (capacity or otherwise) as a result of the Program? Which one? • What is the (expected or unexpected, positive or negative) impacts of the Program in the identified regions, and in the country? • Are there alternative approaches for reaching the expected outcomes of the Program (for example, individual work, partnerships among Colombian schools, partnerships between Colombian universities, communities, NGOs. or partnerships with courts or a ministry or the police, etc)? What results did the Program have that could not be achieved by other entities? Sustainability Program effects Do you think that Program effects (development results and/or host-county and U.S. higher education partnership) are likely to continue over time after funding has ceased? If so, what aspects of the Program have the best chance of continuing? In • What parts of the Program contribute to the sustainability of its objectives? • Do the schools expect to continue the engagement they had during the Program with protection of human rights in Colombia? Why? How? • What initiatives have the partners taken to ensure the sustainability of the Program? • Does the Program continue to be relevant in the Colombian context? Why? Is there any change that should be introduced to US-Colombia Partnership Program – Evaluation report Page 7-5 Criteria Sub criteria Guiding Questions Specific questions applied to the US-Colombia partnership Program your opinion, what would be the major factors that would influence these lasting effects? such a Program? US-Colombia Partnership Program – Evaluation report Page 7-6 Annex 2. List of the persons interviewed Nombre (Name) Institución (Institution) Posición/Cargo (Title) Antioquia Max Gil Corporación Región Olga Xiomara López Flores PNUD Asistente Técnica Pablo Emilio Angarita Universidad de Antioquia Profesor Dora Saldarriaga Universidad Autónoma Latinoamericana Profesora Alexandra Martínez Ramírez PNUD Asistente Técnica Lina Estrada Universidad Pontificia Bolivariana Profesora Experta en NNA Gildardo García Pastoral Social, Rionegro Coordinador Delegación de Promoción Humana Rubén Darío Jaramillo UCO Decano Zeller Álvarez Alianza Antioquia/UMN Coordinador de Proyecto Sandra Gómez Alianza Antioquia/UMN Coordinadora del Proyecto Astrid Osorio Consorcio Antioquia/UMN Coordinadora de Proyecto Maribel Ocasiones UCO Coordinadora Alianza Ana Milena Montoya UDEM Profesora de la Clínica Jurídica Catalina Rojas UCO Coordinadora Alianza Martín Palacio Universidad Pontificia Bolivariana Estudiante Verónica Cadavid Universidad Pontificia Bolivariana Estudiante Isabel González UDEA Profesora Antropología/Experta en Conflicto US-Colombia Partnership Program – Evaluation report Page 7-7 Carlos Morillo UDEA: Sonsón Estudiante Dina Paula Marín UDEA: Sonsón Estudiante Edgar Suárez UDEA: Sonsón Estudiante Cecilia Redondo UDEA: Sonsón Estudiante Jesús Antonio Bravo UDEA: Sonsón Estudiante Verónica Vega UDEA: Sonsón Estudiante Silvia Redondo UDEA: Sonsón Estudiante Ana María Sánchez UDEA: Sonsón Estudiante Alirio Sánchez UDEA: Sonsón Estudiante/Egresada Alex Mesa UDEA: Sonsón Estudiante Luis Fernando Álvarez Universidad Pontificia Bolivariana Decano Cathalina Sánchez Universidad Pontificia Bolivariana Coordinación de la Clínica Laura Zambrano Universidad Pontificia Bolivariana Experta en redes y gestión de proyectos académicos de la UPB Cesar A. Molina Saldarriaga Universidad Pontificia Bolivariana Director Clínica Constitucional Ángela María Mesa Universidad Pontificia Bolivariana Juan Carlos Vásquez Universidad de Medellín Decano Erika Castro Universidad de Medellín Directora Clínica Jurídica Catherine Vega Universidad de Medellín Semillero DDHH Carolina Londoño Universidad de Medellín Estudiante/Egresada Nelson Arango Universidad de Medellín Estudiante US-Colombia Partnership Program – Evaluation report Page 7-8 Jaime Jiménez Universidad de Medellín Estudiante Ángela Vanegas Universidad de Medellín Estudiante 4 semestre. Clínica asuntos de género y derecho. Lina Estrada Universidad Pontificia Bolivariana Directora Proyecto El Pinar Oswaldo Gómez Universidad de Antioquia Gerente Administrativo Alianza Diana Carolina Sánchez Universidad de Antioquia Profesora, experta en clínica. Agustín Vélez Universidad de Antioquia Coordinador del Proyecto Clemencia Giraldo Universidad de Antioquia Decana Juliana Vélez Universidad de Medellín Egresada Zara Marin UCO: Clínica Penitenciaria. Cárceles Oriente antioqueño. Estudiante Marta Lucía Gorgia Rendón UCO: Clínica de niños Estudiante Arley Díaz UCO: Clínica Penitenciaria. Cárceles Oriente antioqueño. Estudiante (9 semestre de derecho) Estefany Castro UCO: Clínica Penitenciaria. Cárceles Oriente antioqueño. Estudiante (9 semestre de derecho) Celenia Osorio UCO: Clínica de niños y clínica de personas de la calle Estudiante (10 semestre de derecho) Paula Marcela Valencia Pérez UCO: Clínica penitenciaria Estudiante (9 semestre de derecho) Alejandra Orozco UCO: Clínica Penitenciaria. Cárceles Oriente antioqueño. Estudiante (10 semestre de derecho) Zunibeth Manco Torres UCO: Clínica Penitenciaria. Cárceles Oriente antioqueño. Estudiante Fundadora de la clínica US-Colombia Partnership Program – Evaluation report Page 7-9 Zara Magdali Gallo Castrillón UCO: Clínica penitenciaria Estudiante (7 semestre de derecho) Sergio Rodríguez Universidad de Antioquia Director zona Sonsón Jaime Agudelo Universidad de Antioquia Director Clínica Jurídica Agueda Torres Universidad de Antioquia Directora Consultorio Jurídico Jhon Jairo Castañeda Comunidad Granizal Edil/Líder Comunitario Diana Salcedo Comunidad Granizal Lidereza Valle del Cauca Laura Giraldo Universidad de Santiago de Cali Estudiante de Derecho Diana Fernández Universidad de Santiago de Cali Docente Profesora de DDHH y DI Público Juan Batero Universidad de Santiago de Cali Estudiante de Derecho Ana Inés Lopéz Ramírez Universidad de Santiago de Cali Estudiante de Derecho Gustavo Jaramillo Universidad de Santiago de Cali Profesor Procesal Civil Sofía Quintero Universidad de Santiago de Cali Estudiante de Derecho Hermes Gregorio Araujo España Universidad de Santiago de Cali Abogado Litigante Leonardo Sabogal Universidad de Santiago de Cali Profesor Diana Quintero ICESI Profesora William Rodriguez Universidad del Valle Fundación Esperanza por Colombia. Jésus Darío González Experto Observatorio de Realidades. Obispo Cali Carlos Fernando Torreros Universidad Santiago de Cali Profesor Diana Marcela Bustamante Universidad de Santiago de Cali Coordinadora de Proyecto Emma Peña Universidad Pontificia Javeriana de Cali Estudiante US-Colombia Partnership Program – Evaluation report Page 7-10 Adriana Tenorio Ocampo Universidad Pontificia Javeriana de Cali Psicóloga Sigifredo Toro FENCOSER Representante Legal Modesto Martínez Cardona FENCOSER Vocal Raúl Núñez Universidad Pontificia Javeriana de Cali Coordinador DDHH Luis Edison Bertinni Universidad Pontificia Javeriana de Cali Grupo Focal Alejandro Ariza Universidad Pontificia Javeriana de Cali Grupo Focal Sebastian Canal Universidad Pontificia Javeriana de Cali Grupo Focal Costa Caribe Jesús Fuemayor UniMagdalena Asistente Centro DDHH Lybni Pacheco UniMagdalena Estudiante de Derecho Andrea Méndez UniMagdalena Estudiante de Cine Álvaro Padilla UniMagdalena Representante estudiantil Raquel García UniMagdalena Coordinadora Programa Opción Legal Pablo Vera Salazar UniMagdalena Vicerrector de Extensión Funcionario UniMagdalena Vicerrector de Docencia Rito Pineda UniMagdalena Estudiante Programa de Derecho Noe Segundo Saavedra Director de Cárcel Judicial Director Cárcel Judicial de Santa Marta Herminia Puche UniMagdalena Docente programa de Derecho US-Colombia Partnership Program – Evaluation report Page 7-11 Hans Rangel UniMagdalena Suplente Representante estudiantil ante el Consejo Académico Iván Amaya UniMagdalena Ayudante Facultad de Humanidades José Baena UniMagdalena Estudiante proyecto Carcelario Eduardo Vélez Soto Defensa Civil Comandante Defensa Civil Colombiana Milton Montes Fonseca UniMagdalena Estudiante proyecto Carcelario María Toloza UniMagdalena Rectora IED Aluna Ledys Buelvas Ojeda Defensa Civil Psicóloga Defensa Civil Laura Cera Uninorte Asistente Centro DDHH Viridiana Molinares Uninorte Profesora/Directora Derechos Constitucional Juan Pablo Sarmiento Uninorte Profesora Investigadora Silvia Gloria Uninorte Decana Laura Briceño Uninorte Coordinadora del Proyecto Wilson Castañeda Caribe Afirmativo Director Ariel Cáceres KUSUTO Director Paúl González Uninorte Profesores de Consultorio Jurídico Edenia Orozco Uninorte Profesores Consultorio Jurídico Katherine Sanchez Alcaldía de Barranquilla Programa de Mujer y Género Silvana Insinares Uninorte Directora Aérea de Comercio Exterior Shilrley Llain Uninorte Directora de Moot Court Javier Tous Uninorte Profesor US-Colombia Partnership Program – Evaluation report Page 7-12 Judith Echevarria Uninorte Profesor Grupo de Estudiantes de Consultorio (10 estudiantes) Uninorte Otras personas Néstor Raúl Correa Experto Derechos Humanos Magistrado CSJ Beatriz Londoño Red Colombiana de Clínicas Jurídicas Universidad del Rosario Adriana Benjumea Corporación Humanas Directora Ejecutiva HED Diana Paez-Cook HED Program Officer, Latin America and the Caribbean Omri Malul HED Senior Specialist, Program Quality and Strategy Anne Guison Dowdy HED Senior Specialist, Monitoring, Assessment, and Reporting María Consuelo Ramírez HED HED/Colombia Jaimie Salazar HED Program specialist Universidades estadounidenses Barbara Frey UMN Directora Proyecto UMN Melissa Aguilar AU Directora Adminsitrativa AU USAID/Colombia Andrea Guardo USAID Human Rights Specialist, Democracy, Human Rights and Governance Donald Chisholm USAID Deputy Director, Office of Democracy and Human Rights Todd Sloan USAID Director, Office of Democracy and Human Rights US-Colombia Partnership Program – Evaluation report Page 7-13 ANNEX 3. Partnership level findings Antioquia partnership Relevance The choice of Antioquia as one of the regions for the implementation of the Program is unequivocally ascertained. Antioquia is economically and politically a pivotal region in Colombia. It has been particularly affected by the different waves of violence that hit the country over the last decades, both in urban centers (like in the Valle de Aburrá where Medellin is located) and in rural areas. It has been the scenario of appalling cases of violence, and many people have been caught between varieties of armed groups that deployed their brutality in the region. It has been the center of expulsion, reception and resettlement of large number of population. One of its public universities, University of Antioquia (UDEA), was singularly targeted during the peak of violence (including through the assassination of faculty and students, threats and violent repression). Other than interesting but stand-alone initiatives, universities have not played a critical role in human rights debates in the region. Strengthening their capacity to train in human rights is undoubtedly pertinent, taking into account the relevance of Antioquia’s higher education institutions as one of the seedbeds of future influential professionals countrywide. The choice of the universities that form the consortium enabled the Program to reach not only the city of Medellin but also the nearby Oriente antioqueño, a region particularly victimized during the internal armed conflict.51 The involvement of the recently established UCO law school as well as of UDEA rural extension in Sonsón has provided the schools with the possibility to foster their human rights focus, and the students with the opportunity to learn about human rights and participate in the opportunities offered by the Program. There is no doubt that, as underlined in HED mid-term assessment report, the three objectives defined by the Antioquia partners were relevant. In particular, regarding objective 2, Antioquia universities have developed some still limited initiatives in the communities. In that sense, promoting the development of outreach activities with the population from the universities is a positive initiative: not only does it allow them to be closer to reality, and motivate their students to learn about and get involved in their immediate surroundings, but it also encourages the schools to continue, resume or initiate their work with the communities. Some external counterparts also found in universities the guarantee for long-term commitment to the community (different from NGOs that depend on projects), and for less radical views. 51 This aspect was underlined in the Initial Institutional Assessment Report, p.7. US-Colombia Partnership Program – Evaluation report Page 7-14 Furthermore, developing or strengthening the law clinics from a human rights perspective and using methods that increase their learning purpose benefit the universities as relevant actors in human rights debates as well as reinforce their capacity to adequately prepare future lawyers. At results levels (objectives and outcomes), the Antioquia partnership planning is logical and does not present critical issues. Some aspects could be debatable, but overall the planning makes sense. A certain disconnect exists, however, between the outputs and activities foreseen and the results. For example, expanding the curriculum on human rights would entail a different focus of the activities, and would have implied going much further than the anticipated actions. The process the Program followed to establish the partnerships might explain this situation, since it oriented US universities to focus their contributions in terms of activities and resources, and did not stimulate the realization of serious analysis of the needs and preconditions of the universities. In Antioquia, the partnership focused the first part of its implementation towards clinical work, probably based on UMN strength in that area rather than as a strategic path to strengthen institutional capacity. Although they recognize the relevance of the partnership’s objectives, Antioquia partners seem to have been involved very late and insufficiently in the design of the Program, situation that provoked resistance in some universities and delayed the implementation. Furthermore, the choice of working through a consortium of four universities was never really explained to them and came as a surprise. Some interlocutors defined the Partnership as “daring”, “adventurous”. If the Partnership triggered interesting outcomes, they did not come as a result of a previous analysis of its feasibility. Studies of each university were carried out during the prior assessment, but it did not cover the possibility of universities working together. Efficiency As the survey and interview respondents indicated during the mid-term assessment, most challenges were related to developing a concrete and functional community among a diverse array of institutions and individuals, ensuring continued commitment at the institutional level, and managing changes to adapt to new processes and procedures. The daring gamble to associate four very distinct universities in Antioquia ended up being an efficient mechanism to ensure the implementation of the partnership. The establishment of a three-member coordination team for the consortium (composed of a Clinical Director, a Human Rights Professor and an Administrative Coordinator) was key to coordinating the Antioquia Partnership, and organize and carry out all the activities: short courses (52 training sessions were reportedly given), joint strategic litigations, and externships in UMN. The Program management team counterbalanced the slim (human) resource dedicated by the participant universities to the Program. US-Colombia Partnership Program – Evaluation report Page 7-15 These many activities would not have been feasible with the resources the Antioquia universities put at the disposal of the partnership. As recognized in the mid-term assessment: “With balancing teaching, partnership, and other responsibilities seen as a necessity, respondents indicated that the time allotted to faculty participating in and implementing partnership activities was a challenge” (p.12). First of all, only in UDEM and in UCO, professors could dedicate most of their time to the partnership. In the rest of the universities, the availability of the professors was much more reduced. Second of all, turnover among the focal points for the partnership in the universities represented a real challenge for the activities’ sustainability. The coordination team played an essential role in ensuring the coordination between all the partners for the realization of the activities. A weekly operative meeting that gathers dedicated faculty at each partner law school allowed for building trust, keeping the partners up-to-date regarding the activities, ensuring their active participation, and strategically plan the activities. The Antioquia partnership involved the operation of a complex and multilayered management system that included the universities, the coordination management team, UMN, HED/Colombia, and HED/Washington. Financially, UDEM has to be added since it managed (not without difficulties) the funds for the four Antioquia universities. It certainly guaranteed the implementation of the activities, and a strict control over it. Such a burdensome mechanism did function owing to the efficiency that characterized many Antioquia professionals, but also to the institutional adjustments that took place in the administration of partner institutions to facilitate partnership implementation. These adjustments reflect important institutional efforts to adjust their structures to achieve the partnership’s intended results. However, it implied an overload of administrative work for the coordination team to the detriment of substantive initiatives. Two of the members of the coordination team were assigned the task to substantially support the universities in clinical work and curriculum modifications. They contributed to building the joint cases as well as working with universities on curricular modifications. However, the organization of so many activities and the Program’s administrative demands took over a lot of their time to the detriment of more substantial tasks. The pressure to show concrete results also contributed to losing the sight of the Program’s overarching goals. The staffing model of the Antioquia partnership has two different effects. On one side, it enabled to booster the coordination of the activities, and thus improving the efficiency of the partnership between the four universities. The approach seems logical and strategic to ensure the work with four universities at the same time. Exceeding simple coordination, it also promoted joint work between the universities. It went to the point that the Partnership as such seems to have gained a proper life beyond each of the universities. On the other side, the Partnership might have overshadowed the results each university has individually achieved through the partnership. US-Colombia Partnership Program – Evaluation report Page 7-16 Effectiveness The UMN did have a previous experience working with Latin American Universities (especially in Mexico). However, it did not have knowledge of the Colombian context nor of the operation and idiosyncrasies of Colombian universities. Although the situation was ameliorated owing to the patience, persistence, arduous work, and flexibility of UMN’s representatives, who regularly visited Antioquia and maintained regular contacts with all the partners, this situation caused some starting hiccups and initial mistrust. For instance, some of the partners expressed the view that their previous human rights initiatives were not taken into consideration and that the project was initiated as if they had not worked previously on the matter. They also regretted that UMN did not have enough knowledge of the work and operation of Colombian law clinics. Despite these initial misconceptions, the UMN quickly overturned the situation and was able to constructively work with its partners to adapt the partnership to their mutual needs. One of the key examples repeatedly highlighted is the substitution of one scholarship for a faculty to do a LL.M. at the UMN by four scholarships to complete a Master Degree in UDEA. The UMN played a key role in providing new knowledge, networking opportunities, and accompaniment to its Colombian counterparts. The importance and experience of UMN in human rights is recognized, the quality of its professors undisputedly acknowledged, and the externships considered as unique opportunities for faculties and students. If the role of UMN was clear training wise, it does not seem so discernible regarding its participation in the selection and writing of the clinic cases, apart from the two joint cases. It is also regrettable that UMN never accompanied university faculty and students in their outreach activities. It would have given them the opportunity to adequately tailor their clinic teaching.52 Taking advantage of UMN’s expertise, the core of Antioquia partnership’s activities was geared towards the strengthening (or establishment in the case of UCO) of the universities’ legal clinics. It was achieved through training to the professors and the students, externships, and the identification of cases that could be jointly advocated. Following that logic, results were achieved at two levels. At the individual university level, UCO and UDEM took extensive advantage of the partnership: UCO created its legal clinic (that is now institutionalized), and UDEM was able to significantly strengthen the work carried out on its own.53 The partnership achieved some but less visible results in UDEA and UPB. 52 For example, it would have been important that UMN accompany UCO in its prison visits to be able to assist them better with tailored methodologies and training. 53 However, in UDEM, the law clinic has not yet been linked to the legal practice center, possibility that would have allowed it to be permanent. In the universities where the legal clinic is inserted in the consultorio jurídico, a better degree of degree of effectiveness could be observed. To the contrary, in universities where they were separated, they missed an opportunity to achieve greater effectiveness in training students. US-Colombia Partnership Program – Evaluation report Page 7-17 At the partnership level, two joint initiatives were developed: a shadow report directed at the Committee of the Rights of the Child, and a joint presentation of the situation of communities resettlement in Antioquia. Both actions required a great level of coordination, which was possible owing to the steady work of the coordination team, as well as the dedication of some of the universities’ focal points. It is undeniable that these two litigation processes allowed the achievement of many different results: working jointly; strengthening legal clinic work; introducing new training methods; motivating the students; improving the knowledge of the faculty and the students involved; opening new networking possibilities; influencing public policies. As such, these very positive initiatives could be pursued. The question remains: would the development of joint litigation processes in human rights contribute to strengthening law schools capacity to train human rights? Is this a backdoor strategy? If so, what conditions are required to make it an effective backdoor strategy? The focus of the first part of the project was geared towards clinical work, leaving aside curricular modifications. The mid-term assessment made a specific recommendation to that respect, and triggered a major concentration of the work in that respect with some small and concrete results. For example, in UCO, the curriculum reform is going to initiate in July 2015, and took advantage of the assistance of the Partnership, through the human rights professor assigned to this task. However, the modification of the curriculum is not an objective that can be attained so easily. Each university has its own process, conditions and objectives. For example, UDEA was finishing a seven-year process it was difficult to impact on. On the other hand, mainstreaming human rights into the curriculum, as is UCO wish, entails structural modifications and training that cannot be reached through a short project. The work on the curriculum would have required conducting a previous assessment of the needs and conditions in each university, and much more dedication than the project allowed for. Regarding the work with communities, each of the universities has developed outreach activities within the framework of the partnership. Some strengthened the initiatives they were already carrying out (like UDEA and UDEM). The partnership allowed them to introduce a human rights-based approach to their actions, involving better the community, implementing new analytical methodologies, and increasing their focus on accountability mechanisms. However, the implementation of some of the outreach activities seems to have responded to Program demands rather than to universities goals. As a result they were either built on pre-existing activities (such as the training in the Granizal community in Bello) or implemented without much preparation or reflection (such is the case for the training in the school of El Pinar), beyond good intentions. The intervention of UMN in the preparation and implementation of outreach activities is not clear. Its implication through the accompaniment of its partners in the field would certainly have enabled its teachers to tailor specific training sessions to the reality students and professors meet (for example, on popular education), and to assist in reflecting on the role of the university and its expected and unexpected impact in the population. US-Colombia Partnership Program – Evaluation report Page 7-18 In Antioquia, the Partnership has limited, even scarce relationships with local civil society organizations. Isolated attempts exist. For instance, UDEM has been participating in an initiative led by Corporación Humanas on sexual/domestic violence and strategic litigation. UCO is the secretariat of Mesa de Derechos Humanos del Oriente, an inter-institutional mechanism that gathers official institutions, the Church, and different sectors of civil society (including UCO, NGOs and business representatives). The persons interviewed from universities and NGOs coincided in highlighting the relevance of an interrelation between two sectors that have a crucial and complementary role to play in human rights promotion and protection. However, they continue to act separately and in parallel. Afar from the discourse, a constructive dialog between the universities part of the Partnership and NGOs has not been undertaken. Finally, gender equality has not been absent of Antioquia partnership activities: specific courses have been dedicated on women’s rights, the UDEM law clinic has developed a specific line on this thematic area. However, they still remain isolated initiatives with restricted scope. On another topic, the lack of a gender specific perspective might have limited the results of the efforts deployed. For example, some parts of the CRC shadow report adopted a clear family￾oriented vision, discussable from a gender perspective. The effort was there; the expertise might not. In sum, the Antioquia partnership carried out many activities, certainly achieved concrete results, and definitively was important to strengthen the knowledge of the persons involved. These results depend greatly upon the characteristics of the universities (with greater impact in a small university, such as UCO, than in a much bigger university, such as UDEA). It provided the opportunity to all the partners to explore the opportunities and understand the challenges that exist to strengthen the training on human rights of their law schools. Its results correspond to a good exploratory process. As underlined by the mid-term evaluation, “more time is needed for the partnership to fully achieve its objectives – the type of systemic changes the partners would like to see in their institutions are considered long-term and will require additional time to take root.” Results The “daring partnership” between the four Antioquia universities showed that common interest could lead to joint work with substantial results. The idea of working through a consortium of four universities did not come from the universities, but was imposed on them.54 For a small 54 The evaluation team was unable to find a written document explaining the reason the consortium was established (the Initial Institutional Assessment report does not mention the consortium; it analyzed each of the four universities). No one could provide an explication either. US-Colombia Partnership Program – Evaluation report Page 7-19 educational institution, such as UCO, a partnership with larger universities and a US university provided it with the possibility to position itself, to network, and to gain a strategic vision. In the words of its dean, the participation in the Partnership “enormously renovated the school.”55 The work Antioquia universities were able to jointly conduct will benefit from future collaborative efforts. A “truly collaborative relationship” has developed between the Antioquia universities and the UMN. The Antioquia partnership has allowed to:  strengthen the faculty capacity (in human rights and education methodology), through short courses, externships, training from international experts – including from Latin America-, contribution to joint litigation. Short courses involved other faculty from the law schools and faculty.  “internationalize” the universities: The universities recognized that the partnership provided them with the possibility to open up internationally. The partnership with an American university with the experience and trajectory of UMN provided them with opportunities that they would not have had without the Program.  strengthen the legal clinics. The strengthening of the legal clinics (especially through the time allocation for professors, new and specialized knowledge) has contributed to strengthen the importance of human rights education in the participating schools;56 it contributed to the pertinence of the clinics within the universities; and it motivated the students to work on human rights issues.  provoke evident changes in students that took part in the activities (enthusiasm, motivation, passion, possible career in human right). The possibility to influence the revision of the university curricula was possibly overstated, especially in bigger universities (such as UDEA). This long-term endeavor requires a clear commitment from the university leaders. A better knowledge and more consultation with the identified universities might have enable to define this work better. 55 Interview with UCO Dean, April 22, 2015. 56 “UDEM faculty indicated that before the partnership the clinic had a fundamentally local approach to strategic litigation, and through the capacity building activities made available by the partnership they have been able to look beyond the local venues for human rights protection (such as through their work with the Inter￾American Commission on Human Rights), to strengthen their work with the community, and to change their perspective on the teaching of human rights. They now realize that they are able to engage in human rights advocacy and litigation at the international level.” (monitoring document) US-Colombia Partnership Program – Evaluation report Page 7-20 Noticeable Antioquia Activities, Products and Intermediate Results --Objective by Objective of the Program-- Under Objective 1, Antioquia results included short courses on specific human rights issues; faculty externships in and visits to the UMN, the enrollment of faculty in UDEA Master in Law Program, the establishment of a curriculum committee focused on human rights to review and improve the curriculum; mainstreaming of the human rights perspective in the curricular reform of the UCO; creating university learning processes in human rights, in the rural zones of Antioquia that have been affected by conflict (Sonsón and Rionegro); strengthening the training of teachers in human rights with an approach to the mechanisms of regional and universal systems of protection of human rights; creating spaces of support and discussion of the human rights agenda in the academia, between partner universities; the UDEM provided contractual guarantees so that the coordinators of the clinic could be devoted to research and to directing the human rights clinic. Particularly, UDEA accomplished the involvement of other schools of the university in research and clinical work. Under Objective 2, the main achievements are the creation of a law clinic in UCO, and the institutional strengthening of UDEM’s legal clinic as well as the creation of processes and methodologies for training in the attention and defense of human tights, through a statute of rules to participate in the clinic by the students, and creating sustainable processes. Additionally, joint litigation cases and advocacy strategies in the universal and regional human rights systems in order to position issues; mobilizing strategies of international advocacy in order to effect the course of judicial processes at the national level, in charge of judicial clinics such as the ones of UDEM and UDE; creating methodologies for serving victims of the conflict; position the universities as centers of socio-legal attention in zones traditionally affected by conflict. Creating alliances by the UCO with the Diocese of the East, as one of the most influential and visible stakeholders in the protection of human rights. It is also relevant to note that the UDEM and the UPB accomplished institutionalizing an office within the law schools, which allowed the offices to be a space of reference to discuss human rights in the universities. Under Objective 3, short courses, externships for students in the UMN, and internship participation in the Inter American Commission on Human Rights, where amongst the experiential learning opportunities provided to students. Furthermore, the UCO and the UDEA encouraged students to combine the obligatory practice of the legal clinic with the cases and project of the clinic, with a human rights approach; new spaces were created and strengthened in the UDEM, regarding culture and human rights, with the support of the students from the clinic; for example, the radio station of the university, and the theatre and singing groups for the dissemination of the contents of fundamental rights; publication of materials created by the students in order to promote and care for the rights of the prison population, women and victims. The possibilities for training abroad for students were amplified, and new abilities of training were created too, such as orality and research. US-Colombia Partnership Program – Evaluation report Page 7-21 Sustainability It is undisputable that the Antioquia universities welcomed the partnerships with US universities, such as with UMN, and have already entered into agreement with other foreign universities regarding other programs. These types of partnership provide regional universities with international recognition, greater weight at national level, and better and greater opportunities for their students. In fact, some universities, such as UDEA, have expressed their interest in furthering the relationship with UMN through a bilateral agreement. Regarding the partnership between the Antioquia universities, it is not clear to what extent the partners involved are interested in pursuing the efforts, and to what purpose. Interestingly, the evaluation team obtained contradictory answers from the persons interviewed. Joint strategic litigations or the strengthening of a regional law clinic network constituted some of the ideas flagged during the interviews for continuing the partnership. Joint litigation could also give rise to training opportunities for prosecutors and judges. However, it is doubtful that the Partnership can be maintained with the actual structure. Furthermore, apart from UCO57 (and maybe UDEM – but for other reasons), it is not clear to what extent university leaders are committed to engaging financial and human resources in a human rights program. A clear commitment of the deans and the rectors is a requisite to allow for the sustainability of any inter-universities efforts. Valle del Cauca partnership Relevance The choice of the Valle del Cauca region is also quite justified. The region is usually understood to comprise the Department of Valle del Cauca and the Northern part of the Cauca department. Situated in the southwest of Colombia, between the Andes and the Pacific, the region counts eight different indigenous peoples and a large Afro-descendent Colombian population. In spite of insufficient information on and investigation of human rights violations in the region, there is enough evidence to argue that a critical human rights concern is the number and type of 57 UCO took two important measures to guarantee the continuity of the Partnership’s results: it compromised itself to pay the salary of the two persons in charge of the law clinics; and the law clinic is now officially established as part of the consultorio jurídico. US-Colombia Partnership Program – Evaluation report Page 7-22 violations against indigenous people.58 Furthermore, according to the United Nations, in 2014, 9.63 % of the internal displacement for the entire country occurred in Cauca region.59 The selection of law schools did not adequately capture their immediate concerns, and does not seem to have had sufficient analysis of the administrative risks of the universities. The choice of USC appears to owe to its high number of Afro-descendent students and the socio-economic background of the student body; the prior existence of a “semillero” in Afro￾descendent matters; and the school’s focus on criminal law. However, other aspects, such as logistical deficiencies, fragile legitimacy or administrative and financial limitations, that could present some risks for the implementation of the Program, do not seem to have been duly taken into account, or at least were not reported. They ended up affecting the level of relevance of the partnership for the school and for the Valle del Cauca, and later restricted the individual and overall results of the project. Additionally, USC’s recruitment and hiring practices were not considered: the inexistence of full time teachers in the law school severely limited the intensity of their commitment and jeopardized the Program’s sustainability. Similarly, the university’s complex, slow, centralized and often unpredictable decision-making process weakened its effectiveness and deviated the immediate attention of faculty and administrative staff to deciphering the way to secure decisions necessary to the partnership. The relevance of the Program to the local universities and its overall regional impact in Valle del Cauca were also restricted by the lack of familiarity of AU with the local context. As we mentioned in Program-level analysis, the relevance of the Program is determined, largely, by the context and situation of human rights in the country. In this sense, this section of the evaluation must determine to what extent the partnership in this region was relevant to confront and address the specific needs of the region. Therefore, field questions were aimed to show how the logical design of the Program by each university was relevant and appropriate to address the situation of human rights in the Valle del Cauca. The law school services to the communities did not appear to target some of the most visible and influential cases of human rights violations in the region. Though it can be argued that any potential violation of human rights makes a good entry point for law school training and institutionalization, a weak linkage between most urgently needed community demands and law school service-practices also weakens relevance and potential interest and commitment from faculty, students and regional communities. The design and initial proposal of the partnership were focused on the goal of institutional strengthening and enhancement of the capacity of faculty and students. To this effect, a series of workshops and legal trainings to teachers, community and beneficiary students were organized. The training was of high quality and appreciated by participants. Still, it left out of the roadmap other dimensions of the objectives as institutional strengthening or securing improved capacity 58http://cms.onic.org.co/2015/03/balance-situacion-de-vulneracion-de-ddhh-e-infracciones-al-dih￾subcomision-de-ddhh-garantias-y-paz-informe-de-seguimiento-0101-0803-2015/ 59 Source DPS/Unidad de Atención y Reparación Integral a las Víctimas by year of expulsion cut-off date: Jan 1, 2015, report date: February 5, 2015. US-Colombia Partnership Program – Evaluation report Page 7-23 to meet the human rights needs of the community. Still, the effectiveness and efficiency of workshops were conditioned by oversized participation. Perhaps more importantly for relevance purposes, the design of the workshops did not take into account either the obvious differences between the two Valle law schools or the specific demands of the local population in terms of human rights violations. The Valle partnership counted on the support of the installed capacity of the law school at AU. AU did offer the Valle law schools initial training or consolidation in some of AU’s recognized areas of competitive advantage, including English summer courses, the moot court competition and the academy of Human Rights and Humanitarian Law. Yet there appears to be little effort to capture the particularities of the regional human rights context or the capability differences between the two selected Valle schools. It seems that the Project design did not require –nor did management supervise-, effort by the US partner School to: i) think about the design and implementation of the Project as a function of the needs of the local partners; ii) document the human rights situation in the region so as to offer creative strategies, or adequate training and litigation addressed and tailor-made to local reality; iii) draw lessons from successful, domestic strategies for the protection of human rights in the country. That void in the design allowed the universities to initially focus the Program on those areas in which they already had greater extent capacity and in which they wanted to excel –with or without Program support. In the case of PUJC, the design of the partnership was essentially circumscribed to training and strengthening existing activities. A clear example of this is the high number of hours, time and resources dedicated to training and participation of students and teachers from PUJC in the moot courts led by AU. The results framework for the project allows for this sort of trade-off in emphasis. A more closely defined higher-level objective might have further stimulated the partners to supplement training with other specific objectives of the Program while developing synergies among project components and pursuing a common higher level objective –as proposed in the overarching objective of the Program. Efficiency Efficiency and effectiveness were both increased and limited by: i) the selection of law school in the early phase of program planning and implementation and, ii) management during implementation at full scale. USC The selection of the universities did not take into account the institutions’ management gaps and capacity. The facts showed that efficiency in the management of resources could hardly be expected, and therefore conditioned the level of effectiveness and impact of activities and deliverables. For example, in USC payment of project coordinator fees appeared to be some US-Colombia Partnership Program – Evaluation report Page 7-24 months late. The evaluation team was also informed that management and administrative delays had forced the suspension of law school services in the field as well as the “semillero” brigades. On one level, built baselines focused on a series of training activities and visits to the Washington College of Law (WCL) summer school without realizing the shortcomings to be overcome and the strengths to leverage. Additionally, the support of AU in the report and review of the indicators and activities was merely an exercise in gathering quantitative information sent by the national coordinators, which does not convey the reality of the difficulties of implementing the partnership. At first glance, one might say that the agreement between USC and AU for the preparation and reporting of quarterly reports was efficient as it was left at the head of the AU coordination the collection of information. This mechanism of cooperation was limited to account for the activities from a purely quantitative approach, leaving aside the identification of the causes of delay in the implementation, cost-benefits of activities, the impacts of the “semillero” activities and the conceptual needs of the initiatives. In relation to risk management as a factor of efficiency of the Project, it is important to mention that the USC team was the only one who informed that the selection of areas and topics to cater for the brigades were limited by recommendation of security contractor. Finally, in the costs of investment of resources and time of the visits by teachers and AU team, there was a lost opportunity for the international team to have a deeper knowledge about the fieldwork of the University, the difficulty and scope of the brigades, and the adjustment of the necessary activities to redefine the project implementation. In particular, participants observed that more days in the field were needed if the AU team is to have direct observation of communities’ needs and potential. PUJC As mentioned, the design of the Program lost degrees of efficiency by focusing on the training of teachers in the academic offerings of the WCL, and especially, in activities designed for summer in the city of Washington. In that sense, the Program missed the opportunity to develop creative projects in other areas, which impact on the effectiveness and results of the alliance with PUJC. Additionally, the initial logic of the Program sought to create a consortium, as happened in Antioquia. However communication difficulties and differences of views, which were not originally identified during the selection process, affected the level of efficiency in the first year, and led to a review of activities by University. This process caused a loss of effort and time in management and results. The administrative relationship between the PUJC and AU coordinators assumed that teachers were interested, but PUJC did not assumed their duties as part of quarterly activities report. Although AU facilitates such management, it was not compensated for the AU academic support US-Colombia Partnership Program – Evaluation report Page 7-25 coordinators. Therefore, there was not an efficient discharge without compensation report of the academic support. In addition, the support of AU was focused on gathering information without an analysis of the effects of the reported activities on the results. During the interviews, the evaluation team was informed that the mid-term evaluation did not meet the expectations of the local partners to be a space for self-reflection, diagnosis and adjustments of activities to achieve the objectives of the Program, and therefore the opportunity was lost to define new academic program and proposed methodological strategies. Effectiveness Assessing the level of effectiveness of the partnership of Valle del Cauca requires starting out with a reading of expected impacts for this region. So, it is reiterated that, according to the initial proposal of American University, results were expected as a consortium between the three universities as equal parts. However, as reported by AU, since the construction of the baselines it became evident that it was necessary to design a separate work for each university; otherwise, the execution of the activities and objectives of the Project ran the risk of not being implemented. In that sense, it is worth asking whether the selection itself of the two universities in Cali and the initial Program implementation efforts decreased or pushed the effectiveness of each partnership. A first deeper question is whether the vision of partnering the USC and PUJC (on the one hand, the USC with a view of formation of political actors and an emphasis on criminal law, and, on the other hand, PUJC with a training mission in private law) conditioned the effectiveness of the overall objectives of the Program. Emphasis or specialization on criminal or corporate law is not by itself restrictive of the potential of the Program. Those strengths can be and have been capitalized elsewhere in favor of the penetration of human rights in institutional arrangements and human resource capacity at the law schools. What is needed –and was not present in either Valle del Cauca cases-, was a deliberate strategy to build bridges between those areas of specialization and the culture of human rights within the law school: for instance, bridges between the new two human rights faculty at PUJC and the curriculum, the constitutionalization of other legal areas or the rest of the faculty. One is the impact of having a solid yet relatively isolated human rights curriculum, another is the impact of permeating the human rights imprint in every dimension and at every corner of the law school. The comparison, for instance, between PUJC and UN is telling in this regard. While the PUJC program remained relatively restricted to one area of law (human rights) plus experiential training in competitive contests and the legal clinic, Uninorte imprinted every action US-Colombia Partnership Program – Evaluation report Page 7-26 with a human rights perspective, be it the curriculum, research, the clinic, other extension and dissemination activities that cut across all faculties of the university. Additionally, it is worth noting that each university selected began the execution with different capabilities, institutional and human strengths and weaknesses. However, expectations and Program goals were set equally for the two partnerships despite the difference in the realities they came from. There was of course room for customization. Within the general standardized framework, schools could creatively introduce customized indicators for intermediate goals of their own. But creativity requires not only talent but also relevance and ownership. When a school, such as USC, is submerged in larger institutional risks and inefficiencies, the priority is survival at all costs, including protecting the mere survival of the Program. USC From the evaluation of the USC, it is evident that there was the investment of a high level of effort in time, planning, monitoring for the implementation of activities, particularly training of teachers, students and community, without achieving concrete results in strengthening of human rights culture in the region, better services to the community or sustainable institutional capacity. Regarding internal management of the Program, the benefits were limited by a lack of ownership by the leaders and teachers of the law faculty and of the University (which is highly centralized). The level of effort rested on few individuals –not with the institution. It went to the point that some of the initiatives of the legal brigades were carried out under the initiative and direction of a private law firm dedicated to pro bono activities in the defense of human rights. There was no guidance from the law school leadership on selection of priorities, linkages with regular curriculum or research or any other university activity. Neither there was participation of the school in monitoring or providing feedback to this firm’s pro bono services. The lack of logistical backing and coordination to support the work of the coordinator of the Program’s activities was also a restricting factor to the Program’s effectiveness at USC. As a matter of fact, the Program coordinators at USC had to do everything by themselves, from the smallest details of each activity to reporting and evaluation. Administrative and substantive functions were too heavy a burden –especially when the individuals involved are not full time employees of the university. It also reduced the hours dedicated to assess context, actions and dimensions of impact of their actions with the community regarding the results expected from the partnership. PUJC In the case of the PUJC and AU partnership, a key success factor for the effectiveness of the results can be linked to the creativity and time commitment of those faculty members in charge of the Project. As a matter of fact, the University provided the necessary incentives and support when it created two full time positions for human rights (there were none before) and appointed US-Colombia Partnership Program – Evaluation report Page 7-27 program leaders to these two academic positions. That action is, by itself, a contribution to the strengthening of the culture of respect for human rights in the region. On the other hand, personalizing the Program and, more broadly, human rights activities within the school in the two recently appointed faculty members, without strong linkages to the rest of the law school, limited the effectiveness of the achievements of results. For example, the Program coordinators imprinted their (restrictive) view of the legal clinic to primarily self￾awareness and training of students, and thus they lost sight of the broader options of the human rights legal clinics –as promoted, for instance, by the Colombian Network of Legal Clinics. The particular focus of the assistance of AU also limited the vision and the range of results at PUJC. Since the AU teamwork focused on quantitative data collection for submission of reports and offering training spaces at WCL, those were the brand of the Program most developed by PUJC. No wonder PUJC had little visibility and weak connections with other institutions –even other Catholic church institutions-- devoted to the protection of human rights in the region, such as the Archdiocese of Cali. Results The impacts of Valle del Cauca Partnership should be viewed as two sides of the same coin: on one side the regional impact of the Project and, on the other side, the impacts in the universities. Regionally, internationalization of both universities in the framework of the activities and summer programs at WCL is highlighted. Additionally, PUJC law program geared towards strengthening the skills of its students in oral presentations, briefs writing and moot court skills (as tool of experiential education) is also recognized. Additionally, international training at AU and the internships at the IACHR and Inter-American Court generated great opportunities for all beneficiaries of the Project. Although the importance of training within the framework of the Project is recognized as an exercise to strengthen the students’ skills, the emphasis and dedication during the three years in the activity crowded out other potential activities with more direct and more visible impact on the enhancement of human rights in the region. The selected initiatives had limited scope of training for a group of students, and are restricted to the university area. Therefore, the universities appeared to have lost sight of the formulation of relevant actions to achieve institutional strengthening and care of the problems of human rights. During the fieldwork, the evaluation team had the opportunity to meet with other law professors and key players responsible for working with communities of Cali on human rights issues, who underscored the disconnection of the universities from the social reality in the region. In particular, the participating universities underestimated the social reality; they worked on projects and issues with little demonstration or precedent effect. The Program as such had little or no US-Colombia Partnership Program – Evaluation report Page 7-28 visibility for those entities, such as church service units, whose mandate is to enhance human rights in the region. In that sense, the internationalization of both universities reflects the academic and political strength and value-added of AU, in particular regarding the inter-American human rights system. It expanded the strategic capacity of teachers and students in the design of litigation strategies and legal action, and allowed them to refine their national and international litigation. However, overweighting the importance of the globalization has also the potential to reduce the level of impact at national and community level. Therefore, the approach on training and activities in the inter-American system reduced the possibility for local universities to expand their networks with NGOs, leaders and key players in the region to strengthen the specific needs of the Valle del Cauca on national actions regarding human rights. US-Colombia Partnership Program – Evaluation report Page 7-29 Noticeable Valle Activities, Products and Intermediate Results --Objective by Objective of the Program— In reference to objective 1, the teachers in USC integrated in their law pedagogy, the methodology of practical solution of cases and experiential training. Moreover, the USC extended the teaching of human rights to students that live in rural zones. Both universities accomplished offering the human rights courses in the first semesters of law school as an elective subject. In particular, the PUJC created an area of concentration in human rights, which is offered to the law students. The PUJC strengthened the institutional capacity by appointing as full time professors the coordinators of the human rights area, including more dedication and incentives to research in this area, as well as the creation of a physical location for the office of the human rights clinic in the university. Regarding objective 2, the USC made agreements with private offices to position their socio￾legal work in the communities; the PUJC consolidated the “consultorio jurídico” through the work of the legal clinic, which generated internal processes of training for the students, in caring for victims of and research on human rights, as well as developing internal regulations in order to differentiate between the work in the clinic and the “consultorio jurídico”.The PUJC has generated some trust with some sectors of the community, in order to position the “consultorio jurídico” and the clinic as a center of legal aid and conciliation. The same university has generated spaces of dialogue and assistance with the social responsibility program at the university, as means of mainstreaming the human rights approach. The PUJC has created its own methodology; its legal clinic has developed methods to train law students through experiential training. It’s also important to note the development of an observatory and an online portal on human rights that allows for the dissemination of news, events and research on human rights in the region, all at the PUJC. Under objective 3, the students have generated their own fundraising strategies to participate in international training scenarios, such as competitions and internships; enhanced oral capacities of the students from both universities that actively participate in research centers and human rights competitions. The students of the USC have learned to analyze the contents of the areas of criminal law with a human rights approach, which allows vivid discussions in the classrooms. An important amount of students of both universities have had the possibility of training in the summer courses of the international law school of the American University. The students of the universities understood that the practice of human rights implies risks that should be valued in every case and project. US-Colombia Partnership Program – Evaluation report Page 7-30 Sustainability The sustainability of the Project in USC is very limited. The University and the law school, while having a high interest in continuing with the Project, did not create incentives to ensure adequate human and financial resource for it. There is a high interest in the new deanship to expand international relations through agreements with foreign universities. However, when the Dean was asked about the ultimate goal to expand networks, it appeared for him to be a measure of positioning of the University outside the general and specific objectives of the agreement. The level of sustainability of the“semilleros” research program and implementation of the legal clinic is very weak due to: i) the high level of hierarchy and bureaucracy for the decision-making and implementation of such projects; ii) the small number of university students who are part of the “semillero”; iii) the lack of full-time faculty to internally lead the activities carried out so far; iv) the low positioning of the USC to solve the demands of the community, since the University is not positioned as a center for conflict resolution and advocacy. According to one of the coordinators, it is highly unlikely that the topics of research in human rights developed by the Program continue without the USAID support. This emphasis comes at a very inopportune time Program wise because, until now, the institution has not yet realized the dimension and benefits of the partnership. For PUJC, the following institutional results are recognized as sustainable effects: • The construction and institutionalization of processes of experiential legal training; • The expansion of the human rights approach with the Social Responsibility Program of the University and other university programs, which is an important discussion for the region in terms of the role of the business sector in human rights; • The strengthening of human rights in faculty and the law curriculum. US-Colombia Partnership Program – Evaluation report Page 7-31 The Caribbean Coast partnership Relevance The Colombian Caribbean region is composed of eight departments.60 Its population consists in a blend of indigenous people, Afro-descendants and persons coming from various waves of migrations (especially from Europe and the Middle-East). The situation of human rights is of great concern because of the presence of military forces, paramilitary forces and the guerrilla. It particularly relates to political persecution, misfeasance and social intolerance.61 Human rights defenders, trade unionists, displaced persons, students and teachers, LGBTI persons are amongst the most affected categories of persons. The lack of information, investigation and data on human rights constitutes, in that regard, a particular concern. The design of the Caribbean Coast partnership62 appeared to be adequate, affecting positively the level of impact and benefits of activities and partnerships built from both Colombian universities. The clarity of its objectives translated into actions by the three universities (the two Colombian universities and the University of Florida) allowed achieving the overall objective of strengthening human rights in the region. Various factors contributed to it: first, the process of selection and design of the proposed partnership of the Caribbean Coast made evident the needs of each university and the situation of human rights in the region; second, UF previous knowledge of the region, and its recognition of the different installed capacities between UM and UN to leverage them during the term of the contract; third, a multidisciplinary vision of the partnership that integrated three units at UF: CGR, CLAS and UF COE. UM There was a high level of relevance of the partnership between UM y UF to the ultimate goal of strengthening the culture and protection of human rights. On the one hand, it responded to the absence of a human rights culture in the communities and the need to foster a culture of peace, reconciliation and respect for the fundamental rights. The partnership is even more relevant in the context of a peace process. Additionally, the legal clinics tackled relevant issues for the context: rights of detainees, including minors, or the rights of conflict-related victims. Verbatim, faculty emphasized that the agreement with the UF allowed addressing the situation of human rights from the methodology of popular education to address the situation of conflict victims and prison conditions. The Program showed from different angles how a HEI could 60 Departments are one of the administrative territory subdivisions. 61http://www.cederhnos.org/home/documentos/violacion-a-los-derechos-humanos-en-el-caribe-colombiano￾incidencia-al-interior-de-la-universidad-publica 62 In this part, “partnership”, “Program” or “Project” are used to refer to the sub-award betwen UF, UM and UN. US-Colombia Partnership Program – Evaluation report Page 7-32 generate strategies for promotion and attention in a community greatly affected by stereotypes and needs in the areas of discrimination, gender and violence. UN The partnership in UN presents a number of special features that increases its relevance. First, its political influence transformed the school in a human rights think tank on human rights and a legitimate actor to influence public policy. During the visit, all groups working with the law program concluded that working with UniNorte legitimizes the work of local impact and may have potential impact on legislation and public policies. A second aspect was determined by the relevance of the investigations carried out under the Partnership and its outreach initiatives.63 Efficiency The UF took the time to getting to know the context and the reality of each of the universities it would work with, which greatly contributed to the success of the Program in the Caribbean coast. It allowed orienting and potentiating the activities and results from the strengths of each university, as well as modulating resources and investments. The nature of the selected public university, UM, was a foreseeable risk that, as in Antioquia, was not properly valued and that could have affected the management efficiency and results of the Partnership. However, despite the difficulties, UM management was able to circumvent the university’s cumbersome administrative procedures and reduce the risk and uncertainty of budget and faculty decisions. While the baseline was built with similar characteristics of the other Program regions, this partnership, especially UniNorte, managed to successfully implement the three broad objectives of the Program. The main mechanism that positively articulated all activities for a common purpose was the research activities. In that sense, the information collected in different studies allowed the training of the students, managed to justify the actions of legal action from the center of human rights for better decision-making process, and strengthened the knowledge of teachers on the human rights-based approach. This effort was supported by UF assignment of an expert professor in each line of research at the UN human rights center. 63 NGOs and community leaders acknowledged the great impact of the actions carried out by the Human Rights Center. For instance, the presence of five students in Kusuto NGO office has increased the services offered to the population. On the other hand, Caribe Afirmativo asserted that the partnership between this minority group and UniNorte allowed to include the agenda of LGBTI rights in national law regarding victims of the conflict. US-Colombia Partnership Program – Evaluation report Page 7-33 In relation to the resources invested in the mid-term evaluation, the evaluation team had the perception that the methodology used, the time and the findings of this evaluation were not adequate to produce a significant review and/or an adjustment of the Program activities. Likewise, it did not meet their expectations to be a space for strengthening regional and national partnerships. In any case, the ongoing high priority and the quality of thematic investigation were likely to trigger the expected Program impact. UM Concretely, for UM, it is worth emphasizing that the coordinators had a perception of the lack of support by UF to the concrete actions from the human rights center and its brigades. However, they recognized that the monitoring and the activities UF team conducted were very effective in promoting some initiatives that were temporally frozen. It gave confidence to the university leaders and a reason to students, teachers and the law school dean to achieve the planned activities in the best possible way. During the fieldwork, it was stressed to the evaluators that the level of detail, time, complexity and lack of training in the preparation of reports by activities reduced the ability of the law program in planning strategically for results management. This generated high levels of stress, which limited the time for the coordination of brigades and curriculum strengthening. However, the dean and coordinator acknowledged that, after being part of this Program, they had more tools to measure their own goals, so they had acquired a monitoring capacity as the result of participation. Finally, the coordinator acknowledged the efficiency of the relationship and support that UF team had to improve evaluation reports. According to the dean, his visit and stay in the UF in the first semester of 2015 was one of the activities that gave a strong boost to the project because: i) it strengthened the relationship with UniNorte; ii) it allowed strategically plan activities of law program for strengthening human rights within and outside the university campus; iii) it allowed an assessment of achievements and future challenges. During field interviews, it was felt that the support of the Red Colombiana de Clinicas Jurídicas and especially the legal clinic of the Universidad del Rosario (Bogotá) was a boost that benefited and supported program activities. Finally, the staff of UM reiterated that the midterm evaluation process did not meet their expectations and therefore missed an opportunity to re-conceptualize activities, partnerships and goals for the final project. UN During meetings in UN, it was informed to the evaluation team that although management of reports was a dimension of project implementation, such activity was headed by the coordinator to ensure that the teachers and the dean were devoted to the overall objective and were not limited to conducting and reporting activities or processes. In addition, it was highlighted that the monitoring matrix did not allow for accounting for the process and key steps to achieving US-Colombia Partnership Program – Evaluation report Page 7-34 overall objective. Therefore, they often stopped reporting those creative activities leading to high level, qualitative achievements. For the Program team, the level of complexity of the monitoring matrices was made manageable owing to the experience in the coordination of monitoring and evaluation, as well as to the understanding of the dean and the whole team that the activity report was a measurement tool but not a roadmap for the main goal of positioning the human rights program within and outside the university. Effectiveness The high level of effectiveness of the Caribbean coast partnership was largely explained by:  The level of commitment by UF to capture and analyze the regional context and human rights situation. It allowed UF to weight and stimulate priority activities leading to the achievement of overarching Program objectives. Additionally, it allowed building a very close partnership to advance relevant research among US and Colombian teachers.  The high level of commitment of the partnership’s coordinators is evident in the quality and time devoted to students and project’s activities.  The high level of understanding and vision of the coordinators to implement the Program from its three dimensions: institutional strengthening, human resources and community outreach work as a single unit to reduce levels of inequality in the region and improve the situation of human rights in Santa Marta, Barranquilla and the surroundings.  The ability to capitalize on an experienced team of teachers and researchers.  The particular research capacity, knowledge and professional experience of the teams of professors at the law clinic. UM Regarding UM, it is worth mentioning that the level of effectiveness was very low in the first year of the Program for two reasons. The first one was very predictable: the selection and design of the partnership. The second one is that coordination was unlikely from the onset. It entails that, during the startup phase, implementation and management were conditioned by the timing and lengthy procedures typical of a public university. The low capacity of the initial coordinating groups to run the activities of the sub-award was solved with the arrival of the current coordinator, who is known for her commitment, dedication and ability to generate internal and external alliances for the life of the partnership. Once the UM overcame these obstacles, several factors permitted to reach better results. US-Colombia Partnership Program – Evaluation report Page 7-35 First, the composition of the school of humanities of the University and its integration with the programs of law, anthropology and cinema, were factors that potentiated the partnership. This multidisciplinary vision and training of students allowed: i) to extend the training of graduates in human rights to students of disciplines other than law with high awareness of and impact on community work; ii) to use audiovisual tools to encourage discussion and culture of human rights beyond the classrooms and legal discussions into the University; iii) to develop inter￾disciplinary visions through the discussion of “semillero” projects to grant students great skills to face human rights debates in the region, and iv) to enrich legal approaches with anthropology research methods such as participatory observation. Second, the commitment, knowledge and vision of the teachers in charge of the research “semillero”, legal brigades, and legal clinic research increased the effectiveness towards the overarching goals of the Program. This was made evident by the quality of each teacher’s plan of action with a baseline, indicators and expected community impact. It was further exemplified by the level of investigation about the community needs in the prison of Santa Marta, the students’ strategic prioritization of human rights violations that led to selection of the best alternatives for advocacy and legal strategy to improve the lives of women in the prison of Santa Marta. Other results can be cited: the partnerships that were generated with key actors in the city; the legitimacy of the law program in the community; the creation of a strategy of comprehensive care in prisons with emphasis on health, and short literacy legal courses for prisoners; diagnosis of the situation of prison overcrowding with the possibility of addressing public policy at the local level. The human rights center of the UM did an excellent job for the attention of persons deprived of their liberty, which led it to identify the main human rights violations in the Santa Marta prison. As the result of its advocacy work, women detainees were transferred to another penitentiary center, as a means to secure their health rights and due process rights. A third success factor is constituted by the ability of the coordination team to build relationships outside the Program, specifically with the Rosario clinic and the Red Nacional de Clinicas Jurídicas. During the interviews with students, they repeatedly said that they felt supported by the Rosario Clinic and recognized the benefits of this collaboration. It is also worth mentioning that, towards the end of the Program, UM was strengthening joint actions with Uninorte and the partnership with UF was also moving upwards. For example, the evaluation team was informed that the University of Magdalena would make the proper follow￾up of a Uninorte legal action files in a court in Santa Marta. Such relationships and concrete support generated very positive links between teams of teachers and students. UN UN proved that by planning in a combined way the three areas of Program intervention it is possible to begin to change the human rights environment in the region of influence of the US-Colombia Partnership Program – Evaluation report Page 7-36 university. To this effect, the law school committed itself to goals in each one of the areas of the Program, and combined those goals to reach the overarching expected impact from the Program. For instance, the law school institutional capacity included policy- and action-oriented multi-disciplinary research to protect the environment and the right to life in the Ciénaga Grande de Santa Marta. Quality research involved research training for students in the legal clinic, led by seasoned faculty with a commitment to utilize research results towards preparing precedent￾setting legal cases before the courts. Their research and outreach services were balanced between student training and community priorities in such a way that the human rights￾community issues remained at the center of university action. Those activities were then actively disseminated and discussed with the entire academic community and potential stakeholders, thereby enhancing awareness and sensitiveness within the entire campus. Since Uninorte is a pro-business school, managers from main corporations in the region quickly become aware of the issues and approaches led by the law school and expanded their views on corporate social responsibility. Furthermore, young graduates from Uninorte Law School keep their ties with the clinic and action-research for pro-bono activities even after they have become practicing lawyers in prominent positions in the area. Since the student body of the law school comes from all social strata, low-income communities are also aware of university priority policies pertaining to positioning human rights issues throughout the region. Uninorte exceeded the expected effectiveness of the Program in the areas of language, curriculum, research, activities and awareness-raising throughout the university. This law school has a human rights vision that was advanced by the Program. It also combined the Program components in unifying strategies. For instance, each action of the human rights center was linked to the training of teachers, students and change of the traditional teaching approach. Additionally, the division of functions within the law school team and the high level of expertise of each team member allowed individuals to foster a specific area of the Program while contributing to the common, higher results. In particular, the full time dedication of the Program coordinator, as well as her managerial ability strengthen the coordination and collaborative approach of teachers, researchers and directors at the law school and within the entire university. US-Colombia Partnership Program – Evaluation report Page 7-37 Results The impact of the Program on the Caribbean coast appears to be broad and far-reaching because of the dynamics of management and effectiveness towards the strengthening of culture and promotion of human rights in the region. The future impact on the national agenda can be linked with the possibility that the partnership has to position their research in debates at the national level as well as within city and the Caribbean region. More broadly, the following results are worth highlighting • The high level of legitimacy of the human rights centers in the two universities, including the recognition of their impact by NGOs, victims groups and local leaders who defend human rights, especially in issues such as the rights of detainees; LGBTI; population of the Ciénaga Grande de Santa Marta; Afro-Colombian population and victims of the conflict. • The positioning of the Universidad del Norte as a prominent think-tank and center of regional advocacy regarding the human rights issues of the most vulnerable population. • The positioning of the Universidad de Magdalena as a center of education for peace and human rights training for children and adolescents in high-risk areas of Santa Marta and around. • The positioning of the research and academic programs inside the UN law and human rights including the lectures and seminars on human rights for the entire community. • The depth and immediate relevance (policy and action oriented) of the research conducted at the two universities: in the specific case of UM research on the situation of persons deprived of liberty; in the case of UN, the investigations related to violations committed by the warring parties on the Caribbean LGBTI population. These investigations have the ability to be unique documents that can elevate regional issues to national policy debates. US-Colombia Partnership Program – Evaluation report Page 7-38 Noticeable Caribbean Coast Activities, Products and Intermediate Results --Objective by Objective of the Program-- Under objective 1, it is worth noting the commitment of the UM to open two faculty positions for the human rights area; courses were consolidated (specialized courses) in order to offer human rights training throughout the university as well as to external participants; human rights obligatory courses were opened, for the students of the faculty and also elective courses, for all the students of the UM. Likewise, UN signed a cooperative agreement with the CLAS of the UF to support research on human rights; human rights issues have been positioned in university events, teacher’s visits and in areas belonging to private law; the consolidation of an institutional web page to position and disseminate the research on human rights; books and articles were published, which position the areas of research in human rights as a crucial matter for the faculty. Regarding objective 2, the UM structured, created and launched the human rights center as a space for discussion and meeting for the students interested in issues such as prison rights, LGTBI, peace processes, among other topics led by the human rights center. UM teachers have defined areas of attention and research in key matters to the population, such as: human right education; protection of the rights of persons deprived of liberty; the rights of LGBTI rights. It has strengthened the popular/civic education among adolescents and socio-economic marginalized groups. Alliances were built between the UM’s human rights center and social leaders. Meanwhile, UN strategically defined the different lines of action in the human rights center and consolidated the relations with key stakeholders for the promotion of rights. Under objective 3, the center of human rights of the UM has attracted interest and participation of anthropology and film students in human rights issues; the skills in oral argumentation and research have been improved among the students of the UM and UN. Methodologies for training in interviewing and caring for victims of violence, were identified and disseminated. Students’ publications in human rights issues have been encouraged in both universities. The spaces of support for students have been generated by both universities. US-Colombia Partnership Program – Evaluation report Page 7-39 Sustainability There is enough evidence to expect sustainability of some/most features of the Program at the two universities. Some factors that can threaten sustainability are briefly presented below. Because of the high level of relevance and impact of the work of two legal clinics, the sustainability may be threatened by potential risks to the safety of students and teachers project leaders in the territories; possible pressure from economic groups, state agents and the Ombudsman; the graduation and subsequent employment and professional career of those students that have benefited from and have enthusiastically participated in the Program. UM Concretely, the progress made with “semillero” and the human rights center of UniMagdalena can be threatened by the withdrawal of funds from the Project, especially because the financial capacity of a public university in Colombia is limited for purposes of compensating Program coordinators or full time teachers. In this regard, it appears that an opening of two positions for full-time professors in the human rights area is a sine qua non condition for the sustainability of the Program in the medium term. Additionally, the persistent barrier of language and cumbersome management of a public university make it more unlikely that UM can receive supplementary funding from USAID or other development cooperation agency. It should be underscored that, at this point in time Program, sustainability appears to depend largely on the decisions the “Vicerrectoría de Extensión” since this office of the University has funding resources from private companies who may be interested in the project’s objectives. Regarding the future partnership with UF, it was said that, if a second stage is conceived, it would be essential to rethink the areas of support and funding. Supplementary partnerships may be sought with the support of the “Red Colombiana National de Clínicas Jurídicas” or of the UniNorte. UN The Uninorte is at a higher level of individual sustainability compared to other national partnerships (Antioquia and Valle del Cauca) due to availability of a full time Program coordinator and this person’s high capacity in the areas of program management and financial resource management. At the closure of the Program, the evaluation team was informed that an assistance agreement was reached between the UN law school and UF Latin-American Studies Center, which clearly constitutes an indicator of f the excellent relationship built between the two institutions. US-Colombia Partnership Program – Evaluation report Page 7-40