EVALUATION REPORT COLOMBIA TRANSFORMA PROGRAM MID-TERM PERFORMANCE EVALUATION May 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Luis Bernal, Adam Isacson, and María Victoria Whittingham for Social Impact, Inc. Photo Credit: Adam Isacson i COLOMBIA TRANSFORMA PROGRAM MID-TERM PERFORMANCE EVALUATION EVALUATION REPORT USAID/Office of Transition Initiatives (OTI) AID-OAA-I-13-00006 QOl 1-OAA-15-00012 PDQ III TO 10 Submitted: May 30, 2018 Luis A. Bernal, MPP (Team Leader) Adam B. Isacson (Senior Policy Expert) María V. Whittingham, Ph.D. (Monitoring & Evaluation Specialist) Social Impact, Inc. 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201 Telephone: 703.465.1884 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This Page Intentionally Left Blank i TABLE OF CONTENTS ABSTRACT........................................................................................................................................................................III ACRONYMS..................................................................................................................................................................... IV EXECUTIVE SUMMARY..................................................................................................................................................V INTRODUCTION............................................................................................................................................................1 BACKGROUND AND CONTEXT.........................................................................................................................1 PROGRAM DESCRIPTION .......................................................................................................................................1 EVALUATION PURPOSE AND EVALUATION QUESTIONS.............................................................................3 EVALUATION PURPOSE...........................................................................................................................................3 EVALUATION QUESTIONS.....................................................................................................................................3 EVALUATION METHODS AND LIMITATIONS.....................................................................................................5 SAMPLING .....................................................................................................................................................................5 DATA COLLECTION METHODS ..........................................................................................................................5 DATA ANALYSIS .........................................................................................................................................................6 BIASES AND LIMITATIONS......................................................................................................................................7 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ..............................................................................8 EVALUATION QUESTION 1A. ...............................................................................................................................8 EVALUATION QUESTION 1B...............................................................................................................................12 TOPICS OF INTEREST 1C.......................................................................................................................................14 RECOMMENDATIONS FOR EVALUATION QUESTION 1. ........................................................................17 EVALUATION QUESTION 2. ................................................................................................................................18 EVALUATION QUESTION 2A. .............................................................................................................................18 EVALUATION QUESTION 2B...............................................................................................................................22 EVALUATION QUESTION 2C. .............................................................................................................................25 EVALUATION QUESTION 2D..............................................................................................................................28 EVALUATION QUESTION 2E. ..............................................................................................................................31 TOPICS OF INTEREST 2F........................................................................................................................................33 RECOMMENDATIONS FOR EVALUATION QUESTION 2. ........................................................................35 ANNEXES .........................................................................................................................................................................37 ANNEX A. EVALUATION SCOPE OF WORK.................................................................................................37 ANNEX B. EVALUATION TEAM MEMBERS......................................................................................................46 ANNEX C. DATA COLLECTION TOOLS.........................................................................................................47 ANNEX D. EVALUATION DESIGN MATRIX...................................................................................................66 ANNEX E. INTERVIEWEE LIST..............................................................................................................................71 ANNEX F. DOCUMENT REVIEW SOURCES ...................................................................................................76 ANNEX G. DISCLOSURE OF CONFLICT OF INTEREST.............................................................................78 ii LIST OF TABLES TABLE 1. STRATEGIC FRAMEWORK OF COLOMBIA TRANSFORMA TABLE 2. DISTRIBUTION OF DATA COLLECTION METHODS (PARTICIPANTS BY SEX AND REGIONS) TABLE 3. AWARD CONTACT PERSON DISTRIBUTION BY SEX 2 5 34 LIST OF FIGURES FIGURE 1. COLOMBIA TRANSFORMA DECISION-MAKING TIME 10 FIGURE 2. TIMING AND LOCATION OF INFRASTRUCTURE ACTIVITIES 14 iii ABSTRACT The United States Agency for International Development (USAID)/Office of Transition Initiatives (OTI) engaged Social Impact, Inc. (SI) to conduct a mid-term performance evaluation of the Colombia Transforma program (2015-2019), which aims to increase the Colombian government’s ability to plan and execute rapid response elements of the Peace Accord and to promote collaboration between territorial actors in support of the Accord’s implementation. The evaluation assessed the program’s strategic approach, relevance, and contributions, while offering suggestions to improve its implementation and inform United States Government (USG) assistance in Colombia. The evaluation was conducted by a three-person team between January and April 2018. The evaluation concluded that while the program has focused appropriately on rapid response activities relevant to the Peace Accord’s success, its performance is slowing down due to the volume of activities and other constraints. The program has been more effective in improving the presence of Colombian government entities created for the post-conflict period, and some local entities, than that of the national government as a whole. It has promoted collaboration among local actors, particularly through small infrastructure activities and resources otherwise unavailable. The program, however, faces challenges due to a changing political environment, concerns about the viability of Peace Accord implementation, interpretation of some USG counter-terrorism statutes, and operational considerations. The evaluation offers recommendations to improve the program’s performance. iv ACRONYMS ANT Agencia Nacional de Tierras ADR Agencia de Desarrollo Rural ART Agencia de Renovación del Territorio ASCAMCAT Asociación Campesina del Catatumbo ASOJUNTAS Asociación de Juntas de Acción Comunal CISCA Comité de Integración Social del Catatumbo DO Direct Observation ELN Ejército de Liberación Nacional (Guerrilla group) ET Evaluation Team EQ Evaluation Question FARC Fuerzas Armadas Revolucionarias de Colombia (Guerrilla group) FARC Fuerza Alternativa Revolucionaria Común (Political Party) FGD Focus Group Discussion FTO Foreign Terrorist Organization GOC Government of Colombia (the three levels thereof: national, departmental, and municipal) INL U.S. Department of State Bureau of International Narcotics and Law Enforcement Affairs KII Key Informant Interview LAC Latin America and the Caribbean LOE Level of Effort MCP Movimiento para la Constituyente Popular MEROS Mesa Regional de Organizaciones Sociales MSI Management Systems International NGO Non-governmental organization OACP Oficina Alto Comisionado para la Paz OHCHR Office of the High Commissioner for Human Rights OTI Office of Transition Initiatives PE Performance Evaluation PDET Programa de Desarrollo con Enfoque Territorial PIC Projectos de Pequeña Infraestructura Comunitaria PNIS Programa Nacional Integral de Substitución (de cultivos ilícitos) PPR Program Performance Review SCO Security Cooperation Office (U.S. Embassy, Bogota) SI Social Impact, Inc. SOW Scope of Work SRS Strategy Review Session UN United Nations USAID United States Agency for International Development USG United States Government v EXECUTIVE SUMMARY Program Background. In 2016, after more than 50 years of armed conflict, the Government of Colombia (GOC) and the largest insurgency group, the Revolutionary Armed Forces of Colombia (FARC), signed a Peace Accord in Havana, Cuba (hereinafter the “Peace Accord”). Consequently, the FARC has formally demobilized and the GOC has begun to enact wide-ranging reforms related to root causes of the conflict: rural development, inclusive political participation, transitional justice, and the reduction of illicit crops such as coca. The United States Agency for International Development (USAID)/Office of Transition Initiatives (OTI) launched the Colombia Transforma Program through a task order issued to Management Systems International (MSI) for the period of July 24, 2015 through July 23, 2019. Part of the United States Government (USG) effort to support a sustainable and inclusive Colombian peace, the program has operated with a USD 43.2 million budget through Fiscal Year 2018. It aims to increase the GOC’s ability to plan and execute rapid response elements of the Peace Accord while promoting collaboration between territorial actors for its implementation. The program operates in Bogotá, Arauca, Norte de Santander, and Putumayo. Evaluation Purpose. OTI engaged Social Impact, Inc. (SI) to conduct a mid-term performance evaluation of Colombia Transforma. The objective of the evaluation was to assess the program’s strategic approach, relevance, and contributions. Evaluation findings, conclusions, and recommendations will be used to improve program implementation and inform future USG assistance in Colombia. Audiences and users of the evaluation include OTI, MSI, USAID/Colombia, U.S. Embassy in Bogotá, local stakeholders including the GOC, and other interested groups in Washington, DC. Evaluation Design and Methodology. A three-person team conducted the evaluation from January to April 2018, using document review, key informant interviews, focus group discussions, and direct observation. In total, the Evaluation Team (ET) collected data from 164 key informants (61 female and 103 male) in Washington DC, Bogotá, Arauca, Norte de Santander, and Putumayo. In addition to the potential for response bias and selection bias, generalizability of data is limited given that the sample of key informants was determined, in part, by geography, accessibility, and availability. Triangulation enabled the ET to verify and cross-validate its findings to determine Colombia Transforma’s overall effectiveness. EVALUATION QUESTIONS (EQS), CONCLUSIONS, AND RECOMMENDATIONS EQ 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? Colombia Transforma’s strategic approach is largely relevant to Colombia’s local context and closely aligned with U.S. foreign policy objectives. The program succeeded in executing activities at a pace far more rapid, and with a greater level of transparency, than that to which communities and state entities were accustomed, although the pace slowed somewhat after 2016. Colombia Transforma has mainly helped lay the groundwork for the Peace Accord’s first chapter on rural development and reform. However, the current interpretation of U.S. statutes prohibiting material support for members of foreign terrorist groups has complicated work in communities whose members include ex-guerrillas. EQ 1A. ASPECTS OF THE PEACE ACCORD Of all chapters of the Peace Accord, Colombia Transforma is most oriented toward implementing Chapter 1 (“Comprehensive Rural Reform”). The other chapters receive less emphasis due to program design, various conditions, or external policy restriction. This focus appeared to be the correct choice under the circumstances. The program has mostly succeeded in responding rapidly, but its speed is affected by the pace of decision-making, volume and complexity of activities, staff turnover in some regions, purchasing from vendors in territories where informal economies predominate, and other operational problems. Colombia Transforma has a window of opportunity in Norte de Santander because important processes have begun in one of the most complex and insecure regions in Colombia. Throughout territories, the vi program’s relevance, in terms of strategic coherence of activities, is weakened by insufficient communication, exchange, and learning among beneficiaries, staff, GOC, and USG partners. EQ 1B. PROGRAM MISSED OPPORTUNITIES U.S law penalizes all who knowingly provide material support or resources to a foreign terrorist organization. FARC is considered by the USG to be a terrorist organization, although, according to Colombian law, it is now a legal political party and local actor. Now that 14,000 ex-FARC members are at large—many in communities where Colombia Transforma seeks to improve rapid response—the likelihood of inadvertently conferring a benefit upon them has increased. This threatens to paralyze some activities in an absence of clarity about the statute’s applicability to low-rank individual ex-combatants. Meanwhile, Colombia Transforma generally avoids crop substitution, defined as assistance conditioned on prior voluntary eradication of illicit crops. This is in part due to the USG prohibition against supporting the Colombian government substitution program established to fulfill Chapter 4 of the Accord. Instead, by backing state presence and economic production, support for Chapter 1 activities may have the salutary effect of encouraging long-term, if slower and less direct, reductions in coca cultivation. EQ 1C. OTHER TOPICS OF INTEREST Especially after the Peace Accord’s implementation began at the end of 2016, Colombia Transforma made few adjustments to its strategic approach. The major one was an increase in infrastructure activities, which is more an operational process than an adjustment of the program’s strategy. Colombia Transforma has reduced local resistance to U.S. assistance programs, even in areas strongly dominated by guerrilla groups. Stakeholders said that the USG has learned from the shortcomings of Plan Colombia, its 1999-2015 aid, diplomatic, and military initiative against drug cartels and guerrillas. Colombia Transforma could fertilize the ground for future National Liberation Army (ELN) negotiations by creating a “demonstration effect” in zones of heavy ELN influence. Stakeholders’ concern about the continuity of activities and support after Colombia Transforma indicates a risk of increased community frustration about the lack of continued presence of GOC and foreign aid after the program ends. EQ 1 RECOMMENDATIONS: 1. Continue to focus on the rural development and state presence aspects of Chapter 1 of the Peace Accord. Focus less on the chapter’s land tenure aspects, except for those related to ethnic communities. 2. Continue work in Norte de Santander. A “window of opportunity” exists in Norte de Santander due to Colombia Transforma support: mayors are collaborating on a common development agenda to an extent rarely seen anywhere in the country, Community Action Boards and producers’ associations have emerged as key social leaders, youth groups and an active church are playing a vital role, and the departmental government has reached out creatively to previously uncontacted victims. Despite persisting conflict, the program has managed to work in townships and municipalities that would have been unthinkable during the conflict’s worst period. Because of its security troubles, which are emblematic of post-Accord Colombia, and its status as a strategic border zone, the need to keep engaging has grown. It would be a mistake to pull back. 3. Raise awareness within the U.S. interagency about the need for common-sense guidelines for interpreting material support provisions so that they are congruent with Colombia’s current context. Trust that USAID and its implementing partners know not to strengthen a group on the Foreign Terrorist Organization (FTO) list. These guidelines should permit low-level engagement with ex-FARC who are (1) not top leaders; (2) not awaiting war crimes trials in the transitional justice system; (3) not facing U.S. indictments or extradition requests; and (4) reasonably determined to have abandoned violence. Keep a database of supported events with FARC presence for reporting and monitoring purposes. Only halt an activity if it violates the four criteria above, or if the database detects a pattern indicating an attempt to take over the space. Continue to support activities with non-FARC citizens around demobilization sites. vii 4. Urgently determine how to reduce delays at bottlenecks. Colombia Transforma staff and beneficiaries identified delays resulting from administrative bottlenecks at the “yellow light to green light” decision-making phase, and in purchasing and contracting in the field. Solutions may involve reviewing administrative burdens and streamlining bureaucratic processes. It is likely, however, that the number of person-hours these tasks require is unalterably high. If so, the program must increase staffing to meet the need. 5. Disseminate more information about program activities, outcomes, and lessons, including their replicability and relevance to the Peace Accord. • Grantees: Convene grantees and program consultants from different localities/regions, social groups, or types of activities to share experiences, skills, outcomes, and lessons learned. • GOC: Convene government representatives at the three levels, horizontally and vertically, to learn from each other, share experiences with Colombia Transforma, and explore opportunities for collaboration around the program. • USG: Develop materials for public dissemination on the OTI or USAID/Colombia website. Distribute occasional memos about select activities among U.S. mission staff and other international donors, to not only highlight successes but also explain what contributed to those outcomes, while frankly discussing challenges and lessons learned so that others may avoid “reinventing the wheel.” Meet more often with representatives of other sections of the U.S. mission whose security or counter-narcotics interventions may involve work in Putumayo, Norte de Santander, and Arauca. 6. Develop a systematic plan for program closure, with involvement by all stakeholders. In the remaining period of performance, engage potential sources of funding—donors (including USAID/Colombia), national government agencies, the private sector—to help sustain fragile processes. 7. Continue to support activities in ELN-influenced areas and around demobilization sites. EQ 2. To what extent has Colombia Transforma achieved its program objectives? Colombia Transforma support increased GOC rapid response capacities to a moderate degree and with substantial variation among national agencies, departmental governments, and local jurisdictions. There was little that a program of Colombia Transforma’s size and scope could have done differently to help the national government overcome longstanding incapacities for rapid response, although promising steps occurred with agencies created to implement the Accord. At the regional level, collaboration between local actors underwent a net improvement, although it is uneven, and its sustainability is very hard to gauge. Infrastructure activities and logistical support for events contributed significantly to progress toward both municipal government capacities and increased local collaboration. Much remains to be done, meanwhile, to help local actors understand and fulfill their roles in Peace Accord implementation, especially where other armed groups’ activity dulls any sense of being in a “post-conflict” environment. EQ 2A. GOC ABILITIES FOR RAPID RESPONSE Colombia Transforma has contributed to improved planning capacity of the national institutions created for the post-conflict period; however, improving implementing capacity of government at all three levels is too much to expect of a rapid-response program. The upcoming political transition in 2018 will complicate program efforts to improve government abilities to implement the Peace Accord, as it will involve large￾scale turnover of staff and the potential arrival of officials who do not share the same dedication to rapid compliance with Accord commitments. For the remaining program period, working with mayors and governors—who will be in office longer than national government officials— represents an opportunity for the continuity of Colombia Transforma. EQ 2B. PROMOTING COLLABORATIONS Colombia Transforma has made progress in strengthening capacities of local actors due to choosing appropriate partners, listening to local stakeholders, using a tripartite methodology (contributions from community, government, and Colombia Transforma), and bringing resources otherwise unavailable. With frequent success, the program identified credible, independent, and diverse actors—most with limited viii prior contact with U.S. assistance programs—and encouraged many to work, for the first time, with government. However, the development of collaborative relationships among different kinds of stakeholders was uneven in Arauca, Norte de Santander, and Putumayo. Reasons for insufficient collaboration among local actors might be the provision by Colombia Transforma of very few opportunities for mutual communication, exchange, and common learning, as well as occasional distrust, personal or political disagreements, or institutional rivalries among local actors. EQ 2C. MOST EFFECTIVE ACTIVITIES/APPROACHES Small-scale infrastructure activities—especially those that rely on the tripartite, shared-responsibility model—provide tangible evidence of state presence and a model for all (and especially local) governments to carry out rapid response in communities with which they had no prior relationships. Logistical support from Colombia Transforma made possible national government entities’ arrival in many communities, and cultural activities served as a pretext for initial rapid response in polarized communities. A common sentiment shared with the ET was that rapid response is most effective when it builds economic opportunities, mostly agriculture-based. Infrastructure activities foster collaboration between local actors that have had little prior contact or relationships. In the process, community members practice leadership, problem-solving, accountability, and how to relate to local government. Logistical support provided by the program connected people and made possible events that would not have happened otherwise, and Colombia Transforma found much interest among beneficiaries in forming and strengthening producers’ associations to work together. Community radio proved to be an educational and organizational tool for improving local capacities in all three territories. EQ 2D. PERCEPTION OF GOVERNMENT PRESENCE In general, local partners and beneficiaries in the territories perceived that government presence has increased at all levels: national, departmental, and municipal. But the perception is that presence has occurred to different extents; the municipal government has the greater perception of presence, followed by some national government entities, and trailed far behind by the departmental government. Colombia Transforma has helped to increase a perception that the Peace Accord matters for beneficiaries’ local context, although the national state’s sluggish arrival dulls this perception. EQ 2E. LOCAL UNDERSTANDING OF ROLES/RESPONSIBILITIES Through capacity-building activities, Colombia Transforma helped organizations to some extent to understand their new post-Accord roles. Program efforts to increase traditionally excluded actors’ engagement in public policymaking are a mechanism to foster leadership and responsibilities. Sub-national governments have limited understanding of what the Peace Accord means for their development plans and administrative responsibilities. In areas where armed groups continue to operate, a lack of a sense of being in a post-conflict environment hinders civil society leaders’ assumption of roles in Accord implementation, despite Colombia Transforma efforts. The program has, however, helped civil society organizations carve out space taken up by larger, more militant campesino confederations. EQ 2 RECOMMENDATIONS: 8. Reconsider the current selection of governmental partners to leave behind more installed capacity, giving priority to municipal governments that have shown will and are devoting resources. As the national government is undergoing vast personnel changes with an upcoming electoral transition, and since many national agencies have not responded assiduously to rapid-response imperatives, reduce investment in national government activities in favor of sub-national government for the rest of the program’s life. Remaining investment in national entities should focus on the new agencies created for Peace Accord implementation, and priority should be given to those that work more closely with the department and municipal governments, as well as with local communities, to maximize local capacity after the program. Watch these entities closely, however, for signs that the government assuming power in August 2018 is weakening or de-prioritizing them. Indicators would include reduced budgeting, effective demotions within institutional architecture, the naming of ix managers according to political criteria rather than expertise, or reduced commitment to consultation with communities. 9. Increase Colombia Transforma’s effective promotion of collaboration among local actors by choosing groups of current beneficiaries (for instance urban youth, indigenous, women’s, victims, or Afro-descendant groups) with which to undertake initiatives that (1) are designed and implemented with the express purpose that they work with each other, (2) involve peace-building, and (2) have potential to continue after the program closes. 10. Highlight the legitimate role of social leaders and the importance of protecting them while seeking to include victims as partners in activities. The moment that a beneficiary receives a credible threat, act aggressively to involve Colombian government entities with protection responsibilities (including but not limited to the Interior Ministry National Protection Department, Defensoría and Early Warning System, Fiscalía, and National Police). The urgency and assertiveness of the program’s response in the face of a threat should serve as an example to Colombian government counterparts of how seriously such threats deserve to be taken. 11. Continue pursuing activities that help Afro-Colombian groups take the first of many steps to legalize community councils. Emphasize women’s activities among civil society organizations and indigenous, Afro-Colombian, and youth groups while increasing overall support to women’s groups. In addition, increase the probability of “green-lighting” activities with partners from all social sectors or groups whose leadership is female. 12. Continue infrastructure activities and encourage use of the tripartite model in which municipal government, community leaders, and Colombia Transforma are all expected to make contributions. At this phase in the Accord implementation process, carrying out tangible actions like infrastructure and community-building activities is the best way to educate target communities about their roles and rights. 13. Continue providing logistical support, especially to efforts of municipal government, while finding legal, administrative, and budgetary ways to undo bureaucratic entanglements that prevent agencies from using their own resources for these purposes. The ART most needs help in getting beyond meetings and generating “early wins.” The Small Community Infrastructure activities (PICs) are an important contribution, but Colombia Transforma must help ensure that the PDETs are more than just a paper exercise. 14. De-emphasize top-down efforts to help national government agencies “arrive” in rural areas. Instead, focus on bottom-up approaches starting with municipal governments. Continue engaging with mayors’ offices, increasing support for those most inclined to respond rapidly and transparently, and all while strengthening civil society capacity. For the remainder of the program, activities with the national government should focus on new post-conflict agencies’ efforts to deliver on commitments already made in meetings, rather than additional planning, workshops, and events. 15. Community Action Boards should continue to be a central partner in Putumayo, Norte de Santander, and Arauca. Efforts to help the government “arrive” should foster—not weaken—the boards’ performance. In addition, continue strengthening independent civil society organizations and upholding their role with respect to more hardline and larger organizations that often have more interlocution with the national government. 1 INTRODUCTION BACKGROUND AND CONTEXT For more than 50 years, the GOC battled insurgency groups in an armed conflict that claimed 220,000 lives, displaced millions of people, and hampered the economy of a country rich in natural resources and human capital. The civil war exacerbated longstanding state weakness and structural inequities, which broke the country into two: a modern and urban Colombia; and a poor, neglected, and sparsely-populated rural Colombia, where most of the conflict has taken place. The rural regions have long suffered from a lack of government services, poor infrastructure, the presence of criminal groups, and illegal economies based on the coca trade. Some of the hardest-hit rural areas have been in the country’s indigenous and Afro-Colombian communities, which represent a disproportionate number of the victims of conflict violence. In 2012, the national GOC and the largest insurgency group, the FARC, started peace dialogues in Havana, Cuba. After four years of negotiations, on November 30, 2016, the two parties ratified the “Peace Accord for the End of Conflict” (the Peace Accord). Consequently, the FARC has formally demobilized and the GOC has begun to enact wide-ranging reforms related to root causes of the conflict: rural development, inclusive political participation, transitional justice, and the reduction of illicit crops such as coca. PROGRAM DESCRIPTION Tangible evidence that Colombia has changed for the better will contribute to a stable and lasting peace and will help build momentum for the Colombian government to implement the medium- to long-term post Accord transformation. With that in mind, USAID/OTI launched the Colombia Transforma Program through a task order issued to MSI on July 24, 2015. With a four-year period of performance through July 23, 2019, the program has operated with a USD 43.2 million budget through Fiscal Year 2018. The program is part of the USG effort to support a sustainable and inclusive Colombian peace. Colombia Transforma aims to help the Colombian government during the critical post-Accord period to follow through on the promises made and demonstrate immediate, tangible benefits of peace in rural conflict regions. Likewise, the program believes that increasing collaboration between territorial actors will build on existing potential and capacities at the municipal level that the national government has underestimated or dismissed. Such collaboration requires that community and civil society actors work with each other and with local and national government institutions to overcome longstanding tensions stemming from the conflict and confusion over their new roles and responsibilities under the Accord. The Theory of Change that underpins the Colombia Transforma Program assumes that contributing to Colombian readiness to implement rapid response activities in the immediate post-Accord period is critical. Demonstrated quick wins will generate confidence in the Accord when its acceptance is most fragile and will help convince Colombian stakeholders that the agreement was worth the major sacrifices made through negotiations. While the overall goal of the program is to enhance Colombian ability to implement rapid response during the 36 months following the signature of the peace agreements, program activities are designed to contribute to the achievement of two objectives: 1. Increase the ability of government to plan and execute rapid response aspects of the peace process. 2. Promote collaboration between territorial actors to lead local implementation of the Peace Accord. The program’s objectives and intended results can be found in Table 1 below. 2 Table 1. Strategic Framework of Colombia Transforma (September 2017) PROGRAM GOAL Enhance Colombian ability to implement rapid response during the 36 months following the signature of the peace agreements PROGRAM OBJECTIVE 1 Increase the ability of government to plan and execute rapid response elements of the peace process. PROGRAM OBJECTIVE 2 Promote collaboration between territorial actors to lead local implementation of the Peace Accord. R E G I O N A L A P P R O A C H E S ARAUCA RESULT 1 Government, grassroots organizations and social organizations participate in the peacebuilding in the territory RESULT 2 Grassroots organizations and social organizations find new ways of relating between each other RESULT 3 Government leads actions to implement the peace agreements PUTUMAYO RESULT 1 Civil society organizations are strengthened and work together to influence peace actions RESULT 2 The government carries out actions to implement the peace agreement by coordinating institutionally and with civil society organizations NORTE DE SANTANDER RESULT 1 Government entities lead Accord implementation in the territory RESULT 2 Civil society, community￾based organizations and other social actors invest and increase their participation in the construction of peace in the territory RESULT 3 Government entities and civil society organizations work collaboratively on actions that promote the construction of peace in the territory NATIONAL RESULT 1 National government agencies with Accord implementation responsibilities interact and articulate their actions with the territories RESULT 2 National level entities implement strategic rapid response actions for the implementation of the Accord RESULT 3 Government and civil society act jointly for Accord implementation. Colombia Transforma operates in four regions: Bogotá, and the departments of Arauca, Norte de Santander, and Putumayo. In each location, it pursues region-specific results. The program implements small grants to help prepare all three levels of government (national, departmental, and municipal) to lead rapid response activities. Particular attention is paid to the local level where Colombia Transforma develops many of its activities with key actors in civil society, religious groups, academic networks, and grassroots organizations to highlight the significant roles that each play in the post-Accord process. Most activities involve information sharing, dissemination and media; infrastructure and access to services; and capacity building and technical assistance to the GOC. 3 EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The United States Agency for International Development (USAID)/Office of Transition Initiatives (OTI) engaged Social Impact, Inc. (SI) to conduct an independent mid-term performance evaluation (PE) of the Colombia Transforma Program. The primary audience for the PE is USAID/OTI, which will use the findings to inform the direction of the program during the final year of implementation. Findings will also be used to inform future USG assistance and programming in Colombia. The evaluation report will be shared with the USAID/Colombia Mission, U.S. Embassy in Bogotá, MSI, local stakeholders including the GOC, and other interested groups in Washington, DC. The purpose of this PE was to assess the strategic approach, relevance, and contributions of the Colombia Transforma Program toward enhancing Colombia’s ability to implement rapid response following the signing of the historic Peace Accord with the FARC guerrillas. The PE covered program performance from August 2015 to January 2018, with more emphasis on activities implemented after the final Peace Accord was ratified on November 30, 2016. This weighting was intended to offer the program suggestions about mid-course corrections with greater practical value at this stage of implementation. EVALUATION QUESTIONS This mid-term PE focused on the lines of inquiry or Evaluation Questions (EQs) below, which were offered by USAID/OTI and revised in consultation with SI and MSI, as stated in the Scope of Work (SOW) included in Annex A. 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? a. Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? b. If there are missed opportunities due to U.S. foreign policy constraints outside the program’s own manageable interests, what are those constraints? c. Potential Topics of Interest: • To what extent has Colombia Transforma adjusted its strategic approach to respond to shifting local context in Colombia? • To what extent has Colombia Transforma adjusted its strategic approach based on lessons (positive and negative) from past Colombia Transforma activities in accordance with OTI’s iterative action learning model? • Has the program’s strategic approach led to unintended outcomes, either positive or negative? 2. To what extent has Colombia Transforma achieved its program objectives? Please review the program’s four portfolios equally: (Arauca, Norte de Santander, Putumayo, and National Grants), with greater emphasis on activities that began after the Accord was signed in November 2016. While the evaluation team may not be able to attribute outcomes solely to Colombia Transforma, when answering the questions below, please consider to what extent the program contributed to the outcomes. a. Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 4 b. Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? c. What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? d. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? f. Potential Topics of Interest: • Has the program led to unintended outcomes, either positive or negative? • Is the program working with the governmental and non-governmental partners who are most relevant to rapid response, both nationally and locally, in the program’s three departments (Arauca, Norte de Santander, and Putumayo)? If the program missed any key stakeholders or significant opportunities related to its chosen partners, which partners and stakeholders should the program engage moving forward? • Has the program affected indigenous groups, Afro-Colombians, men, women, and youth in any specific ways? Are these groups adequately considered in program implementation? • Has the program effectively shared relevant programmatic lessons with GOC counterparts? • Which lines of inquiry would be most relevant for future data collection efforts to inform programming and measure the program’s impact? 5 EVALUATION METHODS AND LIMITATIONS SI conducted this PE from January to April 2018. The period included four weeks for document review, development of data collection protocols and the Inception Report, and mobilization; four weeks in Colombia for data collection and site visits; and six weeks for data analysis and writing. Data collection, analysis, and report-writing was completed by a three-person ET, with participation from an OTI Resource Person. Please see Annex B for profiles of ET members. SAMPLING The ET conducted 23 days of data collection across all four regions of Colombia Transforma activities: Arauca, Bogota, Norte de Santander, and Putumayo. To determine the sample of stakeholders to engage, the ET used a multilevel, sequential process supported by OTI’s activity database. Below, in the order applied, is the criteria the ET used to filter from the universe of stakeholders and activities: 1. Year of implementation (preference for activities initiated after the Peace Accord was signed) 2. Geographic location determined in part by logistical and security considerations 3. Stakeholder type 4. Activity type: information dissemination and media; infrastructure; technical assistance to government institutions In total, the ET collected data from 164 key informants (61 female, 103 male). Please see Annex E for a list of respondents. Table 2. Distribution of Data Collection Methods (Participants by Sex and Regions) Region Key Informant Interviews Focus Group Discussions Direct Observations KII # Female Male FGD # Female Male DO # Washington DC 6 6 7 0 0 0 0 Bogotá 16 11 14 1 7 7 0 Arauca 15 8 21 1 8 6 3 Norte de Santander 12 3 12 3 7 17 2 Putumayo 11 5 9 2 6 10 1 Total 60 33 63 7 28 40 6 DATA COLLECTION METHODS The ET used a qualitative evaluation design consisting of document review, key informant interviews (KIIs), focus group discussions (FGDs), and direct observation (DO). Data collection protocols (Annex C) consist of questions that address and derive from the EQs, as well as from the ET’s document review, its discussions with OTI and MSI staff, and its evaluation design knowledge. See the Evaluation Matrix in Annex D for a summary of data sources and data collection methods used to answer each EQ. DOCUMENT REVIEW The ET conducted a review of documents produced by OTI and MSI to better understand Colombia Transforma program design and implementation, extract findings relevant to the EQs, and inform data collection protocol development so that instruments appropriately supplemented or cross-checked information in the background documents. Data sources for document review are detailed in Annex F. 6 KEY INFORMANT INTERVIEWS (KIIS) Findings from KIIs contributed to the ET’s responses to all EQs. The ET conducted KIIs with 96 key stakeholders at national and subnational levels. Table 2 outlines the number of KIIs by location. The ET conducted KIIs one-on-one or in small groups, as appropriate, with the target groups below: 1. U.S. Government / Implementing Partner (MSI) 2. Colombian Government Actors (National, Municipal, Departmental, Ombudsman)1 3. Local Beneficiaries (community groups, non-governmental organizations (NGOs), Community Action Boards) 4. External Stakeholders The ET used a purposive sampling approach to identify key informants. The ET did not anticipate achieving complete gender parity in the number of men and women included in KIIs. However, the ET made concerted efforts to include both sexes in the KII sample as much as possible through direct invitation, scheduling interviews at convenient times for respondents, and providing transport as needed. FOCUS GROUP DISCUSSION (FGDS) To inform its findings related to program effectiveness, the ET conducted FGDs with local non￾governmental beneficiaries of Colombia Transforma (community groups, NGOs, and Community Action Boards). The ET conducted seven FGDs with a total of 68 participants. The ET identified a tentative list of local beneficiaries to participate in FGDs, including representatives from women’s groups, Afro￾Colombian groups, campesino/producers’ organizations, churches, and community radio. After its fieldwork in the regions, the ET conducted one FGD in Bogota with program staff from OTI and MSI to reflect on and ground-truth findings related to program implementation, challenges, and successes. To minimize the risk that certain individuals might dominate the discussion, the FGD facilitator solicited the opinions of all participants, and the ET conducted several KIIs with participants who were less vocal in the FGD setting. Working closely with MSI and its partners to select locations, the ET considered time, logistics, and safety when scheduling FGDs. DIRECT OBSERVATION (DO) To inform findings related to EQ 2, the ET conducted six DOs (three in Arauca, two in Norte de Santander, and one in Putumayo). As infrastructure is one of the largest components of program activities, the ET conducted DOs of facilities to: (1) verify whether they contribute to the objectives of Colombia Transforma, (2) inquire about the community’s perception of the facility, and (3) verify community ownership of the facility and their interactions regarding the facility. DATA ANALYSIS ET members transcribed KII and FGD notes in real time, cleaning and sharing electronic summaries on a rolling basis throughout fieldwork. Team members conducted internal debriefs during fieldwork to discuss evidence collected, patterns, and discrepancies. The ET captured preliminary findings, conclusions, and recommendations in a matrix to: (1) ensure that the ET prepared a systematic and thorough response to each EQ, (2) verify that preliminary analysis accounted for gender and social dimensions, (3) identify any gaps where additional clarification or analysis may be necessary, and (4) serve as the basis for developing the evaluation report. Triangulation enabled the ET to cross-verify and cross-validate findings that emerged from the above data sources and collection methods, identify correlations between findings, and ultimately determine Colombia 1 Throughout this report, “Colombian government” and “GOC” refer to any or all levels of the government. 7 Transforma’s overall effectiveness. The ET utilized methodological triangulation to develop parallel protocols with the same or similar questions across its KIIs and FGDs. This enabled greater data triangulation because each method addresses sub-sets of the same EQs, and their findings were validated or refuted by the other techniques. Methodological triangulation also enabled the ET to strengthen the potential linkages and accuracy of its data when the results obtained through one method were less conclusive than those obtained through another method. The ET employed several data analysis methods to identify key findings from the collected data, draw conclusions, and make recommendations: 1. Content Analysis – Content analysis entailed the ET’s intensive review of KII and FGD data, as well as program documents, to identify and highlight evidence of Colombia Transforma results (or lack thereof) that contributed to or inhibited achievement of program objectives. 2. Trend Analysis – Trend analysis enabled the ET to examine stakeholders’ perceptions on presence of the three levels of government (national, departmental, and municipal) in their regions, relevance of the Peace Accord to their communities, relevance of implementation to their priorities, and contribution of Colombia Transforma to those results. 3. Comparative Analysis – The ET undertook comparisons of Colombia Transforma results across regions to assess either convergence or divergence in perspectives. In doing so, the ET considered variation in perceptions of government presence, meaning of the Peace Accord in different regions, opportunities for collaboration among local actors, and different understanding of roles and responsibilities with respect of the Accord’s implementation, as well as the definition and application of key program objectives by each program team. BIASES AND LIMITATIONS The ET notes some limitations and bias risks in data collection and analysis: 1. Security and Logistical Limitations: Due to limited resources and security concerns, the ET did not travel to all parts of regions where the program was implemented. The ET only visited urban centers cleared as “safe (without recent safety or security incidents)” by SI and MSI security teams. An ELN armed blockade in Arauca caused logistical difficulties and required schedule changes to avoid placing the ET and data collection respondents at risk of physical harm. Data collection was conducted in urban centers, and the ET arranged for some key informants to travel to participate. The ET also conducted three remote interviews when in-person visits were impractical. 2. Response Bias: Response bias is the risk that key informants may be motivated to provide responses that would be considered desirable or influential in obtaining donor support. The ET observed some degree of response bias among program beneficiaries who seek opportunities to receive more program benefits. The ET mitigated any effects of this bias through triangulation. 3. Selection Bias: Selection bias is an inherent risk when implementers help to facilitate contact with beneficiaries. Given logistical difficulties and security concerns, the ET coordinated closely with MSI to organize KIIs and FGDs. The ET combatted potential selection bias as much as possible by using multiple sources of data and its evidence matrix to triangulate data. 4. Generalizability and Attribution: Given that this non-experimental evaluation design did not involve a comparison group, the ET is not able to attribute impact to the Colombia Transforma program; rather, the ET focused on how the program may have contributed to change. As well, generalizability of data is limited given that the ET collected data from a sample of key informants that was informed, in part, by geography, accessibility, and availability. 8 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EVALUATION QUESTION 1. HOW RELEVANT IS OTI COLOMBIA’S STRATEGIC APPROACH TO THE LOCAL CONTEXT AND TO U.S. FOREIGN POLICY IN COLOMBIA? The Colombia Transforma program’s strategic approach is largely relevant to Colombia’s local context and closely aligned with U.S. foreign policy objectives. It succeeded in executing activities at a pace far more rapid than that to which communities and state entities were accustomed. This pace slowed somewhat, though, as workloads and complexity increased. While the Peace Accord has five substantive chapters, Colombia Transforma has mainly helped lay the groundwork for Chapter 1 on rural development and reform. Other chapters have received less emphasis. This focus on rural governance and civil-society cohesion dovetails with U.S. concerns about insufficiently governed rural areas, which have long been fertile ground for violent groups, criminal activity, and illicit crop cultivation. However, another aspect of U.S. policy, the current interpretation of statutes prohibiting material support for members of foreign terrorist groups, has complicated work in communities where ex-guerrillas have been integrated since August 2017. EVALUATION QUESTION 1A. IS THE PROGRAM FOCUSED ON THE RAPID RESPONSE ASPECTS OF THE PEACE ACCORD THAT ARE THE MOST CRITICAL AND WILL MAKE OR BREAK THE LONG￾TERM SUCCESS OF THE ACCORD? CONCLUSIONS RAPID-RESPONSE ASPECTS OF THE PEACE ACCORD: Of all chapters of the Peace Accord, Colombia Transforma is most oriented toward implementing Chapter 1 (“Comprehensive Rural Reform”). The other chapters receive less emphasis by program design, by circumstance, or by external policy restriction. Most activities related to Chapter 1 have sought to encourage economic development and state presence in rural areas, and they lend themselves to a rapid response approach. They also dovetail with U.S. foreign policy interests, which have long included concerns about the security risks and illegality inherent to ungoverned spaces. Some aspects of Chapter 1, however, deal with land tenure or territorial ordering: mapping and titling of landholdings, for instance, or the designation of lands currently in government possession (baldíos). Though land tenure lies at the heart of the Peace Accord, and although struggles over land were a principal cause of the armed conflict, slow timeframes for land-tenure programs make it impossible for Colombia Transforma to achieve anything more than the most incipient gains before the program ends in 2019. ABILITY TO RESPOND RAPIDLY: In most activities, Colombia Transforma succeeded in responding rapidly, operating at a pace that communities had not experienced before. The program is recognized as the only international effort helping communities in many territories, as well as listening to communities’ priorities in the design and choice of activities to undertake, operating faster than other donors, bringing about tangible results, creating relationships of trust, and fostering collaboration among stakeholders. Colombia Transforma’s performance, however, suffers from the following weaknesses: (1) excessive or unexpected logistical complexity of some activities; (2) overestimating capacity to implement activities rapidly; (3) in some cases, responding to communities’ preferences with activities that are more urgent than strategic; (4) high staff turnover in the Arauca office; (5) difficult purchasing procedures and requirements; and (6) moving comparatively fast to decide to “green light,” or proceed with, an activity (though this process has slowed somewhat over time), but moving less assiduously during the implementation phase. When they occur, the result of these operational delays is often missed “windows 9 of opportunity,” or strain to relations with partners. However, speed carries its own cost as rapid response can prevent Colombia Transforma personnel from perceiving important trends and lessons. CONTINUED WORK IN NORTE DE SANTANDER: The criteria by which Colombia Transforma chose the three departments to operate remain valid. As a border region facing an exodus of Venezuelans—and as a region in which the program has begun promising processes with mayors, Community Action Boards, producers’ organizations, and other sectors—Norte de Santander continues to present a “window of opportunity.” COMMUNICATING OUTCOMES AND LESSONS: Colombia Transforma is not sufficiently communicating to stakeholders about its activities, their outcomes and lessons, and how they might be related to each other at a strategic level.2 Upon completion of activities, the program is not encouraging GOC entities to reflect, in a structured way, on the factors that contributed to outcomes and how to adjust strategies based on those factors. OTI is not sharing the results of Colombia Transforma frequently with USG counterparts. The program appears to lack visibility within the U.S. mission. In Washington and in other U.S. missions, policymakers wrestling with work in ungoverned spaces previously untouched by international programs, with fractious civil society actors, and under severe time pressure would benefit from knowledge of OTI’s experience—but few are likely even to know that the program exists. Though senior field office staff meet monthly, mid-level staff in different field offices meet less often and desire more frequent contact to learn from each other. Colombia Transforma could also do more to foster contact among grantees to fulfill Objective II of promoting collaboration among territorial actors. Colombia Transforma could do more to educate other donors and agencies about the results of its “tripartite” activity model, discussed in 2c below, in which the community, the government, and the donor each make contributions. FINDINGS RAPID-RESPONSE ASPECTS OF THE PEACE ACCORD: Some Colombian government officials, outside experts, and program staff agreed that Colombia Transforma has responded more to Chapter 1 of the Accord (“Comprehensive Rural Reform”) than to any other of its five substantive chapters. This perception reflects circumstances during the Accord’s early implementation phase, rather than a deliberate choice to focus overwhelmingly on a particular chapter at the exclusion of others. Within Chapter 1, the program responded more to rural development commitments than to land tenure and territorial ordering commitments. On the latter issue, the Colombian national government’s procedures and timelines are very slow, especially when landholdings’ boundaries are unclear, records are poorly kept, or ownership is in dispute. Colombia Transforma’s activity with the National Land Agency (ANT), discussed in 2a below, underlines the difficulty of working quickly on Chapter 1’s land aspects. Colombia Transforma is only contributing to the Accord’s second chapter, “Political Participation,” in a preliminary, embryonic way. Program staff, for instance, pointed out that promoting coordination among local actors can lay the groundwork for future greater participation in politics. The program is not working on Chapter 3, which supports the reintegration of ex-combatants, due to the current interpretation of USG legal restrictions on providing support to members of designated terrorist groups (discussed further in 1b below). For policy reasons, neither Colombia Transforma nor the USG are working directly on Chapter 4, which deals with illicit crop substitution. This is the case even though a program document notes that “drugs are the elephant in the room in Norte de Santander and Putumayo.” Though resolving illicit cultivation is a “make or break” aspect for the long-term success of the Accord, U.S. officials are not convinced that the model laid out in Chapter 4, the “National Integral Illicit Crop Substitution Program (PNIS),” is well- 2 A “Topic of Interest” under EQ 2 asked whether Colombia Transforma has “effectively shared relevant programmatic lessons with GOC counterparts.” The ET found very little evidence of such exchange. 10 conceived. Some national government officials suggested working on crop substitution outside of the PNIS framework, perhaps with local governments. Instead, the program focuses on Chapter 1 priorities, rather than linking assistance to the voluntary eradication of illicit crops. (This choice is discussed further in 1b below.) Chapter 5 deals with victims and transitional justice; the Peace Accord’s transitional justice system is barely getting started, however, and Colombia Transforma includes very few victims’ groups among its grantees, as discussed in 2b below. ABILITY TO RESPOND RAPIDLY: Colombia Transforma is the only international presence working for Peace Accord implementation in many rural areas of Arauca, Norte de Santander, and Putumayo. The program has managed to operate in territories that would have been too remote or dangerous for such work in the very recent past. This makes it the first international cooperation that most communities have seen in many years, if ever. Praise for Colombia Transforma’s rapidity, transparency, and consultative nature was nearly unanimous. Stakeholders hailed its rapid delivery on commitments and said it compared favorably both to the GOC and other international donors. A governor said: “this is the first program that brought fast, concrete results” without red tape: “Colombia Transforma has not fooled the people.” Stakeholders praised the program for consulting with them on the design of activities, which they view as collaboration, not an imposition handed down from a desk in Bogotá or Washington. “We’re not doing what Colombia Transforma tells us,” a priest said in Norte de Santander. “They’re helping us do what we want to do. We have our own process.… It’s not like ‘come here and get your money.’ Everything is planned, and we sit together and think it through.” On the other hand, as a national government official said, “there are cases in which the expectation of an agile program is not met.” Colombia Transforma recognizes that it does not always work with desired speed on the implementation of activities. Persistent delays, many of them bureaucratic, are the “principal difficulty for Colombia Transforma,” and they have weakened local leaders’ will to participate according to an August 2017 internal program evaluation. As Figure 1 depicts, delays in the initial decision-making process, in which program staff determine whether to “green-light” a proposed activity, worsened over time. The term “green-light” relates to OTI’s procedure for considering proposed activities. After developing an activity idea with local partners, field staff present the concept to OTI leadership. If OTI agrees that the idea deserves further development, it receives a “yellow-light” status. To gain approval—and advance from “yellow light” to “green light”— Colombia Transforma staff develop the activity’s design, implementation plan, grant agreements, and budget. Some activities never receive clearance in the form of a “green light.” Figure 1. Colombia Transforma Decision-Making Time 11 Delays occurred mainly at the level of field offices as caseloads grew. In Norte de Santander, there were fewer complaints from field staff and local partners about implementation speed or follow-up, although administrative delays became an issue as Colombia Transforma increased the number of activities: “We delay almost 3-4 months in carrying out [activities], and that’s when we’re on time.” Personnel in Bogotá and field offices raised the subject of slowness in disbursements, contracting, and purchasing. Purchasing delays owe in large part to the lack of “formalized” vendors of goods and services in rural areas where the informal economy is the norm. In Norte de Santander, Ocaña staff cited as a source of delays the centralizing of decision-making and administration, especially purchasing, in Cúcuta. In Arauca, Colombia Transforma field staff identified staff turnover as a key factor. To speed purchasing, Colombia Transforma internal recommendations include setting up a database of providers, simplifying procedures, and standardizing criteria. In Arauca, Colombia Transforma staff cited a “need to improve the ‘yellow light’ process to ensure maintained relevance of each activity’s nature as proposed.” Staff said that delayed reviews between the “yellow light” and “green light” stages sometimes caused windows of opportunity to be missed, or that they produced changes in an activity’s design that made it “lose its original essence.” ART staff in Arauca seconded this observation, saying they need decisions in a week or two, for instance to fund PDET pre￾assemblies, but have often had to wait two months or more. CONTINUED WORK IN NORTE DE SANTANDER: According to OTI documents, Colombia Transforma’s three geographic areas outside Bogotá were chosen based on five criteria: least coverage of USAID programs; strong effects of the armed conflict; large presence of illicit crops; strong presence of illegal armed groups (not only the FARC); and being border areas. Despite obstacles posed by continued violence and illegal economic activity, these criteria remain unchanged in Norte de Santander. Interviewees across the range of stakeholders unanimously endorsed continuing the program in Norte de Santander. They cited the emergence, often with Colombia Transforma support, of Community Action Boards, producers’ associations, youth groups, and the Catholic Church Pastoral Social as vital local leaders independent from the political polarization that has beset the conflictive areas within the department. They noted that the departmental government has reached out creatively to previously uncontacted victims. Despite persisting violence, the program has managed to work in townships and municipalities where activities would have been unthinkable during the conflict’s worst period. A national-government official added that border zones like Norte de Santander are “worth gold”: very strategic and with huge development potential. Colombia Transforma staff working in the region cited many “people here who want to change for the better.” COMMUNICATING OUTCOMES AND LESSONS: Colombia Transforma staff believe they do not devote enough effort to communicating about activities. A contract advisor cited the importance of putting on “a bit of a show” with community infrastructure projects, even if it slows the activity down somewhat. An annual report recommends incorporating a communication and dissemination strategy from the outset of activities. Interviews with government officials at all levels revealed very little evidence of a conscious effort on the part of Colombia Transforma to review with them the factors that led to success and are worth repeating in their own future endeavors, or those that undermined progress and should be avoided. While reflections about results appeared to occur formally and informally with local partners in the immediate aftermath of some activities, they were not systematic, and the ET heard no examples of reviews occurring months later, when medium-term results would be more evident. Similarly, there was scant evidence of efforts to encourage any government entity to share lessons with others. Program documents and OTI and MSI staff agreed that Colombia Transforma needs to do more to share its knowledge with other U.S. agencies as well. Mid-level Colombia Transforma staff in Putumayo said that exchanges with colleagues in other regional offices were infrequent. Mid-level Colombia Transforma staff in Norte de Santander’s two field offices both suggested meeting each other more often to share lessons. Grantees, similarly, expressed a desire for the 12 program to help them get to know other grantees. The ET witnessed two Colombia Transforma-funded advisors to the same Bogotá agency engage with each other for the first time. (There are about 55 individuals hired as contract advisors to government agencies.) A Putumayo Victims’ Unit official said, “It is not clear how all the grantees are being coordinated. We remain like ‘islands.’” Colombia Transforma maintains extensive data on program processes and results: each activity generates extensive notes about its impact and preliminary lessons learned. The program’s speed, however, can make learning lessons difficult. “Rapid response doesn’t let you learn right away because you’re moving so fast,” a field office director said. “You often see the lesson much later.” EVALUATION QUESTION 1B. IF THERE ARE MISSED OPPORTUNITIES DUE TO U.S. FOREIGN POLICY CONSTRAINTS OUTSIDE THE PROGRAM’S OWN MANAGEABLE INTERESTS, WHAT ARE THOSE CONSTRAINTS? CONCLUSIONS THE MATERIAL SUPPORT PROHIBITION: The new FARC political party is a local, legal actor resulting from the second chapter of the Peace Accord. The Accord calls for implementation to take place with maximum community participation. As such, it would normally fit within Colombia Transforma’s mission to engage individual members of the FARC political party as actors in communities emerging from conflict. Because all 14,000 ex-FARC guerrilla are considered by the USG to be members of an FTO, Colombia Transforma has instead worked to benefit communities likely to interact frequently with ex-FARC members, without actually working with them. This prohibition is not congruent with on-the-ground reality in post-conflict Colombia, where 14,000 “untouchables” are now circulating freely throughout the country. NON-INVOLVEMENT IN COCA SUBSTITUTION: Although Putumayo and Norte de Santander include some of Colombia’s principal coca-growing zones, Colombia Transforma does not carry out crop substitution, in the sense of requiring that coca be eradicated as a condition of receiving support. This owes in part to the USG decision to avoid supporting the GOC’s PNIS program due to discomfort with the model and the participation of FARC-tied groups. Addressing rural development (Chapter 1 of the Accord), without specifically tying aid to coca eradication, may offer more promise of achieving long-term gains in coca eradication. As noted, most Colombia Transforma activities support the goals of Chapter 1 and are unlikely to bring about a rapid drop in the number of hectares planted with coca, a purported goal of Chapter 4. Program activities do, however, hold greater promise of making reductions more long￾lasting, by contributing to the state presence and basic services on which a legal rural economy depends. FINDINGS THE MATERIAL SUPPORT PROHIBITION: In at least 14 KIIs and FGDs, U.S. and Colombian government representatives, non-governmental partners, and individual experts discussed obstacles posed by the rigid way in which the USG is applying a legal prohibition on working with demobilized guerrillas. Section 2339B of Title 18, U.S. Code imposes a minimum of 20 years’ imprisonment on “whoever knowingly provides material support or resources to a foreign terrorist organization, or attempts or conspires to do so,” if he or she knows that the organization is on the U.S. Department of State (DoS) list of designated foreign terrorist groups.3 The FARC guerrilla has been designated as such since the list’s inception in 1997. Though the guerrilla organization demobilized and became a legally registered political 3 Section 2339A defines “material support or resources” as “any property, tangible or intangible, or service, including currency or monetary instruments or financial securities, financial services, lodging, training, expert advice or assistance, safehouses, false documentation or identification, communications equipment, facilities, weapons, lethal substances, explosives, personnel (1 or more individuals who may be or include oneself), and transportation, except medicine or religious materials.” Whether this covers low-ranking demobilized guerrillas’ one-time participation in an event carried out with U.S. funds is a matter of interpretation: the law as written here is not conclusive. 13 party called the Common Revolutionary Alternative Force (FARC), it remains on the terrorist list, as the USG does not regard the organization and the party to be distinct entities. DoS’ deliberative process will probably maintain that status for the FARC for at least a few more years. As a result, neither the USG nor any private U.S. entity may assist any of the 12,451 individual demobilized FARC members whom the GOC has accredited to participate in the peace process, much less the 1,727 additional members whose cases were under review, or awaiting national identification documents, at the end of 2017.4 All subjects who discussed the issue with the ET unanimously lamented the choice to apply this statute to all individual ex-FARC members who might engage in program-supported activities. Not a single respondent defended the current broad application of the statute. Many contended that such interpretation is unworkable in the context of post-conflict Colombia. Since August 15, 2017, demobilized FARC members have been free to leave the zones where they disarmed. They are now private citizens, cleared by the GOC except for about 4,000 members facing eventual transitional justice trials for war crimes. Key rapid-response aspects of the Peace Accord, such as meetings for the PDETs, require broad participation of communities. As such, the GOC cannot exclude the participation of ex-guerrillas who now reside in these communities. However, these legal citizens in the eyes of the GOC are deemed illegally sanctioned individuals in the eyes of the USG, introducing challenges for Colombia Transforma. As a result, numerous interviewees complained of uncertainty in how to implement the prohibition. “We need a decision about the usefulness of this. You can’t interrupt the course of a whole program just because a guy shows up and eats a snack,” a national government official stated. “It’s not realistic to hold a meeting in Puerto Vega without a FARC presence; they’re everywhere,” said a national government representative in Putumayo, who had to cancel an event for this reason in that township of Puerto Asís municipality. “You’re not supporting guerrillas, you’re supporting a territory,” said a national government official in Arauca. “If you want the state to be in the most remote areas, that’s where these people are.” The need to rapidly establish a state presence and demonstrate benefits of the Peace Accord is greatest in zones previously dominated by the FARC. For that reason, it made sense for Colombia Transforma to support activities in areas near the sites where FARC fighters congregated to demobilize in Puerto Asís, Putumayo; Tibú, Norte de Santander; and Arauquita, Arauca. There, it is urgent to demonstrate to citizens that the government is “arriving,” while dispelling perceptions that Accord implementation assistance mainly favors ex-combatants instead of residents. Supporting nearby infrastructure activities, for instance, helped to meet this need. However, as more ex-guerrillas continue to leave the nearby zones, conducting activities in these areas may be complicated by the material support statute. NON-INVOLVEMENT IN COCA SUBSTITUTION: The USG does not support the GOC’s post￾conflict PNIS program because of the material support restriction. “[T]he FARC has taken over, or created, a number of front groups for coca-growers,” DoS’s senior drug policy official at the time, Amb. William Brownfield said in September 2017. “The government, in turn, is negotiating voluntary eradication agreements with those groups, so that the FARC has basically captured the process.… [O]ur condition is: no involvement by the FARC at all.”5 For this reason, Colombia Transforma canceled an activity with the PNIS program in early 2017. There has been a decision to “draw the line,” as an MSI Washington-based official described, and avoid involvement in activities related to coca or coca substitution. The municipal government and NGOs in Catatumbo (in Norte de Santander, one of Colombia’s top coca-growing regions) contended that it makes more sense to focus on generating income opportunities for farmer families, making the larger economy work, and fulfilling Colombia Transforma Objectives 1 and II, instead of merely “substituting coca.” Some 4 Report of the Secretary-General on the United Nations Verification Mission in Colombia (New York: United Nations Security Council, December 27, 2017): 7 . 5 Adapting U.S. Counternarcotics Efforts in Colombia (Washington: U.S. Senate Caucus on International Narcotics Control, September 12, 2017) . 14 voiced the view that the substitution plan in Chapter 4 is unlikely to succeed in the long term without the rural development services, state presence, and territorial ordering foreseen in Chapter 1. TOPICS OF INTEREST 1C. TO WHAT EXTENT HAS COLOMBIA TRANSFORMA ADJUSTED ITS STRATEGIC APPROACH TO RESPOND TO SHIFTING LOCAL CONTEXT IN COLOMBIA, AND BASED ON LESSONS (POSITIVE AND NEGATIVE) FROM PAST COLOMBIA TRANSFORMA ACTIVITIES? CONCLUSIONS Especially after Peace Accord implementation began at the end of 2016, Colombia Transforma made very few adjustments to its strategic approach. • Before the Accord, the most significant adjustment was replacing Portable Programmatic Teams with regional offices, which increased activities and enabled far better awareness of partners and challenges in the territories. • In the post-Accord period, Colombia Transforma focused more on carrying out a rapid tempo of activities than on adjusting its approach. This period saw a sharp increase in infrastructure activities. This was more a realization of existing plans, however, than an adjustment to the strategy. • Though OTI’s previous experience in Colombia was different, given that the National Territorial Consolidation Plan had a very large security component, staff have sought to apply lessons. A major lesson is that “strengthening local capacity may be the best way to attract national government presence,” rather than prodding the national government to arrive on its own. This hypothesis is compelling but testing it will take time. • Speed, meanwhile, carries its own cost. Despite regular assessments and extensive documentation, the pace of rapid response can prevent Colombia Transforma personnel from perceiving important lessons until much later.6 FINDINGS CREATING FIELD OFFICES: In mid-2016, Colombia Transforma opened three field offices because of a need to be constantly present in the three regions rather than administer everything from Bogotá through Portable Programmatic Teams. The result eased the program’s “ramp-up,” or increase in activities, during the period coinciding with and following the Peace Accord’s signature. MORE EMPHASIS ON INFRASTRUCTURE: As shown in Figure 2, the overwhelming majority of infrastructure activities began after November 1, 2016 (as the Peace Accord was nearing final approval). Ninety percent of activities that beneficiary communities and mayors’ offices request of the program deal with infrastructure or agriculture. Colombia Transforma also has helped finish initiatives that had begun but were on hold due to exhaustion of resources or noncompliance with commitments. Examples include a school in La Carmelita, Putumayo and a river dock in Puerto Angalia, Norte de Santander. 6 The ET offers the following suggestions in response to EQ 2 Topic of Interest “Which lines of inquiry would be most relevant for future data collection efforts to inform programming and measure the program’s impact?”: (1) a large-sample survey of perceptions of state presence in communities where Colombia Transforma did and did not carry out activities; (2) a survey of partners’ perceptions about continuity—whether processes begun by Colombia Transforma activities are likely to remain in motion after 2019; (3) a survey about attribution: whether the government or Colombia Transforma deserves the most credit for results, and why. 15 Figure 2. Timing and Location of Infrastructure Activities LEGACY OF “CONSOLIDATION”: Colombia Transforma hired some staff from the prior rapid-response National Territorial Consolidation Program, which OTI supported and Creative Associates implemented between 2007 and 2011. That program contributed to notable gains in state presence in some historically ungoverned territories. For the most part, though, it did not succeed in drawing non-military national government entities into these territories.7 Colombia Transforma staff with prior “Consolidation” experience said that rather than endeavoring to bring the central state to remote territories, it makes more sense to strengthen subnational government and civil society organization presence in these territories, which could attract the central state. “DOING AND PROVING”: A May 2017 note on a “Rolling Assessment” states, “No changes were made to program-level goal or objectives, the intended cluster results, or the program geography. This is consistent with the March 2017 PPR [Program Performance Review] recommendation to now focus on ‘doing and proving’ versus further changing our approach.” HAS THE PROGRAM’S STRATEGIC APPROACH LED TO UNINTENDED OUTCOMES, EITHER POSITIVE OR NEGATIVE? 8 CONCLUSIONS RAISING EXPECTATIONS: Much more than expected, Colombia Transforma improved local communities’ perceptions of U.S. policy toward Colombia. OTI opened a door for future U.S. assistance and engagement in zones where suspicions of U.S. motives had been high. However, rapid response actions are only occasionally linked to sustainable future development efforts, and the continuity of many rapid￾response activities is uncertain. Colombia Transforma does not expect long-term continuity to be a goal for every activity, though it is for some of them. Cash-strapped sub-national governments’ ability to continue contributing to activities after 2019 is not guaranteed without Colombia Transforma funding for materials, equipment, and logistics. Colombia Transforma risks giving institutions a brief taste of what it would be like to have more resources, only to have those disappear after the program exits, leaving increased but unmet expectations. Where relationships and processes are promising but fragile, dashing expectations could damage trust in government, between local actors, and of the United States. POTENTIAL EFFECTS IN ELN ZONES: In zones of ELN influence, implementing the FARC Peace Accord well could have an important demonstration effect for the ELN peace process, by proving that the state intends to deliver on commitments. 7 Adam Isacson, Consolidating ‘Consolidation’: Colombia’s Plan to Govern Neglected Territories Stumbles (Washington: Washington Office on Latin America, January 2013) . 8 This section covers EQ 2 Topic of Interest: “Has the program led to unintended outcomes, either positive or negative?” 16 FINDINGS POSITIVE PERCEPTIONS OF U.S. ASSISTANCE: Numerous stakeholders of all kinds cited a reduction in local resistance to U.S. assistance programs, even in areas where many social organizations are ideologically opposed to U.S. “intervention.” A professor cited by Colombia Transforma staff in Norte de Santander said that a powerful, radical local campesino federation “still thinks badly of U.S. aid, but now they don’t think as badly as they used to.” In Putumayo and Norte de Santander, stakeholders strongly criticized Plan Colombia—the framework for several billion dollars of assistance in the 2000s— which brought forced eradication of illicit crops while achieving few, if any, security gains in their regions. They said Colombia Transforma was a vast improvement. In Putumayo, stakeholders said that the USG has learned from the shortcomings of Plan Colombia. They praised Colombia Transforma for putting a premium on consultation and inclusion; for entering into dialogue with social organizations that would have shunned the USG in the recent past; for seeking to strengthen civilian government presence and services, including those of local government; and for not promising what it did not intend to deliver. POTENTIAL EFFECTS IN ELN ZONES: Colombia Transforma could fertilize the ground for future ELN negotiations, a Colombian Presidency official said. The idea is to build trust in the Quito process with the ELN by showing that the GOC is complying with the Peace Accord with the FARC. Colombia Transforma Norte de Santander program staff called for showing more results in ELN-influenced zones. In Arauca, the program went to hamlets (veredas) in Saravena inhabited by relatives of ELN commanders and said, “‘please let us talk about what’s in the FARC accord…’ but there was zero interest because the ELN was doing its own messaging. So, we built a common meeting house with the Community Action Board.” That activity was inaugurated in 2017 with the presence of the chief of the government’s ELN negotiating team. Even armed groups had to give assent to activities, because of the community’s strong interest and because of its tripartite in-kind, shared work model. None has been attacked or extorted. CONTINUITY: Staff in one field office cited as a challenge the need to “ensure that rapid response actions are linked to sustainable future efforts.” Departmental and municipal government officials were noncommittal or pessimistic about their ability to sustain processes or services initiated by Colombia Transforma activities after the program ends. “Colombia Transforma has been a sort of ministry inside the mayor’s office,” said an Arauca municipal official. “When this oxygen goes away, we’ll have to cut back.” Norte de Santander municipal governments’ budgets are threadbare, with low revenues and high debt loads, calling into question their ability to sustain activities without program support. Asked whether he might be able to sustain the program’s tripartite shared-work model for infrastructure activities, a Norte de Santander mayor said, “We’ll have to keep going using a ‘magic wand.’ It will require sacrifice.” An official in an Arauca municipality said that as Colombia Transforma strengthens its reach into rural areas, the municipal government receives many more demands from the population. Expectations have increased. But after the program leaves, municipal agencies worry that they will be dealing with these increased demands while reduced to their prior low level of resources. Civil-society representatives voiced similar concerns about continuity. Colombia Transforma’s three regional teams are unsure of the permanence of relationships forged between territorial actors. Asked what happens after the program leaves, a Norte de Santander Community Action Board leader said, “Uh￾oh,” and voiced an uncertain hope that the program will have left behind enough organizational and management capacity. “It’s likely that in three years, all of this experience could be lost,” warned an Arauca NGO. Colombia Transforma with nothing afterward is like “an appetizer without an entree,” said an Arauca women’s group representative. In Catatumbo, a prominent local human rights defender said mayors, Community Action Boards, and independent civil society organizations are in a precarious situation and “have only Colombia Transforma to hang on to,” gesturing with his fingernails. 17 These responses undoubtedly reflect some social desirability bias: respondents have an incentive to convey to an ET a dire scenario of what might happen should a funding source dry up. Nonetheless, the consistency of this reaction, the number of such statements, and publicly available data about the state of most local jurisdictions’ finances all point to questions about the solidity of the rapid-response gains that Colombia Transforma has achieved. RECOMMENDATIONS FOR EVALUATION QUESTION 1. 1. Continue to focus on the rural development and state presence aspects of Chapter 1 of the Peace Accord. Focus less on the chapter’s land tenure aspects, except for those related to ethnic communities. 2. Continue work in Norte de Santander. A “window of opportunity” exists in Norte de Santander due to Colombia Transforma support: mayors are collaborating on a common development agenda to an extent rarely seen anywhere in the country, Community Action Boards and producers’ associations have emerged as key social leaders, youth groups and an active church are playing a vital role, and the departmental government has reached out creatively to previously uncontacted victims. Despite persisting conflict, the program has managed to work in townships and municipalities that would have been unthinkable during the conflict’s worst period. Because of its security troubles, which are emblematic of post-Accord Colombia, and its status as a strategic border zone, the need to keep engaging has grown. It would be a mistake to pull back. 3. Raise awareness within the U.S. interagency about the need for common-sense guidelines for interpreting material support provisions so that they are congruent with Colombia’s current context. Trust that USAID and its implementing partners know not to strengthen a group on the Foreign Terrorist Organization list. These guidelines should permit low-level engagement with ex-FARC who are (1) not top leaders; (2) not awaiting war crimes trials in the transitional justice system; (3) not facing U.S. indictments or extradition requests; and (4) reasonably determined to have abandoned violence. Keep a database of supported events with FARC presence for reporting and monitoring purposes. Only halt an activity if it violates the four criteria above, or if the database detects a pattern indicating an attempt to take over the space. Continue to support activities with non-FARC citizens around demobilization sites. 4. Urgently determine how to reduce delays at bottlenecks. Colombia Transforma staff and beneficiaries identified delays resulting from administrative bottlenecks at the “yellow light to green light” decision-making phase, and in purchasing and contracting in the field. Solutions may involve reviewing administrative burdens and streamlining bureaucratic processes. It is likely, however, that the number of person-hours these tasks require is unalterably high. If so, the program must increase staffing to meet the need. 5. Disseminate more information about program activities, outcomes, and lessons, including their replicability and relevance to the Peace Accord. • Grantees: Convene grantees and program consultants from different localities/regions, social groups, or types of activities to share experiences, skills, outcomes, and lessons learned. • GOC: Convene government representatives at the three levels, horizontally and vertically, to learn from each other, share experiences with Colombia Transforma, and explore opportunities for collaboration around the program. • USG: Develop materials for public dissemination on the OTI or USAID/Colombia website. Distribute occasional memos about select activities among U.S. mission staff and other international donors, to not only highlight successes but also explain what contributed to those outcomes, while frankly discussing challenges and lessons learned so that others may avoid “reinventing the wheel.” Meet more often with representatives of other sections of the U.S. 18 mission whose security or counter-narcotics interventions may involve work in Putumayo, Norte de Santander, and Arauca. 6. Develop a systematic plan for program closure, with involvement by all stakeholders. In the remaining period of performance, engage potential sources of funding—donors (including USAID/Colombia), national government agencies, the private sector—to help sustain fragile processes. 7. Continue to support activities in ELN-influenced areas and around demobilization sites. EVALUATION QUESTION 2. TO WHAT EXTENT HAS COLOMBIA TRANSFORMA ACHIEVED ITS PROGRAM OBJECTIVES? Colombia Transforma support increased government rapid response capacities to a moderate degree, and with substantial variation among national agencies and local jurisdictions. Especially at the national government level, and in the absence of a full-throated push for aggressive implementation from the very top, there was little that a program of Colombia Transforma’s size and scope could have done differently to help the Colombian state overcome longstanding incapacities for rapid response. The most notable national government capacity improvements occurred in agencies created specifically for the implementation of the Accord. At the regional level, collaboration between local actors improved in an uneven fashion, with more progress in communities where either a mayor’s office or a Community Action Board, the church, women’s groups, ethnic groups, or a producers’ association already demonstrated some sort of cohesion, independence, and legitimacy. Infrastructure activities and logistical support for events contributed strongly to progress toward both stronger state capacities and increased local collaboration. While consultation with communities is important, however, many respondents voiced fatigue with additional meetings and a desire for compliance with tangible commitments. Much remains to be done, meanwhile, to help local actors understand and fulfill their roles in Peace Accord implementation, especially in Arauca and Norte de Santander where other armed groups’ activity dulls any sense of being in a “post-conflict” environment. EVALUATION QUESTION 2A. HOW EFFECTIVELY HAS COLOMBIA TRANSFORMA INCREASED THE COLOMBIAN GOVERNMENT’S ABILITY TO PLAN AND EXECUTE RAPID RESPONSE ASPECTS OF THE PEACE ACCORD? CONCLUSIONS RAPID RESPONSE CAPACITIES OF NATIONAL GOVERNMENT: Colombia Transforma has improved some national government entities’ ability to plan their “entry” into territories with no previous government presence, and in some cases to execute some of these first forays. The program has also helped to “bridge” these entities, which tend to operate in silos. The program’s support was more effective in increasing the rapid response capacities of national institutions recently created to implement the Accord (“post-conflict entities”) than in increasing those of pre-existing ministries and agencies, which have barely increased their territorial presence, if at all. This is due largely to institutional challenges, some discussed in the findings below, that have proven too deep-seated for a program like Colombia Transforma to address in a short timeframe with limited funds. Colombia’s national government appears to be replaying many of the same institutional issues that prevented civilian state entities from “arriving” in the framework of the 2007-2011 OTI-supported 19 National Territorial Consolidation Plan.9 The fact that a similar scenario is unfolding calls into question how much learning took place. A deep diagnosis and understanding of obstacles posed by Colombian law and bureaucratic procedures is beyond this evaluation’s scope. Still, these—even more than lack of resources—appear to be the greatest barrier to the national government’s ability to carry out rapid response. For example, where landholding is involved, national government timeframes are especially long. There is no apparent rapid response to land tenure issues. The inability to provide rapid results on titling, registration, and land adjudication is likely to damage perceptions of the Peace Accord among populations in territories recovering from conflict. As Colombia Transforma is not subject to measures like Law 80, its activities were effective and may offer a temporary way around some of these obstacles. The PDET program seeks to get around these structural obstacles by having the Agencia de Renovación del Territorio (ART), a large new agency, guide rapid response in 170 municipalities. However, this program is making uneven progress; it is most advanced in Putumayo and least in Norte de Santander. The ART needs greater coordination between Bogotá and its regional offices. ART’s relationships with local actors vary: some mayors do not understand the “territorial peace” doctrine that underlies the PDET program, viewing it as another channel for central government resources, and some communities say they are tired of meetings with no follow-up. Ultimately, these obstacles are certain to recur when the program ends. Colombia Transforma activities are not oriented toward helping Colombia’s national government find ways to overcome them. OPPORTUNITIES WITH DEPARTMENTAL AND MUNICIPAL GOVERNMENTS: Departmental governments suffer grave deficiencies in capacities and often lack political will to implement the Accord. However, some individual offices within departmental governments have taken on Accord implementation more robustly, with key support from Colombia Transforma. Mayors’ offices vary greatly in rapid response capacity, resources, and will to carry out Accord implementation. Individual mayors’ offices’ performance is extremely uneven, however. Some are hobbled by clientelism, and territorial actors resist working with them. Nonetheless, they are critically important partners. When organized into associations, their planning and execution capacity, and their ability to engage other levels of government, improve. Mayors and governors will be in office longer than the current national government, which offers hope for continuity of processes begun during Colombia Transforma’s first years. In the best case, an August 2018 change in presidential administration will cause delays and the loss of some institutional memory. In the worst case, the next administration could be openly hostile to Peace Accord implementation. FINDINGS NATIONAL GOVERNMENT: Numerous program personnel and other stakeholders reported to the ET that the national government lacks clear lines of authority and decision-making for Accord implementation, complicating the execution of rapid response activities, including those supported by Colombia Transforma. One example among many are the overlapping roles and responsibilities of the Presidency’s High Commissioner for Peace and those of the Post-Conflict Ministry. Agencies, including some created for Accord implementation,10 have redundant mandates, insufficient resources, and no official in a top coordinating role. Without a clear hierarchy between them, “no state entity allows itself to be coordinated from without,” a Bogotá-based advisor to a national government agency said. An October 2016 Colombia Transforma annual report offered this summary: “The main challenges that the National Government faces for effective preparation and implementation of peace agreements are the 9 Caroline Hartzell, Robert D. Lamb, Phillip McLean, and Johanna Mendelson Forman, Maria Cristina Olano, USAID/OTI’s Integrated Governance Response Program in Colombia (Washington: USAID / CSIS, April 2011) . 10 A Colombia Transforma document lists four agencies created or restructured to enable compliance with the Peace Accord in the territories: (1) the Presidency’s Post-Conflict Ministry, whose main function is to organize other agencies with implementing responsibilities; (2) the Agriculture Ministry’s Agency for Rural Development (ADR), which aims to implement agricultural development projects, (3) The National Land Agency (ANT), which distributes and formalizes rural property; and 4) The Territorial Renewal Agency (ART), which seeks to “address the needs of the municipalities where due to incidence of armed conflict there is a lower level of development than the national average.” Other existing national government agencies that have sought to increase post-conflict territorial presence are the Office of the High Commissioner for Peace, the National Victims’ Unit, the National Land Restitution Unit, and the Presidential Cooperation Agency. 20 slowness and inflexibility of government procedures; reduced size of the teams responsible for this preparation; a trend towards centralization of planning despite the urgency of involvement of regional actors; staff turnover in the relevant entities and, historically, the burden of unfulfilled commitments with communities affected by the conflict.” A Framework Implementation Plan for the Peace Accord has been delayed: it was first drafted in April 2017 by a government-FARC committee, with OTI supporting some Colombian government agencies’ participation, and finalized in February 2018, but national agencies are still awaiting assignment of responsibilities. Before the Accord’s signing, due to the negotiations’ complexity and confidentiality, OTI had to move away from its initial objectives of helping the government design institutional rapid response architecture and establishing a rapid-response unit. Today, the government’s rapid-response capacity remains underdeveloped. Colombia Transforma personnel and other stakeholders, both governmental and non-governmental, believe that the program improved the national government’s ability to plan, but that strengthening its bureaucratic capacity to execute turned out to be beyond the program’s scope and manageable interest. Civil-society observers’ critiques were especially strong. A Norte de Santander human rights defender viewed national government dysfunction as more a problem of will or information flow, rather than of resources, speculating that the highest levels of national government may be in a bubble, hearing a distortedly optimistic view of what is happening on the ground. “Different agencies call people to different and overlapping meetings. It’s confusing,” complained a Putumayo women’s group. “You’ve got ART, ANT, ADR, and a bunch of little institutions working without synergy,” said a representative of an Arauca Afro￾Colombian organization. “Feria del Chaleco” (“fairs of people wearing vests with government logos”) is a disparaging term that some respondents used to describe national government entities showing up, offering services, but never following through. ORGANIZING NATIONAL AGENCIES’ “ARRIVAL” IN TERRITORY: Numerous stakeholders representing the program’s various categories indicated that Colombia Transforma helped mayors and the post-conflict national agencies to “enter territory.” The program, they said, served as a “foot in the door” or a “bridge” or an “entry door” into territories for national government agencies, which did not know how to “arrive” on their own. In particular, the program supported the Office of the High Commissioner for Peace’s efforts to educate inhabitants of remote areas about the Peace Accord. When asked with which civilian national government entities they had been in direct contact since the Peace Accord’s signing, stakeholders in all three regions named either post-conflict agencies (ART or, less frequently, ANT or ADR), agencies created within the past few years (the National Victims’ and Land Restitution Units), or the Office of the High Commissioner for Peace. Though Colombia Transforma assisted existing agencies and line ministries, they received little mention in the territories. LAND TENURE AND RAPID RESPONSE: ANT, which leads on land tenure issues, has not yet worked with local actors in the Colombia Transforma zones. The extent of its activity with the program is an effort to “decongest” 1,500 titling processes. As of February 2018, this activity had yet to hire a legal consultant. The timeframe for ANT “decongestion” will be long, likely extending beyond the life of Colombia Transforma. The slowness with which Colombia’s state resolves land tenure issues was a frustration for both the 2007-2011 “Consolidation” plan, and for Colombia’s efforts to implement a 2011 land restitution law. It underlies the conclusion, in 1a above, that the land tenure aspects of the Peace Accord’s first chapter do not lend themselves to rapid response.11 BUREAUCRATIC AND LEGAL OBSTACLES: Program documents show that Colombia Transforma recognized at its outset that the GOC faced bureaucratic and legal obstacles to rapid response, especially contracting and budget planning. These obstacles remain fully in place. When the government has the 11 This evaluation does not intend to discourage support for the ANT and its land-tenure effort. The experience of this “decongestion” pilot activity, though, indicates that such support may be more appropriately provided by an international assistance entity, like USAID, which operates with longer timeframes than OTI. 21 resources available, it is hard to free them up, even to pay for event logistics. A February 2015 OTI assessment discussed the problem, citing “Ley [Law] 80’s severe constraints on public contracting,” and suggesting the use of a legal mechanism called “social investment funds,” or the creation of new funds, to speed rapid response. It appears that this idea was not pursued. A mayor in Arauca said that Law 80, a cumbersome anti-corruption contracting law governing municipalities’ use of national government funds, will make it impossible to carry out the kinds of small, rapid activities that Colombia Transforma has supported after the program ends. THE PDET EXPERIENCE: The PDETs—ambitious state-building and infrastructure projects in 170 municipalities most affected by the conflict—have been a strategy of central importance to Colombia Transforma’s effort to improve national government rapid-response capacity, receiving generous investment from the program. This support goes back to the PDETs’ origin: when the ART had to sell to the Post-Conflict Ministry the idea of its PDETs, Colombia Transforma provided territorial analyses and other supporting data. The program helped ART arrive in parts of Catatumbo and Arauca, as well as the far-off municipality of Puerto Leguízamo, Putumayo, where the government had little prior ability to enter and insufficient budget for logistics to hold meetings. PDETs, however, are still uncoordinated internally between national and local staff, according to Bogotá-based Colombia Transforma staff. ART’s Bogotá headquarters is immersed in technical and policy issues, while field offices are more tied up in day-to-day issues. In Puerto Asís and Putumayo, municipal and departmental governments are playing a sharply limited role in the formation of PDETs. Mayors appear more involved in the Alto (eastern) Catatumbo region of Norte de Santander. Putumayo and Arauca ART officials believe they are close to signing PDET agreements in all designated municipalities. In Norte de Santander, however, the PDET construction effort is running behind. The national government had signed several commitments in Catatumbo following a 2013 agrarian strike, and many communities wish to see those promises honored before entering into new accords. Norte de Santander communities participating in PDET “pre-assemblies” are suspicious. At times, when ART officials hand out sign-up sheets at these meetings, community members tear them up. “I’m not convinced by it,” said a local Community Action Board leader. “We’ve already done a thorough diagnosis of what our needs are, we don’t need another.” Said a Catatumbo mayor, “People say ‘here they go again with more meetings to build a document and then nothing else will happen afterward.” THE JUDICIAL BRANCH: The program recognized at the outset, in a February 2015 assessment, that the judicial branch—a key part of the national government apparatus, especially for dealing with corruption, transitional justice, and human rights cases—was not harmonized with efforts to bring the state into ungoverned areas. Colombia Transforma is carrying out no activities with the judiciary or Prosecutor-General’s Office (Fiscalía), which have virtually no presence outside of larger cities in the three zones. The program has merely supported some local measures for alternative dispute resolution, like conciliators or police inspectors. DEPARTMENTAL GOVERNMENT: As discussed in 2d below, departmental governments have done the least to increase their post-conflict presence. Though Colombia Transforma has carried out some successful activities with departmental governments, the net improvement has been the least of the three levels of government. An August 2017 Colombia Transforma assessment stated that departmental governments’ performance has been “markedly worse” than that of municipal governments, with provision of services often tied to political favors. Though low, departmental governments’ capacities vary across the three Colombia Transforma departments. In Putumayo, the governor had a rough start to her term; her last 6-8 months, though, “have seen an increase in the governor’s own trust of her own subordinates,” a Colombia Transforma official said. In Arauca, the governor’s office at first was closely involved in the planning of rapid response, but contacts in the governor’s office have changed, complicating matters for the program. In Norte de Santander, the governor’s office started well but has faded a bit, according to the same official. However, the Norte de Santander governor’s office’s Victims’ Secretariat received credit 22 from stakeholders for its Colombia Transforma-supported work to reach victims and apply the 2011 Victims’ Law in some of the most remote zones of Catatumbo. MUNICIPAL GOVERNMENT: Colombia Transforma staff tended to identify the greatest progress in improving government rapid-response capacity in the program’s work with mayors’ offices. This was especially the case where mayors participate jointly in associations like Asomunicipios, which includes 15 mayors’ offices in and near the Catatumbo region. Colombia Transforma has partnered with Asomunicipios on more than 10 activities, from territorial ordering plans to fire departments to promotional videos, improving the capacity of a coordinating body first formed in 1993. Work with Asomunicipios enables a focus on a larger regional ordering plan, not only individual municipal interests. However, civil society organizations often provided poor assessments of mayors, especially in larger towns (Puerto Asís, Tibú, Ocaña, Arauca capital). In Putumayo, they accused some mayors of clientelism and criticized Colombia Transforma for seeking to involve them in activities. “We have on several occasions asked [the program] for very specific support,” said a Putumayo non-governmental grantee. “The Colombia Transforma team here says, “you need the local government’s approval. But if we have no relations with a mayor who’s a ‘politiquero’ (political intriguer), nothing happens.” Said another: “Stop making us depend so much on mayors who want political favors in exchange for anything.” EXTERNAL POLITICAL CHALLENGES: Some representatives of national-government post-conflict entities shared with the ET that many other agencies within the national government are quietly not supportive of the Accord. At all levels of government, corruption and clientelism—whether real or perceived—remain as challenges to program achievement. As a national government official in Putumayo said: “politics is often regarded locally as a ‘criminal enterprise.’” Months before Colombia’s presidential elections in May 2018, polls point to no clear favorite to succeed President Juan Manuel Santos. The impending exit of the administration that signed the Accord, and the possibility of implementation passing into the hands of one of the Accord’s vocal opponents, presents a threat to progress. One national-government official based in Norte de Santander noted that, regardless of the result, the transition and new officials’ learning curves will cause bottlenecks in the national government’s implementation capacity, citing the likelihood as an argument to maintain Colombia Transforma beyond 2019. Colombia Transforma staff and a Catatumbo social organization voiced concern that a 2018 deterioration in ELN peace talks, and an associated increase in violence, could reverse some of the progress the program has been making. Finally, sub-national government and Colombia Transforma representatives in both Norte de Santander and Arauca said that the impact of the Venezuela crisis at the border has complicated the government’s ability to implement Accord commitments. About 550,000 Venezuelans entered Colombia through official crossings in the latter half of 2017, and a similar number may have crossed unofficially.12 Most entered through Norte de Santander. Venezuelans made nearly 25,000 visits to Colombian emergency rooms in 2017, up from 1,500 in 2015, which has burdened health systems in Norte de Santander and Arauca.13 Local officials have not received guidance, from the central or departmental governments, for how to deal with the influx. EVALUATION QUESTION 2B. HOW EFFECTIVELY HAS COLOMBIA TRANSFORMA PROMOTED COLLABORATION BETWEEN TERRITORIAL ACTORS TO LEAD LOCAL IMPLEMENTATION OF THE PEACE ACCORD? 12 Carmen Fishwick, Emma Graham-Harrison, John Otis, “Colombia and Brazil Clamp Down on Borders as Venezuela Crisis Spurs Exodus” (London: The Guardian, February 9, 2018) . 13 Christine Armario, “Sick Venezuelans Flee to Colombia in Mounting Refugee Crisis” (Bogotá: Associated Press, ABC News, March 8, 2018) . 23 CONCLUSIONS SUCCESS IN PROMOTING COLLABORATION: Colombia Transforma brought together civil society organizations, government agencies, and other actors with little or no previous contact, resulting in increased trust and collaboration. Colombia Transforma has particularly strengthened the capacities, as well as the autonomy, of Community Action Boards. This was a strategic choice but achieving trustful relationships between Community Action Boards and government will take time in areas of longtime guerrilla influence or where distrust, personal or political disagreements, or institutional rivalries predominate. The program instigated some of the first-ever contacts between government (especially national government) and local civil society organizations. The relationships forged, though, may be fragile and depend on continued enjoyment of tangible benefits from joint work. The most progress in collaboration between local actors occurred in communities where either a mayor’s office or a non￾governmental actor—Community Action Boards, the church, women’s groups, ethnic groups, producers’ associations—already demonstrated cohesion, independence, and legitimacy. ENGAGEMENT OF TERRITORIAL ACTORS: The “map” of local actors engaged in the program looked quite different in the three territories, with different social sectors represented. In the Alto (western) Catatumbo, the variety of non-governmental partners, at least those engaged by the program, appeared to be more limited than elsewhere. Despite Colombia’s “post-conflict” context, a climate of fear is dampening willingness to collaborate; collaboration implies a more visible profile for social leaders, and thus more risk of aggression from armed groups, organized crime, and other violence-prone interest groups. The list of Colombia Transforma grantees includes fewer victims’ organizations than one would expect in three territories with a high population of victims. This may owe to a lack of strong existing victims’ organized groups in the three zones. Still, the participation of victims, and the explicit recognition of their role, is a critical element for sustainable territorial peacebuilding. FINDINGS In the regions where it operates, Colombia Transforma seeks to promote collaboration between local government, citizen organizations, and other actors that had few prior relationships due to the conflict, political polarization, or absence from the territory. The collaboration intends to help local actors assume leadership roles in Accord implementation. The program defines its collaboration objectives differently in the three regions. In Putumayo, it seeks to see local actors “strengthened and work together.” In Arauca, it seeks to help them find “new ways of relating between each other” and to “participate in peacebuilding.” In Norte de Santander, it expects them to “invest and increase their participation in the construction of peace” and “work collaboratively on actions.” Forming closer relationships at the local level is an expected outcome of several tactics, all of which the ET found evidence of some success, among them: • Choosing capable and trusted partners and networks, such as Community Action Boards or the Catholic Church’s Pastoral Social. • Consulting with actors and embracing their priorities for collaboration. • Employing a tripartite methodology, with the community, government, and Colombia Transforma each making a contribution (discussed further in 2c) that increased communities’ sense of empowerment and appropriation. • Helping independent social organizations navigate an environment dominated by larger, more ideologically radical groups. • Offering resources that were otherwise unavailable to communities. SUCCESS IN PROMOTING COLLABORATION: Relationships are difficult to measure; OTI uses network diagrams and mapping as tools to track collaboration among stakeholders. These diagrams 24 became more complex as the number of activities increased, indicating that a broader range of actors are relating to each other in the Colombia Transforma territories. To determine the strength of these relationships (whether to “draw a line” between two actors in a network diagram), the program identified four levels of relationship strength: (1) Making first contact; (2) Sustaining conversations; (3) Maintaining a deeper relationship, for example a training relationship; and (4) Full collaboration, the one that “really counts.” Although those criteria are appropriate to track one- or two-way collaboration between the program and its beneficiaries, they do not suffice to determine collaboration among multiple stakeholders from different social or geographical areas (with or without the program),14 for instance between urban youth and indigenous groups, or between women’s groups and victims, or between women’s groups and Afro-descendant groups. However, the fact that such intended relationships are not consciously targeted does not mean that they do not exist. Pastoral Social in Arauca, for instance, stated that due to Colombia Transforma activities, it gained stronger recognition as a socially committed organization working in local development projects that seeks to help the communities beyond religious purposes: “Centered in designing and implementing community projects, Pastoral Social overcame its fear of collaborating with Community Action Boards, and so did the parish priests.” Development of collaborative relationships was uneven, however. In lower Putumayo, in eastern Norte de Santander, and in the central region of Arauca, Community Action Boards and civil-society groups used similar language about building trust and being on friendlier terms (if not actually being friends) with counterparts in other organizations. The ET heard less evidence of relationship-building, however, in upper and middle Putumayo, where a joint project with women’s groups remains unfinished due in part to non-cooperation, and in western Catatumbo and much of Arauca, where the continued presence of armed groups and accompanying political polarization keep distrust at very high levels. Overall, as noted in a draft Colombia Transforma annual report from October 2017, activities generate networks that can be valuable, “but when they are not identified in a timely fashion or well managed, they are not fully sustained over time.” The report recommends activities that support promotion and strengthening of already-created networks. The ET noted that most activities’ collaboration-building results are bilateral, rather than active collaboration networks that have persisted on their own beyond the scope of Colombia Transforma-supported activities. REGIONAL VARIATIONS IN TERRITORIAL ACTORS: “We found that relations of collaboration are very different in different clusters [regions],” noted a Bogotá-based Colombia Transforma official. In Putumayo, the program worked extensively with civil-society groups, which were organizationally strong but related poorly to each other. While there has been some improvement in Putumayo groups’ collaboration, the progress is “fragile” and the flash-flood tragedy in Mocoa may have caused some decay. Arauca’s central region, especially the area near the FARC demobilization site, has seen greater collaboration between “base organizations,” such as Community Action Boards, that had no history of working together. In Catatumbo, Colombia Transforma worked more with municipal government entities, like the strong Asomunicipios. When asked who the organized “community” of partners is in Catatumbo, local Colombia Transforma officials said (1) producers’ associations, (2) Community Action Boards, and (3) the church diocese. There was no mention of other groups or organizations. THREATS AGAINST SOCIAL LEADERS: The ET found that increasing threats and attacks on social leaders, especially Community Action Board members, are posing an obstacle to collaboration. While Colombia Transforma territories are not the hardest-hit, some civil society representatives expressed fear of what might come. In its final report for 2017, Somos Defensores counted 106 killings of social leaders 14 Enduring criteria to determining the strength of such relationship-building success could be whether those relationships: 1) are due to the program, 2) are currently ongoing, 3) have originated other collaboration initiatives aside or beyond the program, and 4) might survive after the program closes. 25 and human rights defenders, of whom 29 (the largest category) were leaders of Community Action Boards.15 The United Nations (UN) Secretary-General reported in a communiqué issued on 20 December 2017 the killing of 73 social leaders, 18 members of social and political movements, and 14 people during social protests. Eleven additional cases were reported by the UN Office of the High Commissioner for Human Rights (OHCHR) to be in the process of verification. All state agencies, as well as international aid agencies that work with communities in territories, look to link with social leaders, but so do armed groups and organized crime, a Bogotá-based Colombia Transforma advisor pointed out. Armed groups and organized crime want to control social leaders in order to dominate territory, which puts social leaders’ lives at risk. Raising social leaders’ profiles is an unavoidable and often desirable outcome of activities that pursue stronger coordination. While the ET did not probe for evidence of threats against social leaders engaged by Colombia Transforma, with a higher profile comes greater risk. VICTIMS ARE UNDER-REPRESENTED: A national government official in Bogotá and a local official in Norte de Santander pointed out that a rapid-response program cannot, or should not, seek to create new organizations in a short time period by seeking out people who are not already active: it needs to work with existing groups. Perhaps because they are less organized and more dispersed in the three territories, conflict victims are not prominent among Colombia Transforma’s partners, even though they make up as much as half the population. A list of program grantees includes only one local victims’ organization. However, a November 2017 Colombia Transforma assessment says that in Putumayo, social organizations, especially victims, increased their participation. “The government is proposing PDETs and things, that’s good, but it has skipped something,” a priest said in Putumayo. “Most of the participant population is victims, who have lost people close to them. People need to recover and heal.” COMMUNITY ACTION BOARDS: Colombia Transforma personnel in all three departments praised the choice to work with Community Action Boards, which have tended to be made up of recognized local leaders with some concern for community welfare and who are mostly independent from more ideological affiliations. In Catatumbo, program staff said Community Action Boards are a “great protection for mayors” when they come to municipalities’ rural zones, often for the first time, to conduct activities. An Arauca Joint Community Action Board (ASOJUNTAS) president said that Colombia Transforma-backed activities “increased the credibility” of his municipality’s Community Action Boards. In another Arauca municipality, an ASOJUNTAS president said that Colombia Transforma support helped them become more autonomous from the mayor or from armed groups. An Arauca NGO said that the program helped resolve conflicts between Community Action Boards whose memberships had been polarized by the conflict. Community Action Boards in Catatumbo objected to the PDETs’ creation of “nuclei” agglomerating local leaders, eight or nine per municipality, including some non-members of Community Action Boards. They viewed it as weakening the Boards’ traditional leadership role. ASOJUNTAS in one Arauca municipality, however, viewed the nuclei issue more positively, as it mirrored existing municipalities combining Community Action Boards. Trust between Community Action Boards and government often remains low, however, because of the conflict. During the conflict years, a Board in a guerrilla-dominated zone could only operate with the local armed group’s tacit or active support. In part because of the perception of capture by armed groups, government and Boards had very little contact during the conflict years in all three zones, and it remains an issue where armed groups continue to operate today. “The national state treats us like we’re guerrillas,” complained a Community Action Board leader in Catatumbo. EVALUATION QUESTION 2C. 15 Piedra en el Zapato (Bogotá: Somos Defensores, 2018): 81 . 26 WHAT ACTIVITIES (OR STRATEGIC APPROACHES) WERE THE MOST EFFECTIVE AT ACHIEVING EACH PROGRAM OBJECTIVE, AND WHY? WHAT WERE THE LEAST EFFECTIVE, AND WHY? CONCLUSIONS OBJECTIVE I ACTIVITIES AND APPROACHES: Small-scale infrastructure activities, especially those that rely on the tripartite, shared-responsibility model, provide tangible evidence of state presence and a model for government, especially local government, to carry out rapid response in communities with which it had no prior relationships. Within the population, these activities can also convey the sense that the Peace Accord, which may seem like a distant reality, is resulting in a benefit for them. Local-level activities have also limited damage to government credibility caused by the broader government’s chronic delays in “arriving” and delivering on Accord commitments. Some beneficiaries viewed tangible infrastructure activities far more favorably than “pedagogy” and technical support meetings with government officials, including the “pre-assemblies” held to plan PDETs. Logistical support made possible national government entities’ arrival in many communities. Cultural activities served as a pretext for initial rapid response in polarized communities, although some beneficiaries pointed out that these communities have more urgent needs farther down the “hierarchy of needs.” A common sentiment was that rapid response is most effective when it builds economic (mostly agriculture-based) opportunities. OBJECTIVE II ACTIVITIES AND APPROACHES: Infrastructure activities foster collaboration between local actors that have had little prior contact or relationships. In the process of jointly contributing to a tangible product, community members learn leadership, problem-solving, accountability, and how to relate to local government. Logistical support made possible events that would not have happened otherwise, which resulted in relationships between actors who would not otherwise have been able to establish contact. Colombia Transforma found much interest among beneficiaries in forming and strengthening producers’ associations, as a way to work together. The program had positive experiences with community radio as an educational and organizational tool for improving local capacities in all three territories. FINDINGS INFRASTRUCTURE: Colombia Transforma staff and documents often mention small-scale infrastructure activities—meeting rooms, school additions, parks, culverts, renovations to common spaces—as helpful “pretexts” or “excuses for community-building.” Their central purpose is to encourage collaborative processes and relationships between local actors and, at times, national agencies. Numerous interviewees of all social sectors praised the results of this joint work. For instance, an activity to build a communal meeting space in Puerto Jordán, Arauca, successfully brought together two Community Action Boards that had been feuding for many years. These activities also serve as a “hook” to get different government entities to converge. Many infrastructure activities employ a tripartite model in which municipal government provides skilled labor, the community provides unskilled labor, and Colombia Transforma provides in-kind materials and equipment. All three actors participate in planning and design beforehand. Respondents reported that this in-kind, shared-work methodology has generated trust in communities. A Catatumbo mayor said it “creates a sense of belonging and does away with that tendency to ‘ask and ask’… people value the result more when they’re involved in putting it together.” Infrastructure activities also brought Catatumbo mayors’ offices to work in territories with Community Action Boards for the first time. In Putumayo, the tripartite model was in use in some places, like La Carmelita near the FARC demobilization site. It appears to be used less regularly in Arauca, where mayors’ offices are either absent or paying for both skilled and unskilled labor. 27 However, the program sometimes “falls into the cement,” as a Catatumbo priest said. “Sometimes the infrastructure projects end up focused too much on the material part,” rushing to deliver a tangible result. “You have to educate people too. You’re building a bridge with these people, but you must communicate that this is a product of the peace process.” In Arauca, Colombia Transforma similarly recalled a need to “ensure that infrastructure projects are the means, not the end.” A program assessment noted a tendency to rush to show state presence with infrastructure activities, even before there was a real national counterpart in the territory. LOGISTICAL SUPPORT: Numerous stakeholders, especially in Putumayo and Norte de Santander, where distances between populations are greater and the road network is poorer, praised Colombia Transforma logistical support (paying for transportation, meals, and meeting spaces). They said it enabled efforts to improve rapid-response capacities and improve collaboration—meetings, trainings, exchanges, workshops, and initial contacts between government agencies and communities—in remote areas where transportation costs are high, or where government fiscal or bureaucratic restrictions would have delayed funding. Mayors in Catatumbo and Arauca credited logistical support with making possible their arrival in rural parts of their municipalities that had been impossible to access before. In Puerto Leguízamo, Putumayo, Colombia Transforma support helped Afro-Colombian community councils speed up the legal recognition of their organizations and their collective landholdings through their first-ever meaningful contacts with national authorities. Logistical support made this possible in an area of long distances, poor roads, and high fuel costs for the boats transiting rivers. After Putumayo’s Alliance of Women Weavers of Peace held an event in El Placer, a town where paramilitaries committed numerous acts of sexual violence, more women began to speak out about what happened to them. This activity depended on logistical support provided by Colombia Transforma. With the program’s logistical support, Community Action Boards formed a “Catatumbo Communal Council” in December 2017 to coordinate work on political issues, including electoral participation. PRODUCERS’ ASSOCIATIONS: In Catatumbo, Colombia Transforma found that promoting associativity for the production and sale of crops brought together people who had limited contact before. Small producers’ associations are a new actor fostered by the program in several Catatumbo municipalities. Some grantees suggested going further than associativity to include investment in agricultural development. Micro-irrigation was an example given for possible activities in Norte de Santander. Arauca Colombia Transforma staff also noted that many partners want to pursue agricultural development activities for which the program is not designed. The distinction between a rapid-response program and a development program is lost on partners. INFORMATION DISSEMINATION AND MEDIA: Beneficiaries gave a mixed assessment of the national government’s Colombia Transforma-supported efforts to educate about the Peace Accord and its implementation. These often consisted of visits from personnel of the Office of the High Commissioner for Peace, and later from the PDET program. Some recipients found these useful: indigenous groups in Norte de Santander and Putumayo said they helped communities understand their rights under the Accord’s ethnic chapter, while Community Action Boards in central Arauca said that education made them more willing to reconcile with former combatants. However, what the government calls “pedagogy” at first meant visits to rural communities from Bogotá-based officials speaking in cosmopolitan, technocratic language: Colombia Transforma had to help officials adjust their approach. Colombia Transforma staff found community radio to be an effective vehicle for informing hard-to-reach communities about the Peace Accord, their rights, and the national GOC’s plans to “arrive” in their territories. Radio worked for communities, a Colombia Transforma official explained, “not to read to them academically,” but “for having dialogues and discussions.” In Putumayo, listeners said “we wouldn’t have had any space like this,” an outlet that put discussions in the hands of regular people. Community radio is an important vehicle for the department’s indigenous and Afro-descendant communities, women, and NGOs working with them. In Arauca, an organization employs community radio to read books and foster 28 discussions about the Peace Accord with campesinos in remote areas. Youth groups have gained a voice using radio in Norte de Santander. CULTURAL ACTIVITIES: Grantees in conflictive municipalities covered the cost of cultural events like dance, music, and art—especially with children—as activities that everyone could get behind. “Fifteen kids carried musical instruments instead of weapons,” a Norte de Santander mayor said. On the other hand, an Arauca Afro-Colombian group, while supportive of cultural activities, said that improving material conditions should come first: “You need to eat to be able to dance.” TECHNICAL ASSISTANCE TO GOVERNMENT INSTITUTIONS: This is the category with the highest per-activity cost ($72,147). It has supported trainings, planning sessions, the salaries of advisors working from government offices, the creation of policy documents, studies of security challenges, and even support for indigenous and human rights organizations. While it is difficult to assess the long-term contribution of trainings and workshops, respondents often said that their missions (such as the generation of studies, the production of educational campaigns, or the production of detailed work plans) would have been impossible without support from Colombia Transforma. As noted in 2a above, these efforts improved linkages between government agencies and contributed to the first steps of their “entry” into formerly conflictive territories. But they were unable to overcome structural deficiencies hobbling government efforts at all levels. Some stakeholders, too, expressed disenchantment with the frequent hosting of meetings within this category, viewing them as either a substitute for action, or as a preparation for actions that do not end up taking place. An Arauca municipal official derided “big expensive national meetings where people go shopping in Bogotá.” A non-governmental organization worried that what emerges from ART meetings in Arauca “just stays on someone’s desk.” EVALUATION QUESTION 2D. TO WHAT EXTENT DO COLOMBIA TRANSFORMA’S LOCAL PARTNERS AND BENEFICIARIES IN THE TERRITORIES PERCEIVE THAT NATIONAL AND LOCAL GOVERNMENT PRESENCE HAS INCREASED AS A RESULT OF THE PEACE ACCORD? IF THEY PERCEIVE THAT PRESENCE HAS NOT INCREASED, WHAT ARE THE REASONS FOR THAT PERCEPTION? DO THEY PERCEIVE THAT ACCORD IMPLEMENTATION REFLECTS THEIR COMMUNITIES’ PRIORITIES? DO LOCAL PARTNERS AND BENEFICIARIES BELIEVE COLOMBIA TRANSFORMA CONTRIBUTED TO THESE OUTCOMES? CONCLUSIONS GOVERNMENT PRESENCE: In general, local partners and beneficiaries in the territories perceived that presence has increased at all levels: national, departmental, and municipal. They nearly always credited Colombia Transforma for at least some of this increase. But presence does not mean better governance. Stakeholders expect government effectiveness through more investment, less corruption, more accountability, and responses to basic and urgent needs. • At the national government level, the trend among observations from government officials, nongovernmental partners, and Colombia Transforma staff was that agencies’ presence has increased modestly and that Colombia Transforma gets some credit. The Office of the High Commissioner for Peace (OACP), which already existed and did not need to be created from scratch when the Accord was signed, was singled out for failing to meet expectations. It is a political agency, not an operational agency meant to “arrive” and remain in territory. • Respondents mostly perceived departmental government presence to be negligible. This presence has improved the least. 29 • Several mayors’ offices have increased their presence and interaction significantly, often via interaction with Community Action Boards. Colombia Transforma has helped mayors’ offices appear in remote veredas of their municipalities for the first time in memory, if ever. ACCORD IMPLEMENTATION AND LOCAL PRIORITIES: Colombia Transforma has helped to increase a perception that the Peace Accord matters for beneficiaries’ local context, although the national state’s sluggish arrival dulls this perception. In Arauca and Norte de Santander, the persistence of other armed groups makes it difficult to speak of an increase in state presence resulting from an accord with only one of these groups. The difference is most notable in portions of Catatumbo and central Arauca where the FARC had the most influence. CREDIT TO COLOMBIA TRANSFORMA: Beneficiaries credit Colombia Transforma for evidence of increased state presence—perhaps too much. Even though the program seeks to avoid attribution, it rarely succeeds, in part because beneficiary communities have never seen an activity work with the speed of the program. Ultimately, the non-attribution goal is unrealistic. FINDINGS DEFINING “PRESENCE”: When asking stakeholders about how “government presence” has changed in their territories, the ET found that the term means different things to different people. Some are grateful for a visit from officials because this recognition had never happened before. Others want more action and tangible results beyond ceremonies, consultations, and plans. “Presence” varies from Bogotá bureaucrats visiting for a few hours and raising expectations, to a few people newly located in a small office in the municipality office, to constant accompaniment in the countryside. “‘Government presence’ isn’t just some staff in an office,” a Putumayo cultural official explained. A Norte de Santander priest was emphatic (though referring to a national government program that Colombia Transforma does not support): “There’s a will to get the national state to arrive, but sometimes it seems stupid. Sending four people [to Catatumbo] to run a crop substitution program? Just signing a document with people doesn’t generate the ‘change of chip’ [necessary change to a post-conflict mindset].” NATIONAL GOVERNMENT PRESENCE: An August 2017 Colombia Transforma internal evaluation voiced officials’ disappointment with the pace of the national government’s “arrival” in the territories. “There’s a difficulty in that the government is arriving bit by bit. Now the ART [is present], but the ANT, the ADR [Rural Development Agency] haven’t arrived. OACP [Office of the High Commissioner for Peace] is only focused on socializing the accords. SENA [National Learning Institute], ICBF [Child and Family Welfare Institute] have small exercises in the region.” Colombia Transforma staff in Catatumbo said the Office of the High Commissioner for Peace has not been very present in that region, despite their efforts to “pull” it there from Bogotá. And a November 2017 Colombia Transforma internal strategy document says that the High Commissioner for Peace implementation strategy in Arauca remains unclear. Nationally, “With regard to the High Commissioner for Peace, the program has had to reduce its expectations.” In Putumayo, social organizations perceived that the national government had increased its presence more than sub-national governments. Colombia Transforma staff there, however, came to the opposite conclusion. Community Action Boards, indigenous groups, and producers’ associations in Norte de Santander and Arauca said they noted more presence of the post-conflict agencies, especially the ART, which got some praise for “listening.” An Arauca municipal official was more dismissive, arguing that the ART only visits his municipality’s rural areas briefly for consultations, then leaves. He added that national agencies like ART convene people repeatedly to ask the same questions. “A local Community Action Board leader told me, ‘That same guy came as part of Consolidation a few years ago and asked me the same questions.’” A Catatumbo mayor lamented that when the national government visits, it “doesn’t know the context of the regions.” This 30 official insisted that he is still the only government presence in rural parts of the municipality, besides the army, since national and departmental government representatives rarely leave the town’s offices or urban area. DEPARTMENTAL GOVERNMENT PRESENCE: In Arauca, respondents reported that the departmental government has very low visibility. “The governor’s office helps us with other things,” a mayor’s office official said, but due to the governor’s office’s heavy bureaucracy “in projects involving Colombia Transforma, it has been far superior to do it without them.” In the Alto Catatumbo, the overwhelming consensus among stakeholders is that the Norte de Santander government has not shown up at all, even though the current governor won handily in the region. In Putumayo, Colombia Transforma staff and non-governmental organization leaders said that departmental government presence was hampered by a natural disaster in the capital and by corruption allegations against the governor—the first woman to serve in that role—who remains under judicial investigation and whose small Green Party is opposed by long-dominant party structures. MUNICIPAL GOVERNMENT PRESENCE: In central Arauca, in the Alto (western) Catatumbo region of Norte de Santander, and in lower Putumayo, respondents noted an increase in many mayors’ offices presence and rapid-response capacity. “Were it not for Colombia Transforma, these veredas would always be forgotten,” said a representative of an Arauca NGO. Mayors received lower marks in Putumayo’s more populous municipalities and in eastern Catatumbo. In the latter region, Community Action Boards called their mayor’s office “slow and lazy,” and said they have had more interlocution with national agencies. A youth peace organization in that area said it always reaches out to mayors’ offices when it holds events in the field, but rarely gets a response. ACCORD IMPLEMENTATION AND LOCAL PRIORITIES: When asked whether recent activities were made possible by the Peace Accord, a Community Action Board leader in Arauca replied, “Of course.” “Putumayans at first were suspicious about the Peace Accord,” said a national post-conflict agency official in the department. “But when the FARC demobilized, and suddenly you could go anywhere, there was more trust. You feel the difference in government presence in improved security.” However, the official worries that the state’s response is insufficiently rapid. “We’re not taking advantage of these ‘15 minutes’ of tranquility by rapidly increasing state presence.” An August 2017 OTI assessment recognizes that Colombia Transforma still faces a challenge in messaging to beneficiaries that “all activities are a result of the Peace Accord.” In Arauca and Norte de Santander, where other armed groups continue to operate, challenges are even more acute, as beneficiaries expressed little sense of being in a “post-conflict” situation in the first place. A national government official in Norte de Santander noted, “People say, without an ELN accord, “‘la paz de Santos’ is not ours.” An Arauca-based national government official added: “Much of the population sees no difference as a result of the Peace Accord.” ATTRIBUTION: Colombia Transforma seeks to avoid credit for activities that it funds and facilitates. It wants partners, especially government entities, to “show their faces” as evidence that the government is arriving in territories. Still, communities tend not to recognize the government as the driving force of activities supported by Colombia Transforma. A Putumayo-based ART official said that even though Colombia Transforma “doesn’t appear in the photo… people end up thanking Colombia Transforma.” Colombia Transforma’s Arauca staff said, “Partners don’t understand Colombia Transforma’s intention to avoid attribution. Everybody understands where the money is coming from, so…” Beneficiaries often contrast Colombia Transforma’s speed with the sluggish pace of government institutions, but the comparison risks worsening perceptions of the state. This comparison reaches such an extreme that, according to an August 2017 internal program assessment, “Some community leaders expressed concern about having to work through GOC ‘intermediaries’ once Accord implementation began, losing their direct financial relationship to the program.” 31 Colombia Transforma Arauca personnel recommended addressing attribution issues by carrying out activities more often from within the government entities’ facilities. The August 2017 assessment notes that attribution also goes better when a mayor or similar government office visibly accompanies the activity the whole time. EVALUATION QUESTION 2E. TO WHAT EXTENT DO COLOMBIA TRANSFORMA’S LOCAL PARTNERS AND BENEFICIARIES IN THE TERRITORIES BETTER UNDERSTAND THEIR ROLES AND RESPONSIBILITIES FOR ACCORD IMPLEMENTATION? TO WHAT EXTENT, IF ANY, ARE THEY PROACTIVELY ASSUMING LEADERSHIP ROLES TO ENSURE THAT ACCORD IMPLEMENTATION REFLECTS THEIR COMMUNITIES’ PRIORITIES? IF THEY AREN’T ASSUMING LEADERSHIP ROLES, WHAT ARE THE REASONS? DO THEY FEEL THEY HAVE PARTICIPATED IN ACCORD IMPLEMENTATION IN MEANINGFUL WAYS? DO LOCAL PARTNERS AND BENEFICIARIES BELIEVE COLOMBIA TRANSFORMA CONTRIBUTED TO THESE OUTCOMES? CONCLUSIONS ROLE OF DEPARTMENTAL AND MUNICIPAL GOVERNMENTS: Sub-national governments have limited understanding of what the Peace Accord means for their development plans and responsibilities. There is a perception that the Accord confers more burdens and tasks on them, that it is a mandate imposed on them with no opportunity for input. Some actors, including in local government, had unrealistic expectations of the Accord, especially of the resources that would come with it. Colombia Transforma has helped adjust these expectations by supporting efforts to educate about the Accord and by working with mayors and governors on activities, like infrastructure projects, that deliver some new services despite persistent fiscal gaps. Nonetheless, competition for post-conflict resources risks fracturing relationships and associations. ROLE OF CIVIL SOCIETY: In areas now largely free of armed groups, with the FARC no longer making rules and resolving disputes, many communities have a need to understand their role. Their role, in turn, depends on how they are defining “peace”: as a silencing of weapons, or as a broader resolution of many long-felt needs. Colombia Transforma helped organizations at least partially to understand their new roles, but substantial needs in this area remain. Colombia Transforma’s work to increase traditionally excluded actors’ engagement in public policymaking is a baby step toward the political participation foreseen in Chapter 2 of the Peace Accord. A challenge to civil society organizations playing a greater role in Accord implementation is the influence of large, ideologically hardline campesino confederations that share at least some positions with combatants or with the FARC party. Their power, and their stronger interlocution with the national government, dull the sense of anything changing as a result of the Peace Accord. FINDINGS KNOWLEDGE OF THE ACCORD’S CONTENT: There are diverging definitions in the territories of what “peace” means, ranging from a silencing of weapons and greater security, to a whole panoply of historically absent government services. Some observed that creative civic education, “or pedagogy,” is needed to “ground” the Peace Accord and show its relevance to people’s everyday reality. Many communities’ knowledge of the Peace Accord remains low, despite “pedagogy” efforts. This particularly complicates the PDET process, which works better if communities understand Chapter 1 of the Peace Accord, which it intends to implement “Many people still don’t know the peace accords, and it’s hard to demand that they be implemented if you don’t know them,” said an Arauca women’s group leader. Colombia Transforma staff and a Putumayo social leader credited the Office of the High Commissioner for 32 Peace’s Colombia Transforma-supported “pedagogy” efforts with increasing understanding of the Accord at the local level, “but not enough.” UNDERSTANDING NEW ROLES WITHIN SUB-NATIONAL GOVERNMENT: In the view of some national government officials and Colombia Transforma staff, mayors’ and governors’ offices are still not conscious of the Accord’s implications for their tasks and responsibilities, or for their development plans. Sub-national governments at times view Accord commitments as “more bureaucratic processes and obstacles to go through,” another set of burdens and tasks. An August 2017 internal program evaluation hints that mayors’ discourse is generally supportive of the Accord, but “the same doesn’t necessarily happen with their teams. Tying activities with Accord implementation needs to be reinforced.” The same document contended that communities that used to turn to armed groups to solve problems “began to identify the state.” However, Colombia Transforma Arauca staff observed, “Some actors don’t recognize themselves as actors in implementation, like mayors who complain about ‘the government’ without identifying themselves as government.” Colombia Transforma was unable to dispel some municipal governments’ mistaken notion that “Peace Accord” meant “lots of money”—a misunderstanding of their post-conflict role. “We’d thought there’d be more resources post-Accord,” a Catatumbo mayor said, but his administration has ended up having to make do with less. Colombia Transforma has helped fill the gap, particularly by funding small infrastructure projects and paying logistical costs for activities, especially in municipalities’ hard-to-reach rural areas. In Putumayo, a Colombia Transforma document noted, people have had to downgrade expectations that the Accord would bring “a rain of dollars.” As they have adjusted their expectations, Colombia Transforma has helped by supporting activities. In Catatumbo, where a strengthened and more active Asomunicipios has proven to be an effective space for coordination between mayors, some of them are concerned about the divisive effects of a scramble for post-conflict resources from the national government. Mayors whose municipalities are not prioritized for the PDET are very unhappy about being left out. UNDERSTANDING NEW ROLES WITHIN CIVIL SOCIETY: Starting with a very low understanding of roles and very little cooperation between organizations, civil society in all three regions made modest progress toward assuming new post-Accord roles. Leaders of Putumayo social organizations who participated in a Colombia Transforma-supported “Putumayo Space for Territorial Peace” event attested to greater empowerment of communities, a “vision of a Putumayo without fear.… This was an abandoned zone, but people know they need to get out and participate.” Non-governmental groups in all three regions met more, via program-supported activities, to learn about and reflect on the Accord, deciding how to prioritize their work in advocating for its implementation. Social leaders in Putumayo risk being “crowded out” by a more aggressive, FARC-linked campesino organization called the Regional Board of Social Organizations (MEROS) that has strong interlocution with the government. A similar dynamic occurs in Catatumbo, where three campesino organizations (The Campesino Association of Catatumbo [ASCAMCAT], The Social Integration Committee of Catatumbo [CISCA], and Movement for the Popular Constituent Assembly [MCP]) occupy much space. When Colombia Transforma comes and says: “what is your priority,” a Catatumbo mayor said, those large groups respond that the issues (inequality, impunity, an economic model they regard to be an unlevel playing field) are much more fundamental than what can be addressed through individual activities. The government tends to pay more attention to those groups, although it has left them out of the PDET design process. There is a similar dynamic in Arauca, in which two groups, the Mesa Civica and the Movimiento de Masas, “influence all of the civilian population’s actions in the municipalities,” according to a July 2016 Colombia Transforma diagnostic. The program has not worked with any of these large groups and instead seeks to support voices whom the large groups are crowding out. In Arauca, Colombia Transforma staff include among successes having “diminished armed groups’ ideological influence over civil society organizations.” According to a mid-2017 internal Colombia Transforma assessment, “Some of the pretty radical civil society actors started working 33 with us, were shut out by higher-ups, then back to working with us.” One women’s group linked up with Colombia Transforma via a leader of the Mesa Cívica. In Catatumbo, according to a few non-governmental sources, efforts to bring Community Action Boards together have made them stronger and thus more resistant and autonomous from the large campesino federations. They have been more able to say no, for instance, to their calls to participate in massive blockade protests (paros). LOCAL LEADERSHIP AND PARTICIPATION IN ACCORD IMPLEMENTATION: In areas where other armed groups continue to exercise influence, or where large campesino federations occupy much political space, the ET’s inquiries revealed the idea of post-conflict “leadership” to be premature. However, the increased collaboration and capacity resulting from Colombia Transforma-supported activities (discussed in 2b above) has left some groups—Community Action Boards, ethnic organizations, and producers’ associations—expressing a desire to increase their interlocution with the government. However, the ET rarely heard this desire expressed as a result of the Peace Accord. In Putumayo, which has less armed-group presence, strong but separate social organizations have viewed the Accord as an opportunity to increase participation in politics and to make demands of the state. A Colombia Transforma-supported “Space for Territorial Peace” sprung up in Putumayo during the latter phase of the peace negotiations and sought to increase joint action as a direct response to the Accord. It remains active, though a severe natural disaster that hit the department in March 2017 reduced civil society focus on Accord implementation. Ethnic and women’s groups, with Colombia Transforma support, also increased their participation in all three territories; they tended to view this increase as linked to the Peace Accord in Putumayo more than elsewhere. TOPICS OF INTEREST 2F. HAS THE PROGRAM AFFECTED INDIGENOUS GROUPS, AFRO-COLOMBIANS, MEN, WOMEN, AND YOUTH IN ANY SPECIFIC WAYS? ARE THESE GROUPS ADEQUATELY CONSIDERED IN PROGRAM IMPLEMENTATION? CONCLUSIONS INDIGENOUS COMMUNITIES: Indigenous organizations, while generally well-organized, have infrequent interlocution with government, especially at the national and departmental levels. Government too often continues to regard indigenous groups as an obstacle to development, which in turn is an obstacle to peace implementation. Indigenous groups appeared to be adequately represented among awardees of program funds. Indigenous groups give generally high marks to Colombia Transforma for including them and dialoguing with them, although program staff have been concerned about the slow pace of rapid response that has resulted from extensive dialogue and consultation. AFRO-COLOMBIAN COMMUNITIES: Afro-Colombian communities in the Colombia Transforma territories seek greater cultural recognition, especially the legalization of their collective landholdings. Though land tenure is not a rapid response item, Colombia Transforma has usefully helped Afro-Colombian groups to begin the long process of legalizing community councils. Program-supported activities with ethnic groups, especially those with the PDET process in Putumayo, risk exacerbating simmering disputes over land tenure. PDET meetings, whether Colombia Transforma-supported or not, need to be mindful of the risks of reflecting the chosen priorities of a community’s majority at the expense of minority members. YOUTH: Support to youth groups has empowered them, helping them to learn to solve problems on their own, and to engage the state directly. The number of youth groups engaged by Colombia Transforma is small, though, leaving the impression that they deserve more consideration. 34 WOMEN: Women’s participation in Accord implementation has barely begun to be recognized in the three territories, though strong organizations exist in Arauca and Putumayo, where seven of nine program activities with women’s groups took place. Colombia Transforma support increased these existing organizations’ ability to function as peace mediators, to promote income generation, and to draw attention to victims. FINDINGS According to a Colombia Transforma “Program Document” (03/18/2016), the program “will conduct many activities with and through civil society organizations, NGOs, women’s, and indigenous groups. Support to such organizations is not an end in itself but seeks to leverage the significant roles these groups play during the rapid response phase, and as a demonstrative tool for what the government at any level might replicate elsewhere.” The statement in this document is an indication that Colombia Transforma consciously lacks an intentional gender or ethnic-specific approach. However, the program is mindful of the importance of working with such groups. Implicitly, for instance, the three regions outside Bogota chosen by the program meant increased chances of working with indigenous peoples present in those areas. WOMEN: Colombia Transforma’s 83-person staff is made up of 47 women (57 percent) and 36 men (43 percent); among the 10 leadership positions, 60 percent are held by women and 40 percent by men. The program makes conscious efforts to track beneficiaries by sex; for instance, the Year 2 Annual Report (Oct/16-Sept/17) counts 15,921 (43.6 percent) women out of 36,493 total beneficiaries. Table 3. Award Contact Person Distribution by Gender All Activities with an Awardee Contact Men Women Percentage Women Community Group 69 16 19% Host Govt. Municipal 63 6 9% Host Govt. - National 24 23 49% Host Govt. - Departmental 34 13 28% Local NGO 37 14 27% DDGS or No Awardee Type Listed 24 29 55% All Activities Total 251 101 29% Contact individuals for each activity were overwhelmingly male, as indicated in Table 3. Wherever Colombia Transforma has found organized women’s groups, it has endeavored to work with them. In Putumayo and Arauca, efforts to bring together strong but separate women’s organizations did not succeed, due to what appeared to be political disagreements, personal animosities, or competition for recognition. Most other non-governmental grantees, especially Community Action Boards and ethnic organizations, are predominantly male. INDIGENOUS COMMUNITIES: As of March 2018, the program had supported 16 activities with indigenous communities, about 10 percent of all activities with non-governmental partners. National government and Colombia Transforma officials perceived that the national government’s program￾supported attempts to relate to indigenous communities had not gone well. Issues included conflicting definitions of “prior consultation” and these communities’ preference to work within longer timeframes that are not synchronized to “rapid response.” In Putumayo and Catatumbo, however, indigenous leaders mostly praised Colombia Transforma’s efforts and respect for the groups’ slower, more deliberative timeframes. “Historically, we’ve been called a nuisance in the development of the country,” he said. “So, we thank Colombia Transforma for supporting us.” Putumayo indigenous leaders said that they already understand their post-conflict roles due to their “cosmovision” [spiritual and social belief system]. Catatumbo mayors voiced a belief that indigenous communities have not felt benefits of peace and have 35 not been able to participate meaningfully in post-conflict efforts. However, they said, these communities are organized and participatory, and “an important social sector for Colombia Transforma to work with.” AFRO-COLOMBIAN COMMUNITIES: “There was distrust before,” but following Colombia Transforma￾supported activities, an Arauca-based Afro-Colombian group declared itself “now more prepared to gain legal status for our associations.” In Puerto Leguízamo, Putumayo and Tame, Arauca, Colombia Transforma facilitated Afro-Colombian organizations’ first contacts with municipal and national government officials, which generated a process to begin legalizing collective landholdings as “community councils.” Though indigenous and Afro-Colombian groups are key actors in Putumayo, they have different potential roles in implementation of the Peace Accord. A main point of contention between them is control of land. PDET planning may be eroding relations between Afro-Colombian and indigenous communities. At meetings, “communities vote for the projects that they want,” but indigenous participants are a majority that overrules Afro-Colombian communities on choosing priority activities. YOUTH: Stakeholders reflected on the importance of working with young people as critical to “changing the chip” [mindset] of their communities. Representatives of youth organizations in Norte de Santander said they are most interested in the political participation chapter of the Peace Accord. They are becoming increasingly comfortable engaging the state, proposing instead of protesting, and they credit Colombia Transforma-supported activities with contributing to the change. Still, a Norte de Santander youth peace group said that young people in rural towns feel very distant from the governments and have not been called to participate in the design of PDETs. Catatumbo Community Action Board leaders criticized younger community members for seeking to pursue different kinds of activities and not understanding the elders’ “struggle.” In Arauca, Colombia Transforma personnel say that youth groups have been hard to partner with and are very polarized, especially by the two major civil society federations. RECOMMENDATIONS FOR EVALUATION QUESTION 2. 8. Reconsider the current selection of governmental partners to leave behind more installed capacity, giving priority to municipal governments that have shown will and are devoting resources. As the national government is undergoing vast personnel changes with an upcoming electoral transition, and since many national agencies have not responded assiduously to rapid-response imperatives, reduce investment in national government activities in favor of sub-national government for the rest of the program’s life. Remaining investment in national entities should focus on the new agencies created for Peace Accord implementation, and priority should be given to those that work more closely with the department and municipal governments, as well as with local communities, to maximize local capacity after the program. Watch these entities closely, however, for signs that the government assuming power in August 2018 is weakening or de-prioritizing them. Indicators would include reduced budgeting, effective demotions within institutional architecture, the naming of managers according to political criteria rather than expertise, or reduced commitment to consultation with communities. 9. Increase Colombia Transforma’s effective promotion of collaboration among local actors by choosing groups of current beneficiaries (for instance urban youth, indigenous, women’s, victims, or Afro-descendant groups) with which to undertake initiatives that (1) are designed and implemented with the express purpose that they work with each other, (2) involve peace-building, and (2) have potential to continue after the program closes. 10. Highlight the legitimate role of social leaders and the importance of protecting them while seeking to include victims as partners in activities. The moment that a beneficiary receives a credible threat, act aggressively to involve Colombian government entities with protection responsibilities (including but not limited to the Interior Ministry National Protection Department, 36 Defensoría and Early Warning System, Fiscalía, and National Police). The urgency and assertiveness of the program’s response in the face of a threat should serve as an example to Colombian government counterparts of how seriously such threats deserve to be taken. 11. Continue pursuing activities that help Afro-Colombian groups take the first of many steps to legalize community councils. Emphasize women’s activities among civil society organizations and indigenous, Afro-Colombian, and youth groups while increasing overall support to women’s groups. In addition, increase the probability of “green-lighting” activities with partners from all social sectors or groups whose leadership is female. 12. Continue infrastructure activities and encourage use of the tripartite model in which municipal government, community leaders, and Colombia Transforma are all expected to make contributions. At this phase in the Accord implementation process, carrying out tangible actions like infrastructure and community-building activities is the best way to educate target communities about their roles and rights. 13. Continue providing logistical support, especially to efforts of municipal government, while finding legal, administrative, and budgetary ways to undo bureaucratic entanglements that prevent agencies from using their own resources for these purposes. The ART most needs help in getting beyond meetings and generating “early wins.” The PICs are an important contribution, but Colombia Transforma must help ensure that the PDETs are more than just a paper exercise. 14. De-emphasize top-down efforts to help national government agencies “arrive” in rural areas. Instead, focus on bottom-up approaches starting with municipal governments. Continue engaging with mayors’ offices, increasing support for those most inclined to respond rapidly and transparently, and all while strengthening civil society capacity. For the remainder of the program, activities with the national government should focus on new post-conflict agencies’ efforts to deliver on commitments already made in meetings, rather than additional planning, workshops, and events. 15. Community Action Boards should continue to be a central partner in Putumayo, Norte de Santander, and Arauca. Efforts to help the government “arrive” should foster—not weaken—the boards’ performance. In addition, continue strengthening independent civil society organizations and upholding their role with respect to more hardline and larger organizations that often have more interlocution with the national government. 37 ANNEXES ANNEX A. EVALUATION SCOPE OF WORK OTI Colombia Mid-Term Evaluation Scope of Work (FINAL) PERIOD OF PERFORMANCE: January - April 2018 (Fieldwork in February 2018) BACKGROUND For 50 years Colombia experienced an intractable armed conflict against insurgency groups, resulting in 220,000 deaths and over 7 million registered victims. The civil war exacerbated longstanding inequalities between Colombia’s modern, urban centers and the poor, neglected rural areas where most of the conflict has taken place. These rural regions have long suffered from a lack of government services, poor infrastructure and the presence of criminal groups and illegal economies based around the coca trade. After four years of negotiations, the GOC ratified a final peace accord with the largest guerrilla insurgency group, the FARC, on November 30, 2016. Consequently, the FARC has formally demobilized and the GOC has begun to enact wide-ranging reforms related to rural development, inclusive political participation, transitional justice and the reduction of illicit crops such as coca – all root causes of the conflict. While it may take a decade or more to fully implement the Accord, the first few years are the most critical. The USAID/Office of Transition Initiatives (OTI) Colombia program, Colombia Transforma, helps the Colombian government during this critical make-or-break period to follow through on the promises made and demonstrate immediate, tangible benefits of peace in rural conflict regions like Arauca, Norte de Santander and Putumayo. A SWIFT IV task order for Colombia Transforma was issued to MSI on July 24, 2015 and is set to end on July 23, 2019. The program has four programmatic teams based in Bogotá (National Grants) and three departments: Arauca, Norte de Santander (focused in the Catatumbo), and Putumayo. Launched in August 2015, as peace negotiations accelerated in Havana, Colombia Transforma is part of the U.S. Government’s (USG) effort to support a sustainable and inclusive Colombian peace. Colombia Transforma Goal: Enhance Colombia’s ability to implement rapid response during the critical 36 months following the signing of the Peace Accord between the GOC and the FARC. Objective I: Increase the GOC’s ability to plan and execute rapid response aspects of the peace process during the first three years of implementation; Objective 2: Promote collaboration between territorial actors (i.e., local governmental and non￾governmental actors) to lead local implementation of the Accord in conflict regions. The program operates, to a large extent, under the assumption that the success of the Peace Accord depends on how fast Colombians who have suffered the most from armed conflict begin to experience changes brought by peace. Not only must the national government deliver on its promises, but the center of gravity for Accord implementation must also move away from Bogotá to the countryside with local communities and local government authorities taking leadership roles to shape priorities on the ground. This requires that community and civil society actors collaborate with each other and with local and 38 national government institutions, overcoming longstanding tensions stemming from the conflict and confusion over their new roles and responsibilities under the Accord. PURPOSE The purpose of this activity is to conduct an independent mid-term performance evaluation of USAID/OTI’s Colombia Transforma program in Colombia. The evaluation will assess the strategic approach, relevance, and contributions of Colombia Transforma toward enhancing Colombia’s ability to implement rapid response following the signing of the historic Peace Accord with the FARC guerrillas. The final product will be used to inform future USG assistance and OTI programming in Colombia and will be shared with the USAID/Colombia mission, the U.S. Embassy in Bogotá, local stakeholders including the GOC, and other interested groups in Washington, DC. Findings from the mid-term evaluation will be used by OTI and MSI to inform the direction of the program during the final year of implementation. Although the scope of the evaluation will cover the program period of performance to date, more emphasis should be placed on activities implemented after the final Peace Accord was ratified on November 30, 2016 rather than activities implemented during the preparatory phase from August 2015 - November 2016. This weighting is intended to offer the program suggestions about mid-course corrections with greater practical value at this stage of implementation. LINES OF INQUIRY 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? (LOE 20%) a. Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? b. If there are missed opportunities due to U.S. foreign policy constraints outside the program’s own manageable interests, what are those constraints? c. Potential Topics of Interest: • To what extent has Colombia Transforma adjusted its strategic approach to respond to shifting local context in Colombia? • To what extent has Colombia Transforma adjusted its strategic approach based on lessons (positive and negative) from past Colombia Transforma activities in accordance with OTI’s iterative action learning model? • Has the program’s strategic approach led to unintended outcomes, either positive or negative? 2. To what extent has Colombia Transforma achieved its program objectives? Please review the program’s four portfolios equally: (Arauca, Norte de Santander, Putumayo, and National Grants),16 with greater emphasis on activities that began after the Accord was signed in November 2016. While the evaluation team may not be able to attribute outcomes solely to Colombia Transforma, when answering the questions below, please consider to what extent the program contributed to the outcomes. (LOE 80%) a. Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? b. Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? c. What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? d. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace 16 See OTI program maps to see the municipalities where MSI has prioritized programming in these departments. 39 Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? f. Potential Topics of Interest: • Has the program led to unintended outcomes, either positive or negative? • Is the program working with the governmental and non-governmental partners who are most relevant to rapid response, both nationally and locally, in the program’s three departments (Arauca, Norte de Santander, and Putumayo)? If the program missed any key stakeholders or significant opportunities related to its chosen partners, which partners and stakeholders should the program engage moving forward? • Has the program affected indigenous groups, Afro-Colombians, men, women, and youth in any specific ways? Are these groups adequately considered in program implementation? • Has the program effectively shared relevant programmatic lessons with GOC counterparts? • Which lines of inquiry would be most relevant for future data collection efforts to inform programming and measure the program’s impact? METHODOLOGY This evaluation will be non-experimental and largely qualitative in nature, but mixed methods may be used as appropriate. In answering the evaluation questions, the contractor shall utilize data that is disaggregated and analyzed by sex, age, ethnicity (i.e., indigenous, Afro-Colombian, or not), and location, whenever such data is available. Methodological specifics will be agreed upon among the evaluators, OTI/Colombia, and OTI/Washington, and the evaluators are encouraged to suggest creative approaches. OTI’s activity database is a rich source of information on individual activities and should be utilized by the evaluation team. Activity sampling methodology should prioritize activities that began after November 30, 2016 when the Peace Accord was signed and ratified. Sampling methodology should allow the team to evaluate the program’s four portfolios equally (Arauca, Norte de Santander, Putumayo, and National Grants) and should acknowledge relationships between activities, such as activities that led to new activities and/or were adapted or replicated by the program and/or the GOC. Possible methods for the evaluation include: • Facilitated workshop with key program staff (OTI and MSI) to reflect on program implementation, challenges and successes; • Field visits to the implementation areas; • Interviews with key program stakeholders, including U.S. Embassy and USAID staff, MSI staff, GOC officials (national, departmental, local), community leaders, and clients (beneficiaries); • Focus group discussions and interviews with clients (beneficiaries), grantees, and others; • Perception surveys, or secondary analysis of existing perception surveys, if available; • Direct observation; • Documentation review, e.g., bi-weekly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments, and Program Performance Reviews (PPRs), OTI/Colombia’s activity database, the program’s monitoring and evaluation (M&E) tracking tools, an internal cluster evaluation, and other existing data produced by the program or by other third parties, such as the GOC. 40 TEAM COMPOSITION The evaluation team should be comprised per the specifications described below. The positions to be filled by Social Impact require USAID/OTI concurrence of the proposed candidates. In addition to the 2- 3 evaluators engaged by Social Impact, OTI intends to field a staff member to participate in the evaluation as a resource for the evaluation team. Language Skills: All members of the evaluation team must be able to speak, read, write and facilitate interviews and/or focus groups in fluent or very advanced Spanish while in Colombia without need of an interpreter. They should also have strong English writing, reading and speaking skills. Division of Team Roles and Responsibilities: Social Impact, in consultation with the team members, should propose the final makeup of the team and the division of roles and responsibilities and LOE among them. Social Impact should propose whether the senior policy expert or the senior evaluation expert will serve as the evaluation Team Leader. OTI also welcomes the potential subcontracting of local evaluation expert(s) and/or logistics support via a local Colombian evaluation firm, if needed. One senior policy expert who has research experience and at least ten years’ experience following and influencing U.S. foreign policy toward Colombia. He/she must have extensive experience related to, and a deep understanding of Colombia’s political system; the Havana Peace Accord; the GOC’s progress toward implementation; and related peace, security, and counter-narcotics policies within Colombia. He/she should be well connected to Colombian and U.S. policymakers as well as Colombian civil society. If identified as the Team Leader, he/she will be responsible for liaising with USAID/OTI, the preparation of all deliverables, and leading briefings with relevant stakeholders. If not the Team Leader, his/her main role will be to ensure that the evaluation’s findings are relevant to the Colombian political context and U.S. foreign policy context. He/she will support the Team Leader and participate in the pre-deployment OTI orientation (likely in Washington, DC), data collection and analysis, drafting the final evaluation report, and debriefs in the field as well as in Washington, DC. One senior evaluation expert, with research and evaluation experience in complex political or transition environments, including in the local Colombian context. A Colombian national is strongly preferred. Knowledge of OTI-type programming is preferred. If identified as the Team Leader, he/she will be responsible for liaising with USAID/OTI, the preparation of all deliverables, and leading briefings with relevant stakeholders. If not a Team Leader, his/her main role will be to ensure that the evaluation’s design is rigorous and sensitive to the local Colombian context. He/she will support the Team Leader and participate in the pre-deployment OTI orientation (likely in Washington, DC), data collection and analysis, drafting the final evaluation report, and debriefs in the field as well as in Washington, DC. One mid-level evaluation specialist (optional), preferably a Colombian national, with research and/or evaluation experience in complex political or transition environments, including in the local Colombian context. Knowledge of OTI-type programming is preferred. He/she will support the Team Leader and participate in the pre-deployment OTI orientation (likely in Washington, DC), data collection and analysis, drafting the final evaluation report, and debriefs in the field. If a Colombian national, he/she will not be required to participate in person in debriefs in Washington, DC, but can participate remotely when necessary. One OTI resource person: In addition to the 2-3 evaluators hired by Social Impact, OTI intends to field a staff member to participate in the evaluation as a resource member for the team. The role of the OTI Resource Person will be to help the evaluation team better understand, and contextualize data in, the OTI operating model. The exact 41 scope of the resource person will be defined once the individual is identified. He/she will participate with the evaluation team in pre-deployment sessions intended to help orient the evaluation team to OTI’s business model, systems, processes, and approaches to monitoring and evaluation (M&E), and he/she will accompany the team to Bogotá and possibly to some sites in Arauca, Norte de Santander, and Putumayo, depending on USG personnel security restrictions. He/she will serve as a key liaison and point of contact between the Evaluation Team, the OTI Colombia Team and MSI, troubleshooting any logistical challenges, and he/she can help the team arrange certain interviews as appropriate. He/she will be available for regularly scheduled check-ins with the evaluation team during data collection. He/she may also participate as a ‘silent observer’ during certain interviews and/or focus groups, only if needed, or to be on hand for easy consultation on questions and to provide more detailed and timely contextualization of data within the OTI model. However, in order to minimize the potential for bias, the OTI Resource Person should not be involved in data analysis, developing findings or conclusions, or drafting deliverables. His/her main role is that of dedicated resource, liaison, and logistics coordinator. The OTI Resource Person may be called upon to do an initial review of deliverables or a draft report outline for the evaluation team to receive early feedback prior to preparing a complete draft. TIMELINE and LEVEL OF EFFORT (LOE) The fieldwork of the evaluation will take place over approximately 4-5 weeks. The team will travel to Colombia on/about Feb 4, 2018 and depart on/about March 2, 2018 or the following week if more time is necessary. In addition, the team will have a total of two weeks of LOE, over a four-week period in advance of departure for documentation review and some Washington-based interviews. This activity will also include 3.5 weeks of LOE for writing and completion of deliverables. In total, OTI envisions up to 63 days of LOE per member of the evaluation team per the timeline below, including international travel LOE which depends on whether team members are U.S.- or Bogotá-based. However, OTI will look to Social Impact for recommendations to help refine this timeline and LOE before it is finalized based on the final makeup of the evaluation team (total members, division of labor among them, etc.) OTI’s priority is 1) to assemble the appropriate team, working within team members’ availability; and 2) schedule the bulk of the field data collection during February 2018 to mitigate the potential biases that campaigns leading up to the May 2018 Colombian elections may place on respondents during interviews and focus groups. Therefore, if members of the team are only available part-time before or after the field visits, OTI will accommodate adjustments to dates included below. Dates* Task LOE Week of January 1 (timing dependent on team availability) Kick-off call w/ OTI team to refine evaluation questions, initial methodology, etc. Database training should happen early in January to inform design and sampling. 1 Day Before Friday Feb 2 (can start as soon as team is approved) Reading on Colombia context and Colombia Transforma. Key materials to be provided by OTI. Key Washington-based interviews. SI onboarding and EQUI training (2 days) Training on OTI model 7 Days Over a 3-4 week period is recommended. 42 Dates* Task LOE Due Friday, January 19 Drafting Evaluation Inception Report and draft data collection instruments. OTI to reply with comments in one week (by Friday, Jan. 26). 5 Days Week of January 26 Revisions to the Inception Report/methodology. Revision process should include follow-up meeting/call with OTI team to discuss feedback to the Inception Report, refine methodology, travel and logistics, etc. 2 Days Most/all of team will likely need to be in Washington, DC so may need to build in 1 extra day of travel for any Bogota￾based team members to come to Washington. Saturday, Feb 3 Travel to Bogotá, Colombia 1 Day Mon, Feb 5 Bogotá: preliminary meetings with OTI and IP 1 Day Tues, Feb 6 - Wed, Feb 7 Tool piloting and refinement 2 days Thurs Feb 8- Tues, Feb 13 Bogotá-based interviews with senior management, staff, U.S. mission, National entities as appropriate 5 days Wed, Feb 14 - Sat, Feb 17 Travel to Arauca for site visit and interviews 4 days Mon, Feb 19 -Thurs, Feb 22 *Note Feb 20 is a USG holiday, but not in Colombia. Travel to Norte de Santander for site visit and interviews 4 days Friday, Feb 23 - Tues, Feb 27 Travel to Putumayo for site visit and interviews 4 days Wed, Feb 28 - Mon, March 5 Return to Bogotá, initial analysis and report writing, initial debrief on findings to Senior Management Team 5 days Tues, March 6 Travel to US from Bogotá, Colombia 1 day (For US-based team members. Colombian-based team members would use this Travel day later, prior to the debriefs.) Wed, March 7 - Wed, March 21 Report writing; submit first draft to OTI by COB March 21, 2018 7-10 days Over approx 2 week period. 43 Dates* Task LOE Mon, April 2 - Friday April 13 Receive comments from OTI by COB March 30. (OTI has 7 days to review initial draft.) Submit final report by COB Friday, April 13. 5-7 days Over approx 2 week period. Mon, April 16 - Friday, April 20 Or could be later in April if necessary. Debriefs with OTI, MSI and USG interagency as determined by OTI. 2 days Plus 1 extra Travel Day for any Bogotá-based team members who may come back. (2nd travel day was included above). Late April Submit final translated report to OTI (ie Spanish copy if original version is in English or vice versa). TOTAL: 54-59 Work Days (including Colombian field travel) + 2-4 International Travel Days (Varying depending on whether the team member is DC or Colombia based) Total 56-63 total LOE including Travel. *All dates listed above are illustrative and subject to change. Social Impact should propose an updated timeline, taking into account actual international and local travel dates and division of roles and responsibilities/LOE of each team member. A six-day work week is approved during the fieldwork for this activity. TASKS and DELIVERABLES The evaluation team, under the direction of the Team Leader, is responsible for completing the following Tasks and submitting the following Deliverables: Tasks: • Refined or clarified evaluation questions (in collaboration with USAID/OTI) (English); • Evaluation Inception Report, to include the following: o Summary of proposed data collection methods and analysis; o Draft data collection tools (English); o Management framework, including divisions of roles and responsibilities among the evaluation team members and points of contact for liaising with OTI, and MSI (English); o Updated timeline with applicable LOE (English); o Initial interviewee list, activity sample, and site visit locations (to be developed in collaboration with USAID/OTI and MSI) (English or Spanish); o Utilization plan for evaluation findings, based on initial conversations with OTI Washington and OTI Colombia staff. • Materials and written summary of out-briefing to USAID/OTI and MSI before departing Colombia (Confirm with OTI on language of out-brief; will depend on participants, but likely in Spanish if includes some key MSI staff); 44 • Draft evaluation report for comments (content and structure), in English, due March 16, 2018; • Materials (handout or presentation) for and participation in Washington, DC briefing with USAID/OTI, MSI, USG interagency, and other stakeholders as determined by OTI (English); and Deliverable: • Final evaluation report, due March 30, 2018. Final report will be limited to 20-25 pages total, excluding annexes and the executive summary. (English, with Spanish translation) All deliverables will require USAID/OTI concurrence prior to their finalization. The final evaluation report should be in English and in Spanish. The report will be initially drafted in English, then translated into Spanish. Social Impact should consult with OTI about the preferred language of submission for the remaining deliverables and presentations, as noted above. The final evaluation report will be posted on USAID’s Development Experience Clearinghouse (DEC) website by Social Impact. Social Impact will be responsible for the following logistical matters with respect to the evaluation team members hired through this activity: • Social Impact is responsible for all scheduling, arranging meeting locations, and confirming times/dates/locations with the team and interviewees and grantees. OTI (including the Resource Person) and MSI will provide references, contact information, and can assist with scheduling as needed. While the evaluators are in country, OTI’s partner MSI will make available a conference room in its offices for the evaluators to use in Bogotá, Puerto Asís, Cúcuta and Tame. However, meetings in hotel lobbies and coffee shops are sometimes preferable and convenient. USAID/OTI will facilitate access to the U.S. Embassy for meetings with other USAID and Embassy officials, and arrange conference rooms in the Embassy. • Social Impact shall arrange and purchase all international and U.S. travel. TOCOR concurrence is required for all international travel not included in the original activity budget; • Social Impact shall provide per diem (lodging and M&IE) for the evaluation team in Colombia and in the US (based on each member’s travel status); • Social Impact shall fund in-country air travel and ground transportation; • Social Impact will obtain US visas for any Colombian evaluation members;17 • Social Impact and the evaluation team will work with OTI and MSI to arrange interviews in Washington, DC and in Colombia; and • Social Impact will cover the costs of translating the final report (or other materials) into Spanish or English as applicable. USAID/OTI will arrange international travel, provide per diem, secure lodging, fund in-country air travel, for the OTI Resource Person member of the evaluation team. Social Impact will, however, be responsible for including the OTI Resource Person when making ground transportation arrangements for the team while in Colombia. REPORT STRUCTURE The evaluation report (20-25 pages excluding annexes and the executive summary) should adhere to USAID guidelines and be structured as follows: • Cover Page with photo • List of Acronyms • Table of Contents, which identifies page numbers for the major content areas of the report. 17 OTI/Colombia is able to assist in arranging US visas from Bogotá, if required. 45 • Executive Summary (2 to 3 pages): should be a clear and concise stand-alone document that gives readers the essential content of the evaluation report, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. Thus, the Executive Summary should include: maximum of two paragraphs describing the program; summary of objectives and intended outcomes; summary of evaluation methodology and limitations, and key findings including those related to meaningful under- or over-achievement. • Methodology: Describe the data collection and analysis methods used including strengths and weaknesses, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken. This section should also address constraints and limitations of the evaluation process and rigor, including what can and cannot be concluded from the evaluation. All actual or potential conflicts of interests among members of the evaluation team should be noted in this section. In accordance with ADS 203, the report should also state why a performance evaluation was conducted in lieu of an impact evaluation. • Findings: The evaluation team should determine how best to organize this section based on the evaluation questions, in consultation with OTI. • Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to discuss the data and results, and make concrete recommendations for project improvements/changes, pull out organizational lessons learned (both positive and negative), and generally comment on data and results. • Annexes: List of stakeholders interviewed with number and type of interactions; interview data on evaluation questions, the scope of work; qualitative protocols developed and used; any data sets can be provided in electronic format; other special documentation needed. POINTS OF CONTACT The OTI/Colombia Program Manager in Washington will serve as the point of contact for overall coordination of the Washington meetings. There will be some occasions when the OTI Program Manager arranges the meeting and location and others where he/she may provide contact information for the consultants to arrange their own logistics. While in Colombia, the OTI team in Bogotá will identify a point of contact for in-country logistics and scheduling, likely the Acting Deputy Country Representative and relevant MSI program development officers. The OTI Resource Person will be the key liaison coordinating with and communicating between OTI, MSI and the Evaluation Team member. The TOCOR will remain the point of contact for all technical direction requiring formal concurrence. If a Colombian evaluation firm is subcontracted under this SOW, Social impact and the firm should identify a key point of contact at the firm for this evaluation to coordinate with and communicate with OTI and MSI. 46 ANNEX B. EVALUATION TEAM MEMBERS Team Leader: Mr. Luis A. Bernal, MPP is a senior international development and evaluation consultant with more than 30 years of experience designing, implementing, and evaluating private and public initiatives for: economic growth; local development; private sector competitiveness; entrepreneurship; micro, small, and medium-sized business promotion; poverty alleviation, and social inclusiveness. His extensive Monitoring & Evaluation background is comprised of meta-evaluations, performance and ex-post evaluations of multi-site, multi-million and multi-country projects intended to measure a program’s efficiency, efficacy, impact, outreach, and sustainability. Mr. Bernal has led and collaborated on regional projects in Latin America and the Caribbean, as well as projects in: Barbados, Bolivia, Chile, Colombia, Costa Rica, Ecuador, Guatemala, Honduras, Panama, Chile, Spain, Liberia, and the United States, working for IDB, USAID, UN-PBSO and United Nations Development Programme. Mr. Bernal earned his master’s degree in Public Policy from the University of Massachusetts, and a certificate in Evaluation and Applied Research Methods from Claremont University. He also holds a bachelor’s degree in Business Administration from the Universidad Externado de Colombia. Senior Policy Expert: Mr. Adam B. Isacson joined the Washington Office on Latin America in 2010 after 14 years working on Latin American and Caribbean security issues with the Center for International Policy. At WOLA, his Defense Oversight Program monitors security trends and U.S. military cooperation with the Western Hemisphere. Since the late 1990s, Mr. Isacson focused especially on Colombia, the principal destination of U.S. aid in the region. His study of U.S. policy and accompaniment of Colombia’s peace processes has brought him to Colombia about seventy times, including twenty-two of the country’s thirty-two departments. Mr. Isacson has published and co-written dozens of reports and articles, testified before Congress several times, and led several congressional delegations. He holds an MA in International Relations from Yale University. Before WOLA and CIP, he worked for the Arias Foundation for Peace and Human Progress in San José, Costa Rica. M&E Specialist: Maria V. Whittingham, PhD is a Colombian and Spanish citizen with a substantial and wide range of professional experience working with academic, non-profit, private and public institutions, in Latin America, the U.S. and Spain. Her work has ranged from engaging in policy design and implementation, to participating in community building and development interventions. She is an international consultant in evaluation at the Interamerican Development Bank. Currently lives in Colombia and works as researcher at the National University and the ESAP. Miss Whittingham earned a Ph.D. in Public and International Affairs from the University of Pittsburgh and an Advanced certificate in Latin American Social and Public Policy from the CLAS at the same University. Before that she studied at the Universidad de los Andes in Colombia where she graduated as Psychologist, and as Specialist in HR Management. She has published several articles and chapters in books and is member of The Latin American Centre for Development Administration (CLAD), and the Latin American and Caribbean professional network in Management by Results for Development (CoPLAC-GpRD). Additionally, Mr. Jeff Lieberman from OTI joined the team as a resource person and served as an in￾country advisor on OTI approaches and the project cycle. Jeff Lieberman is a Deputy Team Leader with the USAID Office of Transition Initiatives (OTI), currently covering the Europe & Asia Team and Pakistan Team. Jeff joined OTI in 2013 and previously worked 4 years with the Latin America and Caribbean Team, as well as short-term assignments with the Middle East Team and Africa Team. Prior to joining OTI, Jeff worked over 5 years based abroad and in Washington DC for Peacework, a non-profit that promotes international community development. Jeff received a Bachelor of Science degree in Business Management from Virginia Tech and a Master of Arts degree in International Development Studies from The George Washington University. Mr. Lieberman has provided background information to the ET and advised on evaluation design and data collection. 47 ANNEX C. DATA COLLECTION TOOLS MID-TERM EVALUATION COLOMBIA TRANSFORMA Interview Protocol 1. U.S. Government or Implementing Partner Informed Consent and Confidentiality: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, DC area. We are collecting data about the Colombia Transforma program that launched in 2015. As you may know, the program was designed to support a sustainable and inclusive Colombian peace. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. We selected you and other respondents to interview because we understand that you may have perspective on the Colombia Transforma program and/or on relevant subject matter. We expect the duration of this interview to be one hour. We plan to ask you about the peace process in Colombia, as well as program activities conducted by MSI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Colombia’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Do you confirm your consent to participate in this interview? ☐ Yes ☐ No To guarantee accuracy, we find it useful to keep an audio record of the conversation. If you prefer, however, we will not use recording devices. Do you confirm your consent for us to record this interview? ☐Yes ☐No Interview Place and Date: Interviewer(s): Name(s): Sex: ☐ Female ☐ Male Affiliation: Location: ☐ Arauca ☐ Bogotá ☐Norte de Santander ☐Putumayo Stakeholder Group: ☐OTI ☐MSI ☐Other U.S. government 48 EQ 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? EQ 1a: Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? 1. The Peace Accord has five chapters: rural development, political participation, transitional justice, coca substitution, and reintegration of ex-combatants. For which of these chapters is rapid response of implementation most urgent? Why? 2. Which of the Peace Accord chapters is most critical in terms of U.S. foreign policy interests? Why? 3. In which Peace Accord chapters is the Colombia Transforma program contributing to change? How so? 4. Are any changes to Colombia Transforma’s strategic approach needed to improve its relevance to the local context? If yes, please describe. 5. Are any changes to Colombia Transforma’s strategic approach needed to improve its relevance to U.S. policy? If yes, please describe. EQ 1b: If there are missed opportunities due to U.S. foreign policy constraints outside the program’s own manageable interests, what are those constraints? 6. Is there clarity about how the ban on “material support” for members of designated terrorist organizations applies to ex-combatants at all levels? Has this prohibition affected what the program can do in zones of historic FARC or ELN influence? If so, how? 7. Are there other constraints unrelated to “material support?” For instance, does widespread participation in coca cultivation constrain whom the program can work with? 8. Did these limits or constraints force the program to forgo opportunities? What are some examples? 9. How can the most critical opportunities be seized without violating the intention of the constraints? EQ 2. To what extent has Colombia Transforma achieved its program objectives? EQ 2a: Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 10. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, government agencies charged with building state presence in historically conflictive areas? 11. Has Colombia Transforma brought about any working relationships among Colombian government institutions? If yes, please describe. If no, why not? 12. The Peace Accord created many new agencies. Are these newer agencies working effectively with existing ones? How has Colombia Transforma contributed to foster such collaboration? 13. Has Colombia Transforma increased clarity about roles and responsibilities across implementing agencies? If yes, please describe. If no, why not? 14. Have rapid response abilities of the Colombian government to implement the Peace Accord increased due to contributions from Colombia Transforma? If yes, how so? 15. In your perception, will any increases in government capacity be sustained after the OTI “handoff?” Please describe. EQ 2b: Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? 16. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, subnational governments? 49 17. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, civil society groups and other territorial actors? 18. Was Colombia Transforma successful in encouraging territorial actors to work together and develop relationships? Why or why not? Please provide examples of successes and failures. 19. How did Colombia Transforma select the territorial actors that became awardees? Was there a mapping process? Are there sectors of civil society, or prominent local groups, that could not or would not work with Colombia Transforma? EQ 2c: What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? 20. Which Colombia Transforma elements and activities succeeded in increasing government capacity? Why? 21. Which Colombia Transforma elements and activities did not succeed in increasing government capacity? Why? 22. Which Colombia Transforma elements and activities succeeded in increasing local collaboration? Why? 23. Which Colombia Transforma elements and activities did not succeed in increasing local collaboration? Why? 24. Did activities yield unexpected outcomes, either positive and negative? 25. Has Colombia Transforma included gender-specific needs and priorities of both men and women in its programming and assessments? What results have such activities achieved? 26. Has Colombia Transforma included Afro-Colombian and indigenous groups’ needs and priorities in its programming and assessments? What results have such activities achieved? EQ 2d: To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 27. On a scale of 1 to 5, how would you rate the government’s presence in your region before Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 28. On a scale of 1 to 5, how would you rate the government’s presence in your region after Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 29. In your perception, did Colombia Transforma contribute to any increase in state presence? If yes, how so? 30. How has the implementation of the Peace Accord responded to your communities’ needs or priorities? EQ 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership 50 roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 31. Is there specific evidence that local actors better understand their roles and responsibilities in the Peace Accord implementation? If yes, did Colombia Transforma contribute to this understanding? 32. Are there examples of local actors taking on new community leadership roles? If yes, please describe and note how Colombia Transforma contributed to such involvement. 33. If no new local leadership roles have been developed to drive Accord implementation, what could be the reasons? 34. How have local stakeholders participated in the design and execution of Colombia Transforma activities that support Accord implementation? Wrap-Up 35. If Colombia Transforma were to be repeated, would you recommend doing anything differently? If yes, please elaborate. 36. Our task is to provide an accurate and comprehensive assessment of the Colombia Transforma program. Is there anything we did not ask about that is important for us to know? **** 51 MID-TERM EVALUATION COLOMBIA TRANSFORMA Interview Protocol 2. Colombian (National and Subnational) Government Informed Consent and Confidentiality: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, DC area. We are collecting data about the Colombia Transforma program that launched in 2015. As you may know, the program was designed to support a sustainable and inclusive Colombian peace. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. We selected you and other respondents to interview because we understand that you may have perspective on the Colombia Transforma program and/or on relevant subject matter. We expect the duration of this interview to be one hour. We plan to ask you about the peace process in Colombia, as well as program activities conducted by MSI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Colombia’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Do you confirm your consent to participate in this interview ? ⃝Yes ⃝No To guarantee accuracy, we find it useful to keep an audio record of the conversation. If you prefer, however, we will not use recording devices. Do you confirm your consent for us to record this interview? ⃝Yes ⃝No Interview Place and Date: Interviewer(s): Name(s): Sex: ⃝Female ⃝Male Affiliation: Location: ⃝Arauca ⃝Bogotá ⃝Norte de Santander ⃝Putumayo Stakeholder Group: ⃝National Gov. ⃝Departmental Gov. ⃝Municipal Gov. ⃝Ombudsman EQ 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? EQ 1a: Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? 1. The Peace Accord has five chapters: rural development, political participation, transitional justice, coca substitution, and reintegration of ex-combatants. For which of these chapters is rapid response of implementation most urgent? Why? 2. In which Peace Accord chapters is the Colombia Transforma program contributing to change? How so? 52 3. Are there urgent needs in the local context that aren’t included among the Colombia Transforma program strategy’s priorities? 4. What rapid response aspects need more focus to fulfil the Accord’s most urgent needs? EQ 2. To what extent has Colombia Transforma achieved its program objectives? EQ 2a: Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 5. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, government agencies charged with building state presence in historically conflictive areas? 6. Has Colombia Transforma brought about any working relationships among Colombian government institutions? If yes, please describe. If no, why not? 7. The Peace Accord created many new agencies. Are these newer agencies working effectively with existing ones? How has Colombia Transforma contributed to foster such collaboration? 8. Has Colombia Transforma increased clarity about roles and responsibilities across implementing agencies? If yes, please describe. If no, why not? 9. Have rapid response abilities of the Colombian government to implement the Peace Accord increased due to contributions from Colombia Transforma? If yes, how so? 10. In your perception, will any increases in government capacity be sustained after the program closes? Please describe. EQ 2b: Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? 11. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, subnational governments? 12. At the outset of Colombia Transforma, what were some of the chief weaknesses of, and obstacles faced by, civil society groups and other territorial actors? 13. Was Colombia Transforma successful in encouraging territorial actors to work together and develop relationships? Why or why not? Please provide examples of successes and failures. 14. Are there sectors of civil society, or prominent local groups, that could not or would not work with Colombia Transforma? 15. What are perceptions of / suggestions for needed changes to the strategic approach to improve local cooperation? EQ 2c: What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? 16. Which Colombia Transforma elements and activities succeeded in increasing government capacity? Why? 17. Which Colombia Transforma elements and activities did not succeed in increasing government capacity? Why? 18. Which Colombia Transforma elements and activities succeeded in increasing local collaboration? Why? 19. Which Colombia Transforma elements and activities did not succeed in increasing local collaboration? Why? 20. Did activities yield unexpected outcomes, either positive and negative? 53 21. Has Colombia Transforma included gender-specific needs and priorities of both men and women in its programming and assessments? What results have such activities achieved? 22. How has Colombia Transforma included Afro-Colombian and indigenous groups’ needs and priorities in the program’s implementation? How have such activities benefited these groups? EQ 2d: To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 23. On a scale of 1 to 5, how would you rate the government’s presence in your region before Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 24. On a scale of 1 to 5, how would you rate the government’s presence in your region after Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 25. In your perception, did Colombia Transforma contribute to any increase in state presence? If yes, how so? 26. Do you believe that the implementation of the Peace Accord reflect local communities needs and priorities? How so? EQ 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 27. Is there specific evidence that local actors better understand their roles and responsibilities in the Peace Accord implementation? If yes, did Colombia Transforma contribute to this understanding? 28. Are there examples of local actors taking on new community leadership roles? If yes, please describe and note how Colombia Transforma contributed to such involvement. 29. If no new local leadership roles have been developed to drive Accord implementation, what could be the reasons? 30. How have local stakeholders participated in the design and execution of Colombia Transforma activities that support Accord implementation? Wrap-Up 31. If Colombia Transforma were to be repeated, would you recommend doing anything differently? If yes, please elaborate. 54 32. Our task is to provide an accurate and comprehensive assessment of the Colombia Transforma program. Is there anything we did not ask about that is important for us to know? **** 55 MID-TERM EVALUATION COLOMBIA TRANSFORMA Interview Protocol 3. Local Beneficiaries (Community Groups, NGOs, Community Action Boards) Informed Consent and Confidentiality: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, DC area. We are collecting data about the Colombia Transforma program that launched in 2015. As you may know, the program was designed to support a sustainable and inclusive Colombian peace. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. We selected you and other respondents to interview because we understand that you may have perspective on the Colombia Transforma program and/or on relevant subject matter. We expect the duration of this interview to be one hour. We plan to ask you about the peace process in Colombia, as well as program activities conducted by MSI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Colombia’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Do you confirm your consent to participate in this interview? ⃝Yes ⃝No To guarantee accuracy, we find it useful to keep an audio record of the conversation. If you prefer, however, we will not use recording devices. Do you confirm your consent for us to record this interview? ⃝Yes ⃝No Interview Place and Date: Interviewer(s): Name(s): Sex: ⃝Female ⃝Male Affiliation: Activity Supported by Colombia Transforma: Location: ⃝Arauca ⃝Bogotá ⃝Norte de Santander ⃝Putumayo Stakeholder Group: ⃝Community Group ⃝Local NGO ⃝Community Action Board 56 EQ 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? EQ 1a: Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? 1. The Peace Accord has five chapters: rural development, political participation, transitional justice, coca substitution, and reintegration of ex-combatants. Which of these chapters do you believe is most urgent for implementation of the Peace Accord? Why? 2. How would you describe Colombia Transforma’s strategic approach? 3. Are any changes needed to Colombia Transforma’s strategic approach to improve its relevance to the local context? If yes, please describe. EQ 2. To what extent has Colombia Transforma achieved its program objectives? EQ 2a: Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 4. Have you perceived an increase in the government’s capacity and presence in the area? If yes, please describe. 5. Do you think Colombia Transforma contributed to an increase of local government abilities to execute aspects of the Peace Accord? If yes, please describe. 6. Has Colombia Transforma increased your communities’ understanding about roles and responsibilities of the government? The departmental government? The municipal government? 7. Are there improvements that could be made to local government capacity? If yes, please describe. EQ 2b: Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? 8. At the outset of Colombia Transforma, what, in your opinion, were some of the chief weaknesses of, and obstacles faced by civil society groups and other territorial actors? 9. Was Colombia Transforma successful in encouraging territorial actors to work together and develop relationships? Why or why not? Please provide examples of successes and failures. 10. In your opinion, could Colombia Transforma do anything better or differently to increase collaboration among local actors to strengthen implementation of the Peace Accord? EQ 2c: What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? 11. Please describe your involvement in the Colombia Transforma program. 12. What was the purpose of the activity? Please describe the results of the activity. 13. Did activities yield unexpected outcomes, either positive and negative? 14. Has Colombia Transforma included gender-specific needs and priorities of both men and women in its programming and assessments? What results have such activities achieved? 15. Has Colombia Transforma included Afro-Colombian and indigenous groups’ needs and priorities in its programming and assessments? What results have such activities achieved? 57 EQ 2d: To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 16. On a scale of 1 to 5, how would you rate the government’s presence in your region before Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 17. On a scale of 1 to 5, how would you rate the government’s presence in your region after Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 18. In your perception, did Colombia Transforma contribute to any increase in state presence? If yes, how so? 19. How has the implementation of the Peace Accord responded to your communities’ needs or priorities? EQ 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 20. What is your role in the implementation of the Peace Accord? 21. Did your involvement in Colombia Transforma change your understanding of your role? If yes, how so? 22. Do you identify local actors taking on new leadership roles related to the implementation of the Peace Accord? If yes, please describe and note how Colombia Transforma contributed to such involvement. 23. Have other local stakeholders that before didn´t interact found ways of collaboration via Colombia Transforma activities? If so, can you please share an example? Wrap-Up 24. If Colombia Transforma were to be repeated, would you recommend doing anything differently? If yes, please tell us. 25. Our task is to provide an accurate and comprehensive assessment of the Colombia Transforma program. Is there anything we did not ask about that is important for us to know? 58 MID-TERM EVALUATION COLOMBIA TRANSFORMA Interview Protocol 4. External Stakeholders Informed Consent and Confidentiality: Hi, my name is X, and I work for Social Impact, which is an independent research company based in the Washington, DC area. We are collecting data about the Colombia Transforma program that launched in 2015. As you may know, the program was designed to support a sustainable and inclusive Colombian peace. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. We selected you and other respondents to interview because we understand that you may have perspective on the Colombia Transforma program and/or on relevant subject matter. We expect the duration of this interview to be one hour. We plan to ask you about the peace process in Colombia, as well as program activities conducted by MSI, with funding from USAID. There are no known risks or direct benefits related to your participation; however, your inputs may lead to recommendations that benefit actors engaged in Colombia’s peace process—and, thereby, the general public. All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering, end this interview at any point, or withdraw your responses after the interview. Do you confirm your consent to participate in this interview? ⃝Yes ⃝No To guarantee accuracy, we find it useful to keep an audio record of the conversation. If you prefer, however, we will not use recording devices. Do you confirm your consent for us to record this interview? ⃝Yes ⃝No Interview Place and Date: Interviewer(s): Name(s): Sex: ⃝Female ⃝Male Affiliation: Location: ⃝Arauca ⃝Bogota ⃝Norte de Santander ⃝Putumayo Stakeholder Group: ⃝Intl. Org. ⃝Local NGO ⃝Political Org. ⃝Private firm ⃝Other _____________________ EQ 1. How relevant is OTI Colombia’s strategic approach to the local context and to U.S. foreign policy in Colombia? EQ 1a: Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? 59 1. The Peace Accord has five chapters: rural development, political participation, transitional justice, coca substitution, and reintegration of ex-combatants. For which of these is rapid response of implementation most urgent? 2. Do any rapid response aspects need increased focus to fulfill the Accord’s most urgent needs? 3. If you are familiar with Colombia Transforma, are there any local needs that should be better integrated into the program strategy? EQ 1b: If there are missed opportunities due to U.S. foreign policy constraints outside the program’s own manageable interests, what are those constraints? 4. Are you aware of any U.S. foreign policy interests that impede the implementation of the Peace Accord? If yes, could you please elaborate? EQ 2. To what extent has Colombia Transforma achieved its program objectives? EQ 2a: Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 5. Have you perceived an increase in the government’s capacity and presence in the area? If yes, please describe. 6. Have you perceived a decrease in the divide between subnational and national state entities? If yes, please describe. If no, why not? 7. Are there improvements that could be made to local government capacity? If yes, please describe. EQ 2b: Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? 8. At the outset of the peace process, what were some of the chief weaknesses of, and obstacles faced by, subnational governments? 9. What were some of the chief weaknesses of, and obstacles faced by civil society groups and other territorial actors? 10. Have you perceived any changes in the capacity of territorial actors to work together and develop relationships? Please provide examples of successes and failures. 11. What are your suggestions, if any, to improve local cooperation for the implementation of the Peace Accord? EQ 2d: To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? 12. On a scale of 1 to 5, how would you rate the government’s presence in your region before Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 60 13. On a scale of 1 to 5, how would you rate the government’s presence in your region after Colombia Transforma? None Poor Acceptable Good Very Good Don’t know 1 2 3 4 5 14. In your perception, did Colombia Transforma contribute to any increase in state presence? If yes, how so? 15. How has the implementation of the Peace Accord responded to your communities’ needs or priorities? EQ 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 16. What is your role in the implementation of the Peace Accord? 17. Do you identify local actors taking on new leadership roles related to the implementation of the Peace Accord? If you are aware, please describe and note how Colombia Transforma contributed to such involvement. 18. Have other local stakeholders that before didn´t interact found ways of collaboration via Colombia Transforma activities? If so, can you please share an example? Wrap-Up 19. If Colombia Transforma were to be repeated, would you recommend doing anything differently? If yes, please tell us. 20. Our task is to provide an accurate and comprehensive assessment of the Colombia Transforma program. Is there anything we did not ask about that is important for us to know? **** 61 MID-TERM EVALUATION COLOMBIA TRANSFORMA 5. Focus Group Discussion Protocol FGD Place and Date: Moderator: Sex: ⃝Female (Number: ___ ) ⃝Male (Number: ___ ) Location: ⃝Arauca ⃝Bogotá⃝Norte de Santander ⃝Putumayo Stakeholder Group: ⃝Community Group ⃝Local NGO ⃝Community Action Board Welcome, Informed Consent, and Confidentiality: Good morning, and welcome to our session. My name is X, and I work for Social Impact, which is an independent research company based in the Washington, DC area. We are collecting data about the Colombia Transforma program that launched in 2015. As you know, the program was designed to support a sustainable and inclusive Colombian peace. Our evaluation is intended to inform the program’s implementation and to guide the U.S. Government’s strategic decisions about future efforts in Colombia. Thank you for taking the time to join us to talk about activities conducted through the Colombia Transforma program, in which you all have participated as grantees. You may or may not know each other; this will be an opportunity to know more people involved in advancing their communities and building peace in Colombia. You were invited because we are interested in hearing your perspective about the Colombia Transforma program (past, present, and future). All information that you share will be kept confidential. We will aggregate and present our findings to USAID in a way that cannot be attributed to any individual or organization. Therefore, please feel free to speak openly and candidly with us. There are no wrong answers but rather differing points of view. Please feel free to share your point of view even if it differs from what others have said. Keep in mind that we’re just as interested in negative comments as positive comments, and at times the negative comments are the most helpful. You’ve probably noticed the voice recorder. We’re are recording the session because we don’t want to miss any of your comments. People often say very helpful things in these discussions and we can’t write fast enough to get them all down., and we won’t use any names in our reports. Please let me know if you have any problem with the recording or if you prefer that we don´t use it. Your participation is voluntary. Please feel free to ask to skip any question that you do not feel comfortable answering or withdraw from the discussion at any point. Rules: 1. We would appreciate if you please turn off your cellphones. 2. The person who has the “microphone” (empowering object) is the only one that talks; if anyone wants to intervene, please raise your hand and I will keep track of the order for the interventions. EQ 2. To what extent has Colombia Transforma achieved its program objectives? 62 EQ 2a: Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid response aspects of the Peace Accord? 1. Have you perceived an increase in the government’s capacity and presence in the area? If yes, please describe. 2. Do you think Colombia Transforma contributed to an increase of local government abilities to execute aspects of the Peace Accord? If yes, please describe. 3. Are there improvements that could be made to local government capacity? If yes, please describe. EQ 2b: Program Objective II: How effectively has Colombia Transforma promoted collaboration among territorial actors to lead local implementation of the Peace Accord? 4. Was Colombia Transforma successful in encouraging territorial actors to work together and develop relationships? Why or why not? Please provide examples of successes and failures. 5. In your opinion, could Colombia Transforma do anything better or differently to increase collaboration among local actors to strengthen implementation of the Peace Accord? 6. Do you think Colombia Transforma engaged the appropriate actors? If no, why not? EQ 2c: What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? 7. Which activities were most effective in promoting collaboration among territorial actors? Why? 8. Did activities yield unexpected outcomes, either positive or negative? 9. Has Colombia Transforma included gender-specific needs and priorities of both men and women in its programming and assessments? What results have such activities achieved? 10. Has Colombia Transforma included Afro-Colombian and indigenous groups’ needs and priorities in its programming and assessments? What results have such activities achieved? EQ 2d: To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 11. In your perception, did Colombia Transforma contribute to any increase in state presence? If yes, how so? 12. How has the implementation of the Peace Accord responded to your communities’ needs or priorities? EQ 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation 63 in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? 13. What is the role of local communities in the implementation of the Peace Accord? 14. Did your involvement in Colombia Transforma change your understanding of your role? If yes, how so? 15. Do you identify local actors taking on new leadership roles related to the implementation of the Peace Accord? If yes, please describe and note how Colombia Transforma contributed to such involvement. 16. Have other local stakeholders that before didn´t interact found ways of collaboration via Colombia Transforma activities? If so, can you please share an example? Wrap-up 17. Suppose that you were in charge and could improve the Colombia Transforma program. What would you do differently? 18. Is there anything that anyone would like to share on the topics covered in this meeting? Are we missing something important? **** 64 MID-TERM EVALUATION COLOMBIA TRANSFORMA 6. Direct Observation Protocol - Training Observation Place and Date: Observation Site: Observer: Number of Participants: Purpose: The purpose of this observation is to (1) verify whether the training activity contributes to the objectives of Colombia Transforma, as well as (2) verify the community’s involvement in the activity and the interactions of different stakeholders. Instructions: Please consider each item carefully before assigning a score. Unsatisfactory Needs improvement Acceptable Satisfactory Very satisfactory Not Applicable 1 2 3 4 5 n/a _____ 1. Relevance of the topic presented (aligned with OTI program objectives, aligned with local needs) _____ 2. Number of people attending the training _____ 3. Representation of diverse stakeholders attending _____ 4. Presence and participation of women _____ 5. Presence and participation of Afro-Colombian and indigenous persons _____ 6. Participant engagement in the activity _____ 7. Development of follow-up activities _____ 8. Overall grade 9. What are your first perceptions of the process observed? 10. What specific suggestions would you make concerning this activity? 11. Additional notes/comments: **** 65 MID-TERM EVALUATION COLOMBIA TRANSFORMA 7. Direct Observation Protocol - Infrastructure Observation Place and Date: Observation Site: Observer: Number of Participants: Purpose: The purpose of this observation is to (1) verify whether the facility contributes to the objectives of Colombia Transforma, (2) assess the community’s perception of the facility, and (3) verify community ownership of the facility and interactions regarding the facility. Instructions: Please consider each item carefully before assigning a score. Unsatisfactory Needs improvement Acceptable Satisfactory Very satisfactory Not Applicable 1 2 3 4 5 n/a _____ 1. Facility status (clean, organized, etc.) _____ 2. Number of people using the facility _____ 3. Contribution of the facility to solve a community problem _____ 4. Community engagement in construction (volunteer labor, fundraising, etc.) and management/care of the facility _____ 5. Involvement of women in designing, using, maintaining the facility _____ 6. Involvement of Afro-Colombian and indigenous persons in designing, using, maintaining the facility _____ 7. Community ownership of the facility _____ 8. Employment created (if applies) _____ 9. Overall grade 10. What are your first perceptions of the outcomes observed? 11. What specific suggestions would you make concerning this infrastructure project? 12. Additional notes/comments: 66 ANNEX D. EVALUATION DESIGN MATRIX LINES OF INQUIRY DATA COLLECTION METHODS & DATA SOURCES ANALYSIS METHODS 1a. Is the program focused on the rapid response aspects of the Peace Accord that are the most critical and will make or break the long-term success of the Accord? Document Review KIIs • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • Municipal and Departmental Governments Mayors’ offices, governors’ offices, ombudsmen • National Government Presidency agencies, line ministries, agencies created to implement accords • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context • U.S. Government or Implementing Partner OTI and MSI staff • Other US embassy (USAID, political section, economic section, Security Cooperation Office (SCO), U.S. Department of State Bureau of International Narcotics and Law Enforcement Affairs [INL]) • Content analysis • Trend analysis • Comparative analysis 1b. If there are missed opportunities due to U.S. foreign policy constraints outside the program’s own manageable interests, what are those constraints? Document Review KIIs • U.S. Government or Implementing Partner OTI and MSI staff • Other US embassy (USAID, political section, economic section, SCO, INL) • External Stakeholders • Content analysis • Trend analysis • Comparative analysis 2a. Program Objective I: How effectively has Colombia Transforma increased the Colombian government’s ability to plan and execute rapid Document Review Direct Observation • Infrastructure projects in three regions KIIs • Content analysis • Trend analysis • Comparative analysis 67 LINES OF INQUIRY DATA COLLECTION METHODS & DATA SOURCES ANALYSIS METHODS response aspects of the Peace Accord? • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • Municipal and Departmental Governments Mayors’ offices, governors’ offices, ombudsmen • National Government Presidency agencies, line ministries, agencies created to implement accords • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context • U.S. Government or Implementing Partner OTI and MSI staff • Other US embassy (USAID, political section, economic section, SCO, INL) FGDs • Two FGDs with grantees in each of three regions • One FGD with key OTI and MSI staff to reflect on program implementation, challenges, and successes 2b. Program Objective II: How effectively has Colombia Transforma promoted collaboration between territorial actors to lead local implementation of the Peace Accord? Document Review Direct Observation • Training, pending availability, in three regions KIIs • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • Municipal and Departmental Governments Mayors’ offices, governors’ offices, ombudsmen • National Government • Content analysis • Trend analysis • Comparative analysis • Gender analysis 68 LINES OF INQUIRY DATA COLLECTION METHODS & DATA SOURCES ANALYSIS METHODS Presidency agencies, line ministries, agencies created to implement accords • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context • U.S. Government or Implementing Partner OTI and MSI staff • Other US embassy (USAID, political section, economic section, SCO, INL) FGDs • Two FGDs with grantees in each of three regions • One FGD with key OTI and MSI staff to reflect on program implementation, challenges, and successes 2c. What activities (or strategic approaches) were the most effective at achieving each program objective, and why? What were the least effective, and why? Document Review Direct Observation • Infrastructure projects in three regions • Training, pending availability, in three regions KIIs • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • Municipal and Departmental Governments Mayors’ offices, governors’ offices, ombudsmen • National Government Presidency agencies, line ministries, agencies created to implement accords • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context • U.S. Government or Implementing Partner • Content analysis • Trend analysis • Comparative analysis • Gender analysis 69 LINES OF INQUIRY DATA COLLECTION METHODS & DATA SOURCES ANALYSIS METHODS OTI and MSI staff • Other US embassy (USAID, political section, economic section, SCO, INL) FGDs • Two FGDs with grantees in each of three regions • One FGD with key OTI and MSI staff to reflect on program implementation, challenges, and successes 2d. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories perceive that national and local government presence has increased as a result of the Peace Accord? If they perceive that presence has not increased, what are the reasons for that perception? Do they perceive that Accord implementation reflects their communities’ priorities? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? Document Review Direct Observation • Infrastructure projects in three regions • Training and/or government service fairs, pending availability, in three regions KIIs • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context FGDs • Two FGDs with grantees in each of three regions • Content analysis • Trend analysis • Comparative analysis • Gender analysis 2e. To what extent do Colombia Transforma’s local partners and beneficiaries in the territories better understand their roles and responsibilities for Accord implementation? To what extent, if any, are they proactively assuming leadership roles to ensure that Accord implementation reflects their communities’ priorities? If they aren’t Document Review Direct Observation • Training, pending availability, in three regions KIIs • Local Beneficiaries: Community Action Boards, Community Groups, and Local NGOs Afro-Colombian groups, Campesino / producers’ organizations, Church, Community radio, Educators and youth groups, Indigenous groups, Business sector, NGOs/service providers, Women’s groups • Municipal and Departmental Governments • Content analysis • Trend analysis • Comparative analysis • Gender analysis 70 LINES OF INQUIRY DATA COLLECTION METHODS & DATA SOURCES ANALYSIS METHODS assuming leadership roles, what are the reasons? Do they feel they have participated in Accord implementation in meaningful ways? Do local partners and beneficiaries believe Colombia Transforma contributed to these outcomes? Mayors’ offices, governors’ offices, ombudsmen • External Stakeholders Participants in coca substitution, Human rights defenders and victims’ associations, Platforms of campesino groups, Labor unions, Venezuelan migrants, Professional service providers (medical, education, etc.), Judicial branch officials, Think-tanks and journalists with knowledge of security and local context • U.S. Government or Implementing Partner OTI and MSI staff FGDs • Two FGDs with grantees in each of three regions • One FGD with key OTI and MSI staff to reflect on program implementation, challenges, and successes 71 ANNEX E. INTERVIEWEE LIST Participants in Interviews or Focus Group Discussions Gender ORGANIZATION POSITION WASHINGTON DC Male USAID-OTI Team Leader for Latin America and the Caribbean (LAC) and Southern/Eastern Africa Male OTI Director Male OTI Deputy Director Female OTI Deputy Chief of OTI’s Field Programs Division Female OTI Chief of OTI’s Program, Learning, and Innovation Division Male USAID-LAC Bureau Acting Deputy Assistant Administrator Female USAID-LAC Bureau Acting Director, South America Office Male USAID-LAC Bureau Colombia Desk Officer Male USAID-LAC Bureau Desk Officer Male US Department of State Foreign Affairs Officer Bureau of Conflict and Stabilization Operations Female US Department of State Colombia Desk Officer Female US Department of State Deputy Director Office of Andean Affairs Female US Department of State Colombia Desk Officer BOGOTA DC Male OTI COL OTI Country Representative Male MSI Key Program Staff Chief of Party Female MSI Key Program Staff Deputy Chief of Party Male MSI Key Program Staff Senior Technical Advisor Female MSI Key Program Staff Field Programs Manager Male MSI Key Program Staff M&E Manager Female MSI Key Program Staff Office Director Female MSI Key Program Staff Grants Manager Male MSI Key Program Staff Procurement Manager Female MSI Key Program Staff Infrastructure Manager/Engineer Male MSI PDO National Grant Team Male Vicepresidencia de la República Director Política Integral para la Lucha contra Drogas Ilícitas Male Oficina Alto Comisionado para la Paz (OACP) Director Temático Female Dirección de Sustitución de Cultivos Ilícitos Male Federación Nacional de Personeros (FENALPER) Director Male Ministerio de Justicia y el Derecho Asesor Asuntos Internacionales. Female Asesora Justicia Transicional Male Agencia Nacional de Tierras Director Acceso Tierras 72 Participants in Interviews or Focus Group Discussions Gender ORGANIZATION POSITION Male FONADE Director Ordenamiento Territorial Female FONADE Jefe Grupo Proyectos Especiales Male AGENCIA DE RENOVACIÓN TERRITORIAL-ART Subdirector Proyectos Infraestructura Comunitaria Female Subdirectora Participación y Planeación Female Subdirectora Coordinación Fortalecimiento Institucional Female Coordinadora Cooperación Internacional Female Fundación Ideas para la Paz Asesora Female Universidad Sergio Arboleda Asesora Male Oportunidad Estratégica Director Male CONSUCOL Director Male Consultor Política de Drogas Male Consultor Encargado de Asuntos Afro Female Consultora Encargada de asuntos étnicos Male Consultor Desarrollo Agrario - Tierras Female Consultora Participación Comunitaria-PDET Female Agencia Presidencia de Cooperación Coordinadora COL-COL PUTUMAYO Male Red Cantoyaco Team Leader member Male Red Cantoyaco Team Leader member Female Colegio Cohembi Rectora Female Cimarron Putumayo Team Leader member Female ASMUN-Ruta Pacifico Team Leader member Male MSI-Regional Team LPA Male MSI-Regional Team LPA Male MSI-Regional Team FO PIC Female MSI-Regional Team PDO Male MSI-Regional Team Infraestructura Male MSI-Regional Team IO Male MSI-Regional Team Ing. Male MSI-Regional Team GA Male MSI-Regional Team MES Female MSI-Regional Team GS Female MSI-Regional Team AA Male Alcaldia de Puerto Asis Alcalde de Puerto Asis Female Gestor Municipal de la Agencia Renovacion de Tierras ART Male Unidad para la Atención y Reparación Integral a las Víctimas PU Director terrtitorial 73 Participants in Interviews or Focus Group Discussions Gender ORGANIZATION POSITION Male Instituto Misional de Antropologia Director Male INDERCULTURA￾PUTUMAYO Male Agencia de Renovación de Territorio – ART Director Regional Putumayo Female Alianza de Mujeres Tejedoras de Vida Member Female Alianza de Mujeres Tejedoras de Vida Coodinadora Female Alianza de Mujeres Tejedoras de Vida Presidente Female Alianza de Mujeres Tejedoras de Vida Legal representative Male Alcaldia de Puerto Leguizamo Alcalde de Puerto Leguizamon Male Organización Zonal Indígena del Putumayo Vicepresidente Male Afroleguízamo Presidente Female Gobernacion de Putumayo Gobernadora NORTE DE SANTANDER Male Alcaldia de Santander Secretario de Víctimas, Paz y Posconficto Male Unidad para la Atención y Reparación Integral a las Víctimas Director Territorial Norte de Santander y Arauca Male MSI-Regional Team PDO Female MSI-Regional Team Oficial de campo Male MSI-Regional Team Oficial de Campo Female MSI-Regional Team Asistenten de logistica y compras Female MSI-Regional Team Oficial de campo PIC-Tibú Male MSI-Regional Team Oficial de campo Female MSI-Regional Team Asistenten de logistica y compras Female MSI-Regional Team Especialista subvenciones Male MSI-Regional Team Especiaalista M&E Male Agencia de Desarrollo Rural – ADR Profesional de Apoyo Female Agencia de Renovación de Territorio – ART Coordinadora Regional Catatumbo ART Male Técnico ART Male ASOJUNTAS La Gabarra Male Presidente Junta Km 60 Male Presidente Junta Vereda La Angalia Male Presidente Junta Vereda San Antonio Male Presidente Asojuntas Pacheli Female Resguardo Catalaura Governadora Resguardo Catalaura 74 Participants in Interviews or Focus Group Discussions Gender ORGANIZATION POSITION Male quinta con quinta Fundador Female U. Fco. de Paula Santander Docente/investigadora Male Pastora Social Tibu Subdirecto de Pastoral Male Pastoral Victimas Delegado Male MSI Regional Team Oficial de campo Female MSI Regional Team Asistenten de logistica y compras Female MSI Regional Team Asisitente Administrativo Male MSI Regional Team Oficial de campo Male MSI Regional Team Oficial de campo Male Junta de Accion Comunal corregimiento San Pablo Presidente Male Junta de Accion Comunal Municipio de Teorama Presidente Male Pastoral Social Ocaña Director de Pastoral Social Male Asomunicipios del Catatumbo Director Ejecutivo Male Asopersoneros Representante Legal Male Alcaldía de Teorama Alcalde Male Alcaldía de San Calixto Alcalde Male Alcaldía de Ábrego Alcalde Male Alcaldía de Convención Alcalde Male Alcaldía El Tarra Alcalde ARAUCA Female Alcaldia Mayor Secretaria de Gobierno y Convivencia Ciudadana Male Secretario de Infraestructura Male Secretario de Educación Male Asojuntas Tame Presidente Male Asociacion de Negritudes Tame Representante Legal Male Consultor Female MSI-Regional Team Oficial de Campo Male MSI-Regional Team Oficial de Campo Female MSI-Regional Team LPA Male MSI-Regional Team Oficial de Campo Female MSI-Regional Team Asisitente de Subvenciones Male MSI-Regional Team Oficial de Campo Female MSI-Regional Team Oficial de Campo Male MSI-Regional Team Oficial de Campo Female MSI-Regional Team Asistente Administrativa Female MSI-Regional Team PDO Female MSI-Regional Team Asistente de Subvenciones Female MSI-Regional Team Oficial de Campo 75 Participants in Interviews or Focus Group Discussions Gender ORGANIZATION POSITION Male MSI-Regional Team Male MSI-Regional Team Male Alcaldia Alcalde Municipio de Saravena Male Alcadia Personero TAME Male Alcadia Secretario de Desarrollo Social Female Alcadia Secretaria de Transito Male Alcaldia Personero Saravena Male Unidad para la Atención y Reparación Integral a las Víctimas Enlace de prevención y atención de emergencias Female Asociacion cultura Cultural letras y voces-Fortulee Presidente Male Asociacion cultura Cultural letras y voces-Fortulee Gerente Male Emisora Radio Antáres del Oriente Fundador y director Male Asojuntas Saravena Representante Female Alcaldia Secretaria de Gobierno Male Alcaldia Secretario de Desarrollo Agropecuario sostenible Female Alcaldia Coordinadora Casa de la Mujer Female Alcaldia Secretaria de Agricultura, Ganadería y Medio Ambiente Male Defensoría del Pueblo Regional Defensor del Pueblo Regional Arauca Male Asociación de ganaderos de panamá de ARAUCA Vicepresidente Male Asociación de productores agropecuarios sector Oasis￾APAMASO Presidente Female Agencia del Renovación del Territorio-ART Gerente PDET Male Coordinador Regional Male Alcaldía de Arauquita Alcalde Male Alcaldía de Arauquita Secretario de Gobierno Male Pastoral Social Arauca Director Regional Arauca Female Asociación de Mujeres Araucanas Trabajadoras Representante Legal 76 ANNEX F. DOCUMENT REVIEW SOURCES 1) OTI Activity Database 2) Start-up Country Program Assessment (Feb 2015) 3) Go Memo (March 2015) 4) Program Extension Memo (April 2017) 5) Program Performance Review Report (March 2017) 6) Strategic Framework - With Regional Cluster Level Results and Regional Approaches 7) Internal Cluster Evaluation August 2017 8) USAID/Colombia Mission – Country Development Coordination Strategy (CDCS) 2014-2018. 9) Colombia Transforma Annual Reports 10) Colombia Transforma Bi-Weekly Reports 11) Colombia Transforma - Organizational Charts/Staffing Structure 12) Transforma Activity Network Visual 13) Analysis de Redes Sociales / Organizational Network Analysis (ONA) 14) Program Level M&E Plan – v1 Jan 2016 and documents describing regional approaches (clusters) 15) Rolling Assessment - Documents/Presentations. 16) Small Community Infrastructure (PICs) activities implemented with ART. 17) Norte de Santantander - Modelo de Trabajo Comunitario (Dec 2017) 18) OTI Colombia Outreach materials (fact sheets, infographics, maps) 19) General Colombia Transforma program documents, notes, materials 20) General OTI Outreach and Training Materials 21) Articles/Blogs/Think Tanks/Congressional Testimony a) Report of the Secretary-General on the United Nations Verification Mission in Colombia (New York: United Nations Security Council, December 27, 2017): 7 . b) Adapting U.S. Counternarcotics Efforts in Colombia (Washington: U.S. Senate Caucus on International Narcotics Control, September 12, 2017) . c) Adam Isacson, Consolidating ‘Consolidation’: Colombia’s Plan to Govern Neglected Territories Stumbles (Washington: Washington Office on Latin America, January 2013) . d) Carmen Fishwick, Emma Graham-Harrison, John Otis, “Colombia and Brazil Clamp Down on Borders as Venezuela Crisis Spurs Exodus” (London: The Guardian, February 9, 2018) . e) Christine Armario, “Sick Venezuelans Flee to Colombia in Mounting Refugee Crisis” (Bogotá: Associated Press, ABC News, March 8, 2018) . f) Caroline Hartzell, Robert D. Lamb, Phillip McLean, and Johanna Mendelson Forman, Maria Cristina Olano, USAID/OTI’s Integrated Governance Response Program in Colombia (Washington: USAID / CSIS, April 2011) . 77 g) Piedra en el Zapato (Bogotá: Somos Defensores, 2018): 81 . 78 ANNEX G. DISCLOSURE OF CONFLICT OF INTEREST 79 80 81 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523