October 2016 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Jairo Núñez, Daniel Wiesner, Keri Culver, Roger Rasnake, Centro Nacional de Consultoría and Management Systems International. EVALUATION Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative General Report Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative GENERAL REPORT Management Systems International A Tetra Tech Company 200 12th Street South Suite 1200 Arlington, VA 22202, USA www.msiworldwide.com October 2016 Contracted under AID-514-C-13-00003 USAID/Colombia Evaluation and Analysis for Learning (EVAL) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Acronyms ...................................................................................................................................................................... 1 Abstract.......................................................................................................................................................................... 3 Executive summary ....................................................................................................................................................... 5 Introduction............................................................................................................................................................... 5 Economic development............................................................................................................................................. 7 Social development .................................................................................................................................................13 Governance and institutions....................................................................................................................................17 Security ...................................................................................................................................................................20 Recommendations...................................................................................................................................................24 Introduction ................................................................................................................................................................. 25 Evaluation purpose & evaluation questions.................................................................................................................29 Evaluation rationale.................................................................................................................................................29 Evaluation questions ...............................................................................................................................................30 Evaluation design ........................................................................................................................................................32 How to read this report............................................................................................................................................32 Methodology and design .........................................................................................................................................33 Quantitative methods..........................................................................................................................................35 Qualitative methods............................................................................................................................................40 Limitations ..............................................................................................................................................................41 Economic development ...............................................................................................................................................46 Findings and conclusions........................................................................................................................................46 Direct CELI beneficiaries...................................................................................................................................46 Rural economic opportunities: View from the grassroots...................................................................................49 Assistance for productive projects...................................................................................................................... 55 Markets and trade................................................................................................................................................57 Credit, savings and financial services.................................................................................................................60 Transportation infrastructure...............................................................................................................................63 Land ....................................................................................................................................................................65 Recommendations...................................................................................................................................................68 Social development......................................................................................................................................................71 Findings and conclusions........................................................................................................................................73 The Index of Organizational Competencies (ICO) .............................................................................................73 Social capital: Trust ............................................................................................................................................76 Social capital: Participation in associations........................................................................................................81 The particular role of the Juntas de Acción Comunal (JACs).............................................................................83 Associations and productive projects..................................................................................................................86 Women’s and victims’ associations....................................................................................................................90 Cultural and other associations...........................................................................................................................91 Recommendations...................................................................................................................................................93 Governance and institutions ........................................................................................................................................96 Findings and conclusions........................................................................................................................................97 Participation, transparency, accountability and trust ..........................................................................................97 Service delivery ................................................................................................................................................102 Recommendations.................................................................................................................................................107 Security......................................................................................................................................................................110 Findings and conclusions......................................................................................................................................111 Synthetic index of insecurity ............................................................................................................................111 Coca cultivation ................................................................................................................................................113 Terrorism and high-impact crimes....................................................................................................................115 Other crimes......................................................................................................................................................117 Security- and justice-related capacities.............................................................................................................119 Military capacities.............................................................................................................................................119 Police presence .................................................................................................................................................120 Availability of justice services..........................................................................................................................121 Recommendations.................................................................................................................................................122 TABLE OF FIGURES Figure 1. Impacts on perceptions of the economic situation, direct beneficiaries .........................................................7 Figure 2. Impacts on reasons they associate with others, direct beneficiaries...............................................................8 Figure 3. Impacts on land issues, direct beneficiaries versus all CELI..........................................................................8 Figure 4. CELI performance, increasing sales...............................................................................................................9 Figure 5. Trends in economic situation perceptions, CELI municipalities..................................................................10 Figure 6. Impacts on perceptions of problems in production and sales, CELI and PNCRT........................................11 Figure 7. Trends in perceptions of problems in production and sales, CELI and PNCRT ..........................................11 Figure 8. CELI Performance: Average improvements by category, CELI N/S...........................................................14 Figure 9. Impacts on the Social Capital Index, CELI and PNCRT..............................................................................14 Figure 10. Impacts on participation in community activities, CELI and PNCRT .......................................................15 Figure 11. Impacts on participation in producer associations, CELI and PNCRT ......................................................16 Figure 12. CELI Performance: Accountability Index, by region.................................................................................17 Figure 13. Secondary data: Integral Performance Index, 2012 (left) and 2014 (right)................................................19 Figure 14. CELI Performance: Governance Capacity Index, total and regional .........................................................19 Figure 15. Secondary data: Insecurity Index ...............................................................................................................21 Figure 16. Trends in perceptions about illicit crops in their communities, CELI........................................................22 Figure 17. Secondary data: Average number of attacks by guerrilla groups, per municipality...................................23 Figure 18. Secondary data: Police presence ................................................................................................................24 Figure 1.1. The three pillars of the PNCRT and their CELI peers...............................................................................26 Figure 1.2. Evaluation questions ................................................................................................................................. 30 Figure 2.1. How to read and interpret the statistical data in the report........................................................................32 Figure 2.2. Sample by sex of respondent.....................................................................................................................33 Figure 2.3. Sample by ethnicity...................................................................................................................................34 Figure 2.4. Sample by poverty proxies........................................................................................................................34 Figure 2.5. Veredas by security threat level ................................................................................................................35 Figure 2.6. Consolidation Index pillars and their components ....................................................................................37 Figure 2.7. Secondary data: Consolidation Index overall scores.................................................................................38 Figure 2.8. Secondary data: Consolidation Index scores for territorial institutionalization.........................................39 Figure 2.9. Secondary data: Consolidation Index scores for good government, citizen participation.........................39 Figure 2.10. Secondary data: Consolidation Index scores for regional integration .....................................................39 Figure 3.1. Impacts on perceptions of the economic situation, direct beneficiaries ....................................................47 Figure 3.2. CELI Performance: Increasing sales.........................................................................................................47 Figure 3.3. Impacts on reasons they associate with others, direct beneficiaries..........................................................48 Figure 3.4. Impacts on credit and financial services, direct beneficiaries versus all CELI..........................................48 Figure 3.5. Impacts on land issues, direct beneficiaries versus all CELI.....................................................................49 Figure 3.6. Trends in economic situation perceptions, CELI municipalities...............................................................50 Figure 3.7. Impacts on perceptions of economic and food security variables, CELI and PNCRT..............................51 Figure 3.8. Impacts on perceptions of serious economic problems, CELI and PNCRT..............................................51 Figure 3.9. Secondary data: Consolidation Index on regional integration, trends 2011-2014.....................................52 Figure 3.10. Trends in reported monthly household expenditure, CELI and PNCRT.................................................54 Figure 3.11. CELI Performance: Estimated monthly household expenditure, total and regional................................54 Figure 3.12. Impacts on perceptions of problems in production and sales, CELI and PNCRT...................................58 Figure 3.13. Trends in perceptions of problems in production and sales, CELI and PNCRT .....................................58 Figure 3.14. CELI Performance: New use of financial services..................................................................................61 Figure 4.1. Secondary data: Consolidation Index scores for social development........................................................72 Figure 4.2. CELI Performance: Average change in ICO score, CELI Central ............................................................74 Figure 4.3. CELI Performance: Average change in ICO-VEO scores, CELI Norte/Sur.............................................74 Figure 4.4. CELI Performance: Average changes by ICO-VEO category, CELI N/S ................................................75 Figure 4.5. CELI Performance: Social Capital Index, total and regional ....................................................................76 Figure 4.6. Impacts on the Social Capital Index, CELI and PNCRT...........................................................................77 Figure 4.7. Impacts on trust between peers, CELI and PNCRT ..................................................................................77 Figure 4.8. Impacts on opinions of the personería; Trends by percent change ...........................................................78 Figure 4.9. Impacts on perceptions of the police and army, CELI and PNCRT..........................................................79 Figure 4.10. Impacts on resolving conflicts in the community, CELI and PNCRT ....................................................80 Figure 4.11. Impacts on perceptions of mayor’s office performance, CELI and PNCRT...........................................80 Figure 4.12. Impacts on participation in community activities, CELI and PNCRT ....................................................82 Figure 4.13. Impacts on participation in the JACs, CELI and PNCRT .......................................................................85 Figure 4.14. Impacts on participation in producer associations, CELI and PNCRT ...................................................87 Figure 4.15. Impacts on reasons they associate, CELI and PNCRT............................................................................88 Figure 4.16. Impacts on participation in victims’ and women’s groups, CELI and PNCRT.......................................91 Figure 5.1. Impacts on the Participation Index, CELI and PNCRT.............................................................................97 Figure 5.2. CELI Performance: Accountability index, total and regional ...................................................................99 Figure 5.3. Impacts on perceptions of problems with municipal administration, CELI and PNCRT........................100 Figure 5.4. Impacts on citizens’ perceptions of public management, CELI and PNCRT..........................................101 Figure 5.5. Secondary data: Integral Performance Index for the four groups of municipalities................................103 Figure 5.6. Secondary data: Integral Performance Index, 2012 (left) and 2014 (right).............................................104 Figure 5.7. CELI Performance: Governance capacity index, total and regional........................................................105 Figure 5.8. Impacts on perceptions of social services, CELI and PNCRT ................................................................106 Figure 6.1. Secondary data: Insecurity Index ............................................................................................................112 Figure 6.2. Secondary data: Evolution of coca cultivation........................................................................................113 Figure 6.3. Trends in reports that there are ICs in the village, CELI and PNCRT ....................................................114 Figure 6.4. Trends in perceptions about illicit crops in their communities, CELI.....................................................115 Figure 6.5. Secondary data: Average number of attacks by guerrilla groups, per municipality................................116 Figure 6.6. Trends in proportion perceiving security problems, CELI......................................................................117 Figure 6.7. Trends in reports of illegal mining, CELI and PNCRT...........................................................................118 Figure 6.8. Secondary data: Police presence .............................................................................................................121 1 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative ACRONYMS ARD Associates for Rural Development BACRIM Criminal bands, Bandas Criminales (in the service of the narcotics trade) CAR Regional Autonomous Corporations, Corporaciones Regionales Autónomas CDCS Country Development Cooperation Strategy CELI Consolidation and Enhanced Livelihood Initiative CELI Central CELI program in the Central region CELI MdM CELI program in the Montes de María region CELI Norte/Sur CELI program in the North and South regions CERAC Conflict Analysis Resource Center, Centro de Recursos para el Análisis de Conflictos CNC National Consulting Center, Centro Nacional de Consultoría CNC-DoD Crime and Narcotics Center, U.S. Department of Defense CSDI Colombia Strategic Development Initiative CSO Civil Society Organization DNP National Planning Department, Departamento Nacional de Planeación DO Development objective DoD Department of Defense (U.S.) DPS Department for Social Prosperity, Departamento para la Prosperidad Social ELN National Liberation Army, Ejército de Liberación Nacional EVIDINCE Research effort associated with the Empirical Studies of Conflict (esoc.princeton.edu), a multi-institutional collaboration studying conflict and insurgency at the sub-national level FARC Revolutionary Armed Forces of Colombia, Fuerzas Armadas Revolucionarias de Colombia FHH Female head of household GLAC Local savings and loans groups, grupos locales de ahorros y crédito GOC Government of Colombia HH Household IAG Illegal Armed Group IC Illicit crops ICO Index of Organizational Competencies, Índice de competencias organizacionales IDI Integral Performance Index, Índice de desempeño integral INCODER Colombian Institute of Rural Development, Instituto Colombiano de Desarrollo Rural JAC Community Action Council, Junta de Acción Comunal LAPOP Latin American Public Opinion Project MADR Ministry of Agriculture and Rural Development, Ministerio de Agricultura y Desarrollo Rural MdM Montes de María 2 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative MHH Male head of household MSI Management Systems International, Inc. PAR Regional Action Plan, Plan de acción regional PMP Performance management plan PNCRT National Territorial Consolidation and Reconstruction Policy, Política Nacional de Consolidación y Reconstrucción Territorial SENA National Learning Service, Servicio Nacional de Aprendizaje SGP General Participation System, Sistema General de Participación SIJIN Judicial police, Seccional de Investigación Judicial SIMCI Integrated Illicit Crops Monitoring System, Sistema Integrado de Monitoreo de Cultivos Ilícitos SISBEN Identification and Classification system for potential social program Beneficiaries, Sistema de Identificación y Clasificación de potenciales Beneficiarios para programas sociales TA Technical Assistance UACT Territorial Consolidation Administrative Unit, Unidad Administrativa para la Consolidación Territorial UARIV Victims’ Integral Attention and Reparation Unit, Unidad para la Atención y Reparación Integral a las Víctimas UMATA Municipal Agricultural Extension Units, Unidades Municipales de Asistencia Técnica Agropecuaria UNODC United Nations Office on Drugs and Crime URT Land Restitution Unit, Unidad de Restitución de Tierras USAID United States Agency for International Development VEO Verification of Organizational Status, Valorización del Estado Organizacional 3 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative ABSTRACT USAID/Colombia designed the Consolidation and Enhanced Livelihood Initiative (CELI) to support the Government of Colombia (GOC) in resuming its institutional presence in areas of the country under the influence of insurgent groups engaged in the production of illegal crops, drug trafficking, kidnapping and terrorist activities. This impact evaluation compares the status or conditions of beneficiaries and their households before and after their exposure to CELI interventions, isolating external factors that could influence changes in these conditions. The evaluation team conducted 19,946 rural household surveys across in 32 CELI municipalities as well as in 16 municipalities from three regions (Catatumbo, Cauca/Valle and Putumayo) in which the GOC intervened through its National Policy for Territorial Consolidation and Reconstruction (PNCRT) activities. Economic Development Nationwide rural development challenges and infrastructure deficiencies continue to affect CELI and PNCRT municipalities, and most respondents report a decline in their economic conditions. Poverty is present at the household, individual and community level, as market failings and lack of road access severely constrain growth. Access to credit remains tenuous across the regions, and land tenancy has made few gains. Income has increased, roughly at the level of (urban) inflation, but data suggest worsening food security. CELI beneficiaries report general improvements in economic conditions. The challenge of sustainably transporting products to markets remains. Social Development The CELIs documented increases in organizational competency scores – especially in business initiatives and services – and surpassed program targets. From citizen perspectives, however, there were no increases in the social capital index and a statistically significant decrease in the trust component. Trust in state institutions fell across the board except for the Ombudsman offices. Participation in the Juntas de Acción Comunal (JACs) declined, but participants who stayed played a greater role in decision-making. Surprisingly, the survey revealed a decline in CELI municipalities’ involvement with producers associations. In both CELI and control municipalities, respondents attested to an increase in productive capacities. Participation in women´s organizations and victims’ associations increased, and ethnically oriented cultural associations strengthened a sense of identity and built social capital. Governance and Institutions Citizen participation at the ballot box remains relatively high, but electoral challenges such as vote-buying continues. Citizen oversight is minimal, but CELI’s work with the Ombudsman (in at least one region) strengthened the registry of veedurías. Service delivery remains a challenge, with citizen perceiving poor access and low quality. Reports of human rights violations by the military decreased but reports of police violations increased in PNCRT municipalities. 4 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Security The data show fewer crimes overall and improved perceptions of security in CELI and PNCRT municipalities. However, extortion is rising and coca cultivation has doubled nationally. Respondents in CELI and PNCRT zones show more tolerant opinions about the growth and sale of coca. Military and police force levels have changed and justice services continue to be absent in many of the areas under study. These characteristics leave the zones unstable as before, with a lack of state presence that continues to put citizens at risk. 5 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative CSDI and PNCRT zones in Colombia Source: 2013, CSDI Impact Evaluation Baseline Report prepared by DevTech Systems, Inc. and Econometría, S.A. EXECUTIVE SUMMARY Introduction After decades of conflict in rural areas with illegal armed groups (IAGs) and with increasing success in limiting their activities, the Government of Colombia (GOC) launched the National Consolidation and Territorial Reconstruction Policy (PNCRT) in 2009 to increase its presence and to reduce inequity in the historically marginalized conflict zones where IAGs and coca production coexist. To support this effort, the U.S. Government created the Colombia Strategic Development Initiative (CSDI) as an integrated, geographically-focused strategy with significant investments in security, anti-narcotics, access to justice, and coca eradication. For its part, USAID/Colombia created the Colombia Enhanced Livelihood Initiative (CELI) to assist in the GOC’s goals of consolidating security gains, reducing coca cultivation and the influence of armed groups, and expanding state services. USAID selected three organizations to work in four regions that were under the influence of insurgent groups engaged in the production of illegal crops, drug trafficking, kidnapping and terrorist activities. CELI Norte/Sur (N/S) is implemented by Chemonics in Bajo Cauca, Sur de Córdoba and Tumaco; CELI Central, operated by ARD Tetra Tech, works in Cauca, Caquetá, Meta and Tolima; and CELI Montes de María (MdM) was managed by Global Communities in four municipalities in that region. At the time of the evaluation, CELI N/S and CELI Central were ongoing, while CELI MdM completed its work in early 2015. The three CELIs were intended from the beginning to be the subject of a rigorous impact evaluation consistent with USAID’s agency-wide Evaluation Policy, and in 2012 USAID charged its monitoring and evaluation (M&E) contractor DevTech Systems, Inc., to design an impact evaluation of the set of CSDI interventions and collect the baseline survey data. DevTech worked with Econometría, S.A., to carry out this work.1 The evaluation 1 DevTech Systems, Inc. and Econometría S.A. 2013. “Impact Evaluation Methodology of the USAID Strategy in CSDI Zones - Final Methodology Report.” 6 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative design included two subsequent rounds of evaluations: a midterm evaluation, and a final evaluation following completion of the program. The current independent midline impact evaluation of the results of CSDI efforts took place after three years of program implementation for the Norte/Sur and Central corridors and four years for Montes de María. USAID charged its Evaluation and Analysis for Learning (EVAL) project, a contract with Management Systems International, Inc. (MSI), to carry out the midterm measure of the evaluation and to report on the CELI program impacts to date. This report represents that effort, based on the baseline evaluation design, additional qualitative fieldwork, interviews with the CELI teams, and an extensive survey of participants. The evaluation took place in the 33 municipalities targeted by USAID and CSDI as well as in 162 municipalities from three regions (Catatumbo, Cauca/Valle and Putumayo) in which the GOC intervened through its PNCRT activities. The midline, structured around the baseline completed by DevTech Systems, Inc., examined changes since baseline across the institutional, social, and economic indicators related to the interventions. The multi-faceted nature of the security and development problems required a coordinated response, as the GOC also planned. The joint CSDI activities were designed to address the multiple challenges remaining across consolidation zones, including continued presence of armed groups and the illicit narcotics trade, enduring poverty, and lack of rule of law. Under the assumption that the government would serve as the main builder and executor of consolidation, the CELIs developed a strategy for institutional, social, and economic development to support, promote, and catalyze the GOC’s PNCRT. All three CELIs included the following components to reach the objectives listed in the results framework: • Support for productive activities and agricultural value chains (technical assistance and training in farming, processing, and marketing) and for youth entrepreneurial efforts in developing skills in business and commerce; • Support for development of credit and financial institutions and programs; • Infrastructure support in building social goods (schools, roads, health posts, improved sanitation, electricity) and productive infrastructure (processing plants, warehouses, and marketing centers); • Assistance in land issues for communities and individuals, including the formalization of land titles and private property rights, claims for restitution, regularizing the status of public lands, and dealing with special rights reserved for ethnic groups; • Social capital and civil society development, with training and support for Community Action Councils (the Juntas de Acción Comunal or JACs), producers associations, community councils in indigenous and Afro-Colombian areas, and others; and 2 18 municipalities were sampled, but security concerns at baseline excluded El Tarra (Norte de Santander) and Toribio (Cauca). 7 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative • Governance support to strengthen local and regional governments, such as through training in planning, budgeting, taxes, royalties, project formulation and management, and accountability. Economic development CELI projects promoted rural integration and economic development. To understand the degree to which their goals were met and impacts felt, the evaluation gathered information from CELI, PNCRT and control zones; from direct and indirect beneficiaries, control groups, and community representatives such as personeros, the alcaldías, members and leaders of JACs, and civil society leaders. Nationwide rural development challenges and infrastructure deficiencies continue to affect CELI and PNCRT municipalities deeply, and most respondents in all studied areas report a significant decline in their economic conditions since baseline. Poverty is not only at the household or individual level, but also at the level of communities, as the municipal and departmental conditions (market failings and lack of road access) severely constrain growth. Access to credit remains tenuous across the regions, and land tenancy has made few gains. Income has increased, roughly at the level of (urban) inflation, but survey and focus group data suggest worsening food security. The bright spot amidst this backdrop are direct CELI beneficiaries, who report improvements in economic conditions generally and in a set of key variables particularly. The challenge of sustainably transporting products to markets remains. Direct beneficiaries Respondents who had participated in CELI productive projects showed important positive effects. First, when asked about whether their economic situation was good or very good, there was a positive impact of approximately 8pp (Figure 1).3 There was also an impact of 6pp fewer respondents saying that their productive activities generated very little money. Figure 1. Impacts on perceptions of the economic situation, direct beneficiaries Variable Direct CELI beneficiaries Report that the economic situation is good or very good 8pp** Their productive activities generate very little money -6pp** Source: Authors’ calculations. *** At 99% confidence, ** at 95%, * at 90%.4 3 See the Limitations section in the full report and its Annex 3 for a full description of direct beneficiaries. 4 Throughout our reports, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are found in Annex 3 to the main report, and data in the text are tabled in its Annex 10. 8 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative CELI direct beneficiaries reported a 9pp positive impact when they were asked if they received support for their productive projects. They were significantly more likely than other respondents to associate with others to buy inputs, to process and sell products, to organize transportation, and to receive training (Figure 2). However, the impact analysis showed no impacts, positive or negative, on direct beneficiaries’ responses about their perceptions of food security and poverty. Figure 2. Impacts on reasons they associate with others, direct beneficiaries Variable Direct CELI beneficiaries To buy agricultural inputs 6pp** approx.5 To process their products 7.5pp** To sell their products 10pp** To transport their products to market 9pp** To received shared training or technical assistance 13pp** To have a shared representation as a group 7pp* To present group projects and seek financing 8pp* To seek credit 5pp* There were no impacts for direct beneficiaries in having a parcel of land for productive projects, but they did have a higher average number of hectares of land in usufruct with title, at more than double the hectarage found among the wider CELI population (Figure 3). There was a positive impact on having land in usufruct without title, similar to that of the other CELI beneficiaries. Figure 3. Impacts on land issues, direct beneficiaries versus all CELI Variable Direct CELI beneficiaries All CELI zone residents Average hectares of land in usufruct with title 14 to 17 HAs** 7.8 has** Do you have land in usufruct without title? 2.8pp** 2.8pp** Did you get your land through restitution? 3.5pp* 0.1pp Did you get your land through agrarian reform? 1.5pp* 0.9pp* The CELI projects track as one of their main indicators of performance the incremental or added value for key products that they support in productive projects. In the most recent reports, this incremental value substantially exceeded the goals set by the CELIs and would significantly help the supported associations and productive projects. (Figure 4) 5 Figures are approximate for the direct beneficiaries because there are a range of responses based on which definition of direct beneficiaries is used. See the Limitations section and Annex 3 for full details. 9 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 4. CELI performance, increasing sales CELI USD % of goal Central $1,723,467 121% North-South $16,735,470 358% Source: Quarterly and annual CELI reports; see Annex 5 N.B. CELI Montes de María did not report on this indicator, adopted as that CELI was nearing its end Regarding credit and financial services, direct beneficiaries were more likely to report having received credit, at nearly 11pp greater than at baseline, while the full CELI sample reported 7.6pp positive impact on having received credit to support their projects. There was also a significant 8pp increase in responses indicating they had current debt. Overall, direct beneficiaries give the CELIs positive grades in their survey responses. For example, when asked to rate the degree to which the projects had met their expectations on a scale of 1 to 5 (where 1 indicates that expectations for the program were not fulfilled and 5 indicates that they were completely fulfilled), direct beneficiaries gave an average grade of 3.7. A full 96% of direct beneficiaries affirm that they would recommend participating in a Colombia Responde6 program to family and friends. CELI direct beneficiaries and the rural economy. Given the panorama discussed below, CELI direct beneficiaries contrast with the rest of the samples in pointing to positive trends in a range of economic possibilities and the perception of the problems they face, such as the lack of opportunities and the low income provided by their productive activities. Female-headed households form an important part of this optimism. The beneficiaries appreciate the technical assistance (TA) that accompanies the CELI programs and other state institutions (e.g., SENA), and the support promotes assets and capital, which they will be able to enjoy in the long term. Wider CELI and PNCRT samples Perceptions of the rural economy. The positive impacts reported by direct beneficiaries are found in an economic context that appears ever more challenging, and CELI municipalities (and the rural areas in general) continue to face conditions that keep the majority of households in poverty. Across the full sample, in control groups and in CELI and PNCRT municipalities, citizens say that their economic situation has deteriorated since the baseline. This is true across municipalities, across various survey questions, and by notable percentage changes from baseline to midline. Since control respondents show the same trends, no impact of the interventions was found. Rather, this trend is likely related to a series of factors beyond the reach of any specific intervention which, for rural populations, constitute a poverty trap almost impossible to escape. These declines apparent in survey results reveal the plight of households which depend on the income of one person, live in precarious housing with limited access to land, and constitute contextual conditions not attributable to any intervention. (Figure 5) 6 Colombia Responde is the name by which the CELIs are known in the communities of implementation. 10 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative We learned to establish rubber and cocoa as crops. We didn’t know how to graft a plant for it to be more productive, now we have the technical methods. We can improve our parcels by our own means, with citrus plants, cocoa, with all these products. Their workshops have given us knowhow that has enriched us socially and culturally. The most important thing is that our vereda has a different perspective of what the countryside can be. (Producer, Antioquia, CELI) 33% 29% 38% 15% 18% 35% 77% 31% 0% 20% 40% 60% 80% 100% The economic situation is good or very good Last week the respondent ran out of food The family is suffering hunger The conditions in which they live are of extreme poverty CELI Baseline CELI Midline Figure 5. Trends in economic situation perceptions, CELI municipalities Despite the difficulties in the consolidation zones, qualitative data collection revealed that when smallholders were asked about their views on their region and its potential, a different and more positive picture emerged. This is an indication of the ownership inhabitants feel regarding their territories, and, in some cases, a community-wide rejection of illegal activities7 in favor of a culture of legality. However, in sum, the results indicate that the economic situation in CELI and PNCRT regions parallels that found in rural areas across the country, with the inhabitants of remote rural zones faced with the same hardships and needs in surviving day to day, facing difficulties even in terms of basic food security. Support for productive projects. CELI interventions aim to support sustainable and licit income generation. The CELIs made great investments to support smallholder associations, to improve the land, to provide technical assistance and inputs for production, and to support commercialization. Support for specific crops included a focus on quality, better prices for products, organizational development, and particular project support. The CELIs provided technical assistance to improve productivity and quality and emphasized organizational training and capacity building for producer associations. The intervention components attempted to counteract the deficiencies that producers faced in their regions. Productive projects in CELI programs showed evidence of economic changes in the households. The projects aim to create long-term value that helps producers provide for themselves through their productive projects and to strengthen savings and investment, as they oblige the beneficiaries to work and look after their crops with a long-term perspective, with all the costs and sacrifices that this implies. 7 Please see the sections on illicit activities, security, and their contribution to structural poverty, later in the report. 11 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Marketing and trade for rural producers. TA and marketing support help counteract problems related to low profitability. For the most part, only local and regional markets exist in the consolidation zones, and this lack of broader markets seriously hampers the tasks of the CELI and of the GOC. Deficient and partially depressed markets, pressure from armed groups, and the lack of access roads all diminish the effectiveness of productive projects in counteracting the incentives to plant illicit crops. Such structural problems – which are the GOC’s direct responsibility – diminish the ability of the CELI and PNCRT programs to produce effects that are sustainable in time. In the survey, a series of items enquired about the problems experienced in production and marketing products. There were positive impacts on three of these questions in CELI and PNCRT municipalities (Figure 6); that is, negative perceptions diminished over time: Figure 6. Impacts on perceptions of problems in production and sales, CELI and PNCRT Variable CELI PNCRT Abuse by intermediaries -0.076* -0.149 Very low prices for the products -0.062*** -0.094* Lack of water -0.021 0.100* The impacts listed in the table represent significant reductions in the perception of problems among the respondents, relative to control sites. Though impacts were only found for these three questions, the trend over time was a reduction in CELI, PNCRT and control municipalities across a range of such problems, though these did not always show significant impact. (Figure 7) Figure 7. Trends in perceptions of problems in production and sales, CELI and PNCRT Credit for rural producers. Although CELI interventions did not directly provide credit, some participating associations created rotating funds to support their members. The revolving funds have proven valuable for small groups of neighbors and acquaintances. Implementation costs are low and there are important economic benefits to the group. However, according to the household survey, formal credit availability has fallen in both CELI and PNCRT municipalities. 12 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Informal credits contained in trading or illegal gota a gota (“drop by drop”) loans continue, filling a gap when the broader economy fails to provide viable alternatives. Community credit or loans which are a product of programs such as CELI provide an alternative to informality, and information gathered in focus groups indicates that revolving savings and credit funds have served as important instruments to support capitalization in rural areas. Rural roads. The Consolidation zones lack access roads that connect to the country’s most dynamic and populated markets.8 The lack of reliable transportation infrastructure means that local producers cannot access markets where they can gain prices that are higher than costs. The data show very negative perceptions and experiences with road quality. Local market deficiencies are closely related to the state of the roads. Building and repairing roads was not originally one of CELI’s central axes, but such activities have increased in recent years in a broad approach to resolve bottlenecks to commerce. The population recognizes the direct benefit of CELI interventions in roads and infrastructure. Local level users, associations and Juntas de Acción Comunal (JACs)9 have taken ownership of road projects. Land rights and access. An important element in the CELI vision is to support land formalization. This includes helping rural residents get formal titles, supporting displaced families and communities to return to their lands, and supporting governmental institutions which, ultimately, grant land tenure rights. The continuing informality in ownership rights reveals the limitations of the State. The communities deal with this problem through precarious ownership rights, based on fragile documents such as sales letters. The absence of formal ownership titles renders such documents invalid for use in guaranteeing credit and discourages the development of productive projects in the long term. Each of the CELIs have provided institutional support to state institutions involved in the land claims, while facing the obstacles that such institutions have been weak, without clear and consistent processes. The CELIs have maintained records pertaining to restitution cases and the formalization of property ownership. They have, in large part, reached or exceeded their goals. The advances in the titling procedures in the CELI and PNCRT municipalities continue to be limited in the face of the huge scale of the problem. Although the CELIs can support such processes, it is ultimately a matter the State must resolve. Recommendations 1. Set realistic expectations for impact at a level that implementing partners can affect. The impact indicators for CELI were cast at a high level of generality in the results framework, relative to the investment and interventions, the two years of implementation, and the problematic contexts of operations in conflict zones. Given that the economic situation reflects 8 Data from the Ministry of Transport. In departments in which the CELI works, reduced coverage is confirmed (8-16% of the area), reduced paved area (<7%), and poor quality (1-13% in “good” state according to the Ministry’s classification). 9 JACs are the principal institutional (and constitutionally recognized) entity of community governance at the local level. 13 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative regions’ critical structural deficiencies, it is important to not consider lack of wider impact as representing a failure in programming. 2. Make simultaneous improvements in production and marketing. Increased production must go hand-in-hand with market improvements and private sector alliances. Small, focused pilots that treat these interconnected structural failings holistically, by donors or the GOC, are preferable to diluting the effects by intervening across broad geographic areas with expansive mandates. 3. Support wider markets. Support commercialization designed to improve both local markets and those that export goods to other regions, which may provide a solution to local overproduction and low prices. Programs must bring producers and buyers together in the productive process to work with broader marketing efforts to improve quality. Involving banks to increase access to working capital and providing better market information, such as through mobile phone access to up-to-date market information on local and regional conditions, can help. 4. Plan for sustainability in productive projects with delayed returns. CELI and USAID should ensure that program support for long-term productive projects also support food security. Productive projects must combine short- and long-term strategies to ensure food security, cash flow and project sustainability. 5. Reach Female Heads of Household (FHHs) with tailored interventions. . Given that FHHs and MHHs learn and use their learning in different ways, models should reflect this difference and provide different services to take advantage of the characteristics and behavior of each. 6. Maintain activities to expand responsible use of financial services. Revolving funds are a suitable instrument to deal with communities’ borrowing needs but they must be protected against the risk of non-payment. To strengthen governance principles to guarantee sustainability, a system that monitors the funds, sets up discussion fora and experience exchanges, creates incentives for performance and draws attention to successful cases is necessary. 7. Continue focused work on roads. The current advances made by CELI N/S and CELI Central in mobilizing communities for tertiary road maintenance provide a space for experimentation for community involvement in this arena in the future. The CELI programs should continue the successful support they provide to the JACs in institutional strengthening and participation to undertake community infrastructure efforts. Social development Strengthening civil society and social development is considered fundamental to the consolidation of conflict-affected areas in Colombia. Social development is understood as the population’s participation in civil society, trust between peers, and trust between citizens and the state. The CELI interventions are intended to develop and strengthen social capital in target populations as a tool for territorial consolidation. Organizational capacity. The CELIs documented substantial and important increases in organizational competency scores, especially in business initiatives and services, surpassing program targets. An outward focus on markets and services appears to be important to sustainability, balanced with support to organizations’ internal management and other features. (Figure 8) 14 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 8. CELI Performance: Average improvements by category, CELI N/S Social capital: Trust. From citizen perspectives, there were no increases in the social capital index and a slight statistically significant decrease in the trust component, led by a greater drop among FHHs. Trust in 25 state institutions fell across the board in all groups surveyed (CELI, PNCRT, and controls), except trust in the personería. As the effect is generalized, it is not attributed to any intervention; rather, the deterioration likely comes from sustained, external events (absence of state presence, lack of services, scandals, etc.). (Figure 9) Figure 9. Impacts on the Social Capital Index, CELI and PNCRT Variable CELI PNCRT Social Capital Index -0.24 -1.16 Trust component -0.99** -1.00 Participation component 0.74** -0.16 Source: Authors’ calculations. *** At 99% confidence, ** at 95%, * at 90%.10 Qualitative data challenged the depth of this decline in trust. With some exceptions, trust in the institutions is a constant among respondents in focus groups and interviews. They report that trust and solidarity do exist between neighbors and that this has facilitated the development of social groups. In regions where armed groups operate, trust between peers not only improves quality of life but also permits the creation of associations and the strengthening of the JACs. This finding is transversal inasmuch as it is reflected in all kinds of efforts to organize, whether productive associations, victims’ organizations, cultural or youth groups, or others. The difference between the survey and qualitative interviews may be explained by the fact that the surveys represent an average opinion of the population while the qualitative information comes from a purposively selected sample of active, participating people. 10 Throughout, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are in Annex 3, and data in the text are tabled in Annex 10. 15 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Social capital: Participation in associations. The survey data show that households participated more in some – but not all – types of community organizations. The impacts were slightly more significant in CELI municipalities, and among MHHs (but not FHHs). (Figure 10) Figure 10. Impacts on participation in community activities, CELI and PNCRT Variable CELI PNCRT Oversight groups (veedurías) 0.01*** 0.004 Community meetings in general 0.07** -0.003 Voluntary groups 0.06*** 0.05* Sporting or cultural groups 0.03*** 0.03* Security groups 0.005*** 0.005 Youth groups 0.007** 0.004 Women’s groups 0.02* 0.04*** Yet in spite of the overall positive impact, fewer households participated in JACs at midline: from 74% at baseline to 57% at midline in the CELI zones, and from 88% to 64% in the PNCRT municipalities. A variety of reasons, from the time requirements for participation to the perception that local politicians had co-opted the JACs for their own purposes, were cited in explanation. Nevertheless, each CELI developed strong commitments to work with the JACs, including directly contracting with them to carry out community projects and supporting their institutional strengthening. Between CELI N/S and CELI Central, dozens of JACs have taken on responsibility for managing and fulfilling the contractual terms of direct grants, providing these civic organizations with important experience in community management and infrastructure construction and maintenance. Despite the fact that on occasions these entities are considered to have been co-opted, they continue to represent the gateway to state services and programs. Where the security situation has allowed, and where the JACs have the confidence of the population, this kind of social organization is central to political life and to participation in the veredas. In these cases the JACs play a central role in providing access to, and enjoyment of, the services offered by the state in terms of political participation, infrastructure improvements, and social control. In the veredas of geographically distant municipalities the JACs are the principal spaces in which disputes between neighbors are resolved, through direct negotiation. By having this local resource, minor social conflicts are resolved at the vereda level, without the need to take cases to municipal or national bodies. 16 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Associations and productive projects The CELI projects have supported producer associations in several ways: technical assistance and training; support to develop marketing and commercialization (also often involving training); and assistance with financial services to support production and marketing. As the implementer of CELI Central put it, “the focus has increasingly been on empowering and mobilizing smallholder producer associations, and increasing private sector investment in rural communities to create economic opportunities for community stakeholders within the context of licit markets and the rule of law.”11 Surprisingly, the survey revealed an apparent decline in CELI municipalities in involvement with producers associations. Respondents reported a significant 7pp reduction in their participation in producer associations in the CELI municipalities, whereas in the PNCRT municipalities, no significant variation occurred (Figure 11). On the other hand, those who did participate reported greater involvement in decision-making throughout all survey groups. In both CELI and control municipalities, participating respondents attested to a significant increase in their productive capacities across several dimensions. For CELI municipalities, participants affirmed joining producers associations because of the support they provided in processing products, sales, transport, and training. On the other hand, respondents recognized that some producer associations were created largely to secure benefits from government and donor funding, with little reference to their sustainability. Figure 11. Impacts on participation in producer associations, CELI and PNCRT Variable CELI PNCRT Household belongs to or participates in producer or marketing associations -0.068*** -0.009 Women’s, victims’, and cultural associations Participation in women’s organizations and victims’ associations has increased. The social capital constructed as a result of the work of victims’ associations has permitted the creation and strengthening of other kinds of organizations, encouraging an expansion of their aims, fomenting a sense of belonging and identifying new resources. Ethnically-oriented cultural associations were found to strengthen a sense of identity and build social capital and cohesion. This has enabled the reconstruction of community values damaged by the armed conflict. Returning to activities such as dance, music, or traditional artisan production has permitted communities to create a cultural identity in which they are able to recognize themselves and feel pride in their traditions; it has also acted as a form of resistance against armed actors. 11 CELI Central Quarterly Report Q1 FY2016, p. 5. We created a project called the Son de Negros. We focused on the recruitment carried out by illegal armed groups and we are going to include this in our danza negra, an African dance that we have been dancing since the time of the palenque of María la Baja. There is a sense of connection with the mountains, with my grandparents. I remember that my grandfather was black and was a drummer in the danza negra. (Bolívar, CELI) 17 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Recommendations 1. The GOC must ensure security. The intervention zones must be secure before interventions on trust will have any effect. 2. Support JAC legitimacy. Leaders should continue to be trained in board election processes and community participation tools to counteract loss of legitimacy. Monitoring the quality of community associations such as the JACs, adapting tools to meet organizational and community expectations for these groups, and ensuring that programs involve the JACs, communities, the participation of victims, women, and other marginalized groups, as well as local government should be priorities. 3. Loosen the requirement that beneficiaries must be grouped into associations. In future programs, the requirement for obligatory membership in an association to receive benefits should be reviewed to determine how best to structure incentives for participation. Otherwise, associations may have little power to consolidate and become self-sustaining. 4. Increase collaboration with associations whose objectives may not be focused on productive activities. Pre-existing social organizations with aims that are not productive are also candidates for interventions like CELI because of their high levels of social capital, and they are increasing in number in the zones of intervention. Such organizations should also be able to gain help to become autonomous and financially sustainable. Governance and institutions Participation, accountability and trust in public management Citizen participation at the ballot box (nearly 90%) and in political organizations remains high and increased in the last presidential election across all groups interviewed, but electoral challenges such as vote buying and other clientelistic practices persist. Combining survey questions in an Accountability Index shows declines since baseline across the regions under study. Changes were uniformly negative for CELI municipalities in how the public rates the accountability of municipal governance, with a drop in score between 5% and 28% (Figure 12). In specific problem areas, the only notable drop in perceived problems in CELI municipalities was in the delivery of public services (water, electricity, sewer, etc.). Citizen oversight is low and has only increased slightly. Figure 12. CELI Performance: Accountability Index, by region Ind. Description Disaggregations Baseline Midline % Change Source DO3- 028 Level of accountabili ty in CELI municipaliti es Total 23.8 19.63 -18% Household survey • Baseline 2013, Econome tría S.A. • Midline 2015, Proyecto EVAL Montes de María 18.6 17.74 -5% Central 27.8 24.96 -10% Norte 25.5 18.89 -26% Sur 22.6 16.17 -28% Participatory mechanisms are not yet widely effective. Meetings convened by municipal institutions are perceived not to respond to needs of community leaders. Community members 18 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative also rate the performance of their local governments poorly, and their opinion of mayoral management in both CELI and PNCRT municipalities declined significantly. In CELI and PNCRT municipalities, trends are negative regarding government management at all levels. They may be ineffective at implementing new resources, may be creating new and unmet expectations, and may simply not have changed practices in response to interventions. But the effect on citizens is significant. Where CELIs intervened successfully on this front, there are localized results – but the generalized impact the evaluation was designed to detect is not present. The confidence in Colombian local institutions worsened in recent years with the survey showing decreasing confidence in all levels of government. Trust dropped from baseline to midline across the full sample. The decline in confidence is parallel across governmental institutions: police, army, navy, justice; national, departmental, local, city council and JACs; ombudsman’s offices; and advocacy and support bodies such as the national training institutes. An exception to this is the work of the personerías. Citizens in CELI and control areas recognize the personería as the main institution with which communities interact, particularly because of its role in the Victims’ Registry and in complaints about human rights violations. CELI work with the personerías to strengthen the registry of veedurías has been positive. Social service delivery The GOC’s Integral Performance Index (IDI for its Spanish initials), applied since 2006 to assess the management of territorial governments, tracks the performance of mayors and governorships in six dimensions: effectiveness, efficiency, legal requirements, administrative capacity, fiscal performance and management.12 It shows increased scores from baseline and midline for all four groups of municipalities studied – CELI, PNCRT, and their respective control groups. The strongest gains are in efficacy, observing legal requirements in budgeting, and management. CELI municipalities continue to have somewhat lower overall ratings, however, than the PNCRT zones. (Figure 13) 12 The DNP explains the results, the methodology and the sources of information in the documents in this website: https://colaboracion.dnp.gov.co/CDT/Desarrollo%20Territorial/Docuemnto%20Desempe%C3%B1o%20Integral%202013.pdf 19 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 13. Secondary data: Integral Performance Index, 2012 (left) and 2014 (right) This contrasts with the Governmental Capacity Index, developed specifically for this impact evaluation. The GCI measures capacity and service delivery from citizens’ perspectives, rating access and quality of services, quality of public policy formulation, and perceptions of municipal actors’ honesty. Citizens report no progress on the GCI indicators. The analysis shows that governmental capacity is a desirable characteristic but difficult to achieve, and very subjective, as it responds to structural and historical factors that can escape the reach of CELI interventions. However, the stagnation of perceptions of government capacity should be taken into account as a sign of the failure of the policies being implemented to consolidate these regions. Results for both CELI and PNCRT show declines since baseline in CELI municipalities of between 5% and 17% in absolute terms.13 (Figure 14) Figure 14. CELI Performance: Governance Capacity Index, total and regional Ind. Description Region Baseline Midline % Change Source DO 3- 013 Governance capacity index Total 40.7 36.2 -11% Household survey • Baseline 2013, Econometría S.A. • Midline 2015, Proyecto EVAL Montes de María 39.2 37.3 -5% Central 39.8 36.2 -9% Norte 42.6 36.2 -15% Sur 40.3 33.4 -17% 13 The difference in trends between DNP data and survey data can be explained because the DNP data measures internal, short-term and administrative decisions and processes that are not necessarily visible to the population in the same period of time. 20 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Social service indicators, tracked at baseline and midline, had little to do with CELI interventions. But they do show that social service delivery remains a major challenge, with citizen perceptions of access and quality objectively very low, and lower than at baseline. Citizens’ opinions of social service access and quality have not changed in CELI municipalities. In absolute terms, HHs in CELI municipalities report a worsening of schools, while there were some improvements and even impacts among PNCRT municipalities. Recommendations 1. Limit the scope of evaluation to the factors under implementers’ manageable interest. The design of the impact evaluation, based on a vision of a cross-government integrated program in CSDI, proposed far higher impact gains than can be expected from the CELIs alone. 2. Use prior diagnostic activities with potential target municipalities to uncover clientelistic and corrupt practices. Detection of clientelistic practices should trigger GOC involvement to precede donor interventions. This suggests focusing first on supporting control bodies such as the Attorney General, the Comptroller or local veedurías, before distributing resources for specific projects, and financing projects that meet high standards for transparency, community participation and oversight. Avoid implementing with intermediaries, which may increase risk. 3. Strengthen accountability to improve public policy orientation, gain citizen trust and provide management oversight through training; expanding spaces for greater review of management and monitoring; supporting the application of sanctions for corruption; encouraging the development of a culture that discredits corrupt practices; simplifying administrative processes; and developing participatory budgets. 4. Continue to work to strengthen the personerías, who have become essential actors in conflict-affected zones, particularly with victims. 5. Continue to work with municipalities to improve service delivery, and involve citizens in oversight activities. Focus on pilot-level activities with significant civil society involvement, and scaffold local governments with consistent TA. Use scorecards to involve citizens in rating service changes, and learn from failures as well as successes which practices work best in different sectors and circumstances. 6. Work with the GOC to strengthen the IDI. The IDI lacks important considerations of the effects and perceptions of service provision. The national imperative to decentralize is an incentive to view only those indicators that show a picture of progress, but the rubber meets the road in citizen perceptions of those services. Security At the outset, it is important to note that the CELI interventions undertook no activities in the sectors of security and justice reform. The impact evaluation was designed when the CELIs were part of the larger CSDI set of activities, along with a promised whole-of-government approach from the GOC. As a result, impacts in these sectors are outside of the CELIs’ manageable interest. Over the last decade, there has been a drastic transformation in security conditions in Colombia in both rural and urban areas. Reduction in some crimes has converged with an increase in 21 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative others, the recovery of legality in some regions, and the return of others to criminal control. The evidence includes a security index, analysis of coca cultivation, trends in major human rights crimes, and new criminal dynamics. CELI and PNCRT interventions continue to operate in challenging security conditions but these have changed with new dynamics of crime and new threats – less overt and major crimes and more coercive, extortionate behavior that affects the population in different ways. Data collection has not yet caught up with these changes. Coca cultivation has doubled nationally and the CELI and PNCRT zones show related changes in perceptions and opinions about coca as an economic activity. Military and police force levels have changed drastically, and justice services continue to be absent in many areas under study. The zones are unstable as before, with the lack of state presence that has – and continues to – leave citizens and their livelihoods at risk. EVIDINCE14 created an index of insecurity, synthesizing averages of criminal acts of major and low impact (thefts, robberies and homicides) with subjective elements of the perception of security, such as the appearance of graffiti in the communities. Figure 15. Secondary data: Insecurity Index Insecurity dramatically increased in CELI municipalities in 2014, placing it significantly above the indicator for the control municipalities (Figure 6.15). Both CELIs and their controls saw declines in insecurity in 2015. In PNCRT municipalities, the insecurity index shows a steady 14 EVIDINCE is a research effort associated with the Empirical Studies of Conflict (esoc.princeton.edu), a multi-institutional collaboration studying conflict and insurgency at the sub-national level. Source: EVIDINCE. The index combines data on homicide, robbery, larceny, illegal mining managed by armed illegal groups (IAG), and the presence of graffiti alluding to IAG in the municipio. Treatment 1 (T1) are CELI municipalities, and C1 their controls. T2 are PNCRT municipalities, and C2 their controls. 22 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 92% 94% 94% 93% 60% 77% 79% 76% 79% 61% 0% 20% 40% 60% 80% 100% Coca, poppy and marijuana shouldn't be cultivated because they're illegal Producing illict crops negatively affects our families and community Growing coca is not the only way for families to guarantee their subsistence My family and friends think growing coca is bad Our community is willing to denounce the presence of illicit crops CELI Baseline CELI Midline decline from the end of 2013 until the middle of 2014, when there is a slight rise, which is also present in their control municipalities. Since August 2015, the EVIDINCE insecurity index shows a generalized decline in all municipalities studied. This is explained by the de facto truce in the final stages of the peace talks between the GOC and the FARC. In terms of coca production, after a dramatic decrease between 1998 and 2004, illicit crops (ICs) remained steady until 2012, when a pronounced increase in ICs was noted. Survey results suggest an increased perception of ICs since the baseline, especially in the PNCRT municipalities. Also notable are the survey results showing the erosion of respondents’ baseline agreement that illicit crops should be avoided – in CELI municipalities, respondents’ agreement that cultivating ICs dropped from over 92% to 77%, for example. (Figure 16) Figure 16. Trends in perceptions about illicit crops in their communities, CELI These impacts reflect a national trend, which has engendered a range of hypotheses as to its origins: i) the low profitability of other legal and illegal activities, such as illegal mining; ii) the reduction in aerial spraying and manual eradication campaigns; and iii) a boom in cultivation prior to the signing of a peace agreement that would enable the legalization of capital obtained illegally (a practice documented by some FARC commanders) (Wilches and Garzon, 2016). Crime statistics. The gradual dismantling of paramilitary groups and the new operational logic of the BACRIM [criminal bands], added to the commencement of the peace talks with the FARC, have helped to reduce the number of cases of human rights violations throughout the county, especially high-impact crimes, such as massacres and acts of terrorism. There has been a 61% reduction in massacres in the CELI municipalities – similar to the national trend – which means an impact of around 7% of the decline in the national total of massacres. Furthermore, there has been a parallel reduction in guerilla attacks over the last decade in the control and treatment municipalities. Following the fragmentation of paramilitary groups (converted into BACRIM) and their sporadic criminal alliances with the guerrillas, extortion has become the fastest growing crime in Colombia. However, communities’ perceptions of extortion as a serious crime are low, and are on a downward trend in both CELI communities and their controls. The proportion who believe that extortion is one of the most serious security-related problems declined from 9.6% at baseline to 5.5% at midline. 23 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 17. Secondary data: Average number of attacks by guerrilla groups, per municipality15 Source: Conflict Observatory, Universidad del Rosario Military presence. It is not possible to gain access to operational reports on the availability of public forces in the areas studied, but the evaluation survey gives one perspective on military presence. Survey results show a drastic decline in communities’ perceptions of military presence in CELI, PNCRT and control municipalities. From 87% at baseline, only 37% of households in CELI municipalities asserted that there were military forces in their communities at midline. The trend was parallel in their controls: a decline from 83% to 29%. PNCRT municipalities showed a similar but less drastic trend, from 77% to 40%. While perceptions of military presence fell drastically, in the CELI municipalities, there was a 6pp positive impact when asked whether military services had improved over the previous two years (32% to 39%), with no change in control municipalities. Police presence. Figures compiled by EVIDINCE in municipal panel data show a leveling off of police presence in treatment and control municipalities over the past three years. The survey results support this decline in perceptions of police presence across the full sample, as drastic as the decline in military force presence. CELI municipalities and their controls saw two-thirds of their respondents citing the presence of police services at baseline, and only a quarter at midline. 15 The data available from the CERAC and the Ministry of Defense indicate a reduction of 92% and 95%, respectively, in the number of FARC attacks in 2015, which shows the impact of the negotiation process on the reduction of guerrilla actions. 24 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 18. Secondary data: Police presence Source: EVIDINCE municipal panel Availability of justice services. Survey respondents from CELI, PNCRT and control households also report a precipitous decline in the presence of formal justice systems in their municipalities, at even a greater rate of decline than the lack of security presence: from some 70% to 80% at baseline, to 8% to 17% at midline. There was no significant change in their perceptions of the quality of the services. Fewer respondents say they would access justice through formal channels than at baseline in absolute terms, even through the JACs (which appear to be a source of conflict resolution services at the local level). However, trends were the same across the full sample, so these changes appear across all regions. Recommendations 1. Guarantee security conditions in all intervention zones. USAID should consider using its international cooperation resources to further encourage the GOC’s defense sector to prioritize a minimum level of security, both in forces and conditions, in the territories where CELIs and future programs operate. 2. Support the creation of specialized police units and judicial police in the intervention zones, coordinated with the military and focused on specialized units and properly vetted judicial police. 3. Adjust interventions to reach the areas with the greatest coca cultivation. Very few of the coca growers access state programs, and productive projects designed for these citizens do not generate the necessary incentives for returning to the path of legality. 4. Update security indicators to capture changing dynamics, capturing new criminal dynamics and providing an updated diagnosis of the security challenges, including new forms of criminality. 25 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative INTRODUCTION After decades of conflict in rural areas with illegal armed groups (IAGs) and with increasing success in limiting their activities, the Government of Colombia (GOC) launched the National Consolidation and Territorial Reconstruction Policy (PNCRT) in 2009 to increase its presence and to reduce inequity in the historically marginalized conflict zones where IAGs and coca production coexist. The concept of “consolidation” refers to the expansion into the conflict zones of Colombia of the legitimate presence of the State. The policy, designed for implementation once minimal conditions for public safety were established, would create the necessary institutional mechanisms to provide basic human rights protections to the populations in those zones historically affected by conflict and illicit crops. The PNCRT was oriented to establishing the presence of State institutions capable of efficiently bringing necessary public goods needed for development. The policy conceives of social, economic and institutional consolidation and social reconstruction as the result of a process of mobilization and continual strengthening of three pillars: (1) the territorial institutionalization of state institutions focused on guaranteeing security and creating the necessary capacities to institutionally manage the area; (2) the strengthening of citizen participation; and (3) the fostering of regional integration.16 The interaction of these three pillars on a base of territorial security and combined with a coordinated collaboration with state entities would be the path to guarantee the fundamental rights of the citizenry. The U.S. Embassy, for its part, developed the Colombia Strategic Development Initiative (CSDI) to support the strengthening of government presence in strategically important parts of the country and reduce inequity in historically marginalized regions where illegal armed groups and coca production coexist. CSDI was conceived as a whole-of-government, geographically￾focused approach aimed at maintaining advances in security and development and at the reduction of illicit crops and the presence of illegal armed groups in the conflict zones. It included significant investments in security, antinarcotics, access to justice, and eradication activities: USAID is already coordinating closely with other Embassy entities, including the Narcotics Affairs Section (NAS), the Military Group, and the Department of Justice as well as the Political and Economic sections of the State Department to develop an integrated USG approach for the priority CSDI consolidation zones. Programs will include peace and security, illicit crop eradication, access to justice, justice sector reform and humanitarian and development interventions targeted at a wide range of populations.17 16http://www.consolidacion.gov.co/themes/danland/descargas/entidad/planeacion/POLITICA_NACIONAL_DE_CONSOLIDAC ION_Y_RECONSTRUCCION_TERRITORIAL_PNCRT.pdf. 17 USAID Colombia Solicitud de Aplicaciones Número 514-09-000005, “Montes de Maria – Enhanced Livelihoods Initiative;” Septiembre 15 de 2009, p. 5. 26 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative The multi-faceted nature of the security and development problems required a coordinated response. The joint CSDI activities were designed to address the multiple challenges remaining across consolidation zones, including continued presence of armed groups and the illicit narcotics trade, enduring poverty, and the lack of rule of law. Working with the GOC and PNCRT, the goal of uniting USG agencies toward consolidation was: To reach a “tipping point” where state presence and positive perception of the government is sufficiently well-established so that communities decide to align with their government rather than illegal armed groups and the coca industry.18 USAID/Colombia designed the Colombia Enhanced Livelihood Initiative (CELI) as one of the CSDI components and as the backbone of this integrated plan to bolster the PNCRT by supporting efforts to consolidate the State’s institutional presence in areas of the country under the control or influence of insurgent groups engaged in the production of illegal crops, drug trafficking, kidnapping, extortion and terrorist activities. The basic premise of the CELIs is that the armed conflicts and illicit economy are closely linked to the weak presence of the State, and their activities are oriented to strengthening state institutions, generating confidence in them, constructing relations between communities and their local governments, and supporting the improvement of sustainable ways of life. The CELIs clearly are no substitute for the State. The existence of a minimum provision of public goods, property rights and justice services in the consolidation zones is a sine qua non condition for enabling the CELIs, and any international cooperation effort, to find a place in which they can operate and ensure that the resources will serve as catalysts for development and consolidation. The CELIs, or Colombia Responde as the projects are known in the implementation zones, supported the GOC through three pillars that are similar (though not identical) to those of the PNCRT: Figure 1.1. The three pillars of the PNCRT and their CELI peers PNCRT pillar CELI pillar CELI pillars defined as: 1 Territorial institutionalization Strengthening institutional development Improving local government response; formalizing property rights; processing cases of victims’ property rights; implementing a rural development strategy 2 Citizen participation and good government Strengthening social development Strengthening civil society organizations (CSOs); promoting alliances between the community and the State 3 Regional integration Catalyzing economic development Implementation of rural development programs and improving competitiveness 18 Ibid. 27 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Three implementers were convened to undertake the work in four of the seven corridors comprising the Consolidation Zones. The first, Chemonics International, has worked through the CELI Norte/Sur project in the northern (Bajo Cauca in Antioquia and Sur de Córdoba) and southern (Tumaco) regions; the second, ARD Tetra Tech, implements the CELI Central project in 23 municipalities of the departments of Cauca, Valle del Cauca, Caquetá, Meta, and Tolima; and the third, Global Communities, finished a similar set of tasks in early 2015 in the municipalities of Ovejas and San Onofre (Sucre) and Carmen de Bolívar and San Jacinto (Bolívar) through the CELI Montes de María (MdM) project. With this shared mandate and the assumption of the GOC’s leadership role, the CELIs developed strategies for institutional, social and economic development in these four regions. Within USAID’s Country Development Cooperation Strategy (CDCS) for Colombia, the CELIs were first situated within the Mission’s Development Objective 1: Civilian government present in CSDI zones consolidated, which focused on supporting the process of consolidation – strengthening of local government, formalizing property rights, social development and the alliance between community and state, and rural development. In the updated CDCS FOR 2014-2018, the CELI projects were placed under a revised DO 3, Improved conditions for inclusive rural economic growth. CELI investments are targeted to Intermediate Results in supporting Development Objective 3, and have much in common with their original results framework in spite of the shift toward economic activities. The current framework, tied to the CELIs’ common Performance Management Plan (PMP) prioritizes the following: IR 3.1 More equitable and secure land tenure Sub IR 3.1.1 Land restituted to displaced victims of conflict IR 3.2 Increased private and public investment in the rural sector Sub IR 3.2.1 Strengthened local governments’ capacity to access and manage public funds for productive infrastructure Sub IR 3.2.2 Increase private sector investment in target rural communities IR 3.3 More effective producer associations benefitting smallholder farmers Sub IR 3.3.1 Improve institutional capacity to producer associations CSDI and PNCRT zones in Colombia Source: 2013, CSDI Impact Evaluation Baseline Report prepared by DevTech Systems, Inc. and Econometría, S.A. 28 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Under the assumption that the government would serve as the main builder and executor of consolidation, the CELIs developed a strategy for institutional, social, and economic development to support, promote, and catalyze the GOC’s PNCRT. All three CELIs included the following components to reach the objectives listed in the results framework: • Support for productive activities and agricultural value chains (technical assistance and training in farming, processing, and marketing) and for youth entrepreneurial efforts in developing skills in business and commerce; • Support for development of credit and financial institutions and programs; • Infrastructure support in building social goods (schools, roads, health posts, improved sanitation, electricity) and productive infrastructure (processing plants, warehouses, and marketing centers); • Assistance in land issues for communities and individuals, including the formalization of land titles and private property rights, claims for restitution, regularizing the status of public lands, and dealing with special rights reserved for ethnic groups; • Social capital and civil society development, with training and support for Community Action Councils (JACs), producers associations, community councils in indigenous and Afro-Colombian areas, and others; and • Governance support to strengthen local and regional governments, such as through training in planning, budgeting, taxes, royalties, project formulation and management, and accountability. The CELI investments are shared across multiple municipalities and several different sectors. For example, through September 2015 CELI Norte/Sur invested US$56 million in activities across 16 municipalities with financial inputs broken down as follows: 51% in economic development, 18% in infrastructure, 17% in governance, 11% in social capital, and 3% in land￾related activities. Broad dispersion of development resources also took place in the zones under the PNCRT: the government designed a strategy based on the interaction and coordination of public sector entities, but in practice each spent its resources in accordance with its own priorities, interests and historical allocations. Public sector coordination is extremely difficult to accomplish because doing so requires personnel resources and a level of institutional authority that challenges existing patterns and interests, both within independent ministries and agencies and with local governments. Observers also point to weak local institutional capacity. Furthermore, these same competing interests made it difficult for the GOC to fully realize its prioritization within the Consolidation Zones, especially given demands and pressures for nationwide development investments. Thus, the strategy involved good intentions but faced grave challenges in coordination and implementation. Obviously, the CELIs are no substitute for the State. They were called upon to provide support to the pillars of the PNCRT. The existence of a minimum provision of public goods, property rights and justice services in the Consolidation Zones is an essential condition for enabling the CELIs, and any international cooperation effort, to find a place in which they can operate and ensure that the resources will serve as catalysts for development and consolidation. 29 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative EVALUATION PURPOSE & EVALUATION QUESTIONS The three CELIs were intended from the beginning to be the subject of a rigorous impact evaluation consistent with USAID’s agency-wide Evaluation Policy, and in 2012 USAID charged its monitoring and evaluation (M&E) contractor DevTech Systems, Inc., to design an impact evaluation of the set of CSDI interventions and collect the baseline survey data. DevTech worked with Econometría, S.A., to carry out this work.19 The evaluation design included two subsequent rounds of evaluations: a midterm evaluation, and a final evaluation following completion of the program. The current independent midline impact evaluation of the results of CSDI efforts took place after three years of program implementation for the North/South and Central Corridors and four years for Montes de María. USAID charged its Evaluation and Analysis for Learning project (EVAL), a contract with Management Systems International, Inc. (MSI), to carry out the midterm measure of the evaluation and to report on the CELI program impacts to date. This report represents that effort, based on the baseline evaluation design, additional qualitative fieldwork, interviews with the CELI teams, and an extensive survey of participants. EVAL hired a subcontractor, the Centro Nacional de Consultoría (CNC), to collect field data and provide analysis. The impact evaluation was originally linked to the full range of interventions within the framework of the CSDI. Unfortunately, funding for the other programming was curtailed, leaving the CELIs alone as only one component of the full strategy. Given this, it should not be expected that an impact evaluation designed for the larger initiative would detect results of aspects of the CSDI strategy that were never carried out. Evaluation rationale The evaluation took place in the 33 municipalities targeted by USAID and CSDI as well as in 1620 municipalities from three regions (Catatumbo, Cauca/Valle and Putumayo) in which the GOC intervened through PNCRT activities. The midline examined changes since baseline across institutional, social and economic indicators related to the interventions. The key intended audiences of the final evaluation report are the USAID/Colombia Mission Front Office, Technical Offices, and Program Office; and implementing partners at prime contractor and subcontractor levels. The following three principles guide the use of this evaluation: 19 DevTech Systems, Inc. and Econometría S.A. 2013. “Impact Evaluation Methodology of the USAID Strategy in CSDI Zones - Final Methodology Report.” 20 Eighteen municipalities were sampled. Security concerns at baseline eliminated El Tarra (Norte de Santander) and Toribio (Cauca). 30 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 1. Measure the change in impact indicators, and determine the extent to which the CELIs are achieving their goals with target populations. 2. Inform decision-making processes with timely data, to identify components needing strengthening to achieve program goals. 3. Identify best practices and lessons learned to date, to inform future USAID programming. Evaluation questions The 2012 baseline design identified the evaluation questions; these were later amended per USAID request (see Annex 2 for the SOW) and are presented below. Figure 1.2. Evaluation questions QUESTION EVALUATION TYPE STRATEGIC LEVEL 1. What specific impacts have the CSDI interventions produced among the target population in the areas of interventions? Impact IR3.1, 3.2 and 3.3 2. Do CSDI projects show the expected level of performance in support of the consolidation process? Performance Cross-cutting 3. Does the GOC show expected level of investments in consolidation areas following the assistance from USAID in PCNRT municipalities? 21 Performance IR 3.2 4. What are the CELIs' specific effects of in changing the capacity and service delivery of local government? (To serve local people, be accountable, expand social services.) Impact Sub-IR 3.2.1 5. What changes have been produced by CELI interventions to strengthen the organizational capacity of local CSOs in local areas? Impact IR 3.3 6. How much progress has been achieved in the recovery and reconstruction of territory from armed conflicts (IAG presence) and illicit economic activities? Impact Cross-cutting 7. To what extent have CELI interventions contributed to participation, governance and institutionalization of the territory? Impact Sub-IR 3.2.1 8. To what extent have CELI interventions contributed to regional integration and economic development? Impact IR 3.2 IR 3.3 These questions serve both impact (1, 4-8) and performance evaluation (2, 3) goals and the evaluation reflects this blended purpose with a mixed methods design (see section on Methodology and Annex 3 for additional details.) The interdisciplinary team used data from the 21 This question is answered in Annex 6 with data and analysis on GOC investments in PNCRT and CELI municipalities. 31 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative quantitative household survey (in a quasi-experimental pre-post design) and a range of qualitative instruments, secondary data search and multi-faceted analyses to answer these questions in this report. 32 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative EVALUATION DESIGN How to read this report An impact evaluation is a powerful tool but also includes important challenges and differences from other evaluation research. The Limitations section below outlines these, and Annex 3: Methodology contains additional detail. It is important to read the report with these issues in mind, and to understand the different types of findings an evaluation like this generates and how they should be weighted or interpreted. Types of results. There are four types of findings in the report, shown below and in Figure 2.1: 1. Impacts. These come from the survey and are the result of the difference-in-differences design, comparing treated and control groups on key outcomes of interest.22 Impact figures are read as the percentage point (pp) difference between baseline and midline that is attributable to the intervention. Data tables on impacts show statistical significance with asterisks (*) and the title of the table begins with the word Impact. 2. Trends. These figures also come from the survey and they show how one sample changed over time – that is, without the comparison and the calculation of impact. These are shown in graphics rather than tables, and the figure title always begins with the word Trends. These are referred to in the text with the percent sign (%), and in terms of absolute values. 3. CELI performance data. These data come from the CELIs and they track performance indicators linked directly to the CELI interventions. Figures with such data have a title beginning with CELI performance. 4. Secondary data. These data come from other sources like government figures and indices, and they help to describe the context in which CELI interventions operate. Tables with these data have a title that begins with the words Secondary data. Figure 2.1. How to read and interpret the statistical data in the report Table title: In the text: How to read: Impacts pp (percentage point) Relative change over time (compared to control groups) attributable to the intervention Trends % (percentage) Change over time in one subsample Secondary data Secondary data Show context to support findings and/or to explain challenges CELI performance CELI performance CELI progress toward performance goals Whole sample and four sub-samples. From the survey, there is one large sample (19,946 households) divided into four subsamples. These are respondents from CELI municipalities, their 22 This method computes impact on outcome variables, subtracting the estimator (e.g., income) across time (mid-term – baseline) and then between groups. This provides, as a result, the net effect of the program. 33 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative controls, PNCRT municipalities, and their controls. We also discuss contextual conditions, referring to the “full sample,” when results are similar across the entire sample. Direct and indirect beneficiaries. A final clarification regards the direct and indirect beneficiaries of interventions. Due to design challenges (see the Limitations section, below), the midline evaluation could not confirm that those identified as direct beneficiaries at baseline did indeed receive CELI interventions. As a result this evaluation report looks at all residents of veredas in treated municipalities as indirect beneficiaries. There is one exception, and that is in the economic development chapter, where we have been able to identify a population of direct CELI participants, and track their progress as distinct from the other citizens of those municipalities. Methodology and design EVAL and CNC used a mixed methods – qualitative and quantitative – approach to examine a set of indicators for each CELI result. The evaluation sought to understand changes in economic opportunity, social development and civil society, government and institutionalization, and security as a result of the CELI interventions. The design followed the baseline design from 2012-2013 very closely, with additional qualitative methods to gain insights into perceptions, experiences and opinions among treated (CELI and PNCRT) and control groups. The mid-line evaluation was carried out in 33 CELI municipalities and 16 municipalities in three regions (Catatumbo, Cauca / Valle and Putumayo) in which the GoC intervened through PNCRT activities (hereafter referred to as PNCRT municipalities). It is worth mentioning that the results of this evaluation are also presented at the cluster level. Clusters are individual municipalities or small groups of municipalities with similar geographical and socio-economic indicators whose analytical results often differ from regional results. Clusters were integral to this study because they helped to collect more focused data. The evaluation team conducted 19,946 rural household surveys. Survey respondents (who may or may not have been the head of household) were nearly evenly divided between men (51%) and women (49%). When disaggregated by sex of the head of household, an average of 25% of respondents were from female-headed households (FHHs) and 75% from male-headed households (MHHs). There are four groups of households (HHs) in this sample: those in CELI municipalities and their control municipalities (Control 1), and those in PNCRT municipalities and their control municipalities (Control 2). The heads of these HHs, who may or may not have been the person who answered the questionnaire, were 50 years old on average. Figure 2.2 shows the sample by the sex of the HH head, and the total sample size. Figure 2.2. Sample by sex of respondent Male Female Totals Montes de Maria 1607 1264 2871 Central 2732 1915 4647 Norte 2127 2155 4282 Sur 1058 1417 2475 Putumayo 940 983 1923 Catatumbo 1061 863 1924 34 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Male Female Totals Cauca 717 1107 1824 Totals 10242 9704 19946 The ethnicity with which respondents self-identified is shown in Figure 2.3. A plurality of respondents across all four samples self-identified as mestizo. Figure 2.3. Sample by ethnicity The sample was asked a set of questions on conditions of poverty, regarding insufficient housing materials, public services and overcrowding. Insufficient coverage of basic sanitation and water services were the poverty indicators most frequently cited by the four samples, while insufficient walls and overcrowding were the least frequently mentioned. Although the magnitude of these problems is greater among CELI respondents, the proportion of households with each condition is similar in all samples. Figure 2.4 shows how the subsamples compare on these poverty proxies. Figure 2.4. Sample by poverty proxies 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% CELI Control 1 PNCRT Control 2 White Mestizo Indigenous Negro, Afro-Colombian or mulatto Palenquero, Raizal, Gitano, or Rrom 47% 27% 12% 34% 5% 4% 3% 1% 93% 91% 75% 96% 73% 59% 69% 69% 10% 11% 4% 7% CELI Control 1 PNCRT Control 2 Insufficient floors Insufficient walls Insufficient sanitation Insufficient water service Overcrowding 35 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Insufficient sanitation and water service were the most frequently cited poverty markers among the four samples, while insufficient walls and overcrowding were the least common poverty markers. Precise sample amounts are available in Annex 10. Though the magnitude of these problems is greatest among CELI respondents, the proportion of HHs with each condition is similar throughout the samples. At baseline, the veredas in CELI and PNCRT municipalities were classified as low, medium, high or critical security risks. This characterization of the veredas (often called the stoplight rating) was carried out by the GOC and, particularly, its military and Consolidation actors in these regions. At baseline, there were two municipalities that the evaluation never was able to enter, El Tarra in Norte de Santander and Toribio in Cauca, which reduced the number of clusters by two. At midline, after the evaluation team conducted a security review at the vereda level, which showed the following breakdown of the number of veredas surveyed at each levels of security threats: Figure 2.5. Veredas by security threat level Quantitative methods The impact evaluation design included a quantitative survey of households in Consolidation Zones, both with and without CELI interventions. Control municipalities were selected using an advanced matching protocol, described in detail in the annexes to the original baseline report. The original design included an oversample of 12% for each follow-on survey to allow for attrition. Following a panel design, midline evaluation field teams revisited the same households in order to ensure comparability and lost approximately 13% of the sample to attrition. At baseline, the evaluation was designed to capture information from both direct and indirect beneficiaries. Indirect beneficiaries were selected at random from lists provided by local leaders; direct beneficiaries were selected at random from lists provided by the CELIs. The distinction between the direct and indirect beneficiaries was not found in midline data collection (see the discussion of direct beneficiaries in the Limitations section.) However, the evaluation team was able to capture and identify a sub￾sample of probable direct beneficiaries, about whose results we report in the economic development chapter. PNCRT municipalities only had indirect beneficiaries. Timing The timing of the midterm evaluation was first planned for summer, 2014, but was postponed for two concerns. The USAID Mission and EVAL worked together to address these two concerns in order to make best use of evaluation funding. The first concern was that of seasonality. In impact evaluation literature, one of the most troublesome errors is undertaking rural surveys during different seasons. Key indicators about income, farming, sales, access to services and other themes differ greatly by season. If measurements are taken in two different seasons, impacts Low 172 Medium 372 High 530 Critical 212 36 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative (positive or negative) may be incorrectly attributed to program interventions instead of seasonal changes. The baseline was carried out over a variety of seasons over nearly two years’ time.23 The midterm was timed to be compatible with the bulk of data captured at baseline, and to set up a seasonal timeline for the endline as well. In the same discussions, the USAID COR felt that the midline would be more useful nearer to decision-making for follow-on programming, and the new calendar supported that goal. Finally, since the endline could never be on time for follow-on planning, the endline would actually be most useful after the conclusion of the program, to capture whether program results had been sustained. Second, had the midterm evaluation been undertaken in 2014, there would have been just one year between baseline and midline. USAID and EVAL discussed the need for the CELIs to intervene long enough to give the greatest possible program maturity and, therefore, estimations of impact. This was concordant with the function and purpose of the impact evaluation design, which is less about performance and feedback, with a greater emphasis on returning impact estimates. In the end, midterm data collection began at the end of July 2015, and continued through the end of November 2015. It is important to note that local elections were held on October 25, 2015; it is certainly possible that the increasingly politicized atmosphere up to that date would have affected respondents’ opinions, but as they were affected in treatment and control zones alike, it is assumed that the matching design cancels out that influence. Instrumentation The survey instrument includes modules on demographics, justice and security, licit and illicit economic opportunities and performance, government legitimacy, social capital and participation in grassroots organizations, electoral behavior, social and public services, land rights and ownership, victims’ rights, roads, social programs, and opinions about the future and about peace. Several indices are created from the data, including indices of social capital, government capacity, and government accountability. In general, the survey measures perceptions (as in the case of security) which are important but more sensitive to news and recent events. The survey is less adept at capturing actual experiences, in part because most respondents will not have had direct contact with the changes in institutions or security, for example. In this regard, the indicators designed at the time of the baseline survey are weighted toward citizen perceptions. The evaluation added several new items for the midline on satisfaction with interventions and a short set on victimization from the Latin American Public Opinion Project (LAPOP).24 The 23 From the baseline report: “The survey interviews occurred in two phases, with the first phase providing interviews for 65 percent of the indirect beneficiary and control households between March 27 and July 5, 2012. The second phase, providing the remaining 35 percent of the indirect beneficiary and control interviews, plus 100 percent of the direct beneficiary interviews, occurred between November 7, 2012, and May 25, 2013, with a break for the Christmas and New Year holidays.” Impact Evaluation of USAID’s Colombia Strategic Development Initiative (CSDI) Baseline Report. Volume 1: Summary of Findings. June 30, 2013. 24 The Latin American Public Opinion Project (http://www.vanderbilt.edu/lapop/), which has been implemented throughout Latin America by a team at Vanderbilt University for the past three decades. 37 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative survey was administered in person in households, workplaces and, when security prevented deployment, in heads of municipalities. Full details on methods used in this impact evaluation and the calculation of difference-in￾differences is included in Annex 3, including the calculation of differences-in-differences used to detect impact. Other quantitative measures Other quantitative measures included the examination of key secondary data sources. These came from government sources and the EVIDINCE consortium,25 which collects quarterly municipal level data on outcomes of interest, such as police and judicial staff presence, among many others. GOC sources also provided updated data on major crimes including homicides, acts of terrorism, and massacres. The CELI implementing partners also provided data on their achievements to date in their performance management plan (PMP), including such indicators as numbers of organizations and government offices treated, sales of productive project outputs, and leveraged funds. A further source is the Consolidation Index, created in concert with USAID and collected by the GOC since 2011, on a set of economic, social and institutional indicators. The Consolidation Index shows one considered viewpoint of the progress to date in the Consolidation municipalities. This index is built from a set of 37 indicators in the three pillars of the National Policy for Territorial Consolidation and Reconstruction (PNCRT for its Spanish acronym), each of which is in turn made up of a set of components, as shown in Figure 2.6 below. Figure 2.6. Consolidation Index pillars and their components PNCRT pillar Components 1 Territorial institutionalization Security, justice, property rights, and human rights 2 Citizen participation and good government Good government, citizen participation, and social capital 3 Regional integration Connectivity, social services, and economic growth Each pillar is composed of a set of indicators that are reliably available, primarily from government sources – line ministries, defense and police bureaus, departments and municipalities. Indicators in Pillar 1, for example, include indicators on security challenges and incidence of crimes, in particular conflict-related crimes, and indicators on access to justice mechanisms, human rights, and property rights. Pillar 2 includes data from municipal governments on their provision of spaces for citizen participation, their effective tax collection, 25 EVIDINCE is a research effort associated with the Empirical Studies of Conflict (esoc.princeton.edu), a multi-institutional collaboration studying conflict and insurgency at the sub-national level. 38 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative their coverage of citizen oversight committees, the status of local civil society and Juntas de Acción Comunal (JACs), and the Procuraduría General de la Nación’s Open Government Index. Pillar 3 on regional integration includes indicators on roads, electricity and cellular coverage, access to education and health services, hectares under cultivation and the presence of financial entities like banks. Scores on these indicators are summed to create scores for each pillar, and each pillar makes up one-third of the final score on a scale from 0 to 100. Since 2011, data for the Consolidation Index have been compiled into scores for each Consolidation municipality, along with a set of comparison municipalities to which these are matched. The scores are disaggregated by pillar or by component indicators in each pillar, as well as over time, by geography (municipality, department and region), and service provision versus sustainability indicators. The Consolidation Index reports on the overall Consolidation region (disaggregated by municipalities that are PNCRT only, in red below, and those that also include CELI interventions, in blue below) in an aggregate score that has increased, on average, between 10 and 14 points over 100. This positive trend is shown in Figure 2.7: Figure 2.7. Secondary data: Consolidation Index overall scores Source: Consolidation index data, authors’ calculations Both sets of municipalities have improved in the overall Index, as well as by pillars (except in Figure 2.9, Pillar 2 between 2013 and 2014), as shown in the three figures that follow: 53,4 56,2 61,9 62,6 53,7 56,7 64,3 67,6 2011 2012 2013 2014 Total PNCRT CELI 39 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 2.8. Secondary data: Consolidation Index scores for territorial institutionalization Figure 2.9. Secondary data: Consolidation Index scores for good government, citizen participation Figure 2.10. Secondary data: Consolidation Index scores for regional integration It is important to consider that the Index, like any measure, is composed of indicators that have strengths and weaknesses, as well as a particular point of view that differs from the evaluation fieldwork in several ways. First, the indicators in the Index generally come from government perspectives, and in many cases those reporting the raw data are municipal, departmental and national officials. This differs fundamentally from the evaluation’s focus on citizens’ perceptions 43,8 44,3 54,9 61,0 41,9 42,4 49,9 46,9 0,0 20,0 40,0 60,0 80,0 100,0 2011 2012 2013 2014 CELI PNCRT 44,0 53,2 62,8 60,5 47,0 53,9 65,7 58,2 0,0 20,0 40,0 60,0 80,0 100,0 2011 2012 2013 2014 CELI PNCRT 73,1 72,7 75,2 81,4 71,4 72,4 70,2 82,8 0,0 20,0 40,0 60,0 80,0 100,0 2011 2012 2013 2014 CELI PNCRT 40 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative of security, services, participation, etc. Neither one is more “correct” than the other, but they also are not precisely comparable when measuring these same phenomena. Data collection and reporting strategies also differ between the Index and the evaluation data. The basis for many of the Index’s indicators is “principal population centers” – which are determined by those who report the data, including municipal officials or even local transportation companies who report on road conditions. By contrast, the evaluation sample is 79% rural, 18% in population centers other than the municipal capital, and 3% in the municipal capital. A very rural sample like this survey is likely to have very different perspectives than those in urban capitals or principal population centers captured by the Index. Second, the scores on many indicators are weighted to be compared to municipalities that have similar socioeconomic levels, and as a result may be given high marks even when their absolute scores are among the lowest in the nation. For example, in the indicator on open government (using the Colombian government’s own Open Government Index figures), the average 2014 score among ten municipalities in Antioquia is 56.9, but this is calibrated to 87.4 by the comparison with like municipalities. It is important to take into consideration that the scores are altered in this way, before being added together to create the Index. Qualitative methods Qualitative methods included in-depth interviews with key informants at national, regional and local levels, in CELI and PNCRT sites as well as control municipalities. Twenty-one sites were selected across these samples. The sites needed to reflect PNCRT sites without CELI interventions, and municipalities with both PNCRT and CELI interventions, and control municipalities. The sites selected also needed to include all four CELI regions and to be accessible with reasonable travel times and surmountable security challenges. National-level informants were selected for their relationships with CELI or PNCRT interventions to gather an understanding of the interventions’ designs. Evaluations in the field, then, were more about how those designs were implemented. At regional and local levels, the evaluation team interviewed staff at the personerías (Ombudsperson offices), because of the salience of these interventions in the CELI activities, and secretaries of government, to capture a range of other CELI interventions. In each of the 21 visits, these two leaders were interviewed, using a semi-structured instrument that allowed for the comparison of their responses. Also interviewed were heads of civil society organizations and producers’ associations. Focus groups covered themes parallel to the survey and the participants were citizens in CELI and PNCRT zones, some of whom had received or participated in interventions, and others who had not – the qualitative team worked to balance these populations in each site visit in a CELI region, with one focus group of CELI participants and one group of what the design termed “indirect beneficiaries” – those who live in the region but were not directly involved in an intervention. Qualitative data were analyzed in Atlas.ti using thematic coding that mirrored the evaluation questions and allowed for emergent codes to cover issues not foreseen in the evaluation design. These codes were correlated to understand coincidence and patterns of importance to the evaluation’s response to the research questions. All instruments are included in Annex 8 to this report. 41 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Limitations The evaluation design followed in this study has certain limitations. These are grouped and discussed below, and a more detailed discussion can be found in Annex 3, Methodology. Factors related to the evaluation design 1. The CELIs were to be evaluated as part of a much larger and more comprehensive effort that failed to materialize. The outcomes of interest are at a high level of generality in the results framework, relative to the investment, the interventions, and the duration of implementation to date. The evaluation was created to assess a set of programs, formerly called the Colombia Strategic Development Initiative (CSDI), with broader funding and goals than the current set of interventions. CSDI included a whole￾of-government approach from both the U.S. and Colombian governments – anti￾narcotics, security and justice, regional and local institutions, economic development, regional integration and roads, and civil society interventions, across the same geographic intervention area. The Initiative as a whole dwindled to comprise only the Consolidation and Enhanced Livelihood Initiative, run by USAID, with significant funding but without the wider approach initially envisioned. As a result, the impact evaluation design includes several indicator sets for which few or no targeted interventions were completed. Not surprisingly, when interventions were not carried out, no impacts can be detected. For CSDI to have produced detectable effects would have been an enormous feat; for the CELIs to do it without the other CSDI interventions, including the interventions promised by the GOC, was far too much to expect. As a result, impacts found by the evaluation as measured by rigorous statistical techniques are few and mostly at the local level with particular direct interventions with producer associations. 2. Selecting the panel of respondents at baseline was a challenge, resulting in discrepancies in the list of “direct beneficiaries” at midline. A further limitation in terms of determining and attributing impact involves the panel design of the baseline, followed closely at midline. Baseline data collection was carried out over 18 months, crossing various growing seasons and the three different starting dates of the CELIs. In order to respond to the baseline team’s request for lists of direct beneficiaries, CELI Central approached local leaders and called for meetings of potential local participants, who signed attendance lists. CELI N/S, which was already in progress, provided lists of those who were participating in their interventions at that time. These lists served as the sampling frames for the treatment sites, but they created a problem: attendees knew they would receive a program, thus violating the basic assumption of an impact evaluation baseline that the intervention has not yet begun. Knowing about the future benefit changes the population’s perceptions, perspectives, decisions and expectations. To some extent, the changes begin to appear from the time of the meeting invitation, and such changes can no longer be observed when the baseline information is collected, so we do not know how this knowledge may have changed their responses, but the relevant literature suggests a social desirability bias in which respondents who think they are to receive some benefit will respond more positively, in an attempt to be amenable to researchers – who they may presume to be involved with the intervention. This makes it very difficult to detect small effects on certain variables, since the baseline measurement has already incorporated, or lost, such effects. 42 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative The midline evaluation found that some baseline respondents originally identified as direct beneficiaries did not participate (or did not recall or report participating) in CELI interventions. There were significant discrepancies between people who were classified as direct beneficiaries at baseline and the people who eventually participated in CELI programming. The evaluation team therefore recognized three definitions of “direct beneficiaries”: (1) direct beneficiaries per the baseline evaluation design (4,426 households); (2) respondents who answered a survey question about receiving an intervention from Colombia Responde, as the CELI program was known in the field (2,198 households); and (3) direct beneficiaries from the baseline who are on the CELIs’ own participant lists (2,015). Random households in the municipalities were then selected as indirect beneficiaries, to complement the sample and to represent the wide-ranging set of indicators that the CSDI was to have impacted, such as economic integration through roads and markets, security and justice, and increasing and improving government presence. None of the three “direct beneficiary” definitions provides a sufficiently reliable gauge of participation, and as a result, for analysis of the more diffusely applied components of CELI activities in governance, infrastructure support and social development, the indirect and direct beneficiaries of the baseline were combined into one group of beneficiaries. However, for certain activities, especially with respect to participation in producers associations and other economically focused interventions, analysis was conducted on a separate “direct beneficiaries” subgrouping based on the three definitions just specified. When taken together, with no double counting (that is, counting a household only once, for example, when a participant fulfills more than one of the definitions such as by appearing on the original list and by responding “yes” to the survey question on Colombia Responde participation), there are 5,445 unique respondents among the three definitions. The evaluation uses this larger group to examine economic impacts on participants, which represents a more rigorous standard. Where impacts are found on this group, they are generally more positive than the rest of the citizen population. This qualitative team’s experience in the field conducting focus groups helps to understand why so few of those identified as direct beneficiaries actually reported being participants. The first and key questions in the focus groups were regarding participation in Colombia Responde activities, and few focus group participants said they had participated. The reasons for the problems with the Colombia Responde name are not precisely known, but the evaluators speculate that the following could have contributed: i) at GOC request, USAID hid its participation under the Colombia Responde name so that the beneficiaries would attribute the effects of the program to the GOC; ii) subcontractors have other names, and beneficiaries may have been confused about the identity of the benefactor in the first instance; iii) the results may suffer from recall bias, since the baseline was established between 2012 and 2013, and certain benefits ended a long time ago; and iv) a combination of these and other factors that are unobservable. 3. The category of indirect beneficiaries is imprecise, and capturing impacts at that diffuse level will take more investment and more time. Several categories of CELI and PNCRT interventions – essential though they may be – are likely to be invisible to average respondents, and do not reach a minimum treatment threshold, for one of two main reasons. First, some public goods affect few residents of a given area, like the 43 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative strengthening of justice services or work with municipalities or personerías, or some social or productive infrastructure projects. Internationally, only an average 6% of a given population ever accesses justice services in their lifetime, and only a fraction of that figure do so in a given year. Detecting perceptions of change among this “fraction of a fraction” is therefore very difficult. Second, as discussed above, the CELI programming had important goals and benchmarks across the pillars of social, economic and institutional development – such that the significant funding is fragmented across geographic sites and pillars. While a member of a treated productive association or civil society group knows well that they have benefited from an intervention, the “indirect” beneficiary is indeed very distant from the benefits. Their inclusion in the sample dilutes the impacts that can be detected by the evaluation design. Had the robust set of CSDI and GOC whole-of-government interventions continued as planned, perhaps more impacts in these higher-order indicators would have been detected. As it stands, however, the “dose” of intervention is a fraction of what was planned, in far fewer sectors, and at great distance from the beneficiaries, particularly the indirect beneficiaries. Factors outside the control of the interventions 4. Local contexts and the absence of public entities condition the effectiveness of the CELIs. The intervention municipalities are historically economically isolated, with stagnant local markets, very limited road connectivity, armed illegal group activity, minimal (and sometimes corrupt) government institutions, and deep intergenerational poverty. The households in these zones have limited access to land – in terms of either size or low productivity – and a deficit of human capital (education, health, nutrition, as shown in baseline statistics on education coverage, perceptions of service quality, and food security challenges26). The main consequence is low productivity and income from land and work, which are rural families’ two main factors of production. This constrains households in a logic of poverty perpetuation – the poverty trap – in which poverty is reproduced as a consequence of low savings capacity, which prevents changing the initial amounts of capital, land and work. The cycles are reproduced in the short-term, limiting the capacity to change the initial situation, and in the long-term, limiting changes in the welfare of the next generation: young people cannot find opportunities and are pressured into migrating or joining illegal armed groups. In this context – and others related to unemployment and informality – violence, in its different forms, reinforces poverty traps. The other face of this poverty trap context is communal: the municipalities lack access roads, political representation and market formation. Weak political representation often dilutes attention by the central or departmental government, such that the investment in roads is insufficient to develop markets in which small farmers can sell their products 26 See, for example, Impact Evaluation of USAID’s Colombia Strategic Development Initiative (CSDI) Baseline Report Volume 1: Summary of Findings – All PNCRT Zones p. 30 and pp. 42-43; Volume 2 pp. 47-50 and pp. 69-70; plus the regional baseline reports. 44 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative above production and transport costs. As such, poverty is not only individual: it is poverty at a municipal level that encompasses the collective. There is an evident absence of public goods that are essential for the development of communities and entire regions. The capacity of rural interventions to generate significant effects on economic and social development is closely linked to land tenancy and property rights. However, the State’s policy has had little success on this front, and the CELIs faced the same – or worse – institutional roadblocks. 5. Exogenous contextual factors cannot be excluded. Among exogenous factors, 27 it is important to mention the inflation caused by the global economic slowdown and the sharp drop in oil prices, resulting in recession in economic activities associated with extraction. Colombia is the fourth largest oil producer in Latin America, and crude oil is the country’s largest export and source of foreign exchange.28 The drop in oil prices – down 11% between 2015 and 2016 – occurred at the moment that beneficiaries’ perceptions of the performance of the CELIs and the PNCRT were sought. 6. The CELIs’ relationship with the Consolidation Unit was not uniformly strong, based on political and other factors. The Unidad Administrativa para la Consolidación Territorial (UACT) was a new government body as the CELIs began, which benefited from some CELI support for opening offices in the territory and staff capacity building. Being new, the Unit had to establish itself and had less authority than was likely necessary to impel relevant Ministries to undertake work to meet the Unit’s own goals. The national level and regional level offices were not always in synch, according to some stakeholders, and one CELI COR reported better and more collaborative relationships at the regional than at the national level. The UACT was closed and changed to be an office within the Department for Social Prosperity (DPS, for its Spanish initials) in December of 2015. The UACT faced great obstacles in successfully carrying out its role in the territory, including limited budgets for meeting the needs in a complex geographic area with deep social, economic and institutional challenges. The UACT faced the bureaucracy and limitations of a new government agency charged with responsibility for “coordinating” more powerful ministries, but without the authority and power necessary to guarantee that collaboration. It was under these conflicts and complexities that the CELIs had to coordinate their tasks with the national-level GOC (though regional and local contacts were reportedly much more operational.) Now that the UACT has closed, the CELIs are working to articulate their efforts with the changed institutions. 27 The role of the control municipalities is to ensure that exogenous factors do not bias results. Under perfect conditions, exogenous factors apply to both types of municipalities, so in principle these factors would “cancel out.” However, no quasi￾experimental design is ever perfect, and as a result, the evaluation uses control co-variables and clusters in order to minimize possible biases. 28 http://af.reuters.com/article/commoditiesNews/idAFL2N17O0W3. 45 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Factors related to implementation 7. The selection process for veredas in which to work faced challenges that could affect the evaluation results. The selection of veredas is a crucial foundation for implementation and for evaluation. The selection of municipalities and veredas in which the work is ultimately performed has substantial effects on the results of the strategy – and, therefore, on the impacts measured. Implementing partners are guided in selection by USAID and GOC preferences, and the baseline team for this quasi-experimental design followed up by employing a municipality matching system. Such systems, while imperfect, allow for the modeling of a robust counterfactual – what would have happened in municipalities that did not receive the interventions. One hypothesis generated during the evaluation was that the selection – guided by the “traffic light” process – tended toward the veredas closest to the municipal centers, with greater access, more resources, greater associativity, and greater social capital, etc., while the problems of illicit crops and armed groups were extended in more remote and dispersed geographic areas. In general, it can be asserted that the poorer the population, the greater the impacts of an intervention of this type. Thus, focusing on what was easiest operationally may have been counterproductive in terms of impact. At least at the outset of the programs, the selection of veredas depended on the “stoplight” process, which relied on Colombia’s military for impartial classification of the danger of each vereda where the CELIs might potentially intervene. Stakeholders in USAID and the CELIs eventually sought a way out of this requirement, as they found the system highly politicized and, at times, the stoplight decision unjustified. Over time, the CELIs’ contracts were amended to allow implementers to amplify their reach to veredas in which local dynamics suggested potential positive relations – such as a shared productive project. In this way the CELIs could serve communities that were divided administratively but shared common agricultural goals, and could expand to veredas that they themselves had established as safe for project activities. 8. CELI intervention expanded, so that seven municipalities included in the PNCRT sample are now CELI intervention sites. This means that, when the evaluation discusses the PNCRT municipalities, seven have been subject to CELI intervention and may show progress that cannot be attributed to PNCRT, but rather to CELI Central. This is discussed in detail in the PNCRT regional report. 46 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative ECONOMIC DEVELOPMENT Evaluation question: To what extent have CELI interventions contributed to regional integration and economic development? SUMMARY: Nationwide rural development challenges and infrastructure deficiencies continue to affect CELI and PNCRT municipalities deeply, and most respondents in all studied areas report a significant decline in their economic conditions since baseline. Poverty is not only at the household or individual level, but also at the level of communities, as the municipal and departmental conditions (market failings and lack of roads) severely constrain growth. Access to credit remains tenuous across the regions, and land tenancy has made few gains. Income has increased, roughly at the level of (urban) inflation, but survey and focus group data suggest worsening food security. The bright spot amidst this backdrop are direct CELI beneficiaries, who report improvements in economic conditions generally and in a set of key variables particularly. The challenge of sustainably producing and selling products remains. The goal of this chapter is to understand the role of the CELI projects in promoting rural integration and economic development. The information collected was analyzed for the main findings in two priority areas: i) the limitations of the municipality and of the households in terms of developing their economic activities and generating incomes; and ii) the role played by the CELI interventions in supporting and promoting local capacities and eliminating said limitations in a joint effort with the national government. To assess the economic situation, the evaluation team gathered information from a broad range of CELI, PNCRT and control respondents. These included direct and indirect beneficiaries, control groups, and representatives of the communities at large, including personeros, secretaries of the alcaldías, members and leaders of JACs, and civil society leaders, among others. Findings and conclusions Direct CELI beneficiaries Respondents who participated in CELI productive projects showed significant improvements in their economic situation. When asked about whether their economic situation was good or very good, there was a 6 to 10pp positive impact on their responses (Figure 3.1).29 There was also an impact of 6pp fewer respondents saying that their productive activities generated very little money. 29 Please see the Limitations section in this report and Annex 3 for a full description of direct beneficiaries in the sample. 47 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 3.1. Impacts on perceptions of the economic situation, direct beneficiaries Variable Direct CELI beneficiaries Report that the economic situation is good or very good 6 to10pp**30 Their productive activities generate very little money -6pp** The CELIs report incremental or added value for key products that they support in productive projects, as one of their main indicators. In its most recent reports, this incremental value exceeded the goals set by the CELIs substantially and would significantly help the supported associations and productive projects. (Figure 3.2) Figure 3.2. CELI Performance: Increasing sales CELI USD % of the goal Central $1,723,467 121% North-South $16,735,470 358% Source: Quarterly and annual CELI reports; see Annex 5 N.B. El CELI Montes de María did not report on this indicator and it has been closed There were no impacts, positive or negative, on direct beneficiaries’ responses about food security and poverty. Among direct beneficiaries there was a 9pp positive impact when they were asked if they received support for their productive projects. They were significantly more likely than other respondents to associate for buying inputs, processing and selling products, transportation, and training. (Figure 3.3) 30 A range of impact coefficients is offered here, between the three possible definitions of direct beneficiaries. Please see the Limitations section and Annex 3 for a full description of these definitions in the sample. 48 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 3.3. Impacts on reasons they associate with others, direct beneficiaries Variable Direct CELI beneficiaries To buy agricultural inputs 6pp** approx.31 To process their products 7.5pp** approx. To sell their products 10pp** approx. To transport their products to market 9pp** approx. To received shared training or technical assistance 13pp** approx. To have a shared representation as a group 7pp* approx. To present group projects and seek financing 8pp* approx. To seek credit 5pp* approx. Regarding credit and financial services, direct beneficiaries were more likely to report having received credit, at nearly 11pp greater than at baseline, while the full CELI sample reported 7.6pp positive impact on having received credit to support their projects. There was also a significant 8pp increase in responses indicating they had current debt. There was no impact on the amount of credit since baseline. For financial services, there was a positive impact beyond that of the wider sample of CELI respondents in terms of having a bank account. (Figure 3.4) Figure 3.4. Impacts on credit and financial services, direct beneficiaries versus all CELI Variable Direct CELI beneficiaries All CELI beneficiaries Have you gotten credit to support your productive projects? 11pp** approx. 7.6pp** Do you have debt? 8pp** approx. 3.4pp Do you have a checking account? 3.5pp** approx. 2.3pp** Do you have a savings account? 8pp** approx. 0.1pp There were no impacts for direct beneficiaries in having a parcel of land for productive projects, but they did have a higher average number of hectares of land in usufruct with title, at more than double the hectareage found among the wider CELI population. There was a positive impact on their having land in usufruct without title, parallel to that of the other CELI beneficiaries. There was no impact, as with the rest of the CELI sample, on being willing to pay predial taxes. They were also more likely to say they had not paid predial taxes because costs were prohibitive, or because they were currently in a succession process. Fewer say they would not pay predial taxes 31 Figures are approximate for the direct beneficiaries because there are a range of responses based on the definition of direct beneficiaries used. See the Limitations section and Annex 3 for full details. 49 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative because they already paid someone else (Figure 3.5). For both direct beneficiaries and the wider population, the proportion of respondents who say they paid predial tax in the last two years fell by 4pp. Figure 3.5. Impacts on land issues, direct beneficiaries versus all CELI Variable Direct CELI beneficiaries All CELI beneficiaries Average hectares of land in usufruct with title 14 to 17 has** 8.9has** Do you have land in usufruct without title? 2.8pp** approx. 2.2pp** Did you get your land through restitution? 3.5pp* approx. 0.16pp Did you get your land through agrarian reform? 1.5pp* approx. 1.26pp Would you be willing to pay predial taxes? 0.07pp approx. -0.98pp If you have not titled your land, was it because you don’t have resources to do so? 7pp** approx. 5.6pp** If you have not titled your land, was it because you are currently in a succession process? 2.8pp* approx. 2.5pp** If you would not pay predial taxes, is that because you’ve already paid other groups? -2.7pp* approx. -2.7pp* About a third of the direct beneficiaries reported that Colombia Responde had carried out infrastructure projects in their veredas, such as the construction or improvement of roads and schools; one-quarter said that Colombia Responde had improved or constructed parks, cultural spaces or sports centers. Almost one-fifth (between 17 and 20%) of direct beneficiaries reported that the Colombia Responde interventions included improvements or construction of spaces for community meetings. Overall, direct beneficiaries give the CELIs positive grades in their survey responses. First, they were asked to rate the degree to which the projects had met their expectations, on a scale of 1 to 5 (where 1 indicates that expectations for the program were not fulfilled and 5 indicates that they were completely fulfilled), and the average grade given by direct beneficiaries was 3.7. A full 96% of direct beneficiaries affirms that they would recommend participating in a Colombia Responde program to family and friends. Rural economic opportunities: View from the grassroots The remaining sections of this chapter examine the wider populations in CELI, PNCRT and control municipalities: that is, the direct and indirect beneficiaries and their controls. Both quantitative and qualitative data show that the general socioeconomic situation of households in the Consolidation municipalities is difficult and has worsened since baseline. The stories and experiences of focus group participants shed light on the mechanisms that stop people, despite their efforts, from being able to rise out of poverty. These are structural problems that are difficult to change in the short- and medium-term. The income derived from the products cultivated by the households or the daily wages they receive for working the land is a subsistence income, leaving households with nothing to save 50 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 37% 43% 38% 15% 18% 49% 77% 31% 0% 20% 40% 60% 80% 100% The economic situation is good or very good Last week the respondent ran out of food The conditions in which they live are of extreme poverty The family is suffering hunger CELI baseline CELI midline and no way to acquire assets or working capital. On the whole, the households depend on the income of one person, they live in precarious housing, and they do not have access to land: Do you see? We don’t have anything, our house isn’t even finished. If my husband has any [money] … he’ll pay rent [for land]; because, when you don’t have any land to grow things on, then you have to work like this. Even now, you have to work day labor to be able to earn a living, but this is very poorly paid. Do you see what I am saying? From 6 to 4 in the afternoon, and for a week, they pay $125.000. That is not enough for the family. (Producer, Meta, Control) Across the full sample, in control groups and in CELI and PNCRT municipalities, respondents say that their economic situation deteriorated since the baseline. This is true across sites, across various survey questions, and by notable percentage changes (trends) from baseline to midline. Since control respondents show the same trend, no impact of the interventions was found. Perceptions from CELI municipalities are shown in Figure 3.6, which parallel perceptions in PNCRT and control municipalities. Figure 3.6. Trends in economic situation perceptions, CELI municipalities Source: Authors’ calculations. *** At 99% confidence, ** at 95%, * at 90%.32 There was a positive impact of 7.7pp in CELI municipality respondents reporting that their economic situation was good or very good (Figure 3.7). Food security variables showed negative trends (Figure 3.6) but no impact (Figure 3.7), since treatment and control municipalities had similar results. 32 Throughout, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are in Annex 3, and data in the text are included in a table in Annex 10. 51 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 3.7. Impacts on perceptions of economic and food security variables,33 CELI and PNCRT Variable CELI PNCRT Report that the economic situation is good or very good 0.0774** 0.0115 Report going hungry -0.0141 -0.0507 Report that they feared running out of food in the last week for lack of money 0.00338 0.0410 Report running out of food in the last week for lack of money 0.00414 -0.0499 Report living in conditions of extreme poverty -0.0656 0.0766 For CELI municipalities, male heads of household (MHHs) showed a positive impact of 5pp. That is, among these households, the economic situation is perceived as deteriorating, but somewhat less so than in their controls. The households with female heads (FHHs) saw no impact. These pessimistic perceptions seem to be a contextual condition rather than attributable to any intervention. Survey questions regarding the economic situation also focused on serious problems related to unemployment or the lack of opportunities, the profits from productive activities, and incentives for illegal activities. No impact was found on the households’ perceptions of the economic development of their regions, with one exception – that of profit from productive projects. For this variable, the CELI interventions improved the situation by 5.7pp for households in CELI municipalities, specifically FHHs, as compared to households in control municipalities. (Figure 3.8) Figure 3.8. Impacts on perceptions of serious economic problems, CELI and PNCRT Variable CELI PNCRT Unemployment and lack of work opportunities are serious problems -0.0293 -0.0233 Their productive activities generate very little money -0.0571* -0.0185 One of the few attractive productive options is that of illegal activities 0.0326 -0.0612 There were no impacts in the PNCRT municipalities for any of these variables (Figure 3.8). In absolute terms, PNCRT households and controls had the same deterioration for all variables. Positive changes in perceptions were noted in several issues relating to economic development. By sex, the FHHs in CELI municipalities reported a positive impact of 6pp on the perception that 33 These figures estimate project impact compared to control groups and are read as the percentage point (pp) difference between baseline and midline that is attributable to the intervention. In the first variable, for instance, a statistically significant and positive 7.7pp impact was found in the CELI municipalities. Statistical significance is denoted by asterisks, per the previous footnote. 52 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative their productive activities generate little money. For PNCRT, this perception improved even more, falling by 8.5pp in FHHs and 0.5pp in MHHs. In spite of some negative perceptions about the economic situation, qualitative data collection revealed that, when the smallholders were asked about their views on their region and its potential, a different picture emerged. This is an indication of the ownership inhabitants feel regarding their territories, and, in some cases, a community-wide rejection of illegal activities34 in favor of a culture of legality. Ten years ago Costa Rica was all about coca; all over the place, there was even coca in cell phones; a terrible war. Thank God, we live in peace now. We have planted citrus fruits, chocolate, and there is farming, milk and grasslands. (Producer, Meta, CELI) It is magnificent here, especially in the countryside. Everything grows in the countryside, whatever you want to do, to plant, farming, everything. People can enjoy the peace, the climate, everything is perfect. I would say that life is perfect here. (Producer, Meta, CELI) Consolidation Index data on regional integration show gains in economic growth (measured by growth in cultivated hectares, and number of financial entities), increased connectivity (roads, electrical and cell coverage) and positive and stable social service provision (measured in terms of school and health system coverage.) The trends and data points are shown in Figure 3.9. Figure 3.9. Secondary data: Consolidation Index on regional integration, trends 2011-2014 These data show a very positive picture for the Consolidation zones, while survey data show a worsening situation across the wider population. This has much to do with the indicators in question, and their perspective – the Consolidation Index looks at indicators for which it has municipal-level access, and these are not from the more rural citizen perspectives of the survey data. 34 Please see the sections on illicit activities, security, and their contribution to structural poverty, later in the report. 78,5 88,8 85,1 87,1 47,7 61,8 0,0 20,0 40,0 60,0 80,0 100,0 2011 2012 2013 2014 Connectivity Social services Economic growth 53 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative The complex question of household income Estimating household income among rural small-scale agricultural producers has long been recognized as difficult and imprecise. 35 Nevertheless, CELI projects are required to report on income estimates for their PMP, based on data collected in the baseline and in the current midline research. Estimated monthly expenditures are used as a proxy for income, because of the challenges of collecting income data. Reported household expenditures have increased in absolute terms since the baseline. However, the increases are similar in the control group municipalities, so it is not possible to attribute these increases directly to CELI and PNCRT interventions. The increases are relatively parallel to the inflation rate36 in three of four CELI regions, so while the increases are positive (Figure 3.10), they are not likely to have significantly changed respondents’ living conditions relative to household expenses. This is supported by their greater reports of food insecurity in response to other questions (Figure 3.6, above). The exception is in Montes de María, where a larger leap in reported monthly expenses is recorded (66%). Still, the increase is parallel in control zones, and cannot therefore be attributed to the CELI interventions. The expenditure by region and the percentage change are reported in Annex 5. The CELIs met their targets for this indicator. Disaggregating by sex shows no difference between FHHs and MHHs. 35 The survey asks about the monthly expenditure instead of the income, which has been shown to be unreliable in household surveys. This phenomenon has been documented for decades in developed countries and across much of Latin America. Rural surveys tend to present the worst performance in correctly capturing family incomes, for a number of reasons: income is received from various sources; households receive payment in kind; they are paid irregularly and their income changes according to the season; if there are any illegal payments, they are not likely to be reliably reported, etc. Estimations show that the reported expenditure is in some cases greater than the income reported in the same survey (Rogers and Gray 1994; Jencks 1997; Sabelhaus and Groen 2000); de Meyer, Bruce D., and James X. Sullivan. (2003). Measuring the wellbeing of the poor using income and consumption. National Bureau of Economic Research Working Paper Series; Working Paper 9760 accessed February 2, 2014 at http://www.nber.org/papers/w9760, p. 18. Expenditure has been considered more precise because there is less reason to under￾report. However, in order to obtain good estimates, very detailed instruments are required, adding 20 or 30 minutes to an interview. It was decided not to take that type of measurement, but rather to continue to use the measure established in the baseline, to maintain comparability and to minimize the burden of the instrument. See, for example, Pettersson, Hans. (2005). Chapter XXIV: Survey design and sample design in household budget surveys, in Household Sample Surveys in Developing and Transition Countries Department of Economic and Social Affairs Statistics Division, Studies in Methods Series F No. 96 al http://unstats.un.org/unsd/pubs/gesgrid.asp?id=349. 36 The inflation rate is based on urban data at the national level, and so is not precise enough for close comparisons. But, in general terms, the increased reported expenditures across CELI regions do not exceed that rate of year-on-year inflation, or do not do so by much, with the exception of the Montes de María region (as this section goes on to explain). 54 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 3.10. Trends in reported monthly household expenditure, CELI and PNCRT Regional expenditure changes over time do look positive, though they are not statistically significant relative to the control municipalities. (Figure 3.11) Figure 3.11. CELI Performance: Estimated monthly household expenditure, total and regional Ind. Description Region Baseline Midline % Change Source DO3- 005 Avg monthly household income of USAID's beneficiaries37 Total 393,93638 504,106 28% Household survey • Baseline 2013, Econometría S.A. • Midline 2015, Proyecto EVAL Montes de María 402,032 667.645 66% Central 451,601 498.240 10% Norte 334,978 374.798 12% Sur 426,725 510.188 20% Conclusion 1: The perception of the economic situation in CELI and PNCRT municipalities has worsened, but the trend is the same in control municipalities. This trend is certainly related to a series of factors out of the reach of any intervention and that constitute the poverty trap. Despite the difficulties throughout the Consolidation zones, it is not uncommon to find communities’ positive values with respect to their regions. This is an indication of the ownership inhabitants feel with respect to their territories and productive projects. In some cases, they also demonstrate community level rejection of illicit activities in favor of a culture of legality. However, in sum, the results indicate that the economic situation in CELI and PNCRT regions parallels that across the country, with the hardships and needs of the inhabitants of remote rural zones who survive from day to day and face difficulties even in terms of basic food security. 37 Figures are for average or mean household expenditures, as the evaluation team judged expenditures in the rural households surveyed to be a more reliable measure than reported income. 38 Figures used for the baseline differ slightly, as we are now combining indirect and direct beneficiaries (BI and BD, respectively, for their Spanish abbreviation used at baseline). Percent change is calculated on the combined BI + BD figures. 55 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Conclusion 2: Direct CELI beneficiaries are an important exception to this negative perception. The contrast with the rest of the samples is noteworthy and is statistically equivalent between households headed by men and women. Also among CELI households, there are positive trends in the perception of the problems they face, such as the lack of opportunities and the low income provided by their productive activities. The female-headed households form an important part of this optimism. Assistance for productive projects CELI interventions aim to support sustainable and licit income generation. The CELIs made great investments to support smallholder associations, 39 to improve the land, to provide technical assistance and inputs for production, and to support commercialization. In Antioquia, for example, the greatest investments were made to support rubber, cocoa and coffee production; in Córdoba, to support cocoa, plantain, rubber and papaya; and in Tumaco mainly for cacao, coconut and shrimp. CELI Central reports beginning the program with these types of primary inputs, and then focusing more on facilitating government collaboration to involving the state in these projects. They focused on quality, better prices for products, organizational development and particular project support, such as cold chain development in Meta. The CELI worked in concert with Regional Autonomous Corporations (Corporaciones Regionales Autónomas, or CAR), mayors’ and governors’ offices, and the productive project partners. Together they worked to develop shared plans of action to establish necessary infrastructure and processes to help productive projects succeed. The CELIs provided technical assistance (TA) in efforts to improve the productivity and quality of specific crops. In Tolima, for instance, CELI Central has worked with coffee producer associations, “firstly, in offering technical assistance in knowledge transfer and capacity building for producers attempting to fulfill international quality standards; secondly, in the donation of supplies and materials such as pedagogical materials for training; and thirdly, through a process of support for management, negotiation, and training in the services offered by the association.”40 In La Macarena, the project brought together members of the producers association in La Cristalina de Lozada for an “Agricultural Best Practices” course, during which they covered the issues of insemination, vaccination, rotation, pasture capacity, hygiene, quality in milk management, and commercialization.”41 All three CELI projects have emphasized organizational training and capacity building for producer associations. For instance, towards the end of the MdM project, a CELI partner operator helped strengthen ten producer associations through training in accounting, organizational, tax and business topics, and office management.42 In CELI N/S, the team reported throughout FY 2015 that it continued to improve the institutional capacity of producer 39 A section on producers associations can be found in the chapter on social development. 40 CELI (Central) Annual Report FY2015, p. 52. 41 Ibid., p. 46. 42 MdM Quarterly Report FY2015 Q1, p. 20. 56 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative associations to provide effective services to their members and their communities. In Q3 2015, for example, they described supporting eight associations producing cocoa and three that had sold shrimp, rice, and honey.43 In Tolima, CELI Central was joined by the local government and the private sector to provide training to 30 producer associations working with cocoa and coffee to strengthen both their technical expertise and their organizational capacity.44 On the whole, each of the CELI intervention components was directed at counteracting the deficiencies or weaknesses that the producers faced in their regions: TA, for example, aimed to counteract low productivity and provide products with greater added value. Similarly, productive projects were scaffolded with support to their associations and with agreements with local governments to facilitate local market conditions. Producer associations were also supported in their negotiation power with the buyers or intermediaries. The people and communities interviewed in the CELI municipalities positively recognize the impact of the support received in agricultural inputs, credit, courses and TA. Their range of products has been extended, providing a real alternative to illicit activities. Household economies have been transformed at the local level, and their parcels are more valuable. There is qualitative evidence of the positive changes in the regional economies, through the productive projects supported by CELI programs and, it appears, other interventions:45 You can strengthen your stable, stockyard, your cattle. They don’t give us cash but it is a great help. The cocoa growers, there are 32 of them, they have been given… fertilizer and cocoa seeds… and lime… fungicide for the mixture… and a fumigating machine… (Producer, Meta, CELI) Initially we grew chili peppers, eggplant and rice… thanks to the different programs… we are selling our own brand of rice now. (Producer, Antioquia, CELI) In PNCRT municipalities, 11pp more households (almost all male-headed) reported that they received support for their productive projects, when compared to the control municipalities. In CELI municipalities, the sample did not report impacts in this period. Globally – including treatment and control sites – respondents report a decrease in support for productive projects. Families who do receive help – particularly in plant material, courses, or training – more frequently report a practical component to the support, which coincides with better practices for adult learning. In focus groups, respondents told stories about CELI learning and support for productive projects. The following text describes the process in one CELI N/S case: I think that the training has helped us a lot; I think that they help society change its way of thinking. Before, we didn’t know how to establish a crop; with these projects we have learned to establish rubber and cocoa as crops. So where we didn’t know how to graft a plant for it to be 43 CELI N/S Quarterly Report FY2015 Q3, p. 5. 44 CELI Central Annual Report FY2015, p. 11. 45 USAID readers of this report should note that fertilizer and seeds were not part of the CELI interventions. There were likely other programs also intervening in the same place, and that beneficiaries do not always distinguish between them. 57 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative more productive, now we have the possibility to have the technical methods, the technical guidelines for us in the fields… now we can improve our parcels of land by our own means, with citrus plants, cocoa, with all these products. So all their training workshops have given us the knowhow that has served to enrich our social and cultural level. We have enjoyed this part, it has been useful, and I think it is the most important thing… the most important thing is that our vereda has a different perspective of what the countryside can be. (Producer association leader, Antioquia, CELI) Female-headed households reported several positive impacts. There was a 10pp positive impact on FHH responses that they had received credit for their productive projects in CELI municipalities. Similarly, FHHs in the CELI and PNCRT municipalities reported important and positive impacts in associations for processing, sales and transport of their products (for more details, see the chapter on social development and producer associations). On the other hand, FHHs reported receiving fewer courses or trainings in CELI municipalities (18pp less), and 30pp less in PNCRT municipalities. With respect to CELI productive projects’ support for long-term crops, in focus groups some producers talked about their subsistence problems while they waited for such crops to produce: I don’t think it has been great [the intervention] because what Colombia Responde has given us is not producing yet; they are long-term [crops]. So we still have to work and save from what we have to invest in this to sustain it. (Producer, Caquetá, CELI) This crop [rubber] should be done differently; for example, it should be grown with another crop that produces sooner than rubber so that we can subsist from the other crop, we could live off of the other crop. It is very [difficult] to sustain rubber… [T]hey should have planted it with another crop like they did for the cocoa growers, they cultivated cocoa at the same time as plantain, timber products, they planted three products in one. (Producer, Caquetá, CELI) Conclusion 3: The productive projects in CELI programs showed evidence of economic changes in the households. The beneficiaries appreciate the TA that accompanies the CELI programs and other state institutions (e.g., Servicio Nacional de Aprendizaje, National Learning Service, or SENA), as well as the support for assets and capital that they will be able to enjoy in the long￾term. Conclusion 4: CELI support goes directly to the strengthening of value that will mature in the medium- or long-term. Households with these crops, even before maturity, now have greater financial worth because of the production potential they have developed. The goal, along with improved institutional support, was to create long-term value that helps producers provide for themselves through their productive projects. The purpose of these instruments is to strengthen savings and investment, as they oblige the beneficiaries to work and look after their crops with a long-term perspective, with all the costs and sacrifices that this implies. However, respondents did express concerns that they are currently experiencing short- and medium-term food insecurity. Markets and trade The main objective of rural development is the development of markets in a broad sense; that is, spaces where the transaction of products continues to expand. Small and underdeveloped markets have high transaction costs and communities become trapped in a vicious cycle in which small markets do not encourage greater production and, simultaneously, the increased production that exceeds the size of their markets collides with rigid local markets and with the lack of 58 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative preparedness for exports to other regions. Some focus group participants – in both treated and control municipalities – explain the serious problems smallholders face in the absence of trading mechanisms: If we do not transcend this part of the transformation, of giving added value, of trade, we continue being the same. This is so not only with rubber but any product. (Producer, Antioquia, CELI) They could give us all the crops you want, but if we don’t have anywhere to sell them, what can we do? There is a big gap as far as this goes. (Producer, Antioquia, CELI) We plant our crops, but there is no one to sell them… the big problem we have here is one of trade. (Producer, Putumayo, Control) In the survey, a series of items enquired about the problems experienced in production and marketing products. There was a positive impact on one of these questions in CELI and PNCRT municipalities (Figure 3.12); that is, negative perceptions diminished over time: Figure 3.12. Impacts on perceptions of problems in production and sales, CELI and PNCRT Variable CELI PNCRT Abuse by intermediaries -0.0395 -0.15 Very low prices for the products -0.0485* -0.0911* Lack of water -0.0736 0.0952 The impacts listed in the table represent significant reductions in the perception of the problem of low prices for their agricultural goods among the respondents, relative to control sites. Though impacts were only found for these three questions, the trend over time was a reduction in CELI, PNCRT and control municipalities across a range of such problems, though these did not always show significant impact. (Figure 3.13) Figure 3.13. Trends in perceptions of problems in production and sales, CELI and PNCRT 59 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative In contrast, respondents in PNCRT sites had a greater perception of the problems of high input costs, IAG interference, soil quality, and the availability of water. For FHHs in particular, there was an important reduction in the perception that market prices are low for their products, in both CELI (9pp) and PNCRT (16pp) municipalities. This was the only significant difference between FHHs and MHHs, but it is another piece of evidence showing that the perceptions and expectations are not always parallel between male and female household heads. CELI also conducted notable work with the private sector to improve value chains. CELI Central reported that rubber in Caquetá and coffee in Sur del Tolima and Norte del Cauca provide important lessons in establishing win-win arrangements between producers and the market. In Caquetá, CELI forged an alliance between 1,100 rubber producers from 16 municipal committees and an international trader in order to establish procedures for quality certification and export. A coffee alliance linked nine producer associations with services to market their product, conduct tasting training, strengthen organizational capacity in high-standard purchasing, and improve access to niche export markets. These activities with the private sector extend to other value chains, regions and new markets, helping producers become partners in the process with added value, services and better, more stable incomes.46 The markets in the regions continue to be local and regional in most cases. Markets that export goods to other regions are small in terms of their sales, with few exceptions. The survey indicates that there were no great impacts in the places where the interviewees sold their products. In the CELI municipalities, people reported selling more from their farms, in the closest populated center, to cooperatives or trade associations, in the municipal center, in near-by veredas, to the general public, or in fairs and the market square. They reported selling less outside the municipality, though CELI respondents were 7.7pp more likely to have sold to intermediaries or wholesalers than at baseline, compared to control municipalities. For the PNCRT municipalities, it was parallel and just as mixed. These trends were the same or stronger in the control municipalities, so the statistical impact for most of the cases was null. In sum, the data indicate that local markets are the most important for smallholders. The proportion of sales in the markets located in municipal centers, nearest population centers, and neighboring veredas, combined, was from 83 to 90% in the CELI municipalities, and from 73 to 89% in PNCRT municipalities. Sales to wholesalers and intermediaries declined, but sales to cooperatives do not compensate for this reduction (which was expected with certain types of associational activity). Conclusion 5: TA and marketing support help counteract problems related to low profitability. However, trading such products requires markets with enough buyers and sellers and the establishment of prices that are higher than production costs. The lack of markets in the Consolidation zones seriously hampers the tasks of the CELI and of the GOC. Similarly, the productive projects counteract the incentives to plant illicit crops, but with the deficient and 46 Data from conversation and e-mail with CELI Central COR, 30 June and 27 July, 2016. 60 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative partially depressed markets, pressure from armed groups and the lack of access roads (see the section on roads later in this chapter), it is difficult for the policies and programs to have the desired effects. It could be concluded that in the presence of structural problems – which are the GOC’s direct responsibility – it is difficult for CELI and PNCRT programs to produce effects that are sustainable over time. Conclusion 6: The markets in consolidation territories continue to be local and regional markets, and export markets are small in terms of the proportion of sales. As such spaces are limited, the producers tend to be affected by the high costs of trading that go along with the deficiencies typical of small markets. It is a circular relationship: small markets do not encourage greater production and, in contrast, greater production that exceeds the incentives of these markets clash with the rigidity of the local market and the lack of facility to export to other regions. Credit, savings and financial services Access to financial services and credit to strengthen the productive and licit value chains has been an intermediate goal of the three CELIs. They have created savings and credit groups, developed revolving funds within producer associations, and offered TA to associations to expand their skills in the use of financial information and tools. For example, CELI Central reports activities with funding that reached USD$608,000 to support 14 producer associations in Tolima, Caquetá, Meta, and Cauca whose goal is to improve financial services, from the improved management of revolving funds to the use of financial tools to improve accountability and information management. CELI Central reached agreements with Colombian financial institutions (Bancolombia, Banco Agrario, Banca de las Oportunidades, and Colpensiones) to expand mobile banking, create more than 100 savings schemes, and support and train 12 associations to access FINGARO loans for agriculture and farming.47 CELI MdM worked with the microfinance institution CREZCAMOS to deliver 7,352 microcredit loans, 11,000 micro-insurance policies, and almost USD$9 million to the local economy to fund productive activities. Offices were set up in each of the project municipalities, and 116 local savings and loans groups (grupos locales de ahorros y crédito, or GLACs), to provide credit to their members.48 CELI N/S reports that it has spent over half of its total investment or USD$32 million49 on economic development activities, which include rural financial services, strengthening producer associations and other local associations, and value chain support. 50 Throughout the project life cycle, more than 20,000 men and women have been mobilized in the 16 target municipalities to participate in GLACs sponsored by the Banca de las Oportunidades in an effort to reduce poverty. A 2013 case study on the growth of these schemes in Bajo Cauca showed how the veredas used the shared resources to address community needs such as buildings, pedestrian 47 CELI-Central Annual Report, September 2014 – September 2015, p. 12. 48 Colombia Responde Quarterly Report July-September 2014, p. 26. 49 CELI N/S FY2016 Quarterly Report (Q1) p. 5. 50 CELI N/S Quarterly Report (Q2, FY2015), p. 21. 61 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative bridges or ventures such as drugstores or bakeries. 51 In 2015, CELI N/S expanded the GLAC program to help its members use the savings to generate incomes and create other small ventures such as micro-franchises for small goods.52 The CELIs have also worked to improve access to credit and bank use among their beneficiaries. In terms of bank use, they have helped approximately 52,000 people to gain access to financial products. Figure 3.14 shows their performance compared with the goals for this indicator: Figure 3.14. CELI Performance: New use of financial services CELI Number % of the Goal Montes de María 7.201 142% Central 7.927 104% North-South 36.689 141% Source: Quarterly and annual CELI reports; see Annex 5 They have also worked with productive projects in accessing credit and by supporting their associative efforts. Many associations have a revolving fund that offers smallholders the possibility to obtain loans to deal with disasters (the conditions and rules of the loans are negotiated within each association), serving as a strategy for microcredits and micro-insurance: Today, we have a revolving fund of around 60 million pesos; because we decided to use a rotating system for all the inputs that arrive [and] harvests that we sell. We take out the inputs and we are very clear about the fact that there is a small profit margin, allowing our resources to continue to grow. So it has helped consolidation, especially organizational. (Producer association leader, Antioquia, CELI) The possibility to access some type of credit is an important benefit for the households, given that their low savings and investment capacity and their lack of assets53 stops rural households from being able to access services in the traditional financial sector. Banco Agrario gives loans. I am going to give you an example in case you don’t know; the Banco Agrario loans you money, it loans money to the man with a plot of land; but it can’t lend me money. Why? Because I don’t have any land. (Producer, Antioquia, Control) Formal banks are starting to appear in some but not all territories, and there are areas they have not yet reached. There were reports in Meta of people having to walk for four hours to reach a municipal center where there is a bank. However, informal access to credit is also present outside the formal banking system. In terms of marketing, credits can be linked to trading contracts or product delivery. The financial cost is hidden in the commercial transaction. This type of credit is based on trust between the parties and, as such, it is very common in rural areas. This type of credit can also be more formal, for example, through the middleman that buys the products: 51 Transforming Lives: Rapidly Growing Community Banking Initiative Is About Much More Than Money 52 CELI N/S Quarterly Report (Q2, FY2015), p. 27. 53 Explained by the previously described poverty rate. 62 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative In my vereda, the man has a car and he travels around in it every day, he takes the produce and he lends you 10,000, 20,000, whatever you need [to pay] per harvest, in one installment. (Producer, Montes de María, CELI) Given the lack of access to credit and family income levels, the population is obliged to resort to informal or illegal loans, also known as gota a gota. Debtors have to pay usurious interest rates which expose rural households to risk. Paying [monthly] interest rates of 10% and 20%... apart from the fact that this is illegal money, because most are illegal resources and they lend with usurious interest rates. They also attack the person, or they threaten him, if they don’t pay the interest. (Functionary, Antioquia, CELI) CELI interventions did not directly provide credit, though some participating associations created rotating funds to support their members. HHs in CELI municipalities infrequently cite credit as part of their productive projects, and that frequency fell since baseline: in CELI municipalities, the proportion of households reporting that the help they received was in the form of credit fell from 23% to 18%. This decline was much less, however, than in control municipalities, for an impact of 10pp for the CELI population; in particular, this difference seems to have been felt most in female-headed HHs (10pp). In terms of other financial services, the survey results show a reduction in the use of savings accounts of around 10% across the full sample. Checking accounts, which comprise only a small fraction of banking services, did see a small increase. When disaggregating impacts in CELI municipalities by sex, there is a reduction of 8pp in FHHs, while in PNCRT municipalities, the reduction for this same group is of 12pp. Across the full sample, payments for public services through banks or non-bank branches have increased. Both in CELI municipalities and in PNCRT, requests for credits in banks, cooperatives, and NGOs have increased; however, FHHs in CELI municipalities report a significant decrease of 4pp. There was also a decrease in the frequency of credit across the full sample and a drop in the average amount of household debt (almost 6 million COP in CELI municipalities, and almost 10 million COP in PNCRT municipalities). As mentioned earlier, the revolving funds have proven to be a valuable instrument for small groups of neighbors and acquaintances. The implementation costs are low and there are important economic benefits to the group. As explained by a beneficiary, a common type of revolving fund is the farming fund: Our fund is of 168 cows… We give cows to people who can feed and sustain them. For example, if someone says they want no more than five animals, then that is what they will get. If they give him 5 newly calved cows, in four years, he delivers 5 new, newly calved cows and the 5 that we gave him he keeps; and the calves these cows have in these five years... and the heifers are branded with the name of the association… they are the ones that will grow and be given back, while the bulls are settled for, 50-50. So if in five years you are going to deliver the cattle… and there are at least 15, right? Because the original cows’ heifers have given birth and there are 15 animals; so we take out the 5 that belong to the association, the five that were given to the man, and is there are any left, they split them 50-50. (Producer, Meta, CELI) However, the revolving funds are not sustainable unless they are well regulated and internally supervised. Given that the seed capital almost always comes from support programs (a subsidy), the beneficiaries sometimes claim that they don’t have to pay such support back, and thus 63 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative endanger its sustainability. It is for this reason that, as already mentioned, the instrument is very well suited to small groups54 in which members trust each other. Conclusion 7: According to the household survey, credit fell in both CELI and PNCRT municipalities. However, the information gathered in focus groups indicates that financial support has fulfilled a vital function for the development of savings and credit. Revolving savings and credit funds have served as important instruments to support capitalization in rural areas. Conclusion 8: Informal credits contained in trading or illegal gota a gota loans will always exist unless financial services progress in rural areas. Theoretically, illegal credit problems can be resolved judicially, but the informal modalities arise when the broader economy fails to provide viable alternatives. Community credit or loans which are a product of programs such as CELI provide an alternative to informality. Conclusion 9: Bank formalization has not increased; in fact, it is moving backwards across the full sample. However, households are increasing their use of credit, through cooperatives or NGOs. The negative impacts in FHHs in terms of accessing credit is a matter of concern. Transportation infrastructure The Consolidation zones lack access roads that connect rural residents to the country’s most dynamic and populated markets.55 Despite high marks for connectivity in the Consolidation Index (see Figure 3.9, above), the Ministry of the Interior data convey a different story about coverage and quality. The baseline survey data are telling as well: only 16% of the households belonging to the CELI municipalities access their houses from a paved road (in the control municipalities, this value was 32%); 56% from dirt roads, 20% from footpaths, and 8% from rivers. The lack of reliable transportation infrastructure to reach the veredas and municipalities means that local producers cannot access traditional markets with a stable supply and demand that can guarantee prices that are higher than their costs (including that of transporting their products). Not only does this result in higher production costs for private and public goods, but it also facilitates and conceals illegal activities. When the State arrives to implement roads in the regions, this is well received by the inhabitants because it favors the competitiveness of producers’ activities and it improves quality of life for rural inhabitants. There are lots of veredas that don’t even have horse trails; you have to go on the river or along footpaths. (Producer, Putumayo, PNCRT) In my case, I have to walk for two hours just to get to the nearest road, and unless it is on the back of an animal, there is no way to move anything. (Producer, Tolima, CELI) 54 See chapter on social development, regarding producer associations and communities. 55 Data from the Ministry of Transport. In departments in which CELI works, coverage is minimal (8-16% of the area), there is limited paved area (<7%), and poor quality (1-13% in “good” state according to the Ministry’s classification). 64 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative If we have good roads, the products [can be sold]. (Producer, Caquetá, CELI) All the counterparts have come to establish crops, not for infrastructure… now that we have our products, we need roads to get them out. (Producer association leader, Antioquia, CELI) The quality of road connectivity is directly related to incomes; the higher the transport and trading costs, the lower the producers’ earnings: With 60 liters of latex a day, he earns $75.000 and he is paying his helper $25.000. He is left with $50 [thousand], but then he has to pay to manage all this: he has to have it transported on an animal for 4 kilometers, 5 kilometers to get it to the road edge. (Producer association leader, Antioquia, CELI) To build or repair roads was not one of CELI’s central axes, but they have acted in this sector and such activities have increased in recent years, in a comprehensive approach to resolve bottlenecks to commerce. For example, CELI Central designed an activity that sought sustainability within a framework that linked civil society with public entities. Support committees were set up to follow up on the maintenance of the improved roads. CELI helped to legally formalize the JACs so that they could publish invitations to tender and contract the work. The inhabitants of the vereda often contributed by providing labor.56 At last report, CELI Central spent US$1.5 million57 and leveraged three times this amount from local governments. The goal was to rehabilitate 113 kilometers of road, but rather than focus on kilometers repaired, projects were chosen in which targeted investments would make a big difference. For example, in an area of Meta, residents used a market road that had four areas that became impassable in certain seasons. The project did not replace the entire road, but rather, with help from the residents, found sustainable fixes for the four critical points that were identified. Now the road remains passable year-round, at a much lower cost, and the residents themselves have the tools to maintain it. Although this way of working on specific repairs does not please the construction contractors – who tend to prefer large projects – it is an efficient manner of using the funds for a common goal. Across the full sample, survey respondents recognize a slight improvement in the roads inside their municipalities and a relative standstill in the situation relating to inter-municipal roads. Some 34% of PNCRT households consider that the condition of the inter-municipal roads is very good, and 23% considers that the inter-vereda roads are good. The situation has improved more in control municipalities than CELI municipalities. The survey also reveals perceptions that transport costs have gone down across the full sample. Even so, more than half of survey respondents (58% in CELI, 56% in PNCRT, and more than 52% in the controls) indicate that transport costs are a serious problem for production and trade. Where we grow things on the farm, it takes two hours to get the fruit out on an animal, sometimes the animal can hardly get through because it is in bad condition… in the high part. (Producer, Meta, Control) 56 Source: In-depth interviews with CELI teams and their managers at USAID. 57 Ibid. 65 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative At the same time, existing roads are not often well-maintained. Some evidence surfaced in qualitative interviews about corrupt activities with regard to road maintenance: Two years ago a compacting roller went in in the morning and came out in the afternoon. And what did they say? The road has been compacted. But it hadn’t been. It was a waste of money, the roller went in at 6am and came out at midday and the only thing that mattered was to send a photo to the local government. (Producer, Antioquia, CELI) The missed opportunities are clearly expressed by this agricultural worker in Caquetá: In this area, very little is grown to sell… in the area we live in, everyone grows crops that they are going to use… Because we lack roads, we cannot sell our product, milk, which is what we sell the most, at a good price. (Producer, Caquetá, CELI) Conclusion 10: The data show very negative perceptions and experiences in terms of the quality of the roads network. The deficiencies of local markets are tightly related to the state of the roads. The productive projects can attempt to break poverty traps, but the lack of roads reinforces the traps. It is obvious that the task of CELI is complementary to government actions, but it is not a substitute: the infrastructural problems must be solved by the State, and what CELIs do without this urgent GOC work may end up being a short-term palliative. Conclusion 11: When there are roads and infrastructure, the population recognizes the direct benefit of CELI interventions. Local level users, associations and JACs have taken ownership of roads projects. Land An important element in the CELI vision is support to land formalization. This includes helping the rural residents ensure formal titling, supporting displaced families and communities to return to lands they had to abandon as a result of the armed conflict, and support governmental institutions who, ultimately, grant land tenure rights. This work began prior to the Mission’s Land and Rural Development Program activities. When that program came on line in March, 2014, the CELI work had built a foundation. For example, CELI Central began work in 2013 to formalize lots for public use, such as schools and health clinics. LRDP picked up on that to develop similar and complementary activities in some regions of CELI Central coverage. For example, in Tolima in 2013, CELI supported the agriculture ministry’s administrative work which LRDP has continued into a juridical phase.58 The main obstacle has been weak state institutions in this sector without clear and consistent processes. El Instituto Colombiano de Desarrollo Rural (INCODER) proved unable to fulfill the mandates of several attempts at land reform and was relatively impotent in the face of political processes resulting from extreme land concentration. The challenges in restoring land tenure rights or compensating the victims are great. The relatively positive data in the Consolidation Index show an increase in scores (over 100) for property rights indicators (whether the 58 Data from conversation and e-mail with CELI Central COR, 30 June and 27 July, 2016. 66 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative municipality has been “microfocalized,” the first step in land restitution; and progress in municipal zoning) from 26 points in 2011 to 46 points in 2014. But these data do not reflect titling challenges, land concentration, or progress on restitution once the government begins the process with microfocalization. The CELIs maintained records pertaining to restitution cases and the formalization of property ownership. The CELIs have reached or exceeded their goals in most, but not all, cases. 59 The CELIs have supported State institutions involved in the land claims, despite the fact that the latter posed considerable challenges at times. 60 At the beginning of its project, CELI MdM found hundreds of abandoned properties in its municipalities due to displaced communities and the many problems of informal titling and a lack of understanding in terms of ownership rights. CELI already worked closely with the relevant agencies, prior to the adoption of the Victims and Land Restitution Law, in 2011. CELI MdM launched the “Historical Analysis of Land Rights” Project61 to document the connection between the campesinos and the land, particularly victims of violence and displacement. This study served as a basis for subsequent work in terms of creating methods, checklists and templates to trace a map of the multiple routes that the communities and the families would have to follow to obtain legalization and compensation.62 In FY2013, CELI N/S sponsored the establishment of a local land titling office in alliance with the mayor’s office in El Bagre, Bajo Cauca. Within a short time span, the office “received 100 applications from campesinos for unregistered derelict land or unoccupied land owned by the State requested before INCODER, as well as 11 land titling applications for the public use of derelict land.”63 In the same period, this CELI helped the Unidad de Restitución de Tierras (URT) in Valdivia to process 627 cases in the Register of Confiscated and Abandoned Land,64 and, in Tumaco, it supported the updating of the property register and the establishment of a land committee to supervise the formalization and regulation of property titles.65 CELI Central also worked on strengthening the URT’s institutional capacity in the areas in which it works by contracting 17 arbitration experts in 2014 to support restitution cases. 66 CELI MdM similarly provided support in the positioning of the URT in its focus municipalities, granting funds for personnel, office support and technical assistance to set up offices in Sincelejo and Carmen de Bolívar.67 CELI Central also reported the experience of providing TA to INCODER from 2012 to 2014, in topographic surveys and cadastral appraisals. INCODER then 59 CELI N/S “The goal of 2600 formalized properties supported in the CELI municipalities has not been reached,” and there are only six months of the project left. For details on PMP indicator goal attainment, please see Annex 5. 60 For example, in 2012 CELI N/S reported difficulties it faced reaching cooperation agreements with any State agencies involved in land tenure issues (CELI N/S, Quarter IV and Annual Reports 2012, Chemonics, Inc., p. 13). 61 Análisis Histórico de Derechos sobre la Tierra. 62 Colombia Responde MdM Final Report, Global Communities, April 2015, pp. 22-26. 63 CELI N/S, Quarter IV Report, July-Sept 2013, Chemonics, Inc., p. 18. 64 Registro de Tierras Incautadas y Abandonadas. 65 Ibid. 66 CELI Central October-December 2014 Quarterly Report, Tetra Tech ARD, p. 44. 67 Colombia Responde MdM Final Report, Global Communities, April 2015, p. 26. 67 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative bought 107 lots totaling 8,267 hectares which they distributed to indigenous groups in the Consejo Regional Indígena del Cauca (52 lots), the Autoridades Indígenas de Colombia (2 lots), and the Asociación de Cabildos Indígenas del Norte del Cauca (1 lot), as well as to campesinos (36 lots) and Afro-Colombians (16 lots). The effort was designed to resolve land-related conflicts among these groups in Cauca.68 Beyond institutional support, the CELIs have provided assistance to communities and organizations in specific cases of the legalization of property titles, and on supporting families in building their cases to request restitution. They supported civil society organizations (JACs, producer associations, and women’s organizations) to help people understand the often complex steps involved in formally registering their land and guaranteeing their titles. For example, in 2014, CELI N/S mobilized the Universidad Católica del Norte to create a specialized training program with 61 participants from seven producer associations who received 120 hours of training in the norms and rights involved in land ownership and how to gather the necessary documents to present to the authorities to obtain property titles and registers.69 CELI Central worked with producer associations in the north of Cauca and southern Valle, where the campesinos frequently only have informal titles, to carry out formalization workshops on the legal processes involved in restitution and titling. The participants visited priority parcels to collect information, and these cases are already being considered.70 At baseline, 24% of HHs in CELI municipalities had titled lands (averaging 5 hectares) and 44% said they had land but no title. In 2015, 42% of respondents in CELI municipalities said they had titled land, and 40% said they had land but not title. In control municipalities the same dynamic appeared, with 35% said they had titled land at baseline, and 57% at midline. As the trend was the same in control and treated municipalities, there is no impact attributed to any intervention. Land ownership is informal across CELI, PNCRT and control municipalities. The priority problem continues to be that of legalizing the acquired rights through succession or the land market (sale or purchase); such lands represent up to 85% in CELI and 94% in PNCRT municipalities, indicating that the problem is not one of access but rather one of formalization of titles. There is a predominant absence of formal deeds and land tenure relies on precarious titles, as highlighted by members of the focus groups: Very few people have deeds and most only have a sales letter… like 40% have deeds, or 50-50 more or less. (Producer, Meta, Control) Generally the people don’t have formal titles here. So much so that the municipal administration has lands that still aren’t in the municipality’s name. (Functionary, Antioquia, Control) 68 Data from conversation and e-mail with CELI Central COR, 30 June and 27 July, 2016, referencing their activity LND-02385- 12 - APOYO AL PROCESO PREVIO DE ADQUISICIÓN DE TIERRAS POR PARTE DEL INCODER EN CAUCA. 69 CELI N/S Annual Report 2015, AID-514-C-12-00001, Chemonics, Inc., p. 11. 70 CELI (Central) October-December 2015 Quarterly Report, AID514-C-11-00002, Tetra Tech ARD, p. 26. 68 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative If you don’t own land you are nobody. You go to a bank and the first thing they will ask you for is the deed. No, no, I have a sales letter. Can’t be done. (Producer, Tolima, CELI) Without land titling, how is anyone [going to do] a productive project? (Producer, Meta, CELI) However, as can be seen from the above, no strong counterpart to this has existed, given that INCODER was not titling at the required pace. In addition, since the beginning of 2016, INCODER has been replaced by two new agencies which are only beginning to function. Thus, the problem of supporting land titling is rooted in the government agencies rather than in the interventions. The beneficiaries themselves are aware of the efforts made by CELI, but they also express their desire for the process to be improved: Titles are formalized… some deeds maybe. Not [as much as] we aspired to. But we have moved forward. At least the seed has been planted. (Producer association leader, Tolima, CELI) Here we really haven’t had the required support from the corresponding entity, INCODER. There are very few parcels with titles in the Macarena, which makes many interventions avoid the area, because having title is a requirement. So, yes, we need more help on this. (CSO leader, Meta, CELI) We made progress with Colombia Responde and Consolidación. Work was carried out in 2012... [but] the program could have been better. (Producer, Tolima, CELI) Conclusion 12: The informality in ownership rights sheds light on the limitations of the State. The communities deal with this problem through precarious ownership rights, based on fragile documents such as sales letters. The absence of formal ownership titles renders such documents invalid for use in guaranteeing credit, and discourages development of productive projects in the long-term. The advances in the titling procedures in the CELI and PNCRT municipalities continue to be limited in the face of the huge scale of the problem. Although the CELIs can support such processes, it is ultimately a matter for the State to resolve. Recommendations 1. Maintain realistic expectations at a level that the operators can affect (conclusions 1-3, 6, 10 and 12) Impact indicators are at a high level of generality in the results framework, relative to the investment and interventions, the short duration of implementation, and the problematic contexts. Given that the economic situation reflects regional trends and the regions’ critical structural deficiencies, it is important to not consider lack of impact as representing a failure in programming. 2. Make simultaneous improvements in production and trade (conclusions 2-3, 5-7, and 10) Increased production must go hand-in-hand with market improvements. Low quality, intermittent and seasonal production is not attractive for a market that exports goods to the rest of the country. Some CELI activities striking alliances between the private sector and producers’ associations have made important qualitative gains for both sides, and should be emulated. In contrast, good quality and regular production has better results when that production is directed at adequate markets. Transport to markets is never guaranteed. The GOC must endeavor to resolve market and infrastructure inadequacies and establish programs that improve relationships with commercial allies and intermediaries. This is and must be one of the most important concerns for post-conflict policies in Colombia. While international funding is certainly important, it may be wise for the latter to lead small focused pilots that treat these interconnected 69 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative structural failings holistically, or to support GOC efforts that do so, rather than dilute the effects by intervening across broad geographic areas with expansive mandates. 3. Support an array of markets (conclusions 5, 6 and 8) It is important to support commercialization projects designed to improve both regional markets and projects that export goods to other regions, which may provide a solution to local overproduction and low prices. In such cases, specific programs are needed that intentionally bring small-scale producers and intermediaries together in the productive process to work with broader marketing efforts to improve quality or to involve banks to increase access to working capital. The intermediaries, if necessary, can constitute the starting point for a good system of trade in the regions given that they fulfill an economic function that would be difficult to replace. It is important, however, for the intermediary not to become monopsonistic in a way that reduces smallholders’ bargaining power. Intermediaries risk their capital in money and equipment and come up against insufficient quality and quantity in terms of what they buy from the producers. As such, specific programs have to be structured to understand their function and improve it to benefit the regions. Better market information adjusted to local conditions would contribute to improving the communication of supply and demand, and to bridging the cost difference produced by intermediation. Regional market information systems should involve provincial centers or the Ministry of Agriculture and Rural Development’s (MADR) Unidades Municipales de Asistencia Técnica Agropecuaria, or UMATA. Mobile phone access to up-to-date market information is easy to implement and can function well, as seen in mobile data improvements in other countries. 4. Plan for intermediate food security in productive projects (conclusion 4) It is recommended that CELI and USAID make concerted efforts to ensure that programs supporting productive projects also support food security. This situation should be anticipated by providing TA and traditional farm or other sustenance activities that minimize dependency. The productive projects must combine short- to long-term returns in order to ensure cash flow and project sustainability at the household level. 5. Reach the FHHs with interventions that work (conclusions 2 and 9, and the relevant section in the chapter on social development) Overall, the benefits have not been the same for FHHs and MHHs. Given that the two household types learn and use their learning in different ways, models should be developed that reflect this difference and provide different services to take advantage of the characteristics and behavior of each. For example, the FHHs attend training workshops and courses to a lesser extent, but they report taking greater advantage of associations for different productive ends and for marketing. 6. Maintain activities to expand financial services (conclusions 7-9) Revolving funds are a suitable instrument to deal with the communities’ borrowing needs but they must be protected against the risk of non-payment. While communities have access to other programs such as MADR’s Productive Alliances program, the communities should be supported in strengthening these revolving funds in the long-term. Credit decreased globally, according to the household survey. However, according to focus group interviews, such forms of support have fulfilled a vital function for the development of savings and credit schemes, and they are important as long as they are conditioned or linked to the creation of collective savings and credit funds that 70 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative maintain strong principles of governance to guarantee their sustainability. Such a framework avoids diluting resources by creating dependency. Steps to undertake would include creating a system that monitors and follows-up the communal funds or revolving funds; sets up discussion forums and exchanges of experiences; creates incentives or awards for performance; and draws attention to successful cases. 7. Continue focused work on roads (conclusions 5, 10 and 11) Although the improvement of the roads system is the responsibility of the central and local governments, the CELI programs should continue to reinforce the successful support they provide to the JACs pertaining to institutional strengthening and community infrastructure. In particular, it should be a participatory community-based model for the management of the tertiary road network, in line with international experience, taking into account the national and environmental contexts. 8. Continue to expand the work in terms of land tenure formalization (conclusion 12) It is a priority to facilitate relationships with the new state organs that replaced INCODER in order to continue work on land tenure as part of USAID’s ongoing portfolio. To supplement current information, the evaluation team recommends a transparent monitoring system to focus attention on the issue. USAID should work with the GOC on their database of petitions, assessing the data over time and comparing regional performance. The database should register the location, the number of hectares, produce management and resolution rates. A corollary issue with land restitution is that, in the event of an eventual peace accord and a greater number of successfully resolved land restitution processes, conflict over given parcels may increase as new or changed IAG fight for local dominance. It will be important to consider these kinds of conflicts in connection with the security recommendations in that chapter. 71 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative SOCIAL DEVELOPMENT Evaluation question: What changes have been produced by CELI interventions to strengthen the organizational capacity of local CSOs in local areas? SUMMARY: The CELIs documented important increases in organizational competency scores – especially in business initiatives and services – and surpassed program targets. From citizen perspectives, however, there were no increases in the social capital index and a slight statistically significant decrease in the trust component, led by a greater drop in FHHs. Trust in state institutions fell across the board except for the personería in survey results, though qualitative data challenged the depth of this decline. Participation in the JACs declined, but participants who stayed played a greater role in decision￾making. Surprisingly, the survey revealed an apparent decline in CELI municipalities’ involvement with producers associations. In both CELI and control municipalities, respondents attested to a dramatic increase in productive capacities. Participation in women´s organizations and victims’ associations increased, and ethnically oriented cultural associations were found to strengthen a sense of identity and build social capital. Strengthening civil society and social development is considered fundamental to the consolidation of conflict-affected areas in Colombia. Social development is understood as the population’s participation in civil society, trust between peers, and trust between citizens and the state. It may also be understood as the degree of trust characterizing the relations between social actors, the prevailing norms of civic behavior, and the level of association activity (Kliksberg, 1999, p. 85). To complement this position, Kliksberg cites James Coleman’s theory (1990) that social capital is manifested individually and collectively, explaining his position in the following terms: The first of these is connected with an individual’s degree of social integration and his network of social contacts: it implies reliable relations, expectations of reciprocity and forms of behavior, and improves effectiveness at the private level. It is also a collective good, however. Thus, for example, if everyone in a particular neighborhood shares tacit standards of non-aggression and concern for others, children will be able to walk to school in perfect safety and social capital will be producing public order (Kliksberg, 1999, pp. 85-86). Social development thereby acts to strengthen trust between peers, which may be transformed into processes of cooperation and associativity depending on local characteristics. The CELI interventions are intended to develop and strengthen social capital in target populations as a tool for territorial consolidation. It is made up, therefore, of …[t]he aggregate of the actual or potential resources which are linked to possession of a durable network of more or less institutionalized relationships of mutual acquaintance and recognition (…) The volume of the social capital possessed by a given agent thus depends on the size of the network of connections he can effectively mobilize and on the volume of the capital (economic, cultural or symbolic) possessed in his own right by each of those to whom he is connected (Bourdieu, 2011, 221). Three approaches are used to examine the impact of interventions using the CELI model to strengthen the capacity of civil society organizations (CSOs). The first approach used a tool drawn from the CELI toolkit: the Índice de competencias organizacionales, or Index of Organizational Competencies (ICO) or the ICO-VEO (Index of Organizational Competencies- 72 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Verification of Organizational Status).71 The CELIs used these tools during implementation to assess the organizations before and after implementation, to understand the changes that occurred over time as a result of their activities. The indices are composed of variables that capture organizational capacities, weighted for their perceived importance. The indices provide information on significant changes related to the CELI interventions. The second approach calculates impacts relative to control areas, based on the survey questions. In the survey, social capital was measured using different batteries of questions, combined to create a social capital index. This index examines two aspects: trust between peers and between citizens and the state; and participation in social organizations. The index allows an analysis of impact. The third approach used in the analysis of social capital involves the subjective experiences of citizens, some of whom were members of CSOs in the areas of intervention. Qualitative interviews and focus groups were carried out in 21 municipalities. The instruments used to capture these perceptions and experiences included topics related to direct participation in social organizations and productive associations, and in particular worked to capture changes in social capital resulting from participation in interventions, the reasons for which organizations had been established, the challenges they faced, their achievements and the effects they may have had on the population. One final, external source is the Consolidation Index data on participation and social capital. The former is measured by the operativity of certain municipal spaces for participation, the percentage of municipal funds subject to legally constituted citizen oversight mechanisms, and an electoral risk index. Social capital is measured in the Index by the degree to which Juntas de Acción Comunal and local social organizations have registered themselves annually. Figure 4.1. Secondary data: Consolidation Index scores for social development Social capital has had a net gain of less than one point, while participation as defined by the Index has improved significantly. It is important to note that one of the indicators (electoral risk) 71 CELI interventions employ variations of the instrument according to their needs and priorities. In this report, the abbreviation ICO is used to refer to all variations, as the indicator is shared by all the CELI interventions and has that name. Details on the CELIs’ attainment of their goals in this and other indicators are included at Annex 5. 30,4 59,7 58,5 60,8 0,0 20,0 40,0 60,0 80,0 2011 2012 2013 2014 Participation Social capital 73 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative used by the Consolidation Index has not changed since 2011; therefore that data point is null. Overall, the municipal perspective, represented by the Index, is somewhat positive on social development. There are, at times, differences between the qualitative and quantitative findings presented below. These emerge from the different sources consulted in the field using each method. The qualitative work was carried out with an intentional sample, chosen for their participation in producer organizations, their status as conflict victims, or their functions in working with civil society or cultural organizations. By contrast, the survey results represent an average of the impacts across the quantitative sample, chosen at random within veredas. Findings and conclusions The Index of Organizational Competencies (ICO) CELI interventions are intended to create concrete improvements in the competencies identified as essential to organizations’ effective functioning. The CELIs created and refined tools to monitor changes in these organizational competencies: internal organizational democracy, service provision, financial development, management, administration and human development. This package of variables generated a scoring system for each of these capacities that were then weighted and added together to obtain a maximum score of 100 points. This score was a key indicator for the CELIs’ performance measurement in social development, and all of the CELIs met or surpassed their goals. The CELIs used the resulting indices to grade the producer organizations, establishing a baseline that enabled the organizations to measure their subsequent evolution. The application of the tool was participatory insofar as the leaders of the organizations were invited to express their opinions on the achievements of their organizations in each category. Although there were no comparison data available for control organizations, it is a useful tool for analyzing progress against a defined set of indicators. CELI staff met their contractual obligations by using these indices to measure the progress of the actions vis-à-vis the organizations. Consultants or members of each regional CELI applied the ICO in a participatory manner that involved the leaders and members of the organizations in order to ensure high levels of representativeness. This process also reduced the subjectivity and bias that might otherwise be associated with the process. The tool serves not only to measure the capacity of the associations but can also be used to develop an action plan for each organization that takes into account their strengths and weaknesses in each subset of variables. CELI Central reported to USAID in 201672 on the results and average changes in performance during the interventions for a total of 52 organizations. On average, there was an approximate 72 ICO-VEO scores (for CELI N/S) and ICO scores (for CELI Central) will be updated at the end of Fiscal Year 2016, in October, 2016. These later scores will show progress in the current fiscal year. 74 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 50% increase in the ICO. (Figure 4.2)73 Figure 4.2. CELI Performance: Average change in ICO score, CELI Central Source: Operational data, CELI Central The scores may be disaggregated by battery of competencies, as in the case of the scores for democracy and management. Unfortunately, this information was not available in digital form at the time the evaluation was conducted for this CELI. The ICO-VEO is based on information concerning democracy and participation, management and administration, business initiatives and services, and assets and human development. 74 CELI N/S reported that it had worked with 89 organizations in 2016 with similar scores and levels of growth. (Figure 4.3) Figure 4.3. CELI Performance: Average change in ICO-VEO scores, CELI Norte/Sur Source: Operational data, CELI N/S The data include scores for individual competency sets. The overall point increase is therefore divided into the following categories, as shown in the following figure: 73 The ICO score equals 100, so the scale used in the whole of this section and in the figures runs from 0 to 100. 74 Taken from the VEO document: Una herramienta de diagnóstico para el fortalecimiento de organizaciones de productores, n.d., USAID, Perú. 41,1 60,6 19,5 0 20 40 60 80 Start End Change 43,2 67,8 24,5 0 10 20 30 40 50 60 70 80 Start End Change 75 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 4.4. CELI Performance: Average changes by ICO-VEO category, CELI N/S The most notable increase for this CELI was in the category of business initiatives and services (essential for sustainability), for which the organizations covered by the initiative scored an average of almost 8 points. This CELI’s ICO uses a diagram of a house75 to visualize the scores, alongside a weighting that assigned a higher score to this component (25% of the total). The situation was similar for the human development component, which showed the second highest increase, while the components of assets and management were emphasized less in the weighting protocol and also showed lower levels of increase for the period examined. However, all scores were positive, representing an increase of almost 25% over the course of the interventions. CELI N/S also reported that the visualization of a house aided organization participants to take ownership of the instrument, and that the process of supporting organizations to develop the tool was highly participatory: “we looked at organizational development from their perspectives… and left installed capacity on how to use it. It’s not just that we adapted it, but how we went about it,” according to Chief of Party José Félix Montoya.76 With the software and the other tools made available to the leaders, it is hoped that these groups will continue constructing their organizational “house”. One organization, Asocaval, supported by CELI Norte, has become adept with the instrument to the point where the government of Antioquia has contracted them to support other producer associations in the appropriation of the instrument. Although the measure used by CELI MdM was not available to the evaluation team (because the project closed before the evaluation began), the progress figures appear in its reports and demonstrate a similar positive trend. Working with 128 organizations during the life of the project, the reports state an average change of 54 points up to September 2012, of 76 points up to 75 For an explanation of the diagram see “Una herramienta de diagnóstico para el fortalecimiento de las organizaciones de productores,” Verificación del Estado Organizacional. 76 In a meeting on August 30 with the CELI N/S team and Chief of Party José Félix Montoya at their headquarters, translated by the authors from the Spanish conversation. 76 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative September 2013, and of 105 points to the end of December 2014.77 Some respondents noted the diagnostic utility of the index for identifying strengths and weaknesses in the organizations and for encouraging members to appropriate proposed changes. Another respondent noted that the ICO prioritizes the examination of the internal workings of organizations, while emphasis could more valuably be placed on the way the organization acts within its environment (services, revolving funds, commercial relations, and/or credit unions). This respondent felt the latter perspective would be more useful in generating concrete actions or improving income, focusing as it does on how the organization should position itself in relation to the market and sell its products. Conclusion 1: This approach provides evidence of significant increases in organizations’ competencies. An outward focus on markets and services appears to be important to sustainability, balanced with support to the organizations’ internal management and other features. The methodology employed by CELI N/S appears to have encouraged organizations’ taking ownership of the tool. Social capital: Trust The social capital index is composed survey questions comprising trust (60 points) and participation (40 points – see the next section of the report), for a total of 100 points. Trust was identified at different levels: family, neighbors, friends, the local Community Action Council (JAC), legal and state oversight bodies, development institutions, local and national government, and the police and military. As is apparent, many of the factors affecting the index scores are beyond the remit of cooperation programs such as CELI and should be considered more as indicators of contextual conditions when employed so soon after the construction of the baseline. Nevertheless, analysis of program level subcomponents provides an increased understanding of program strengths and weaknesses. In general terms, the social capital index did not register any impact either in the CELI or in the PNCRT municipalities, but the percent changes are mostly positive. The percent changes are shown in Figure 4.4 for CELI municipalities, and the index impacts in the table that follows. (Figure 4.5) Figure 4.5. CELI Performance: Social Capital Index, total and regional Ind. Description Regions Baseline Midline % Change Source DO3- 024 Social Capital Index Total 22.8 23.63 4% Household survey Montes de María 20.6 22.60 10% • Baseline 2013, 77 Quarterly reports dated July-Sept 2012, July-Sept 2013 and Oct-Dec 2014 made available by USAID for the purposes of the evaluation. As the Project has ended it is not possible to inquire how the average came to exceed 100 points, but it is assumed that the score was aggregated over the years of the intervention. 77 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Ind. Description Regions Baseline Midline % Change Source Central 23.8 24.33 2% Econometría S.A. • Midline 2015, Proyecto EVAL Norte 23.8 24.92 5% Sur 22.94 20.82 -9% The sub-indices of trust, on the one hand, and participation, on the other, did demonstrate impacts (see the following section for a detailed examination of the participation component). For the component of trust, there was a negative impact of 1pp in CELI municipalities, while in PNCRT municipalities no statistically significant effect was found. (Figure 4.6) Figure 4.6. Impacts on the Social Capital Index, CELI and PNCRT Variable CELI PNCRT Social Capital Index -0.24 -1.16 Trust component -0.99** -1.00 Participation component 0.74** -0.16 Source: Authors’ calculations. *** At 99% confidence, ** at 95%, * at 90%.78 Sex-disaggregated analysis shows that households with female heads of household (FHHs) fell more than households with male heads (MHHs) (by one point out of 60, at a significant level). Viewed by region, in CELI Central, municipalities showed a significant increase of more than 1pp, while in the other regions there was no impact on the index. The trust sub-index was constructed in part from three survey questions, shown in Figure 4.7. With one exception, the data do not register any impact either for the PNCRT or the CELI. Figure 4.7. Impacts on trust between peers, CELI and PNCRT Variable CELI PNCRT High level of trust that a neighbor who promises to help them would do so -0.0356 -0.0185 The tradition of working collectively on community projects exists here -0.0111 -0.092* High level of trust in members of the community -0.0131 -0.0514 There was no significant impact on trust in the CELI municipalities; in general, an absolute decrease appears for these three questions (which also occurred in the control municipalities). There was a negative impact of 9pp in the PNCRT municipalities. For FHHs, the impacts were negative for the question on the tradition of collective work in community projects (-11.5pp for PNCRT municipalities and -6.5pp for CELI municipalities). In PNCRT, these households also 78 Throughout, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are in Annex 3, and data in the text are tabled in Annex 10. 78 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative responded that they did not have much trust in other members of the community (-7pp impact). The trust index also included a battery of questions on how much trust respondents have in state institutions, from the national level to the local. Uniformly, across the 25 institutions that the survey asks about, trust worsened across the full sample (CELI, PNCRT, and their controls). The effect was substantial for all trends at national, departmental and municipal level and for the JACs and social organizations, the legal and control bodies and the educational, agricultural and security institutions. As the effect is generalized, it cannot be attributed to any intervention; rather, citizen perceptions have deteriorated notably, likely affected by sustained, external events (absence of state presence, lack of services, scandals, etc.). One contrast was found in CELI zones, where perceptions of the offices of the personería municipal (Ombudsperson offices) fell less than in the controls (amounting to a positive and significant intervention impact). This is confirmed by the results from other survey questions (Figure 4.8), in which CELI municipalities show positive impact. Results are consistent across the range of survey questions: nearly 5pp of positive impact are found in CELI municipalities relative to controls when respondents are asked about their trust in the personería, 17pp positive impact in respondents’ rating of the personería, and 10pp positive impact when asked about change over the last two years. Figure 4.8. Impacts on opinions of the personería; Trends by percent change Treatment Control Baseline Midline Baseline Midline Impact I have a high level of trust in the personería CELI 28% 22% 28% 17% 0.0465** PNCRT 21% 14% 33% 28% 0.0616 The personería is good or very good CELI 45% 52% 46% 41% 0.172** PNCRT 38% 39% 52% 53% 0.0312 Services at the personería have improved over the last two years CELI 23% 29% 17% 13% 0.101* PNCRT 13% 18% 20% 28% 0.00594 The positive results are concentrated in the CELI Central region, where the positive impact was almost 12pp on improvements in the personería in the last two years, and over 7pp in the proportion of households that rate the personería as good or very good. Other CELIs report not having focused on direct involvement with the personería offices, or having provided support that was more logistical or equipment-based. Those zones show lesser or no impacts on respondents’ perceptions of the personería. While the survey shows no impact on the levels of trust felt by members of the population towards their peers or their government, the qualitative fieldwork showed a different reality. Save some exceptions, trust in the institutions is a constant among the qualitative respondents in focus groups and interviews. They reported that trust and solidarity do exist between neighbors 79 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative and that this has facilitated social groups. In regions where armed groups operate, the trust between peers not only improves the quality of life but also permits the creation of associations and strengthening of the JACs. This finding is transversal inasmuch as it permits all kinds of associativity, whether productive associations, victims’ organizations, cultural or youth groups, or others. In my vereda, I feel good about my community. As I’ve said, I’m the President of the Council (JAC) and when I need to organize a meeting people collaborate; we look out for each other and, I don’t know, I feel at home here in my vereda. (JAC leader, Antioquia, CELI) As pretty much all of us know each other (in the vereda), well, there is more access, right? Because if there’s an event or something, well there it is, and everyone knows each other, so it’s easier there. (CSO member, Cauca, PNCRT) The municipal personerías, secretaries of government and leaders of social organizations say that trust in social and community processes has increased as a result of increased security levels in the regions, enabling people to associate freely without fear of reprisal from the armed groups and the strengthening of associations of victims formed to demand their rights. This contradicts the survey data but likely reflects their experience working with organization members, who tend to participate more and show more trust. In terms of the security situation, and its relation to social capital in Montes de María, it is suggested that the improved levels of security in the region have permitted the emergence of social organizations. When conditions of violence in a region represent risks for the inhabitants, social processes dwindle. An interviewee from Carmen de Bolívar, a CELI municipality, said: Leadership was silenced. They killed the leaders... (When) the conflict was beginning to decline a little, they (re)emerged. (CSO leader, Bolívar, CELI) As suggested in the field, when the security conditions began to change different social processes began to emerge, because there were fewer risks associated with the leadership of community organizations. Similarly, the survey shows a positive impact on population perceptions in CELI municipalities. This contrasts with the impact in the municipalities in which only the PNCRT intervened, where there were no impacts registered. The proportion of HHs that felt that police security services had improved over the previous two years increased by 8.5pp relative to controls, while the households that considered the military security services had improved over the same period increased 8pp compared with their control municipalities. (Figure 4.9)79 Figure 4.9. Impacts on perceptions of the police and army, CELI and PNCRT Variable CELI PNCRT Police security services improved last two years 0.0851** -0.0149 Military security services improved last two years 0.0813** 0.0385 79 The increase is mitigated by overall perceptions of the security situation and of security forces. There was no impact on whether the security situation had improved in the veredas or corregimientos. Nor was there impact on police ratings, nor on the percentage that would recommend that a family member had left the zone should return. 80 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Despite the recognition that the services provided by the police and the army had improved, a range of conflicts is resolved within the community, as the legitimacy of the JACs and the conflict resolution tools they employ lead many people to resort to the JACS for conflict resolution, rather than to higher-level institutions such as the municipal personería or the police. Thus, the impact in the PNCRT municipalities on households that report that they approach other members of the community when dealing with serious problems with a neighbor is an increase of 2pp while there was no impact in CELI municipalities. (Figure 4.10) Figure 4.10. Impacts on resolving conflicts in the community, CELI and PNCRT Variable CELI PNCRT Facing serious problems with a neighbor, they approach other members of the community 0.00424 0.0203** This information indicates that the CELI interventions and the PNCRT have, to different degrees, permitted community ties and resolution tools for minor conflicts to deepen within the community, contributing, for some, to increased trust. Alongside qualitative reports of increased faith in JACs and social institutions, some respondents revealed low levels of trust for municipal institutions. 80 In the CELI municipalities no impact was found on perceptions of the performance of the mayor’s office while in PNCRT municipalities, positive perceptions of mayor’s office performance dropped 6.5pp. (Figure 4.11) Figure 4.11. Impacts on perceptions of mayor’s office performance, CELI and PNCRT Variable CELI PNCRT Mayors’ office performance is good or very good -0.0187 -0.0653* The importance of the JACs to the population resides in the fact that these are the mechanisms through which the population organizes itself to carry out the tasks that are not performed by the State. As the inhabitants of the veredas in one of the PNCRT municipalities expressed it, they feel obliged to organize themselves and work together in order to maintain and repair the roads. In our vereda it’s like this: members have the right… in our case we all have to meet together and if a road has to be repaired then we get together and between us we all contribute, but if someone is not a member, they don’t contribute (JAC member, Putumayo, PNCRT) Survey results showed that 4.4pp more respondents in CELI municipalities trusted their JACs a lot or quite a lot, compared to baseline and controls; however, the qualitative data showed even greater increased trust in the JACs. The greater increased trust among qualitative responses may be explained by the fact that the quantitative information represents the average opinion of the 80 Please see more detail on this in the Governance and institutions chapter. 81 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative population while the qualitative information is a purposively selected sample of active, participating people. In any case, the value assigned to the JACs is related to the fact that for the population they represent access to training, productive projects and other initiatives that the government authorities or international cooperation agencies may develop. The JACs represent the gateway to many state services and projects, as they are the closest point of contact the population has with official institutions. This position is clear from the following testimony: The Juntas de Acción Comunal have been a fundamental axis of this community. It is an honor to be the president of a Junta de Acción Comunal and they are involved in all the training events, they create Councils, there is training on human rights... (JAC member, Meta, CELI) Conclusion 2: The survey data show a marked decline in the level of trust that citizens have in the organs of government, in all the municipalities surveyed during the study, and for all the government bodies included in the survey. This effect is not attributable either to the CELI or to the PNCRT interventions because it is common to the control sites as well. Among the more actively participating community members who were interviewed in qualitative data collection, there are reports of positive change and improved relationships between citizens and the State. However, the latter group of respondents is much smaller. The survey data also show a decline in trust levels between peers and neighbors, especially in FHHs. These figures provide a strong contrast with the experiences narrated by the participants during the qualitative work, who had higher levels of participation and much more positive opinions of the social fabric – but, again, these are active participants and so, by design, they are more likely to perceive and take part in the construction of the social fabric. Conclusion 3: In zones where the security conditions have improved it is also possible that levels of trust between neighbors have, to a degree, improved. Not only does this increase the quality of life but also permits the creation of new forms of association and the use and strengthening of existing organizations, such as the JACs. Social capital: Participation in associations The CELI strategy encompasses three levels: i) the strengthening of social capital in the regions it covers, ii) the generation of relations of trust between neighbors and iii) the promotion of cooperative relations within communities. Working towards this objective, the CELIs have supported and strengthened different forms of association through a range of interventions focused on existing grass roots organizations such as the JACs, which are the prime instance of participation in public life in the municipalities. Equally, productive associations have been created or strengthened in an effort to construct alternatives for communities affected by this phenomenon. Another focus of intervention was the support provided to victims’ associations with the intention of generating collective processes to ensure their rights are respected under the terms of Law 1448. 81 Finally, the CELIs have strengthened the cultural associations present in 81 Law 1448, also known as the Victims and Land Restitution Law, was signed by President Santos in 2011 in order to “facilitate the restitution of millions of hectares of lands abandoned or stolen as a result of human rights abuses and violations” (The 82 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative the areas of intervention; these operate as a way of building trust, collective identity and belonging in the territories, rooted in the intangible heritage of the community. As mentioned above, the social capital index is composed of measures for trust and participation (the former accounting for 60 points out of 100 and the latter 40). Participation is measured in respondents’ reported affiliations: interest groups, producer associations, political parties, the JACs, community organizations and citizen oversight processes (veedurías). The survey data show that, across the full sample, there was a moderate but significant increase in household participation in community activities. The impacts were slightly more significant in CELI municipalities, where this sub-index was positive in almost 1 point (out of 40); there was no impact in the PNCRT municipalities (Figure 4.6). When the results were differentiated by gender there was no evidence of difference between FHHs and MHHs. An analysis of seven specific kinds of participation showed that participation increased by between 0.5pp and 9.3pp in the CELI municipalities (Figure 4.12). The impacts were smaller in the PNCRT municipalities and fewer kinds of associations showed significant impact: voluntary groups (5pp), sporting or cultural groups (3pp), and women’s groups (4pp).82 Figure 4.12. Impacts on participation in community activities, CELI and PNCRT Variable CELI PNCRT Oversight groups (veedurías) 0.013*** 0.0048 Community meetings in general 0.093** -0.0035 Voluntary groups 0.049* 0.069*** Sporting or cultural groups 0.033*** 0.032** Parents associations 0.059*** -0.028 Security groups 0.006*** 0.005 Youth groups 0.008** 0.005 Women’s groups 0.012 0.036*** Political party or movement 0.026* -0.012 Agricultural school groups 0.0149** -0.0885 Data disaggregated by sex shows differences in participation in different kinds of organizations. In general, MHHs reported increases in participation, while their equivalents with FHHs reported more than a 50% reduction in participation. In the CELI municipalities, MHHs increased their participation in the JACs by almost 4pp, while FHHs reported a reduction of 10pp. In the Victims Land and Restitution Law, p. 5). For more information, see the Amnesty International report on the law’s land restitution provisions at http://www.refworld.org/pdfid/4f99029f2.pdf. 82 In six other kinds of organizations there was no impact either in CELI or in PNCRT municipalities (see Annexes for detail). 83 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative PNCRT municipalities, FHHs were 13pp less likely to participate in the JAC or other community groups, while MHHs increased their participation by 10pp. In all CELI municipalities MHHs increased their participation in cultural groups by approximately 4pp, while FHHs presented no change. While CELI municipality HHs reported nearly 4pp increased participation in voluntary groups, the households that participated in decision-making were MHHs (10pp). Notwithstanding the decrease in FHH participation across the full sample, among FHHs in the PNCRT municipalities, participation in one type of organization – women’s groups – increased by 5pp. Even so, no impact was seen among FHHs in terms of decision-making. Individuals who did participate reported more participation in decision-making, by between 25% and 40% more across the full sample. As this increase was general, it cannot be attributed to any intervention, but the change is positive and noteworthy. At the same time, the percentage of persons who report that they take part in meetings but not in decision-making fell by the same amount. While it is true that overall participation has fallen, this finding reflects improved quality in the social participation engaged in by the population. Conclusion 4: In the CELI municipalities, there was an important increase in participation in associations, but FHHs did not participate in this increase. In PNCRT municipalities, no impact was apparent on average participation levels. The particular role of the Juntas de Acción Comunal (JACs) The Juntas de Acción Comunal or Community Action Councils are the most local level of civil society recognized in Colombian law: voluntary associations in urban neighborhoods or rural veredas that can formally constitute for civic purposes. While the three CELIs were not asked to report systematically on the numbers of JACs served, it is clear that each CELI developed strong commitments to work with the JACs over the course of the activities, both in building their capacity to function as more effective community organizations as well as through engaging JACs in direct contractual relationships to carry out specific projects. Each CELI undertook training both of individual community JACs and of associations grouping multiple JACs. CELI Montes de María held 31 sessions in 17 communities, providing orientation in management practices, transparency, and trust building. The JACs learned skills and procedures that they applied to implementing projects and negotiating advantageous agreements with providers.83 In southern Tolima, CELI Central worked to strengthen management capacities of 633 JACs through training in project development, empowerment, contracting, and access to public services. One goal was to prepare JACs for the April 2016 leadership elections to build local-level sustainability.84 CELI N/S engaged the Foro Nacional por Colombia to assist 65 JACs and five umbrella associations in Tumaco to strengthen organizational capacities, work with 83 Colombia Responde Montes de María Quarterly Report July-September 2014. Bogotá, Colombia: Global Communities, p. 11. 84 CELI Central Annual Report September 2014-September 2015. Bogotá, Colombia: TetraTech ARD, p. 21. 84 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative higher levels of government, and develop plans that represent constituent goals.85 CELI N/S trained Bajo Cauca leaders in environmental management, land issues, communications and financial management.86 The training and guidance provided by the three CELI projects have empowered many JACs to enter into legal agreements for carrying out local community infrastructure projects. CELI Central has turned to the JACs as implementers in efforts to improve tertiary roads and social infrastructure projects, delivering approximately USD$6.2 million in contracts. In Meta and Caquetá, JACs have taken responsibility for tertiary road maintenance and improvement. Supported by 10 CELI grants totaling USD$879,000, the JACs mobilized labor, food, and lodging for workers, raised funds and contributed cash.87 CELI N/S made infrastructure grants to JACs throughout Bajo Cauca and its Tumaco intervention sites. In its December 2012 report, the project reported that of the 45 grant agreements approved in Córdoba in that quarter, 34 JACs and two JAC associations received funding to move forward with community projects.88 Between CELI N/S and CELI Central, dozens of JACs have taken on responsibility for managing and fulfilling the contractual terms of direct grants, providing these civic organizations with important experience in community management and infrastructure construction and maintenance. The CELIs report greater efficiency and efficacy in the successful completion of these works and contracts, under budget and within proposed time frames, with citizen oversight committees providing watchdog services. The capacity now resides in the hands of these JACs for future undertakings. The JACs give rise to other organizational forms, such as the producer associations. The JAC’s importance as a starting point for associativity is reflected in the opinions obtained in Antioquia and Meta in response to questions about the resources received from the CELIs: Above all they pass resources through the JAC. There the Goat Rearers’ Association was created for the whole of the municipality of Caucasia. That is, veredas Brasil, Villa Mercedes, Corcovada, Los Tigres 1, 2 and 3, and Delirio. They operate there, and also in Villa Mercedes, there are people who are beneficiaries of the Association of Rubber Producers – that’s the Arias family, who live there; and they’re also in Asocor. (Producer association leader, Antioquia, CELI) At the office of the Secretary of the Municipal Government our role is to strengthen the JACs and associations for different productive areas. Here there’s a bit of everything, so they have organized themselves for example in the area of cattle farming, cacao, rubber, fish-farming… And I know that all these productive associations – some of which are strong – have been accompanied by Colombia Responde. (Functionary, Macarena, CELI) In spite of the overall positive impact, participation decreased for JACs and the producer associations (for the latter, see the next section). This was true for both CELI and PNCRT 85 CELI N/S, FY 2016 Quarterly Report (Q1), December 2015. Bogotá, Colombia: Chemonics, pp. 26-27. 86 CELI N/S 2013 Quarterly Report, April-June. Bogotá, Colombia: Chemonics, p. 16. 87 CELI Central Quarterly Report April 2015-June 2015. Bogotá, Colombia: TetraTech ARD, p. 18. 88 CELI N/S Quarterly Report October-December 2012. Bogotá, Colombia: Chemonics, p. 9. 85 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative municipalities. In absolute terms, fewer HHs belonged to or participated in the JACs: from 74% to 57% in the CELI zones, and from 88% to 64% in the PNCRT municipalities. The decline also occurred in control zones; as a result no impact is seen (Figure 4.13). It appears that FHHs made up an important part of that decline, as MHHs increased decision-making within JACs by 5.5pp. Figure 4.13. Impacts on participation in the JACs, CELI and PNCRT Variable CELI PNCRT Household belongs to a JAC or other community group 0.0249 -0.0593 Household participates in decision-making in JACs or other groups 0.0359 -0.0358 Beneficiaries indicated that participation is difficult because involvement in the JACs requires dedicating a considerable amount of time and unpaid labor. In certain cases, respondents stated that local politicians have sought to co-opt the JACs for their own purposes. Some respondents in municipalities included in the intervention argued that community processes have deteriorated as a result of the actions of politicians who have sought to use the processes as a social base to advance their private interests. A focus group participant in the Guamuez Valley, Putumayo, said: The community processes, whether JACs, associations or cooperatives, whatever: they are all getting weaker because we’ve been weakened first by political maneuvering, second by the violence and third because no one listens to what they have to say. That is, the authorities don’t listen to them despite the JACs’ rights in the organizational process. (Producer, Putumayo, PNCRT) In the previous quotation several different reasons are advanced to explain why social processes have become weaker and trust in the institutions has fallen overall, if not necessarily among CELI-supported JACs. These include the intervention of local politicians, violence, and the reduced effect that this kind of organization and these processes have on decision-making at local level: community processes have a voice but no vote. CELI activities with the JACs worked to strengthen JAC leadership and management structure, and their legal, administrative and social functions. Stronger organizations with better leadership would have greater public recognition and visibility. The CELIs also contracted with the JACs, as described above, which has an empowerment effect and greater respect for the leadership and decision-making with JAC leadership structures. Such empowerment helped to bolster the CELIs against political influences and extortion attempts, which are very common with external firms, according the CELI Central COR. In the veredas the JACs are sometimes the first resort for resolving conflicts between neighbors in processes involving direct negotiation. Beneficiaries who took part indicated that they live in rural areas that are far from the population centers, in municipalities where state presence is scarce. In certain cases the JACs occupy the vacuum left by the State and permit neighbors to construct trust, such as in this case where citizens reported on who they ask for help with a problem with a neighbor: Through the JAC, or you reach some agreement between the victim and the aggressor... it hasn’t been necessary for the Ombudsperson to appear or for anyone to go to court. No, it’s dealt with there in the vereda… The five leaders of the Council meet with the conciliators and they call in the people who have the problems. They all sit down together, everybody gives them advice and they come to an agreement. And that’s it: the problem is dealt with. (Producer, Antioquia, 86 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative PNCRT) Intervention support provided for associations has enabled communities to appropriate their processes through training or the building of infrastructure (activities developed by the UACT or one of the CELIs). The JACs often lack the budgetary provision and infrastructure they require to function. One government employee in Putumayo told the evaluation team: For example, training has been provided to the JACs. [They] have played an important role in the activities carried out in the municipality. Indeed, the Unit has helped in the construction of these small community houses [buildings in which the JACs meet with their communities.] (Functionary, Putumayo, PNCRT) Conclusion 5: Despite the fact that these entities are occasionally considered to be co-opted by local political interests – which may threaten their legitimacy in certain places – they continue to represent the gateway to state services and programs. Where the security situation permitted, and where the JACs have the confidence of the population, this kind of social organization is central to political life and to participation in the veredas. In these cases the JACs play a central role in providing access to, and enjoyment of, the services offered by the State in terms of political participation, infrastructural improvements and social control. In some regions the JACs are weaker, explaining why surveyed households report reduced levels of participation overall. However, JAC participants report more active participation at the decision-making level. Conclusion 6: In the veredas of geographically distant municipalities, the JACs are the principal spaces in which disputes between neighbors are resolved, mostly through direct negotiation. By having this local resource, minor social conflicts are resolved at the vereda level, without the need to take cases to municipal or national bodies. This is the case in regions where the security situation allows it. However, participation can be affected in cases where threats are issued against leaders. Associations and productive projects As described in the introduction, CELI priorities changed from a focus on consolidation of the GOC's presence to what could be characterized as more traditional rural development objectives, with a focus on agricultural production and the strengthening of rural value chains. This is seen most clearly in the reassignment of the CELIs from their original focus on consolidation under DO1, “Civilian government present in CSDI zones consolidated,” to its present articulation as DO3, “Improved conditions for inclusive rural economic growth." Throughout each project and especially since the revised Development Objective was released in June 2015, work with value chains has been a principal means for the projects to achieve their goals. In carrying out this work, the projects supported the producers associations in several ways: technical assistance and training; support work in developing marketing and commercialization (also often involving training); and assistance in financial services to support production and marketing. As the implementer of CELI Central put it, “the focus has increasingly been on 87 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative empowering and mobilizing smallholder producer associations, and increasing private sector investment in rural communities to create economic opportunities for community stakeholders within the context of licit markets and the rule of law.”89 CELI Central worked with 21 producer organizations in the AgroColombia activity, on issues of administrative and accounting, according to the CELI COR. This included supporting the rules behind rotating loan funds, creating a teamwork and leadership culture, financial education, and 1,114 financial products granted from the rotating funds associations. They brought youth into the trainings to generate conditions for sustainability and facilitated the opening of a bank branch where previously there had been no financial services.90 Considerable focus was placed on organizational training and capacity building, and TA on a range of crops, as described in the economic development chapter of this report. Training of producers associations covered a broad range of topics, including improving agricultural production, introducing new crops, strengthening the functioning of their organizations, and training youth to encourage the transfer of knowledge and resources to the next generation. Study tours are a modality used by the project to widen the experience of producers associations, such as the visit organized by CELI Central for members of Asocaferumet in Meta. These coffee producers traveled to Quindío to learn about the productive chain for specialty coffees and met with a range of businesses working with small-scale coffee producers, as well as learning about the certification process.91 Associations in Quindío have since negotiated a contract with a coffee exporter and will expand operations through the Acumen Fund (also supported by USAID).92 In the survey data, respondents reported an 8pp reduction in HH participation in producer associations in the CELI municipalities, whereas in the PNCRT municipalities no significant variation occurred. (Figure 4.14) Figure 4.14. Impacts on participation in producer associations, CELI and PNCRT Variable CELI PNCRT Household belongs to or participates in producer or marketing associations -0.0799*** 0.0128 The percentage of members who reported participating at the level of decision-making rose from around 30% to around 50% in both CELI and PNCRT municipalities. While positive, in the control municipalities the tendency was the same, so this may not be attributed to the interventions. Productive associations established for cultivating cacao (Putumayo, Nariño), rubber (Caquetá) or cattle-raising (Meta), were in zones with voluntary crop-substitution programs. Small 89 CELI Central Quarterly Report Q1 FY2016, p. 5. 90 Data from conversation and e-mail with CELI Central COR, 30 June and 27 July, 2016. 91 Ibid, p. 44. 92 As reported by the USAID COR for CELI. 88 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative producers saw the associations as a way to leave illicit activities behind. An official in Putumayo explained: If you visit the 40 veredas that are free from illicit crops [you will see that] here in the municipality there have been some interesting processes of community participation and leadership. Because it’s been them and it’s been us: we get tired of all this stuff and we are going to rid our vereda of these crops that have done so much damage. But it is in that connection that the State is unable to help us. (Functionary, Putumayo, PNCRT) Survey results suggest there is an economic rationality behind respondents’ associativity. The active members who responded to the questionnaire said that the associations were useful to their productive projects for several fundamental reasons. The results indicate that positive impacts were felt in the CELI municipalities in the processing of products (4pp), sales (almost 5pp), transport (5pp) and training (5pp). For the PNCRT municipalities, more reported associating for product sales (almost 5pp) and transport (12.5pp). (Figure 4.15) Figure 4.15. Impacts on reasons they associate, CELI and PNCRT Variable CELI PNCRT Decided to associate in order to process their products 0.0431* 0.0223 Decided to associate in order to sell their products 0.0496* 0.0486** Decided to associate in order to transport their products 0.0512* 0.125*** Decided to associate in order to engage in joint training 0.0494* 0.0224 The most important aspect of these figures becomes visible when they are disaggregated by sex, in particular in the CELI municipalities, where most of this impact comes from households with female heads. More FHHs reported joining in associations in 2015 to sell their products (8pp) while for MHHs there was no impact. More FHHs also reported working together in 2015 for processing (5pp) and transport (10pp). The only measure in which a positive impact was apparent for MHHs was in association in order to engage in joint training (3pp), and FHHs had no impact on this measure. In PNCRT municipalities, more FHHs reported associating in 2015 to sell products (9pp) and to transport products (17pp). In absolute terms, HHs in CELI and PNCRT municipalities reported more frequently that participation in associations had enabled the households to improve their productive activity (increasing from 48% to 75% for CELI, and from 30% to 67% for the PNCRT). In the controls, the effect was even greater, showing that it is generalized and cannot be attributed to an intervention. While it is true that people enter the JACs as a result of natural processes of communication and the resolution of daily problems that occur in the communities, the emergence of producer associations was also encouraged by state and international cooperation projects and programs. These entities usually demand that the organizations they support are constituted as associations in order to ensure that the benefits reach the population. At that point, the European Union turned up and gave us the idea that if we didn’t form an organization, we wouldn’t be able to get any money, because the government was only going to invest in associations. So it came along and founded five associations. (Producer, Meta, CELI) We produce things now; we have the crops to produce stuff. Where did our vision come from? 89 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative When the FUPAD [NGO, Fundación Panamericana para el Desarrollo]came to our vereda and called a meeting. (Producer, Antioquia, CELI) Yes, I’m a member of an association in the Chicalá vereda called Aprochicalá. We began to set it up a couple of years ago. (…) You can’t receive any help or anything these days if you don’t belong to an association. So, we’ve had to take part in an association. (Producer association leader, Tolima, CELI) As a result, the communities respond by increasing the number of associations and, in some cases, paying little attention to their durability or sustainability. Once benefits have been delivered, the members sometimes lose interest in continuing with the process unless programs and projects are run again in their communities.93 Consequently, it is no surprise that respondents recognized the existence of paper associations, as in these examples: Of all the associations in Vista Hermosa there are many that are just paper associations, nothing more. That is, they have only been registered in order to capture resources and [afterwards] they remain quiet. (Producer association leader, Meta, CELI) In ‘98 I didn’t know of any other associations. But year after year more and more associations were created and now there are something like 72 or 76, something like that. Every time someone from the Ministry or some government body comes to give us training they always insist that if we don’t form an association we won’t receive any resources. That’s why the people have gone crazy forming associations – there are associations with 10 members, 18, 20… (Producer, Meta, CELI) This does not mean that the CELI or the UACT encouraged the creation of paper associations more than other interventions did, simply that an incentive exists that affects all rural interventions. In certain cases, the current interventions are constructed on the basis of earlier associations. An example drawn from CELI N/S provides a good illustration: Colombia Responde turned up and carried on with the projects initiated by FUPAD. Colombia Responde enabled the rubber-growers association to keep going and continued to equip and maintain the plantations; and now we’re being helped by Colombia Responde. (Producer, Antioquia, CELI) Conclusion 7: As a result of the Colombian government’s interventions and of international cooperation programs, the number of productive associations is increasing. Households recognize that the associations have served to improve their productive activities. Among the most important incentives for forming associations are processing, transport, sales, and joint training. These results have been more significant in the CELI interventions and above all among FHHs. People who participated in associations at the time the baseline was constructed continue to take part in associations but are now also involved in decision-making at higher levels than in control municipalities. Conclusion 8: At the same time, overall household participation in these associations has declined. This contradiction might strengthen the hypothesis that households seek out benefits 93 Associativity has its own logic. It is related to the number of beneficiaries, the fewer the better, and to the system of internal rules – the more stringent the better – that minimize the effects of opportunism (the free rider phenomenon) (Olson, 1965). 90 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative and that once these have been achieved, participants lose interest in the organizations. Opportunistic interests (the free rider phenomenon) may prevail over participation for conviction or long-term vision. Women’s and victims’ associations Other forms of association supported by CELI included groups of victims of the armed conflict, facilitated by the provisions of Law 1448. Women founded many of these associations and take on community leadership roles, resulting in their empowerment within the community. Participants emphasize the increased levels of cooperation and autonomy that they have achieved, while new forms of female leadership have emerged, based on increased participation in public affairs and the search for recognition and dignity for the victims of the conflict. Similarly, new organizations have emerged that did not previously exist in the region, such as associations of young people or of Afro-Colombians. According to the qualitative fieldwork, the support of the Department for Social Prosperity (DPS) and the Victims’ Integral Attention and Reparation Unit (UARIV) have been fundamental to this process. The participation of women in public affairs and in productive activities has led to a significant cultural change compared to a generation ago. In addition to the CELIs, other programs such as Familias en Acción and the DPS’s Women Savers have also played a fundamental role. The words of a female community leader from a municipality in southern Tolima illustrate the increased participation of women: The women’s organization has existed for about five years now… I had very few [in my group] at the beginning, very few, but now I have something like 300 women working side by side with me. (Producer association and CSO leader, Tolima, PNCRT) In some cases, the productive associations are made up principally of women who are victims of the conflict and/or living in situations of displacement. These associations have enabled the women to generate an income, create links of solidarity with others and build roots in the territories that have received these people and families. This was the situation in San Jacinto, Montelíbano, and Santander de Quilichao (CELI). A woman from Montes de María told us: [I] am an artisan and I belong to the Luz y Vida group of artisans… I make mochilas, I do a bit of everything: I work, I iron, I wash clothes and that lets me support my family. I am not from here, and I thank God that this place has welcomed me and that things have gone really well – I have friends and we all meet together and when there is some activity or a celebration we take part and we are all happy. We’re fine, thanks be to God, in spite of everything we’ve suffered… We keep on struggling and everything is turning out right. (Producer, Bolívar, CELI) The survey data also shows that victims’ associations are growing. In CELI municipalities, the number of people who reported belonging to organizations of this kind rose from 7% to 12%, while the corresponding growth for the PNCRT was from 6% to 9%. The quality of participation also improved, thanks to increased participation in decision-making. However, these changes are also present in the control municipalities, and were not, therefore, attributable to the interventions. On the other hand, in PNCRT municipalities, women’s group participation and decision-making fell (Figure 4.16); but since the control areas decreased even more, the PNCRT municipalities saw a slight positive impact (3.6pp) by comparison. 91 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 4.16. Impacts on participation in victims’ and women’s groups, CELI and PNCRT Variable CELI PNCRT Participation in groups of displaced, returned or vulnerable populations 0.0107 0.00657 Decision-making in groups of displaced, returned or vulnerable populations -0.116 -0.0654 Participation in women’s groups 0.0356* 0.0117 Decision-making in women’s groups 0.0610 -0.260* In contrast to the quantitative findings, which showed no impact for belonging to or participating in victims’ organizations, the qualitative responses indicated their importance. Organizations for displaced, returned or vulnerable people were said to have transformed into organizations of another kind that have expanded their identities and their sense of what they should seek to achieve, without losing sight of the reasons they were originally founded. Frequently, in order to generate income for their members and for the organizations, victims’ associations established complementary productive projects. We know about victims’ organizations because many different organizations take part in the regional round table and they are interested in expanding into agricultural production. As a victim’s organization we have always focused on farming, while others focus on commercial activities. (CSO member and producer, Putumayo, PNCRT) In Santuario we have Las Líderes who are women who work with shampoo and air-fresheners. There’s also Asovides, which is for displaced people - and they’ve created two more organizations for them in Santuario too. And then we have an association of goat producers, and there’s another of those in Montaña. (Producer association leader, Caquetá, CELI) The members of our foundation are internally displaced. (Producer, Tolima, Control) With these changes, victims’ organizations have been able to attract new funding for increasing members’ sense of belonging and improving household incomes. Conclusion 9: The social capital that has been constructed as a result of the work of victims’ associations has permitted the creation and strengthening of other kinds of organizations in a process that has encouraged an expansion of their reasons for existing and their aims, fomenting a sense of belonging and identifying new resources. Conclusion 10: Overall, participation in victims’ associations and organizations of vulnerable groups increased, and the CELIs have had a small but significant positive impact on participation in women’s groups. Though the change cannot be attributed to the interventions, they represent an important change in the target population. Cultural and other associations In some of the zones in which the fieldwork was carried out, and in particular in areas with large Afro-Colombian populations such as Montes de María and Tumaco, the cultural processes created by civil society have developed and strengthened the population’s sense of rootedness in their territories. These associations help to reconstruct relations of trust, the social fabric and the identity of the communities. In Montes de María, the importance of cultural processes for strengthening social capital and ties with the territory was apparent: 92 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Now what are we doing? We have a dance project called Danza Negra, with gaitas [traditional wind instruments made of cane], and we are beginning to rescue our traditions through drumming; and we also created a Danza Negra group in San Isidro. (Producer, Bolívar, CELI) For their part, in Tumaco and Antioquia, women were found to be leading cultural activities using music, leading to an increased sense of identity among the population. The organization is very close to [women’s group] Asomuca and it is the role of women in this tradition to preserve the Tuna Tambora [traditional singing and drum groups]. (Producer association and CSO member, Antioquia, CELI) 18 women and five men. There we see the role of women… in the field of dance. So the tradition is not lost; because the Tuna is ancestral, it’s existed since the people first arrived in Cáceres… It has been women who have preserved the tradition. (Producer, Antioquia, CELI) Another way of strengthening the social fabric has been through the support provided to associations that preserve and reproduce the intangible cultural heritage of their communities. The CELIs have provided direct support to different cultural associations, enabling the communal values destroyed by the conflict to be reconstructed. In the CELI and the PNCRT municipalities alike, participation in this kind of association increased by three points compared to the controls. However, this participation did not include decision-making (in contrast to what was observed in other kinds of organizations). In focus group discussions, participants suggested that a return to activities like dance, music or traditional artisan production has permitted communities to renew their cultural identity and to feel pride in their traditions, but that these organizations also operate as forms of resistance against armed insurgents. In regions where Afro-Colombian communities predominate, such as San Jacinto or Tumaco (both CELI municipalities), the program has supported cultural associations that had disappeared as a result of the conflict. The associations help to reconstruct identity and ties of trust, and involve youth in processes dedicated to preserving cultural legacy. A cultural leader from Montes de María explains: We created a project called the Son de Negros. We focused on… the recruitment carried out by the illegal armed groups and we are going to include this in our Danza Negra. Danza Negra is African dance that we have been dancing since the time of the palenque94 of María la Baja. There is a sense of connection with the mountains, with my grandparents; I remember that my grandfather was black and was a drummer in the Danza Negra. (Producer association and CSO member, Bolívar, CELI) Cultural associations are fundamental to the Afro-Colombian communities as a conduit for self￾awareness and to feel pride in their history and their traditions. A leader from Tumaco comments in this regard: The organization Tumbo de la Mar was born several years ago in the minds of a group of friends. It was born of the necessity to conserve, preserve and share matters connected with our cultural 94 Palenques are communities of free blacks created in resistance to slavery. 93 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative roots. In about 2009 or 2010 … this desire to develop cultural activities emerged in a meeting, and from that point on the work has continued… with young people from different parts of Tumaco working to preserve and teach our music and traditional dance. Because this is our identity, it is what we are here, here in this part of the southern Pacific coast. (CSO member, Tumaco, CELI) The cultural associations have been successful in preserving tangible cultural heritage by tapping into the knowledge of the elderly and recovering forgotten traditions and practices. We have heard of women’s groups, but they are not exactly cultural but groups of artisans; the artisans are very united; they are groups of old people – I think the youngest is about 65. The group meets here in the vereda. The artisans make handicrafts, and before this, traditional instruments. (CSO leader, Tumaco, CELI) The qualitative analysis showed that CELI support to cultural associations has impacted young people and their participation in community activities, and has made it possible to preserve the cultural heritage of the community and to transmit it to new generations. Cultural activities have also permitted the creation of relations of trust, preserved memories and, above all, helped communities to value their collective identities. Conclusion 11: The CELIs have supported different cultural associations, enabling the reconstruction of community values damaged by the armed conflict. Returning to activities such as dance, music or traditional artisan production has permitted communities to create a cultural identity in which they are able to recognize themselves and feel pride in their traditions; it has also acted as a form of resistance against armed actors. In regions with a large Afro-Colombian population, such as Tumaco and Montes de María, cultural organizations have been strengthened, enabling the emergence of social ties in the regions where the interventions have taken place. The high levels of social capital that have been constructed in the regions have been added to the equation, as a result of the presence of victims’ associations which establish the trust on which the participation of social organizations is based. Conclusion 12: The support provided to associations that preserve and reproduce the intangible cultural heritage of a community is enormously important because it produces transformations in the communities by assigning value to collective identity. Cultural associations have helped to inspire processes of symbolic renewal in communities of victims of the conflict. Recommendations 1. Adapt programs to encourage trust (conclusion 2) The unprecedented overall decline in trust of the government cannot be resolved by development alone, but it is important to develop ways to strengthen community ties to mayors’ offices and local government bodies. Despite international development interventions, the population feels less trust toward local government than they did two years ago, and less trust between neighbors. These indicators are highly sensitive to factors outside the control of any program, but interventions can and should develop tools to confront the problem. 2. Improve work with female heads of household using participatory models (conclusion 4) In terms of participation, the positive results obtained for MHHs need to be extended to FHHs. Efforts should be made to benefit this population that has traditionally been immersed in unfavorable socioeconomic conditions and has had less free time available to dedicate to associations. FHHs who already participate in victims’, women’s and other groups could serve as 94 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative guides for other women to become involved and to increase their participation. Similarly, the community leaders of associations and other groups, whose commitment and participation represent higher levels of social embeddedness, could serve as a resource to advance this aim. The CELIs should continue investigating different ways of using this resource of association leadership and practice in the final months of the intervention, engaging in outreach activities with community members whose behaviors indicate increasing isolation. 3. Support the legitimacy of the JACs (conclusions 1, 5 and 6) In order to strengthen the JACs, the CELIs should continue training leaders in board election processes and community participation tools to counteract the risk of losing legitimacy. Support of the JACs to encourage wider participation in local affairs – including women’s participation, that of victims, and other marginalized groups – should be a priority so as to generate horizontal communication between neighbors and vertical relations between communities and local authorities. Programs should monitor the quality of community associations such as the JACs, in ways similar to how producers’ organizations are monitored. The ICO should be adapted to meet organizational and community expectations for these groups and to ensure that programs involve the JACs, communities and local government. In cooperation with mayors’ offices and officials responsible for community or administrative affairs (the secretarios de gobierno), work should continue to establish special programs focused on the JACs to improve their management capacities and their representativeness. Monitoring results should be made public to generate community incentives to play an active role in holding JACs to account. The component competencies of the ICO should be evaluated to determine which aspects work well and which do not. Given the differences of opinion that exist concerning its usefulness, the Chatham House Rule95 could be used for such a discussion, in which parties express their opinions openly and identify weaknesses or gaps in the measuring tools without fear that their views will become public. 4. Strengthen the role of the JACs in the resolution of conflicts (conclusion 6) The CELIs should support the strengthening of the tools currently available to the JACs in their conflict resolution role with training for leaders in the veredas. Other interventions, local and regional personerías, the police and the Defensoría or even groups such as the Chambers of Commerce could also be involved. 5. Make associations more transparent (conclusions 7 and 8) To improve the quality of the associations, use a refined ICO as a monitoring and ranking system for associations. Make results public and give greater access to programs and benefits to the highest ranking associations, with local authorities’ involvement. 95 When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed. https://www.chathamhouse.org/about/chatham-house-rule. 95 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 6. Loosen the requirement that beneficiaries must be grouped into associations (conclusions 7 and 8) In future programs, the possibilities that intervention might not rely strictly on the creation of associations should be examined. It is not always necessary to require that potential beneficiaries form an association. If an association is to be authentic, it should emerge from the initiative of individuals who decide whether they wish to take part or not and who wish to gain individual advantages (for example, improved sales for their products). This represents a contrast with the JACs, whose objectives are based on public goods and proposals (such as parks, bridges, child-care facilities, road improvements, etc.) Similarly, programs such as CELI should support private initiatives. Small businesspersons improving their businesses may be a ripe target for support, as was sometimes the case under the CELIs. There are also times when the obligatory formation of new associations is not appropriate for cultural, social or historical reasons. 7. Increase collaboration with associations whose objectives may or may not be at root productive (conclusions 9-12) Social organizations are also candidates for CELI-style interventions because of their high levels of social capital, and they are increasing in number in the intervention zones. Such organizations (that are not necessarily producer associations) should be supported to become autonomous and financially sustainable. Such activities could be developed in cooperation with the DPS, the Ministry of Agriculture, SENA and the Municipal Agricultural Extension Units (UMATAs). 96 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative GOVERNANCE AND INSTITUTIONS Evaluation questions: To what extent have CELI interventions contributed to participation, governance and institutionalization of the territory? What are the CELIs' specific effects of in changing the capacity and service delivery of local government? (To serve local people, be accountable, expand social services.) SUMMARY: Citizen participation at the ballot box remains relatively high, but electoral challenges like vote-buying persist. An accountability index from the survey questions shows declines since baseline across the regions under study. Citizen oversight is minimal, but CELI work with the personerías (in at least one region) to strengthen the registry of veedurías has been positive. Service delivery remains a major challenge, with citizen perceptions of access and quality very low as well as lower than at baseline. Reports of human rights violations by the military decreased but reports of police violations increased in PNCRT municipalities. This may reflect a change in force presence in consolidation municipalities, but does not reflect an improvement in citizens’ interactions with public forces. CELI interventions were designed to strengthen local policymaker capacity and to improve management and provision of public goods and services. Only through consistent and fair service delivery will citizens recognize the permanence and legitimacy of the State in areas where IAG influence has been strong and, in many cases, parastatal. CELI interventions varied according to the region, type of municipality and its initial conditions, the quantity and quality of social organizations, and perceived priorities. In general, the CELIs sought to strengthen local governments in accountability processes, formulation and execution of municipal plans including rural development, and guaranteeing victims’ rights. In parallel, the CELIs sought to strengthen civil society to work in articulation with municipal governments, defend their rights and exert control in government actions. This chapter has two main sections. The first examines governance from the citizen perspective: political participation, electoral practices and conditions; transparency, oversight and accountability; and trust in government. This section uses the following sources of information: 1. Participation index composed of survey questions around electoral and political participation. 2. The comparison of survey results between baseline and midline. 3. Qualitative findings from interviews with municipal representatives, personerías and social organization leaders, as well as focus groups with beneficiaries. 4. Related secondary sources. The second section concerns service delivery and the capacity of local governments to deliver services well. Sources for this section include: 1. The Integral Performance Index, a GOC tool that rates governments on a range of competencies. 97 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative 2. The comparison of survey results between baseline and midline, including an index of items on government capacity and service delivery from the citizen’s perspective. 3. Qualitative interviews with personerías, organization leaders and beneficiary focus groups. 4. Related secondary sources, including the Consolidation Index good government sub-index. The evaluation team then makes a set of recommendations for USAID and the CELIs’ ongoing work, with the hope of increasing participation and making interactions with the GOC more effective at promoting decentralization and improving public service delivery. Findings and conclusions Participation, transparency, accountability and trust Political participation and electoral culture: the Participation Index On a scale of 1 to 100, the Participation Index sums variables such as participation in political parties, JAC and oversight groups or veedurías; the perception of participation in elections; voter registration; and whether the respondent voted in the last presidential election. In CELI municipalities, the average score is 36/100 while control sites show 34/100. The CELI municipalities show a positive impact equivalent to 1.8 points over the control group (Figure 5.1) because of the control group’s decline, while the CELI score remained stable. PNCRT municipalities were stable at about 35 points out of 100 at baseline and midline. Figure 5.1. Impacts on the Participation Index, CELI and PNCRT Source: Authors’ calculations. *** Significant at 99% confidence ** 95% * 90%96 Electoral practices In absolute terms, around 90% of respondents across the full sample report voting in the last presidential elections. This is an increase of about 10% from the average reported at baseline. Since the effect was the same across samples, no impact is found. Around 95% of voters said that they voted because their vote counts, or because it was their responsibility as citizens. Across the full sample, about 85% say they voted because they were drawn to a particular candidate. Participation in political organizations increased for CELI and PNCRT by around 6%. Since this change was also seen in control areas, no impact is found. 96 Throughout, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are in Annex 3, and data in the text are tabled in Annex 10. Treatment score (/100) Control score (/100) Baseline Midline Baseline Midline Impact CELI 36.14 36.11 35.82 33.36 1.82** PNCRT 35.85 35.02 35.58 34.23 -0.63 98 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Among citizens who did not vote, the reasons for not doing so are similar for PNCRT and CELI zones. The main reason people gave for not voting was that no candidate was satisfactory, but also cited were the difficulty in voting and a lack of interest in the electoral process. Survey data show no impact from either CELI or PNCRT interventions on these factors. Respondents were asked whether they or someone they knew had been pressured or forced to vote. Only 3% of CELI respondents said yes, slightly lower than at baseline, without significant impact. When asked if they or someone they knew had exchanged their vote for money or gifts, 21% said yes at baseline and 16% at midline. Controls declined even more, resulting in a 3pp negative impact for CELI sites. No impacts are recorded in such practices for PNCRT. Conclusion 1: There is good voter turnout, but clientelistic practices persist. High electoral participation shows that voting has become institutionalized: citizens do vote, regardless of interventions. The rates show that citizens in municipalities affected by the conflict think that the two most recent presidential elections were important – elections that have focused the debate on the end of the conflict. There is a high level of acceptance of formal democratic channels. It is concerning, however, that respondents admit having taken (or knowing someone who has taken) money or gifts in exchange for their vote, in CELI, PNCRT and control regions. This suggests a not uncommon practice in municipalities in the process of institutionalization, where the investment of resources implies, in the vast majority of times, clientelist uses. Clientelism can be explained because there is discontent with the political system for not being efficient in ensuring the satisfaction of citizen needs and service delivery. This leads to the citizens perceiving the elections as their only opportunity to extract benefits from the State, and in turn the citizens demand something in return for their vote. Citizen oversight, accountability and trust in public management Overall, citizen participation in oversight processes has been low, both at baseline and in the midline. For CELI municipalities the participation rate is 2.2% in contrast with their control municipalities (1.6%). However, participation in oversight committees or veedurías in CELI municipalities increased by 1pp compared to the control group, between baseline and midline. In PNCRT municipalities, participation in oversight committees at midline is 2%, in contrast with the control group whose proportion is 1%. However, there is no evidence of impact, since participation at baseline was also higher in PNCRT municipalities than in their controls. The survey instrument contained questions that were combined into an accountability index equal to 100 points, which is shown in Figure 5.2. The survey asked to what degree the mayor kept citizens informed about public expenditures, the presence or absence of public spaces for citizen oversight, and the degree to which local government took citizens’ opinions into account in decision-making. 99 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 5.2. CELI Performance: Accountability index, total and regional Ind. Description Region Baseline Midline % Change Source DO3- 028 Level of accountability in CELI municipalities Total 23.8 19.63 -18% Household survey • Baseline 2013, Econometría S.A. • Midline 2015, Proyecto EVAL Montes de María 18.6 17.74 -5% Central 27.8 24.96 -10% Norte 25.5 18.89 -26% Sur 22.6 16.17 -28% The changes were uniformly negative for CELI municipalities in how the public rates the accountability of municipal governance, with a drop in score between 5% and 28%. Overall, the good government sub-index of the Consolidation Index shows improvement since 2011, with an average municipal score of 54.7 increasing to 62.3 over the three years since data began to be collected. This sub-index is measured with four indicators: the Open Government Index of the GOC, percentage of funding from royalties, and effective tax payments (personal and industry/commerce.) The positive trend to this sub-index is compared to similar, under-performing municipalities, rather than in absolute terms, but at least does show improvements. Qualitatively, CELI support for government has been important, among other things, to strengthen and streamline veeduría registration, as community leaders and personerías report. Here they’ve really increased the registration of veedurías. Before, a project was done ... and no one knew anything; and then someone would come and criticize. Now people are involved and they build oversight and monitoring process into the project. So it has also strengthened the institutionalization a great deal. (CSO leader and producer association member, Tolima, CELI) CELI Central reports working with producers’ organizations and JACs to participate in public spaces to lobby municipal administrations, such as in the Sur del Tolima, or formulated projects to be included in the Project Bank.97 Issues of accountability are important to evaluation respondents, and in general their perceptions of government accountability have worsened at midline. In PNCRT, about 6pp more households consider that one of the most serious problems associated with the municipal administration is that it does not report what it is doing (Figure 5.3). In CELI municipalities, negative responses at baseline regarding public service deficiency dropped by 11pp at midline. All other variables show no impact in CELI, PNCRT or controls. 97 Data from conversation and e-mail with CELI Central COR, 30 June and 27 July 2016. 100 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 5.3. Impacts on perceptions of problems with municipal administration, CELI and PNCRT Variable CELI PNCRT They do not report what is being done 0.0047 0.0568* They do not pay attention to requests from the community -0.0217 0.0705 They do not have the capacity to resolve requests from the community 0.0337 -0.0506 Mistrust of government entities 0.0231 -0.036 Deficiency in public services (water, electricity, sewer, etc.) -0.114*** -0.114 The lack of highways 0.0120 0.0709 Corruption -0.0106 -0.0347 The lack of institutional goods: hospitals, schools, courts, child care -0.00292 -0.0711 Around half of the treated sample (49% in CELI municipalities and 53% in PNCRT) think that one of the most serious problems of the municipal administration is that they do not pay attention to community needs. This problem is evident in focus groups and interviews regarding meetings with the municipality: I am the JAC president and I got tired; I really do not come to meetings anymore. I chose not to go back. They don’t solve anything. They have not given anything to the village. (JAC leader, Meta, CELI) Finally, the survey also shows that around 7% to 8% of CELI and PNCRT households reported that the mayor informs the community on how resources have been spent, with no impact since baseline. The lack of contact between the citizens and the mayor was also evident in the focus groups: We need more connection with the Mayor and the Council. It also depends on us, as a community, to go break the ice and make that relationship. (CSO leader, Antioquia, CELI) Conclusion 2: Accountability is still an emerging practice. Responses to the survey questions about accountability were low and the index declined from baseline to midline. Lack of accountability has a negative effect on citizens’ participation and effective control. Local governments tend not to create spaces for accountability and do not invite citizens. While Law 489 of 1998 established compulsory accountability processes, the public receives little benefit from these processes. From the qualitative data, it is clear that the CELIs contributed to strengthening accountability (and therefore good governance), but the evidence does not show that it is widely practiced. Conclusion 3: Participatory mechanisms are not yet widely effective. While CELI has intervened to ensure that communities’ stated needs are included in development plans at the municipal level, people who claim to have attended the meetings convened by the municipal institutions state that these are not efficient. In general these meetings do not address the needs or complaints expressed by community leaders or do not implement plans they formulate together. This creates expectations and negatively affects citizens’ perceptions of their governments. This deepens a vicious circle, in which ineffective participation erodes perceptions about government management and transparency, which in turn discourages participation, further weakening accountability. The Consolidation Index indicates higher levels of oversight and citizen participation than is detectable 101 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative in the survey data, likely in part for the very rural character of the sample – but this will need to be examined in the future to find ways to bring a wider range of citizens into the process, not only those in municipal or other capitals. The lack of accountability and citizen participation in the formulation, implementation and monitoring of public policy is also reflected in the relatively low ratings citizens give to government management, at both the local and national levels. There is negative impact in both CELI and PNCRT municipalities in perceptions of mayoral and national level management. (Figure 5.4) Figure 5.4. Impacts on citizens’ perceptions of public management, CELI and PNCRT Variable CELI PNCRT Rates the mayor’s management as good or very good -0.0187 -0.0653* Rates the department’s management as good or very good -0.0191 -0.00223 Rates the national government’s management as good or very good 0.00249 -0.131** Trust the mayor’s promise to carry out a project -0.022 -0.0466 Trust the JAC president’s promise to carry out a project 0.0277 -0.0339 Among CELI households at midline, in absolute terms, only 15% rate the mayor’s management as good or very good, while at baseline this figure was 21%. Similarly, for PNCRT municipalities, there was a negative impact of 6.5pp in trusting the mayor’s office to carry out a project it has promised to do. Perceptions of the national government have also worsened in PNCRT municipalities, with 13pp negative impact. These data show clearly that opinions of management have deteriorated. Conclusion 4: In CELI and PNCRT municipalities, impacts since baseline are negative regarding government management at all levels. They may be ineffective at implementing new resources, may be creating new and unmet expectations, and may simply not have changed practices in response to interventions. But the effect on citizens is significant. Where there have been successful interventions on this front, there may be localized results – but the generalized impact the evaluation was designed to detect is not present. The crisis of confidence in Colombian local institutions has worsened in recent years. The HH survey reveals decreased confidence in all levels of government. Trust in government dropped from baseline to midline, across the full sample. The decline in confidence is parallel across governmental institutions: police, army, navy, justice; national, departmental, local, city council and JACs; ombudsperson offices; and advocacy and support bodies such as the national training institutes. Just one rated institution shows positive impact: the personería, and only in CELI sites. In absolute terms, opinions of the personería fell in both CELI municipalities (from 28% to 22%) and in control sites (from 28% to 17%), but more so in the latter. This results in a 4.6pp positive impact in CELI municipalities: that is, had the CELI not intervened, the municipalities would have seen similar declines. In the Central region, the positive impacts are particularly significant. As this 102 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative effect is not seen in PNCRT municipalities, the increase itself is likely linked to CELI interventions. Qualitative data from CELI areas suggest that the personerías do work to meet the requirements of national policies and legislation for victims; this may explain the higher percentages of confidence in institutions directly involved with these processes, as is the personería. Citizens in CELI and control areas recognize the personería as the main institution with which communities interact, particularly because of its role in the Victims’ Registry and in complaints about human rights violations. Personería officials interviewed for the evaluation also recognize the support of the CELIs with personnel and technological resources: We have worked directly with Colombia Responde. They have focused USAID resources in the municipality with the personería. They’ve institutionally strengthened the personería in human resources, in relation to victims, and prevention training. We’re able to cover more people and more institutional offers despite, as everyone knows, not having many resources. (Functionary, Caquetá, CELI) This quote, from the CELI Central region, is illustrative of the particular effort put forth by CELI Central interventions with personerías. Government functionaries told the evaluation team that the CELIs have contributed to better municipal planning, in consensus with communities, on specific issues like victims, taxes and royalties. They also report the added value of local government training and support to communities in developing projects to access resources.98 Conclusion 5: Trust in institutions depends on their ability to meet citizen needs, and that has worsened overall. Despite this general decline, improved perceptions of personerías shows a positive impact of CELI policies, and qualitative sources show that government actors acknowledge CELI contributions to capacity building. They are also registering and monitoring veedurías, as noted in the section on oversight above. However, circumstances have worsened the citizens’ overall trust of government, creating an even worse environment than when the CELIs began. Service delivery The Integral Performance Index The GOC created the Integral Performance Index – IDI for its Spanish initials – to evaluate the management of territorial governments, and has applied it since 2006. It tracks the performance of mayors and governorships in six dimensions: efficacy, efficiency, legal requirements, administrative capacity, fiscal performance and management.99 98 The contrast between quantitative estimates and the qualitative reports may be explained, at least in part, by the difference in samples: as a random exercise, the quantitative research shows the population average, while qualitative research reached out to active community members, who are not only easier to reach but also more likely to have experience with the interventions. 99 The DNP explains the results, the methodology and the sources of information in the documents in this website: https://colaboracion.dnp.gov.co/CDT/Desarrollo%20Territorial/Docuemnto%20Desempe%C3%B1o%20Integral%202013.pdf. 103 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative In this chapter we analyze the evolution of the IDI between 2012 and 2014. For this purpose, we grouped and classified IDI data into the four groups of municipalities under study: CELI, PNCRT and their respective control groups. (Figure 5.5) Figure 5.5. Secondary data: Integral Performance Index for the four groups of municipalities Source: Data from Integral Performance Index of the National Planning Department IDI scores for all groups are “good”100 in both PNCRT and CELI municipalities at midline, up from “low” at baseline. CELI, PNCRT and control municipalities registered increases between 2012 and 2014. Regarding the performance of the index itself, changes from year to year can be large, and are often explained by the existence of strong or weak professionals, such as health and education secretaries, or companies providing potable water and sanitation services. Breaking it down by component, the IDI index’s significant differences appear mainly in three dimensions: efficacy, legal requirements and management. (Figure 5.6) 100 The “good” rating for municipalities is between 60 and 69 points and the “low” rating is between 40 and 59 points. 58 60 60 57 65 67 68 74 0 10 20 30 40 50 60 70 80 CELI Control PNCRT Control 2012 2014 104 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 5.6. Secondary data: Integral Performance Index, 2012 (left) and 2014 (right) Efficacy The efficacy component analyzes the implementation of the development plan and the fulfillment of its goals. In this component, between 2012 and 2014, there is a noticeable improvement in the rating (between 20pp and 40pp) for analyzed municipalities. In 2014, part of the strong performance was that more administrations reported about their compliance with the goals of the development plan. (Only 23 out of 80 reported in 2012, but two years later, 77 of 80 did so.) While there is no way to show that the CELIs are behind this increase in reporting, they did emphasize their support in creating plans for municipal development. Legal requirements The rating of the legal requirements component improved 4pp for CELI and 7pp for control municipalities. This improvement means that the administrations improved in the management of resources of the General Participation System in accordance with current budgetary regulations. In the PNCRT municipalities this score was stable. Management This component assesses municipal capacity to manage resources and turn resources into goods and services for the population. This indicator measures the tools that the municipality has for strategic planning, project monitoring and accountability. The management component score improves in the four groups of municipalities. However, the improvement is much higher in the CELI municipalities than in the control group, around 8pp. The scores of the management component are the highest of the six components of the IDI. Other components The remaining components of the IDI were relatively stable in the studied municipalities. Fiscal performance measures the collection of taxes, debt management and compliance with Law 617 on public finance sustainability. Fiscal performance scores were the most stable, improving by 1pp to 3pp between 2012 and 2014. Efficiency evaluates results obtained with General Participation System, or Sistema General de Participación (SGP) resources in health, education, 105 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative water and sanitation; this component was also very stable. Administrative capacity measures staff stability, professionalization, the availability of computers, process automation and implementation of models and control standards. In this component, scores remained stable and satisfactory. Conclusion 6: The integral performance of municipalities improves across the full sample, with important gains in components of management, effectiveness and legal requirements. This trend is common to the four groups, so it is not possible to attribute progress to any interventions. However, it is reasonable to mention that both CELI and PNCRT interventions included direct and explicit support in the elaboration and implementation of municipal development plans and regional action plans (planes de acción regional, or PARs). Therefore, it is not unreasonable to think that these interventions may have helped the progress of IDI, especially if one takes into account that the management component comprises most of the progress in the index. At the same time, this measure is internal to the GOC. As a contrast, it is useful also to examine the Governmental capacity index from the survey data, in the following section, to balance the government’s own view of its services with the experiences and opinions of service users. From that perspective, much work remains to be done. Governmental capacity index This index measures government capacity and service delivery from the perspective of citizens, rating access and quality of services, quality of public policy formulation, and perceptions of municipal actors’ honesty. The results found for both CELI and PNCRT municipalities show no impacts generally nor by sex or region, and scores declined from 5% to 17% in absolute terms in the CELI municipalities (Figure 5.7). Among the components, the deterioration of scores is worst in respondents' rating of access to services. Figure 5.7. CELI Performance: Governance capacity index, total and regional Ind. Description Region Baseline Midline % Change Source DO3- 013 Governance capacity index Total 40.7 36.2 -11% Household survey • Baseline 2013, Econometría S.A. • Midline 2015, Proyecto EVAL Montes de María 39.2 37.3 -5% Central 39.8 36.2 -9% Norte 42.6 36.2 -15% Sur 40.3 33.4 -17% The difference in trends between DNP data and survey data can be explained because the DNP data measure internal, short-term and administrative decisions and processes that are not necessarily visible to the population in the same period of time. The CELI teams and CORs note the challenges of working with government functionaries who are subject to high levels of rotation with each new administration, and sometimes between administrations. This condition certainly affects all donor-funded projects that involve capacity building or technical assistance to government bodies, and it is to be anticipated. Conclusion 7: From the perspective of the citizenry there is no improvement in government capacity. The analysis shows that governmental capacity is a desirable characteristic but difficult to achieve, and very subjective, as it responds to structural and historical factors that can escape 106 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative the reach of CELI interventions. However, the stagnation of perceptions of government capacity should be taken into account as a sign of the failure of the policies being implemented to consolidate these regions. Social services Citizens’ opinions of social service access and quality have not changed in CELI municipalities. In absolute terms, HHs in CELI municipalities report a worsening of the service of educational establishments (Figure 5.8), with negative impacts on quality perceptions for primary and secondary schools. There were some improvements and even impacts among PNCRT municipalities. Figure 5.8. Impacts on perceptions of social services, CELI and PNCRT Variable CELI PNCRT Schools for children between 5 and 11 years old are good or very good -0.0656** -0.0397* Schools for children between 12 and 18 years old are good or very good -0.0440** -0.0474 The service provided by schools for children between 5 and 11 years old has improved -0.007 0.0822* The service provided by schools for children between 12 and 18 years old has improved -0.004 0.0812* When analyzed by gender, the assessment indicates that there was a negative effect of 5pp in the perception of quality of educational establishments in FHHs. In PNCRT territories, perceptions of schools improved with a positive impact of 8pp. PNCRT municipalities also show a positive 12pp impact on perceptions of primary school improvements in FHHs. In terms of health services, there were no major changes in perceptions of quality. In both the CELI and PNCRT areas and controls, about a third of the population considers health services to be good or very good. Neither report that there has been an improvement of this quality in the last two years. The qualitative fieldwork shows some negative healthcare experiences: One has to be dying to be assisted. I bring patients here because that is my job during the weekend, and there are some of them [healthcare staff] who watch and choose: this one can come in, this one cannot. Health[care] is seriously bad; it is very embarrassing, but one has to tell the truth. (CSO member, Cauca, PNCRT) The Identification and Classification System for potential social program Beneficiaries (SISBEN) coverage has not increased in any of the study areas, and in PNCRT municipalities and their controls, coverage decreased in absolute terms. There is a small increase of 0.1pp of coverage in MHHs. The results of the survey indicate that the government has not completed the first step to improve the provision of social services in the areas of consolidation. Conclusion 8: Perceptions of social services are mostly stagnant, especially with respect to quality. These indicators had little to do with CELI or PNCRT interventions, but do show a fairly stagnant set of perceptions about social services. Combined with the deterioration in public perceptions evident in the government capacity index mentioned above, it is clear that conditions for basic social services are largely unchanged. These conditions contribute to the poverty trap discussed in previous chapters. 107 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Victims' rights In absolute terms, the percentage of HHs experiencing a human rights violation is between 6% and 7% for both CELI and PNCRT households. The only variables that show impact on victims’ rights indicators are in PNCRT municipalities. The first is a 7pp decline in the proportion reporting that the military forces were responsible for human rights violations. Interestingly, the second impact is found in a 5.5pp increase in the proportion reporting that the police were responsible for a violation of human rights. This could simply be a reflection of the change of the occupation forces to the extent that as consolidation advances, the army withdraws, giving way to the police. The qualitative fieldwork conducted shows that the CELIs made active interventions in the personerías, especially supporting technology to register victims, with training, and with personnel. There may be a reduced need for denouncing human rights violations, but there is also the possibility that the effectiveness of the personería offices is reduced due to an increased workload brought by the registration of victims. In this sense, qualitative research provides some clues: Really we didn’t do well... it's not an excuse, but really it is very difficult to complete the victim intake form. To break down one of these forms is exhausting; sometimes you receive three or four reports and there can be 20 reports every week with thousands of functions. (Functionary, Meta, CELI) Conclusion 9: The reports of human rights violations by the military decrease but the reports of police violations increase in PNCRT municipalities. This may reflect a change in force presence in consolidation municipalities, but does not reflect an improvement in the behavior of these forces. Conclusion 10: Qualitative evidence indicates that personerías may be overwhelmed by the workload involved in victim registration. Reported cases of rights violations do not always translate into victim registration and some do not even claim that status. Citizens note the complex procedures to register victims at personerías. Much remains to be done to increase state capacity to improve the attention toward people who were subjected to a violation of their rights. The evidence of the CELI efforts reflects that these efforts, though valuable, have not been entirely sufficient to achieve a real change in the experience of the victims at the time of reporting. Recommendations 1. Limit the scope of future evaluations to factors under implementers’ manageable interest (Low impact findings and contextual knowledge) The CELIs were proposed as part of a whole-of￾government intervention with coordinated GOC support, but on its own it is a complementary intervention. The impact evaluation design posits far larger impact gains than can be measured from the CELI intervention designs alone, and across a much wider population of indirect beneficiaries than the CELIs can hope to affect. This does not imply that the interventions have not helped to improve participation, governance and institutionalization in local contexts; however, there are historic structural failings in the territories that prevent any such intervention from constituting a true model of governance for institutionalization. 108 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Impact analysis should be focused on the reasonable and specific results that the intervention can be expected to make. It is the GOC that must lead efforts for the institutionalization of the territory. Where this institutionalization does not exist, the CELIs can only add support in limited ways. 2. Attempt to ensure through prior diagnostic activities that potential target municipalities are not characterized by clientelistic and corrupt practices (conclusion 1) Detection of clientelistic practices should trigger GOC involvement that precedes donor interventions. Two tools should be employed: • Build efficiency in control entities and sanctions as deterrents for corruption. • Ensure a higher level of anti-corruption control within institutions with transparency measures regarding targets, processes and implementation of social programs. While donor interventions cannot take direct action with such tools, they can modify their own work to reduce the risk of clientelism through its programs, particularly in high-risk sites, as follows: • Focus first on supporting control bodies such as the Attorney General, the Comptroller or local veedurías, as in the JAC trainings, before distribution of resources for specific projects. • Limit financing to projects that meet rigorous standards for transparency and community participation, and avoid implementation through intermediaries. In places where exposure to corruption is high, using intermediaries for implementation increases risk. Municipalities should be required to meet a minimum of socio-political and economic conditions (local government structures, transparency, presence of monitoring bodies and oversight, decentralization schemes, infrastructure, policies to fight poverty and promote social cohesion) that are guaranteed by the GOC. Donor interventions could help by establishing indicators in different areas (e.g. governance, infrastructure) and can also thereby prioritize intervention. Where such conditions do not exist, donor interventions should initiate a stage of state support to meet these standards. 3. Continue to strengthen political participation through social organizations (conclusion 1) The CELIs should continue to support social and political processes for association to strengthen citizens’ knowledge of the channels of political participation and influence in public administration. The following specific strategies and actions would support these ends: • Help women and youth acquire identity documents and register for elections. • Promote women's organizations to encourage participation in all spheres of public life. • Use schools as a site for training, socialization and promoting values, democratic practices and institutions for children and youth with special emphasis on the importance of elections. • Strengthen political parties and ensure their transparency to avoid the concentration of power. 4. Strengthen accountability (conclusions 2, 5 and 6) Strengthen the ongoing practice of accountability to improve public policy orientation, gain citizen trust and provide management oversight: • Train communities and local governments in the practice of setting priorities, performance indicators, and results-based management for day-to-day operations. 109 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative • Generate appropriate spaces in internal, external, vertical and horizontal levels to assist communities to keep tabs on progress on indicators and management. • Strengthen information processes and coordinate with control entities to ensure that communities and authorities can promptly investigate and apply sanctions when mismanagement or corruption are detected. Prosecution is important when funds go awry. • Accompany increased social and legal sanctions with sustained education policies to stimulate a culture of discrediting corrupt practices. • Simplify administrative processes. • Develop participatory budgets. 5. Strengthen citizen consultation and participation from supply and demand sides (conclusions 4 and 5). The fact that citizens do not feel involved and that trust in institutions has diminished requires a renewed focus on consultation exercises and citizen participation. Train and strengthen the exercise of citizenship from one side, while on the other, intervene rigorously to strengthen local government institutions in the same capabilities the citizens are being trained to expect. 6. Continue to work to strengthen the personerías, which have become essential actors in conflict-affected zones, particularly with victim registration The work accomplished in personería offices is an important counterweight to the general decrease in confidence in the government at all levels, most notably in CELI Central. These bodies need broad support to ensure inter-agency linkages and information flows are improved, in particular because of their central role in giving voice to citizens whose rights have been violated, and for whom basic needs remain unmet. 7. Continue to work with municipalities to improve service delivery, and involve citizens in oversight activities USAID and its implementers have a daunting task in their work to strengthen municipal and departmental service provision. There is a national imperative to decentralize, combined with low capacity and funding at municipal levels. Focus on pilot-size activities with significant civil society involvement, and scaffold the local governments with consistent TA. Use scorecards or other measures to involve citizens in rating the service provision changes, and learn from failures as well as successes what practices work best in different sectors and circumstances. 8. Work with the GOC to strengthen the IDI Indicators and indices measuring municipal and departmental performance are necessary and useful for monitoring progress and managing for results. The IDI, however, lacks important consideration of the effects and perceptions of service provision. The same national imperative to decentralize is an incentive to view only the indicators that show a picture of progress, but the rubber meets the road in citizen perceptions of services. 110 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative SECURITY Evaluation question: How much progress has been achieved in the recovery and reconstruction of territory from armed conflicts (IAG) and illicit economic activities? SUMMARY: CELI and PNCRT interventions continue to operate in challenging security conditions but the data show fewer crimes overall and improved perceptions of security in CELI and PNCRT municipalities. However, extortion is on the rise and likely underreported, and coca cultivation has doubled nationally and respondents in CELI and PNCRT zones show more tolerant opinions about the growth and sale of coca. Military and police force levels have changed drastically, and justice services continue to be absent in many of the areas under study. These characteristics leave the zones unstable as before, with the lack of state presence that has – and continues to – put citizens and their livelihoods at risk. At the outset, it is important to note that the CELI interventions undertook no activities in the sectors of security and justice reform. The impact evaluation and this evaluation question were designed when the CELIs were part of the larger CSDI set of activities, along with a promised whole-of-government approach from the GOC. As a result, impacts in these sectors are outside of the CELIs’ manageable interest. Nevertheless, this section aims to respond to the research question through an analysis of the principal security, legal and justice dynamics in consolidation zones. Different modules of the impact survey, information from secondary sources, and transcriptions of focus groups and interviews conducted in municipalities are analyzed. These data are compared with similar data collected in control municipalities. The goal is to present an updated and objective characterization of the risks and challenges that still confront the beneficiary communities of CELI interventions. The chapter describes how CELI and CSDI contributed to consolidation in the regions where it worked and compares indices of violence and legality for treatment and control zones. It analyzes the behavior of the EVIDINCE synthetic security index related to coca cultivation, illegal mining, terrorism and high-impact crimes, and emerging crimes including extortion, theft, and micro-trafficking, among others. The chapter goes on to analyze GOC security and justice-related capacities. This includes a trends analysis of military presence and of public perception of the level of service. It also includes an analysis of the dynamics of the police and justice staffing levels in the municipalities studied. The chapter closes with recommendations based on the findings and conclusions presented. 111 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative The ultimate objective of the CELI interventions is the consolidation of a process for comprehensive and sustainable rural development in the target regions.101 As described earlier in this report, the conceptual framework and USAID’s working hypotheses contemplate that this objective will be achieved through the attainment of three intermediate results: institutional development, social development, and economic development. Security is an exogenous element in the design of the CELIs, and is the exclusive responsibility of the GOC. The CELIs needed to reach areas in which at least a minimum level of security already existed because the CELI operators cannot carry out security-related interventions, which are within the exclusive ambit of the public forces. Security is thus the prerequisite for all CELI interventions. Findings and conclusions Over the last decade, there has been a drastic transformation in security conditions in Colombia in both rural and urban areas. Reduction in some crimes has converged with an increase in others, the recovery of legality in some regions, and the return of others to criminal control. The evidence includes a security index, analysis of coca cultivation, trends in major human rights crimes, and new criminal dynamics. Synthetic index of insecurity EVIDINCE created an index of insecurity, synthesizing criminal acts (homicide, robbery, larceny, illegal mining) with subjective elements of the perception of security, such as the appearance of graffiti in the communities. 101 Consolidation Process for Comprehensive and Sustainable Rural Development: see the Midline Impact Evaluation of USAID’s Consolidation and Enhanced Livelihood Initiative (CELI): Work Plan and Design. 112 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 6.1. Secondary data: Insecurity Index Source: EVIDINCE. The index combines data on homicide, robbery, larceny, illegal mining managed by armed illegal groups (IAGs), and the presence of graffiti alluding to IAGs in the municipalities. Treatment 1 (T1) are CELI municipalities, and C1 their controls. T2 are PNCRT municipalities, and C2 their controls. Insecurity dramatically increased in CELI municipalities in 2014, placing it significantly above the indicator for the control municipalities (Figure 6.1). Both CELI and their controls saw declines in insecurity in 2015. In PNCRT municipalities, the insecurity index shows a steady decline from the end of 2013 until the middle of 2014, when there is a slight rise, which is also present in their control municipalities. Since August 2015, the EVIDINCE insecurity index shows a generalized decline in all municipalities studied. This may be explained by the de facto truce in the final stages of the peace talks between the GOC and the Revolutionary Armed Forces of Colombia (FARC). Data from the Consolidation Index show an increase in the sub-index of security from 43 to 49 points over 100. This sub-index is measured by the degree to which police have supplanted military operations in the area, the existence and implementation of a security plan for the municipality, presence and density of coca, and subversive and terrorist acts. While the latter two have declined precipitously since 2011, coca cultivation has increased and police presence is still inadequate. Security plans are partially in place in most sites. There has been a similarly small but positive change in human rights indicators used in the Consolidation Index, composed of the rate of killings of prominent people, massacres, anti￾personnel mine incidents, displacements, and extortionate kidnappings; and the existence of a municipal human rights committee. This score went from 59.7 to 66.9 from 2011 to 2014. 113 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Conclusion 1: The indices may remain stable while their components have opposite or misleading trends. The advantage of synthetic indices is that they allow for the integration of multiple variables and present an overall image, giving a better picture of how living under the measured conditions is likely to be experienced by residents. The disadvantage is that opposing dynamics are neutralized, and so what may be occurring behind the stability in the trends is the criminal displacement of one activity to another. The index also has fewer components that show new criminal trends, so the decline in crime per se is somewhat misleading. The index serves as a starting point for analysis, with dynamics on major crimes, but is not a definitive or final measurement of security dynamics. Coca cultivation Coca cultivation in Colombia reached a historic low in 2013 (according to both the Integrated Illicit Crops Monitoring System (SIMCI) and Crime and Narcotics Center (CNC) measurements).102 In the two years since, however, the growth has been sustained and drastic. In the last CNC measurement, cultivated hectares doubled, from 80,000 to 159,000, with a proportional increase in the production of cocaine, from 290 to 442 metric tons. The SIMCI￾United Nations Office on Drugs and Crime (UNODC) figures show that the increase in coca crops in 2014103 was greater in the CELI and PNCRT municipalities than in their control groups, as indicated in Figure 6.2: Figure 6.2. Secondary data: Evolution of coca cultivation Source: SIMCI-UNODC 102 The Crime and Narcotics Center of the Central Intelligence Agency (CIA) produces an annual estimate of the area cultivated with coca for the Andean region. Its latest results were presented to the GOC in March 2016. The main conclusions – national statistics – were disseminated in the media. The complete report is not accessible to the public. The statistics cited here are found in the articles published in the El Tiempo newspaper, Semana magazine and El Espectador newspaper on March 15, 2016 (http://www.eltiempo.com/mundo/ee-uu-y-canada/ee-uu-advierte-de-aumento-de-cultivos-de-coca-en-colombia/16537123). 103 2015 data are not yet public, but perhaps will be in time for the regional CELI reports. Nevertheless, the data should be understood to represent the municipality, with no detail on whether the increased coca is actually found in CELI intervention areas. 114 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative We also see that the presence of illicit crops (ICs) was greater in the PNCRT municipalities than in the CELI municipalities since 2010. In addition, the level of ICs in the PNCRT municipalities has been higher than that in their controls. There was a sharp drop in the general trend of ICs in the PNCRT municipalities between 1998 and 2004. Since then, the level of ICs by municipality ranged between 500 and 1,000 hectares. However, beginning in 2012, when the UACT and CELI interventions began, it is possible to observe a steady, growing and pronounced trend of ICs in the PNCRT municipalities. For 2014, the average IC level was close to 1,500 hectares per municipality in the regions in which only the PNCRT operated. The midline evaluation data show an increased perception of IC presence in veredas. (Figure 6.3) Figure 6.3. Trends in reports that there are ICs in the vereda, 104 CELI and PNCRT Treatment Control Baseline Midline Baseline Midline CELI 8% 10.1% 12.3% 10.7% PNCRT 22% 25% 7.5% 6.5% The increase in perception of ICs is statistically significant, and the perceptions corroborate the trend of a greater increase in the CELI municipalities, which is also observed in the SIMCI reports. We also find that in the CELI municipalities, more FHHs than MHHs (by 9pp) perceive this increase in ICs in their areas. In PNCRT municipalities, the proportion of households that reported the existence of ICs in their village rose from 22% to 25% between baseline and midline. In contrast, this proportion declined in the control zones in the same time period. This results in an impact of 5.8pp in perceptions of the presence of ICs in PNCRT municipalities. The impact evaluation also included two list experiment questions: a particular survey technique used to get frank responses to sensitive questions, in this case about interactions with armed groups and with illicit crops. These results, though not attributable to interventions, are available in Annex 7. In sum, they show no impacts from CELI or PNCRT on interactions with IAG or with illicit crops, but a general trend away from these practices. This may be a result of citizens who answer what they believe the surveyors “want to hear,” or may reflect the evaluation sample’s and the CELIs’ less dangerous municipalities and veredas (because of GOC insistence on not working in the most dangerous areas), or some combination of factors. Also notable are the survey results showing the erosion of respondents’ baseline agreement that illicit crops should be avoided. Figure 6.4 shows that their previous rejection of these crops has been eroded since 2012-2013: 104 Throughout, the data source is the evaluation’s survey data, unless otherwise noted. Standard significance notations apply: *** for 99% confidence, ** for 95%, * for 90%. Impact estimation models are in Annex 3, and data in the text are tabled in Annex 10. 115 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 6.4. Trends in perceptions about illicit crops in their communities, CELI Conclusion 2: CELI and PNCRT municipalities are seeing a rise in illicit crops as part of a larger national phenomenon. While the midline evaluation of the CELI and PNCRT interventions found increases in the perception of the presence of ICs, these impacts reflect a national trend. There are many hypotheses about the causes of the increase: i) the low profitability of other legal and illegal activities, such as illegal mining; ii) the reduction in aerial spraying and manual eradication campaigns; and iii) a boom in cultivation prior to the signing of a peace agreement that would enable the legalization of capital obtained illegally (a practice documented by some FARC commanders) (Wilches and Garzon, 2016). In the case of the PNCRT municipalities, for example, ICs have historically been more present in treatment than control zones. This may mean that these zones have a greater risk of an increased presence of ICs, since that is where the know-how, agricultural and geographic conditions, criminal networks, and the institutional gaps that make the proliferation of illegal plantings possible exist. While control municipalities were selected to parallel these conditions, unproblematic comparisons are notoriously difficult to construct. Terrorism and high-impact crimes The gradual dismantling of paramilitary groups and the operational logic of the BACRIM (criminal bands), added to the continuing ceasefire between the GOC and the FARC, have helped to reduce the number of cases of human rights violations throughout the county, especially high-impact crimes, such as massacres and acts of terrorism. Colombia Ministry of Defense figures for the last four years show a 68% reduction in massacres, from 34 to 13 cases, 92% 94% 94% 93% 60% 77% 79% 76% 79% 61% 0% 20% 40% 60% 80% 100% Coca, poppy and marijuana shouldn't be cultivated because they're illegal Producing illict crops negatively affects our families and community Growing coca is not the only way for families to guarantee their subsistence My family and friends think growing coca is bad Our community is willing to denounce the presence of illicit crops CELI Baseline CELI Midline 116 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative and from 171 to 54 victims.105 Massacres are one of the most drastic forms of violence used by illegal IAGs to intimidate the population, generate displacement and control territory. There has been a 61% reduction in massacres in the CELI municipalities – similar to the national trend – which means an impact of around 7% of the decline in the national total of massacres. Furthermore, there has been a parallel reduction in guerilla attacks over the last decade in the control and treatment municipalities. Municipalities saw an uptick in 2012 prior to the FARC truce, and a new upturn in 2014, mainly because of the increase in attacks by the National Liberation Army (ELN). The trend of guerrilla attacks in the PNCRT municipalities was similar to that in CELI municipalities. (Figure 6.5) Figure 6.5. Secondary data: Average number of attacks by guerrilla groups, per municipality106 Source: Conflict Observatory, Universidad del Rosario A recent study by the Conflict Analysis Resource Center (CERAC)107 estimates that the peace process has led to a reduction of 1,500 deaths. This strengthens the argument that the crimes traditionally committed by IAGs are on a downward trend. Absolute figures from CELI municipalities confirm those trends. (Figure 6.6) 105Achievements and Challenges of the Defense Sector (p. 29) https://www.mindefensa.gov.co/irj/go/km/docs/Mindefensa/Documentos/descargas/estudios%20sectoriales/info_estadistica/Logr os_Sector_Defensa.pdf. 106 The data available from the CERAC and the Ministry of Defense indicate a reduction of 92% and 95%, respectively, in the number of FARC attacks in 2015, which shows the impact of the negotiation process on the reduction of guerrilla actions. 107 http://www.cerac.org.co/es/. 117 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 6.6. Trends in proportion perceiving security problems, CELI The trends are parallel across the full sample: perceptions of all of these crimes have reduced dramatically. Other crimes Extortion Following the fragmentation of paramilitary groups (converted into BACRIM) and their sporadic criminal alliances with the guerrillas, extortion has become the fastest growing crime in Colombia. However, communities’ perceptions of extortion as a serious crime are low, and are on a downward trend in both CELI communities and their controls. The proportion who believe that extortion is one of the most serious security-related problems declined from 9.6% at baseline to 5.5% at midline. However, though the statistics do not reflect it, qualitative evidence shows that extortion is a growing concern in CELI municipalities: Extortion is returning in many places. The case here is that the people are now looking at it as very normal, unfortunately. So the people generally pay their extortion, the farmer, the merchant, nothing happens, and they wait to be charged again. (Functionary, Antioquia, CELI) Rates of reporting of extortion are considered low, relative to incidence. For example, only 5,304 cases were reported in all of Colombia in 2015 (Ministry of Defense),108 despite anecdotal evidence of the wide and rapid spread of the crime nationwide. For example, the national media have widely covered the FARC’s change in strategy in this regard. In recent years, this group has 108https://www.mindefensa.gov.co/irj/go/km/docs/Mindefensa/Documentos/descargas/estudios%20sectoriales/info_estadistica/Lo gros_Sector_Defensa.pdf. 19% 46% 7% 16% 4% 9% 42% 1% 2% 58% 8% 20% 4% 10% 1% 6% 21% 1% 1% 57% 0% 10% 20% 30% 40% 50% 60% 70% Homicides Illegal armed groups Anti-personnel mines Displacement Kidnapping or forced disappearance Extortion Citizen insecurity - robbery, muggings, etc. Forced recruitment Sexual violence There are no serious security problems here CELI baseline CELI midline 118 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative given up kidnapping in order to pursue the extortion of merchants and transporters.109 The low rate of reporting is considered related to the inability of public forces to measure and combat the crime. Illegal mining A second crime of relevance for the transformation of territorial security conditions is illegal mining. Though there are no local measurements of illegal mining in Colombia, its growth is a widely recognized issue of national policy. The surveys also reflect changes in perceptions of illegal mining between baseline and midline. (Figure 6.7) Figure 6.7. Trends in reports of illegal mining, CELI and PNCRT For CELI municipalities and controls, perceptions of illegal mining dropped from baseline to midline – from 9% to 6% in CELI sites, and from 7% to 5% in control zones. PNCRT municipalities reported increased illegal mining, from 3% to 6%. Disaggregation by sex indicates that the effects in the CELI municipalities are higher among FHHs. Theft, micro-trafficking and prostitution Other crimes associated with the dynamics of citizen security in large cities also appeared in focus groups: car theft, drug micro-trafficking centers and prostitution. Regarding drugs and certain dealers… the police have made searches and seizures and are after them. But they have had a huge effect on us because they were very healthy kids; nowadays we have some kids who are very lost in that, and I think that now we will have to look at what is going to be done with them: drug use is proliferating a lot. (CSO member, Montes de María, Control) Focus groups indicated the existence of conflicts and criminality that are outside the normal orbit of the armed conflict. They mentioned deforestation, the droughts caused by mining, and domestic violence, which may be factors to be considered for the redesign of the objectives of 109 El Tiempo, Emails Reveal the FARC's Extortion Plan - Justice - El Tiempo, 2014, http://www.eltiempo.com/politica/justicia/plan-de-extorsion-de-las-farc-a-mineros/14606819. 9% 7% 3% 6% 6% 5% 6% 3% 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% CELI Control 1 PNCRT Control 2 Baseline Midline 119 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative the local programs and the mechanisms for capturing and recording the dynamics of illegality, conflict and violence in the territories. The survey data, however, show absolute reductions in the perception of citizen insecurity, halving baseline figures across the full sample. The security challenges in the CELI zones and their respective control groups have changed significantly: the replacement of some crimes for others110 is observable in perceptions and secondary statistics. Increases in some crimes, in some cases (extortion, illegal mining, theft and corruption, among others) have occurred simultaneously with the reduction in other types of crimes (homicides, displacement, massacres and terrorism) more uniformly across the Consolidation Zones. We can track the older manifestations of insecurity and illegality, but the new ones are not visible to, and are under-recorded by, available capture tools. Conclusion 3: Security and violence indicators and their monitoring in consolidation zones have not been adjusted to capture the changed dynamics in the population. The scheme for monitoring the day-to-day security and justice needs of the communities is outdated and misses the expansion of certain organized criminal activity. Further, communities’ vulnerability to criminal organizations is expressed not only in the dynamics of violence, but also in economic arrangements that are disadvantageous for citizens – monopolies and the diversion of resources – dynamics for which there is little evidence and no observation or monitoring tools. Security- and justice-related capacities The security dynamics depend on two general factors: the first is the capacity of IAGs to exert armed control to exploit economic opportunities; and the other is the police, military and justice capacity to neutralize the expansion and transformation of these criminal organizations. These capacities and their evolution and level of interinstitutional coordination are important factors that formed a fundamental part of the territorial consolidation and crop substitution strategies. Military capacities It is not possible to gain access to operational reports on the availability of public forces. Prevailing military doctrine in the Ministry of Defense is one of confidentiality, so as not to reveal local operational capacities, publicize investments or personnel segmented by territory, or disseminate statistics related to mobility and territorial control capacities. The official statistics are limited to the total number of troops and the budget execution. The evaluation survey gives one perspective on military presence. There was a drastic decline in communities’ perceptions of military presence in CELI, PNCRT and control municipalities. From 87% at baseline, only 37% of CELI municipality households asserted that there were military forces in their communities at midline. The trend was parallel in their controls: a decline from 83% to 29%. PNCRT municipalities showed a similar but less drastic trend, from 77% to 40%. 110 The assailants change their behavior in order to avoid risks or because their current activities have ceased to be profitable. 120 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative While perceptions of military presence fell drastically, in the CELI municipalities, there was a 6pp positive impact when asked whether military services had improved over the previous two years (32% to 39%), with no change in control municipalities. More FHHs shared this perception than MHHs (8%). PNCRT and their control municipalities showed similar trends. Operation Sword of Honor, a 2013 military strategy with the objective of capturing FARC leaders, spread the military throughout a larger area, reducing the level of protection for population centers and roads. It has been argued that as a consequence of this operation, in areas that had already been secured, IAGs were able to return and exist alongside the public forces. The dispersion of military capacities may explain the drop recorded in the midline survey in the perception of the presence of military forces in the veredas. Ministry of Defense and General Command statements report that after several years of discussion they never agreed on a definition with the UACT of consolidation municipalities. This led to the creation of local development task forces coordinated by the Army’s Tactical Brigades. Rather than supporting UACT zones and priorities, military policy expanded to include its own development programs. This lowered potential coordination between the military and the PNCRT and CELIs, which helps explain the low levels of perceived presence of the military in the veredas included in the study. Conclusion 4: There is a lack of coordination between military forces and the PNCRT/CELIs. The dispersion of military forces, perhaps occasioned by Operation Sword of Honor, appears to have resulted in a drastically reduced military presence in Consolidation zones. Police presence The most significant transformations in police presence occurred between 2002 and 2004. Permanent facilities were established in 121 municipalities that previously lacked police presence. The expansion included certain veredas where IAGs committed war atrocities. Ministry of Defense records show National Police officer increases from 159,000 to 183,000 from 2010 to 2015.111 This 9% national-level increase was not, however, observed in CELI municipalities and controls. One focus group respondent discussed the lack of public force presence and the continuing difficulty for displaced citizens to return: There’s a structural issue here, that of security, which the State has yet to guarantee, for our safe return. What needs to be done is a set of structural solutions so that these zones are safe for owners to return to, and it’s an issue of the public forces. (CSO leader, Antioquia, CELI) The figures compiled by EVIDINCE in municipal panel data show a leveling off of police presence in treatment and control municipalities over the past three years. 111 Ministry of Defense (2015); Security Strategy for the New Colombia available at: https://www.mindefensa.gov.co/irj/go/km/docs/Mindefensa/Documentos/descargas/Prensa/politica_defensa_nuevacol2015.pdf. 121 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative Figure 6.8. Secondary data: Police presence Source: EVIDINCE municipal panel The survey results support this decline in perceptions of police presence across the full sample, as drastic as the decline in military force presence. CELI municipalities and their controls saw about two-thirds of their respondents citing the presence of police services at baseline, and only a quarter at midline. PNCRT and their control municipalities had about half of their respondents affirming the existence of police services at baseline, with a decline of 35% to 40% reporting the same at midline. Police capacities have two important dimensions: the first is the involvement of police officers in surveillance and patrolling, and the second is specialized services (anti-kidnapping and anti￾extortion, anti-narcotics, and SIJIN [local judicial and investigative police]). In general, small￾and medium-sized municipalities have only surveillance police, and departmental capitals provide judicial investigation capacities, depending on the crisis or specific need. In terms of investigative capacities and the fight against organized crime, the police are profoundly centralized, carrying out only temporary and localized operations in the territories. No evaluation of capacities is complete without the monitoring of these two operational lines, which, although they depend on the same entity, are very different. Unfortunately, the reports on the police presence do not disaggregate the presence by specialty, and so we cannot know whether the increase or reduction in the number of police corresponds to the surveillance component or the judicial police and specialty component. Availability of justice services The Consolidation Index shows improvement since 2011, from 47.7 points to 58.8 points over 100. The sub-index on justice conditions is composed of two indicators: the presence of the judicial apparatus in municipalities, and the use of mobile justice brigades and/or equity conciliators in rural sites. Other data, however, contradict the presence of the apparatus or other options for dispute resolution. Survey respondents from CELI, PNCRT and control households represent a more rural sample than the municipalities reported in the Consolidation Index. These survey data show a 122 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative precipitous decline in the presence of formal justice systems in their municipalities, at even a greater rate of decline than the lack of security presence: from some 70% to 80% at baseline, to 8% to 17% at midline. There was no significant change in their perceptions of the quality of the services. Fewer respondents say they would access justice through formal channels than at baseline in absolute terms, even through the JACs (which appear to provide conflict resolution services at the local level). However, these trends were the same across the full sample. Conclusion 5: Both police presence and access to justice services – key areas of state presence – have declined in all areas surveyed. The decline of police presence is notable in the regions, and information is not available that reveals which type of police capacities are available to rural inhabitants in areas surveyed. At the same time, judicial capacities, which at the local level are represented by various entities and their level of coordination in the territory, also appear to have been greatly reduced. The measurements are not unified or complete enough to establish a comprehensive statistical series of all of the actors who must ensure access to justice (the Attorney General’s Office, Judiciary, the National Ombudsman’s Office, Legal Medicine, Police Inspectors, Family Commissioners’ Offices, and the Public Ministry, among others). For that reason, measuring access to justice from the perspective of the presence of judges may be an oversimplification. These measurements of the justice conditions at a local level have been an ongoing challenge for the Colombian state, and have been placed at the top of the government’s agenda. Therefore, one of the commitments in the National Development Plan is the construction of an Index of Unmet Legal Needs that can be used to assess and adjust judicial capacities. Conclusion 6: There is no convergence between the security and justice agendas and the CELI and PNCRT programs, based on the availability of public forces and judges. Evidence of this is the creation of brigade units within the military that duplicate the work of the CELIs, as well as the absence of a specific strategy for the CELI zones. The UACT and CELI programs ultimately lacked consistent coordination with the security and justice component under which they were originally designed, though some regions may have had better coordination than others. The intervention zones did not have localized security programs, strengthened public forces, or an expansion of institutions for access to justice. They also had no guarantees for the permanent presence of the military and police outside the urban centers. Security assistance disappeared from the intervention map. Recommendations 1. Support the creation of a local index of legal needs (conclusion 5) USAID should consider supporting the development of an index of unmet legal needs. This index would serve as a very important diagnostic tool for strengthening access-to-justice conditions, a dimension for which a comprehensive measurement tool does not yet exist. 2. Adjust the interventions in order to reach the areas with the greatest coca cultivation (conclusions 2 and 3) The drastic growth in coca cultivation throughout the country (which doubled between 2012 and 2015) has had an impact in treatment and control sites and the entire country. There is no single cause for this in all of the regions, but there is a need to review recent policy decisions and strategies for controlling the supply of drugs. Expanding cultivation reduces the chances of success of the restoration of legality in the territories. 123 Midterm Impact Evaluation of the Consolidation and Enhanced Livelihood Initiative There is an enormous territorial gap in the programs for reducing the supply of drugs and replacing illicit crops. Very few of the coca growers access state programs, and productive projects designed for these citizens do not generate the necessary incentives for returning to the path of legality. It is necessary to reconsider development projects’ outreach to the most vulnerable communities, and to expand to reach those that are even less accessible and who may have more vulnerability to the coca value chain. Investigate construction of a new micro-targeting model that is inclusive and viable for all veredas in targeted zones and reflects the changing realities of crime. 3. Update security indicators to capture changing dynamics (conclusions 1 and 3) Performance indicators and EVIDINCE municipal panel security data should be adjusted in order to capture new criminal dynamics and to provide an updated diagnosis of the security challenges, including new forms of criminality. Variables should be weighted to prevent the territorial consolidation index from neutralizing any opposing changes in the security variables, to better reflect the security reality that citizens experienced. Support constant monitoring to guide the management of the security agenda. The criminal transition entails a reduction in direct violence and the use of indirect force, something seen in Antioquia and BACRIM control zones. For this reason, the security analysis should be combined with tools for analyzing market and movement freedom in communities. The GOC and USAID should work together to redefine the concepts of IAG territorial presence. Security tools and indicators will need to identify criminal control in the form of monopolies or oligopolies in affected communities, and not the de facto territorial control that was combated in the past. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523