DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ARTISANAL GOLD MINING ACTIVITY (ORO LEGAL) PERFORMANCE EVALUATION FINAL REPORT December 2021 MEL ACTIVITY Contract Title: USAID/Colombia Monitoring, Evaluation and Learning activity Contract Number: 72051419C00001 Submitted: December 3, 2021 Contractor: Panagora Group, Inc. COR: Omar López Authors: Manuel Fernando Castro (Evaluation Director), Jairo Herrera (Mining Specialist), Luis F. Monroy (Value Chains Specialist), Maria Andrea Rueda (Environmental Specialist), Mauricio Aguilar (Evaluation Specialist), Oscar Quiroz (Quantitative Specialist), Camila Chavarria (Qualitative Specialist), and Juliana Moreno (Qualitative Analyst). Orlando Gracia, Luisa Fernanda Cardona, Juan Guillermo Bedoya, Yazmin Medina and David Doncell. Panagora Group Contacts: Ana María Rivera Chief of Party Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: arivera@panagoracolombia.net Orlando Gracia Lead Evaluation Expert Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: ogracia@panagoracolombia.net Francis Okello, PhD Director, Monitoring, Evaluation, Research, Learning and Adapting Practice 8601 Georgia Avenue, 8th Floor, Silver Spring, MD, USA Email: francisokello@panagoragroup.net CONTENTS ACRONYMS I ABSTRACT 1 EXECUTIVE SUMMARY 2 A. EVALUATION PURPOSE 14 B. BACKGROUND AND DESCRIPTION OF THE ACTIVITY 15 C. EVALUATION QUESTIONS AND METHDOLOGY 18 D. LIMITATIONS OF THE EVALUATION 23 E. FINDINGS AND CONCLUSIONS 24 F. RECOMMENDATIONS 76 ANNEX 1: METHDOLOGICAL DESIGN APPLIED TO ANSWER THE EVALUATION QUESTIONS 89 ANNEX II: QUANTITATIVE DATA COLLECTION METHODS AND CHALLENGES 104 ANNEX III: SUPPLEMENTAL ANALYSIS AND DATA 114 ANNEX III: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX 131 ANNEX IV: DETAILED METHDOLOGY 155 ANNEX V: DATA COLLECTION INSTRUMENTS 216 ANNEXI VI: SOURCES OF INFORMATION 298 ANNEX VII: SCOPE OF WORK OF THIS PERFORMANCE EVALUATION 301 ANNEX VIII: SUMMARY INFORAMTION ABOUT EVALUATION TEAM MEMBERSBERS 327 ANNEX IX: SIGNED DISCLOSURES OF CONFLICTS OF INTEREST FROM EVALUATIONTEAM MEMBERS 330 i | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV ACRONYMS ANM National Mining Agency ASGM Artisanal and Small Gold Mining ASM Artisanal and Small-Scale Miners BGI Better Gold Initiative BioREDD+ USAID/Colombia Biodiversity-Reduced Emissions from Deforestation and Forest Degradation Program CAR Regional Autonomous Corporation CC Community Council CLA Collaboration, Learning, and Adaptation COR Contracts Officer Representative CORANTIOQUIA Central Antioquia Regional Autonomous Corporation CORNARE Negro and Nare Rivers’ Basins’ Regional Autonomous Corporation CSO Civil Society Organization EIA Environmental Impact Assessment EVA Municipal Agricultural Evaluations EVOA Evidence of Alluvial Gold Exploitation FARC Colombian Armed Revolutionary Forces GIS Geographic Information System GOC Government of Colombia Ha hectare IP Implementing Partner MADR Ministry of Agriculture and Rural Development MADS Ministry of Environment and Sustainable Development MEL Monitoring, Evaluation, and Learning Activity MFP Mining Formalization Program USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | ii MME Ministry of Mines and Energy MPU Mining Production Unit NAMA National Appropriate Mitigation Actions NGO Non-Governmental Organization OL Artisanal Gold Mining Activity (Oro Legal) PIRS Performance Indicator Reference Sheets SENA National Learning Service SoW Statement of Work SWOT Strengths, Weaknesses, Opportunities, and Threats Analysis TA Technical Assistance UPA Agricultural Production Unit USAID United States Agency for International Development USG United States Government WWF World Wildlife Fund 1 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV ABSTRACT The Oro Legal (Legal Gold) Activity worked to improve the capacity of the government, local communities, and the private sector to address the informality of artisanal and small-scale gold mining (ASGM) and reduce and reverse its environmental impacts in 22 municipalities in two departments of Colombia (Antioquia and Chocó). The Activity aimed to i) strengthen governmental capacity to facilitate compliance with mining regulations; ii) increase the participation of artisanal mining associations and community groups in mining formalization; iii) provide training and technical assistance to enable miners to formalize their operations; iv) rehabilitate areas that had been degraded by mining; v) identify alternative income-generating activities; and vi) improve the quality of potable water in targeted areas. This evaluation examines whether the achievements of the intervention are perceived as effective and impactful, identifies the factors that influenced its performance, and determines the contextual factors that affect the sustainability of the results, with a specific focus on responding to the six evaluation questions. The evaluation uses mixed methods (quantitative and qualitative) complemented by Geographic Information Systems (GIS) tools and informed by a Collaborating, Learning, and Adapting (CLA) approach throughout. The qualitative portion of the methodology combined a technique drawn from the phenomenology of perception with a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis, drawing on semi-structured interviews with a range of actors involved in the Activity. The quantitative approach included descriptive statistics, spatial econometrics, multiple correspondence analysis, grouping of observations, and text mining from beneficiary surveys and OL’s Monitor system. This evaluation found that beneficiaries have a high level of satisfaction with OL, highlight its strengths, and perceive its interventions as relevant to the context and needs of the targeted territories. However, the evaluation also identified a series of challenges and opportunities for improving implementation, design, and operation of this type of intervention. The evaluation proposes 43 recommendations, with 100 implementation measures to put these recommendations into action. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 2 EXECUTIVE SUMMARY Small-scale and artisanal gold mining (ASGM) in Colombia is an activity that, besides being highly affected by and connected to illegal activities, has a significant negative environmental impact. About 80 percent of the gold produced in the country comes from artisanal and small-scale miners. According to a sector census between 2010 and 2011, 72 percent of mining production units (MPUs) were small-scale, 63 percent did not hold title to their mine, and 75.7 percent did not have an environmental license. The mining sector, moreover, has little capacity to regulate these issues. This situation is exacerbated by the presence of illegal armed groups that control and engage directly in a host of illegal activities, including the illicit extraction of minerals, mercury trafficking, cocaine production, illicit logging, and land grabs. Mercury is widely used in ASGM, despite being prohibited in mining since 2018, and produces a high level of environmental and socio-economic degradation, in addition to enormous risks to human health. In 2011 Colombia was estimated to be the largest per capita mercury polluter in the world. The country is a signatory to the Minamata Convention on Mercury, and Law 1658 of 2013 lays out a roadmap to eliminate mercury use. Nonetheless, enormous challenges in overseeing the chain of control and low regulatory capacity persist. The Office of the Comptroller General of Colombia estimates that the national government has made progress amounting to 46.72 percent toward totally eliminating mercury, with this being about 90 tons per year.1 Reliance on mercury occurs in large part because ASGM is often carried out in remote regions under the control of illegal armed groups, by small producers for whom stopping the use of mercury is a long-term process that implies accepting behavioral changes in their work to achieve economic benefits that are not immediate. In addition, small miners are not typically trained in appropriate techniques and lack access to skills and mercury-free technology. Oro Legal (OL) aimed to improve the capacity of the government, local communities, and the private sector to address informality in ASGM and reduce its environmental impacts. It pursued this goal through two core strategic objectives and six results areas. The first objective was to improve the governance of gold extractive activities through results areas that included (1.1) strengthening government capacity for compliance with mining regulations, (1.2) increasing the participation of mining associations and local communities in the formalization program, and (1.3) building the capacity of the miners through training and technical assistance. The second objective focused on strengthening the capacity of government, communities, and the private sector to manage environmental impacts through (2.1) rehabilitation of degraded areas by illegal mining, (2.2) the generation of alternative livelihoods, and (2.3) improving the quality of potable water. With a budget of $22.1 million, Oro Legal worked in 22 municipalities in the departments of Antioquia and Chocó. It continued the work of the mining component of BioREDD+, a previous USAID initiative that was active from 2011 to 2015. Covering five years of implementation (2015-2021), this evaluation of OL responds to specific evaluation questions about the Activity’s performance. These questions include: i) evaluate to what extent the achievements of the intervention are perceived as effective and as having an impact, ii) identify the factors that affect the components and sub-objectives, and iii) determine the contextual factors that affect the sustainability of the results achieved. The methodological approach examines the support that OL provided to beneficiaries and partners in terms of mining formalization, mining governance and 1 Contraloría General de la Republica CGR (2020). Evaluación de avances y resultados en la implementación del Plan Estratégico Sectorial para la eliminación del uso del mercurio. 3 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV policy, promotion of alternative livelihoods, rehabilitation of degraded areas, and the reduction/elimination of mercury in ASGM. The evaluation uses a mixed-methods approach, which entailed collecting primary and secondary qualitative and quantitative data, and a focus on the collaborating, learning, and adapting (CLA) approach throughout the process. Moreover, the evaluation used Geographic Information Systems (GIS) to interpret spatial and geostatistical information and georeferenced data for analysis, particularly data on the rehabilitation of degraded areas. This report presents findings and conclusions2 in response to evaluation questions and suggests targeted recommendations. The conclusions on formalization and mining governance respond to the following questions: i) To what extent do the formalization processes provide sufficient incentives for miners involved in ASGM to remain formal and improve the environmental performance of their mining operations? and ii) What have been the main internal and external factors that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance? As the process of formalization and its incentives have a close relationship with the policies that govern the mining sector, these first two questions are combined in the same section. The questions about the Activity’s other components are as follows: iii) How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well-being outside of ASGM activities?; iv) the relevance and effectiveness of the different models of land rehabilitation used in the program; v) the instruments and approaches that were most effective in reducing the use of mercury in ASGM; and, finally, vi) the perceptions of different actors regarding the relevance and effectiveness of OL. The evaluation findings are synthesized into 37 conclusions, 43 recommendations, and 100 implementation measures. The conclusions and recommendations seek to identify and disseminate lessons learned to inform USAID/Colombia decision-makers in designing and implementing similar activities. The principal findings respond to the evaluation questions, corresponding to components of the program, as follows: MINING FORMALIZATION AND GOVERNANCE Oro Legal formalized 146 MPUs, 53 in Antioquia and 93 in Chocó reaching an overall score of 108 percent in its performance target. MPUs with mechanized operations, access to public services, and which completed a works and operation plan (PTO), achieved on average, a greater degree of formalization. Twenty of the 53 MPUs formalized in Antioquia were in Segovia, five in Remedios and Vegachí, eight in Barbosa and Don Matías, three in Zaragoza, four in Tarazá, five in El Bagre and eight in Buriticá. Fifty-one of the MPUs formalized in Chocó are in Tadó, 27 in Unión Panamericana, 10 in Cértegui, and five in Condoto.3 The econometric models of the determinants of mining formalization 2 Conclusions are integrated within the findings section as titles under each evaluation question. 3 MELA used the final report of Oro Legal for the total number of MPUs intervened, formalized, and their respective departmental breakdown (see "Final Report Artisanal Gold Mining - Environmental Impact Reduction Activity (Oro Legal" July 2021. https://land-links.org/wp-content/uploads/2021/07/USAID-Colombia_Oro-Legal_Final-Report.pdf). Unfortunately, the numbers included here were not available during this evaluation's data collection and analysis period. Therefore, the evaluation team could not triangulate this information with the other sources of data used. However, MELA's evaluation component included these numbers in the report following a requirement made by the COR of the Activity. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 4 found that MPUs with mechanized operations scored on average 14.7 points higher on the formalization indicator than MPUs with traditional operations. Location and proximity to groups of formalized MPUs created hotspots for formalization. In addition, access to public services and good practices like saving receipts and preparing a PTO were associated with higher levels of formalization. MPUs that saved their receipts and MPUs that prepared a PTO scored, respectively, 10.4 and 8.2 points higher on the formalization indicators than those that did not.4 In general, beneficiaries were highly satisfied with the program and mentioned its strengths. In the survey, 72 percent of beneficiaries reported great satisfaction with the work of OL to facilitate formalization (beneficiaries particularly highlighted efforts to assist with regulatory compliance and cleaner environmental practices). Most mining formalization beneficiaries were satisfied with OL, and particularly highlighted the program’s contribution to implementing PTOs and obtaining mining titles. They also mentioned OL’s contributions to publicizing the benefits of formality and its assistance in overcoming disincentives to formalize. Beneficiaries learned modernized mining techniques and adjusted to greater regulatory compliance, which greatly reduced their risk of enforcement actions by authorities. OL contributed to improve environmental performance by supporting the impact assessments needed for environmental licenses and the labor rights and practices. This was recognized by environmental authorities, who consider these documents as critical guidelines to protect the environment. Similarly, OL helped implement cleaner and more efficient gold processing techniques. The program helped to introduce good business practices that advanced the rights of workers and ethical practices. Beneficiaries reported making greater commitments to the health of their workers and to their constitutional right to physical integrity, a shift that is considered difficult for ASGM. In addition, 28 percent of miners surveyed identified protection from work-related risks as one of the main benefits of formalization. Beneficiaries who obtained their mining title and environmental license improved their incomes. Of the beneficiaries surveyed, 39 (34.5 percent)5 out of 113 MPU declared they got a mining title and environmental license and also increased their income. According to this survey, average monthly incomes for miners who formalized their operations reached $2,377, while the same figure for miners who had not formalized their operations stood at $401. Once they met the requirements demanded by the Unique Registry of Mineral Sellers (RUCOM) to certify the legal origin of their mineral, miners could get better prices for their gold and reduce their risk of law enforcement action. Legal miners can obtain higher prices for their gold on the legal market than informal miners do with selling to illicit actors, as the latter take advantage of the clandestine nature of mining operations and the miners’ consequent fear of the authorities to extract a lower price. Still, legal income can only increase once miners hold their title and environmental license; without these tools, the gold cannot be sold legally and there are no licit profits. This presented a substantial challenge for OL, as it meant that miners needed to have considerable economic capacity to sustain themselves during the time between applying for the title and license and receiving them. 4 More details can be found in Annex I. Supplementary data and analysis – See quantitative annex Formalization and mining governance – Econometric model of determinants of mining formalization – Table 2, pp. 4 to 8). 5 These figures correspond to the data obtained from the beneficiary survey, which found that 39 out of 113 MPUs had obtained mineral title and the environmental license. 5 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV OL was an interlocutor and facilitated linkages between the MPUs and government authorities, but trust continues to be affected by weak institutional capacity. The Activity built bridges of understanding between different actors involved in formalization, interacting with high officials of the national government as well as ASGM miners, and offered small miners a real possibility of achieving legality. Oro Legal developed a methodology for scoring formalization of ASGM in Colombia, which facilitated concrete policy actions to support miners in formalizing. OL’s work on mining formalization policy is widely recognized by gold producers, international mineral marketers, local, regional, and national authorities, academics, and researchers on small-scale mining. Empowering miners by dignifying their activities contributed to OL’s effectiveness in mining formalization. Oro Legal contributed to making visible to the GOC, the mining sector, and the broader public the work and dignity of small-scale gold miners. The Activity contributed to making visible miners' material and technical deficiencies and to better understanding their operational performance. Miners themselves, by being part of Oro Legal, could feel less fear of the justice system and could face the formalization process without feeling stigmatized as criminals or socially excluded. Mining formalization is a process promoted at the ministerial level, rather than being a comprehensive and inter-institutional policy, and this status is one of the main obstacles that the formalization policy faces. A key finding of this evaluation is that mining formalization is not a whole-of-government policy, with an enabling law connecting the different sectors of the GOC concerned with mining regulation. This situation impedes inter-institutional coordination for the governance of the sector. Furthermore, mining formalization is optional for miners and beneficiaries can withdraw at any time if discouraged during the process by the multiple disincentives to formalization such as the length of time it takes to complete the process, convoluted procedures, prohibition on selling gold during the process, and uncertainty about its cost. The evaluation findings also suggest that creating structural changes in miners’ behavior could require longer-term sustained support. Interviews for the evaluation found great uncertainty about what the future might hold for miners and their upcoming attitudes toward formality. This uncertainty is conditioned by the fact that the Activity has concluded, and that many beneficiaries remain in the formalization process, but have not yet obtained critical documents like the mining title and the environmental license. OL beneficiaries who had not graduated by the time of OL’s closing perceived a considerable degree of uncertainty. Some miners interviewed compared the end of OL to the end of the BioREDD+ program and expressed fear that they would not receive subsequent support from donors. According to the model of determinants of mining formalization, length of time in the program is one of the factors that most explains MPUs advances toward formalization. On average each additional quarter an MPU participates in the program is associated with 7.8 additional points on the indicator for formalization of mining operations. At the same time, the model finds that each point that an MPU had at entry was associated with 0.85 additional points, reflecting the importance of the starting point for MPUs embarking on the process of formalization.6 VALUE CHAINS AND ALTERNATIVE LIVELIHOODS 6 For more details, see Annex I. Supplemental data and analysis – See quantitative annex Formalization and mining governance – Econometric model of determinants of mining formalization -– Table 1, pp. 4 - 6). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 6 The income gains from beekeeping and annatto complemented household incomes and may allow for a transition toward productive activities that are more attractive and more viable than ASGM for beneficiaries. According to the beneficiaries survey, 76.4 percent of annatto producers and 44 percent of beekeepers were involved in mining-related activities prior to working with OL. Also, 87.3 percent of beekeeping beneficiaries and 43.4 percent of annatto producers agreed or strongly agreed that value chain activities had improved their incomes. Ninety-two percent of beekeepers and, 68.5 percent of annatto producers said that they consider the value chain projects more viable than mining. Nevertheless, Interviewees mentioned that various families continued to carry out mining activities while participating in value chain projects, due to insufficient incomes from the value chain projects. It is important to note that the beekeeping value chain belongs to a more mature market than annatto. Antioquia has been the second-place department in Colombia for honey production for almost a decade,7 whereas annatto has been a traditional orchard crop in Chocó, with cultural significance but no market development. Moreover, during the past decade, the beekeeping value chain has been supported in the Bajo Cauca region by other USAID programs, whereas annatto has not. OL is the first USAID program to pilot an annatto value chain initiative. Statistical analysis shows8 that the greatest level of satisfaction among beekeepers was positively related with perceived income increases, and that beekeepers with more than one year in the program deemed the activity more viable in comparison with mining activities. Beneficiaries in the annatto value chain, on the other hand, expressed dissatisfaction around income, which was associated with a lower perception of the activity’s viability compared with mining. In the same vein, beneficiaries with less than one year in the program declared the lowest perceptions of viability. In line with these observations, generating sufficient and stable income seems to encourage more producers to move toward involvement in value chains, gradually leaving mining activities. OL strengthened the technical capacity of beneficiaries, increased their productive potential, and improved yields and quality. With beekeeping, OL established 11,360 hives. Between 2016 and 2019 there was a nationwide increment of 34,236 hives.9 Taken together, these figures suggest OL supported 33 percent of nationwide growth in beekeeping over the last four years. Regarding yields, before OL the average productivity was 8.4kgs/year/beehive according to a study undertaken by CONIF (2016);10 Beneficiaries surveyed in 2020 declared an average yield per hive of 14.2 kgs/year, which means a productive potential of 161,312 kgs/year for the totality of the hives installed by OL. In the case of annatto, there were 664 Ha under cultivation across Colombia according to a survey carried out in 2017.11 Between 2017 and 2019, OL established 661 Ha, becoming the largest agro-industrial establishment of annatto in the country. The national yield estimate in 2017 averaged less than 1.5 tons/Ha of annatto annually. While OL’s supported plots are not yet at full productive capacity-estimates of future potential production are roughly four tons/Ha per year. The total productive potential of plantations established by OL was estimated by A&ACH at 3,000 tons/year. Business models were prepared for Agricultural Production Units, but there were no comprehensive business models for the entire value chains. This impeded anticipating all the impacts associated with different components of the value chains, including overproduction of annatto and a lack of working capital in beekeeping. In the case of annatto, it is worth observing that OL identified soon after the beginning of the intervention a particular variety of annatto adapted to the humid conditions and poor 7 Dirección de cadenas pecuarias, pesqueras y acuícolas del ministerio de agricultura y desarrollo rural, 2020 8 Annex 1 – Supplemental data and analysis – See quantitative annex value chains and alternative livelihoods – Descriptive analysis, variable associations and grouping of observations, pp. 6- 7). 9 Ibid. 10 Corporación Nacional De Investigación Y Fomento Forestal – CONIF (2016). Diagnóstico de la actividad apícola en el Bajo Cauca Antioqueño para OL. 11 Evaluaciones Agropecuarias Municipales adelantadas por el Ministerio De Agricultura Y Desarrollo Rural 7 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV soils of Chocó and established the foundations of an agro-industrial annatto value chain in the department. However, production goals were not calculated with a comprehensive business model that considered the cost structure along the productive stages up to commercialization and included demand size. The private commercial partner in the annatto value chain could only absorb a third of the actual production. This led to a very high risk of desertion of beneficiaries, as by the time of the evaluation some producers had already lost up to three harvests. In the beekeeping chain there are greater commercialization guarantees. Still, although the commercial partner for honey has managed to secure loans for COP $250 million for working capital, that partner cannot yet guarantee cash purchase for the entirety of the honey sold by the beekeepers in the region who supply it with honey. OL was able to improve environmental conditions and avoid increased degradation of land and illegal extraction of gold in the targeted areas. Beekeepers and annatto producers surveyed agreed, at 79 percent and 68 percent levels respectively, that the value chain activities had prevented further negative environmental effects in the targeted areas. Producers indicated that they preferred working in alternative livelihoods projects to gold mining, and that the alternative productive establishments had helped to rehabilitate areas degraded by informal/illegal gold mining operations OL's value chain activities strengthened beneficiaries’ sense of resilience in the face of a problematic public order situation and provided hope for the future. The alternative livelihood projects strengthened household resilience by reducing their economic vulnerability, strengthening social networks at the family and community level, increasing perceptions of security, and bolstering perceptions of institutional support. The percent of beneficiaries who felt that the value chain activities had contributed to improving security conditions was 64.6 percent for annatto and 74.7 percent for beekeeping. According to a statistical analysis,12 beekeeping- beneficiaries indicated that they would want to continue with the value chain activity even when presented with a hypothetical situation in which income from the activity decreased by 75 percent. REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING OL undertook rehabilitation in areas of environmental importance, whose recovery contributed to key national and international goals. The polygons targeted by OL overlapped with national forest reserves (Law 2 of 1956) by 96 percent in Chocó and 33 percent in Antioquia; 36 percent of the areas targeted in Chocó and 33 percent in Antioquia were located in priority conservation areas as defined by national policy.13 Eight percent of targeted areas in Antioquia corresponds to priority areas for restoration, and six percent in each department lies within priority areas for rehabilitation, as determined by the Ministry of Environment for the Unique Registry of Ecosystems and Environmental Areas. Oro Legal targeted areas with a significant level of degradation and considerable risks for communities. The informal and illegal mining activity carried out for decades in these areas has led to biodiversity losses and negatively affected environmental goods and services, such as water, air, soil for planting, and regulation of local microclimate. These areas have seen deforestation, soil degradation, and increased vulnerability to natural disasters and climate change. Various municipalities in Chocó have repeatedly 12 For more details, see Annex I. Supplemental data and analysis – See Quantitative Analysis Value chains and alternative livelihoods - Descriptive analysis, variable associations and grouping observations – Figure 7, pp 9-10). 13 Documento Conpes 3680, 2010. This document set out guidelines for the consolidation of the National System of Protected Areas, which aimed to increase the ecological representativeness of the system, through the designation or expansion of protected areas in high priority sites. These guidelines also consider elements to improve connectivity and ecological integrity and to ensure the generation of environmental services. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 8 appeared in early alerts for deforestation,14 with changes in land use for mining and the cultivation of coca being the two main causes. Local actors perceived positive results such as improvements in microclimate, increased biodiversity, and soil recovery as a result of the rehabilitation work of the program. Using satellite imagery to analyze vegetation cover in a sample of 2,077 Ha where OL rehabilitation took place from 2016 to 2020, (the assessment) identified a four percent increase in vegetation cover and a 14 percent decrease in deforestation. There were also increases in vegetation cover of four percent in Antioquia and nine percent in Chocó and a reduction in deforestation of 13 percent for Antioquia and 35 percent for Chocó. The models of rehabilitation developed by Oro Legal in the two targeted departments were appropriate from an environmental and social perspective. OL used Acacia mangium trees in Antioquia for its rehabilitation model; this species has been used and recognized for decades to restore soils degraded by alluvial mining. In Chocó rehabilitation projects used local ecosystems as a reference. The Activity hired community members to carry out the labor of transplanting plants from these intact local ecosystems, using a landscape approach to promote recovery of the ecosystem as a whole. Both beneficiaries and government entities declared that the approach used was appropriate because it was undertaken in coordination with the communities. OL filled gaps in financing in the absence of public investment for rehabilitation and its model achieved a large land extension. Colombian regulations do not have a legal definition for environmental liabilities, which makes it difficult for public institutions to dedicate money to rehabilitation. OL took on this issue and did so in a cost-effective manner. When compared with similar programs, such as the National Restoration Plan (with costs of $1,820/Ha) and World Wildlife Fund (WWF) ($2,000/Ha), OL has lower costs, at $1,700/Ha. The model combined active and passive rehabilitation, employed different species of trees based on the features offered by the targeted landscape, and involved local communities throughout the process, including in watering, planting, and contributing labor to the nurseries. Oro Legal achieved a total extension of 17,028 Ha of rehabilitated lands degraded by illegal and informal gold mining, of which 13,421 Ha were under passive rehabilitation. The sustainability of rehabilitation efforts is not guaranteed, and it runs the risk of losing the efforts and resources that have been invested to date. OL’s implementation contract does not establish an obligation to guarantee the sustainability of the rehabilitated areas, and while the program carried out activities to try to seek stability, interviews found that attention to conservation has waned with the departure of the implementing partner (IP) from these areas. ELIMINATION / REDUCTION OF MERCURY IN THE SUPPLY CHAIN FOR ASGM The technical assistance that OL provided to miners, in particular for PTOs, advanced best practices for mercury-free gold extraction. Ninety percent of the MPUs surveyed declared that the PTO was vital for carrying out processing without mercury, and 88 percent recognized that the program supported access to important techniques and technologies for mercury-free extraction. We were able to conclude that the technical assistance provided by OL, particularly in carrying out deposit studies, facilitated better 14 Caracterización de las principales causas y agentes de la deforestación a nivel nacional período 2005-2015. González, J. Cubillos, A., Chadid, M., Cubillos, A., Arias, M., Zúñiga, E., Joubert, F. Pérez, I, Berrío, V. Instituto de Hidrología, Meteorología y Estudios Ambientales – IDEAM-. Ministerio de Ambiente y Desarrollo Sostenible. Programa ONU-REDD Colombia. Bogotá, 2018.IDEAM, 2017 9 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV practices for mercury-free extraction, and that formalization is positively correlated with the elimination of mercury in mining operations. The most important external factor for the reduction of mercury was the entry into force of Law 1658 in 2018. Based on monitoring records of the OL program, beneficiaries changed their practices from using 7 grams of mercury to produce 1 gram of gold in the second half of 2017, to 0 grams of mercury in the first quarter of 2020. This reduction is most notable in alluvial and underground gold mining. However, we found that 1.1 percent of beneficiaries declared that they continued to use mercury to process gold. Given that a small number of ASGM miners said that they continue to use mercury, purchasing it on the black market or in gold shops, an important challenge for relevant GOC authorities is to focus on control of the trade and illegal access to the element. Education and awareness-raising campaigns have had positive results. Interviews conducted for this evaluation with GOC officials from the mining and environmental ministries revealed that they believed the development of campaigns to disseminate information are the best strategy for reducing the use of mercury. At the same time, more than 70 percent of beneficiaries indicated that health concerns motivated them to stop using mercury. ORO LEGAL’S PERCEIVED RELEVANCE AND EFFECTIVENESS In general, beneficiaries had a high level of satisfaction with the program. On a scale of one to five, with five being complete satisfaction,15 73 percent of OL’s mining formalization beneficiaries declared that they were satisfied or very satisfied with OL, as did 77 percent of beekeeping beneficiaries, 59 percent of rehabilitation beneficiaries, and 40 percent of annatto beneficiaries. Beneficiaries also highlighted positive aspects, such as the relevance of OL to the needs of beneficiaries in local context, the positive effects on the environment, on mining activity, economic development, quality of life, and the participation of women and young people. OL contributed to changing the mindset of miners regarding the possibility of carrying out environmentally, socially, and economically sustainable gold mining activity. Interviews with different stakeholders indicate that awareness raising, and training provided by OL contributed to changing miners’ mentalities regarding mining formalization. Only two percent of those surveyed said that there is no advantage in formalization. The qualitative evidence shows that OL contributed to reversing the skepticism of artisanal miners in the formalization processes and connected them with the mining and environmental authorities. The support that OL provided to mining formalization and the reduction of mercury use increased awareness among beneficiaries about the importance of legality and taking care of the environment, filling knowledge gap of miners in these areas. Quantitative evidence indicates that the desire to protect water sources (49 percent) and to protect the air (42 percent) figured among the main reasons that miners declared for not using mercury. OL created job alternatives for women, improved their incomes, promoted their empowerment, and helped build community ties that improved their quality of life and that of their families. Up to 37 percent of the direct beneficiaries of OL are women. Fifty percent of participants benefited from beekeeping activities supported by OL are women, while in rehabilitation of areas degraded by illegal mining this percentage was 44. Women represent 29 percent of beneficiaries in the annatto initiative as an alternative activity to informal gold mining. Although mining is an activity dominated historically by men, 14 percent of the participants in mining formalization were women. The actors interviewed noted 15 The scale of satisfaction for beneficiaries to rank the program was as follows: (1) Very dissatisfied, (2) Dissatisfied, (3) Neutral, (4) Satisfied, (5) Very satisfied. The reported percentages reference the total for (4) Satisfied and (5) Very satisfied. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 10 that the women who participated had the opportunity to work in different activities, gaining independence and empowerment. Beneficiaries (both men and women) declared that they felt a greater sense of security and peace of mind. This evaluation found that beneficiaries of OL from all the initiatives and both departments felt closer to government institutions; they also highlighted that OL had contributed to the reduction of illicit activities in the targeted areas in favor of alternative sources of income. RECOMMENDATIONS OF THE EVALUATION MINING FORMALIZATION AND MINING POLICY AND GOVERNANCE • Continue to emphasize among miners and authorities alike, the improvements in the environmental, social, and economic performance of MPUs that is gained through the formalization process, and which contributes to dignify the ASGM miners. • Maintain and/or reinforce the adaptive approach of activities like OL working on ASGM to be able to respond to regulatory changes and turnover of officials in government entities. • Increase USAID’s participation in establishing the relationship between the implementing partner and government institutions, to lend official status to the implementer’s proposals and requests. • Strengthen the technical entry criteria and the minimum stage that beneficiaries must reach within the formalization process to guarantee the compliance with the intervention cycle and define what happens for beneficiaries who do not reach the minimum stage set by the program. • Ensure that the design of OL-type programs include, from the beginning, actions to create new channels for the commercialization of gold and associated minerals and to establish partnerships with organizations that promote fair trade, ethical markets, and price differentials that support environmental and social responsibility. • Verify the achievements of BioREDD+ and OL, to determine the most appropriate duration of time for these programs to satisfy the expectations of USAID. This includes coordinating activities with other training providers to streamline, maximize resources, and reduce pressure on miners. • More actively use the position of USAID to promote obligatory mining formalization as an official state policy, drawing on the experiences of OL. • Continue the practice of hiring technical personnel native from the targeted regions. VALUE CHAINS AND ALTERNATIVE LIVELIHOODS • Continue to incentivize the development of agricultural value chains, selected through a comprehensive analysis of vocations and suitability across the selected territory, to advance gradual transitions from mining into new economic activities. • Continue to develop technical capacity in the targeted territories and to design strategies that promote the sustainability of technical assistance over time. 11 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV • Manage support for the continuity of technical assistance from COLTAPICOLA and the technical specialists from the community councils to the producers that participated in OL. • Focus interventions on the development of local and regional production chains that add value and connect with markets in a competitive way. • Guarantee that the incentives for establishments with small producers, especially on a large scale, are supported by a commercial chain of the volumes to be produced and the design of an associative business model that ensures its viability. • Ensure the continuity of financial, technical, and business support to complete the consolidation of the annatto value chain in Chocó. • Strengthen incentives for the participation of women and young people in agricultural value chains and for the active involvement of families and their way of life in the productive activities. • Strengthen strategies for cultural and community ownership of projects, facilitating the empowerment of local agribusinesses and the participation of youth leadership in the community. • Continue to promote value chains that incorporate the agroecological systems in the targeted areas. REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING • Incorporate environmental criteria into the design of activities like OL, in order to allow target areas to be selected in a way that is aligned with the goals of national and local policy and that increases the impact of the rehabilitation activity. • Include soil analysis, measuring pollution and degradation, in the design of activities like OL. • Activities like OL should widely disseminate their results and the effects of rehabilitation of quality of life and society at large to generate community ownership and greater care of the rehabilitated zones by local populations. • Connect rehabilitation interventions of activities like OL with the ethnic development plans of the community councils, to strengthen environmental governance among Afro-Colombian communities. • Connect rehabilitation efforts of activities like OL to local environmental movements and leaders in order to work jointly with them and encourage their work in these areas. • Promote Payment for Environmental Services and/or the sustainable exploitation of acacia wood. • In activities like OL, manage Acacia planting in a way that advances towards ecological succession making use of endemic species that contribute to the rehabilitation of the soil (applies to Antioquia). • In activities like OL, which include passive rehabilitation components, establish alliances with local authorities and grassroots communities to increase local ownership and ensure achieving the results of rehabilitation (applies to Chocó). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 12 • Document the costs of rehabilitation to contribute to the planning of subsequent projects with a similar scope. • Activities that include passive rehabilitation within collective territories should include the development of a strategy for territorial control in concert with the community councils to contribute to the maintenance of the managed areas. • Support the alignment of plantations on private lands to exercise better control of the planted areas and not allow the actions promoted by activities such as OL to be lost through degradation as new pressure factors enter the ecosystem. • Implement plans to connect mayors’ offices, autonomous regional corporations, and IIAP to ensure continuity of the activities carried out by these institutions (e.g., knowledge transfer). ELIMINATION/REDUCTION OF MERCURY IN THE ASGM SUPPLY CHAIN • Continue to deliver training to miners on issues related to the prohibition on mercury usage. • Include a component of support for government entities in the design of activities like OL to help them advance geological studies to look for mineral deposits. • Carry out a technological reconversion program for ASGM miners who have received the training and know how to make use of the new practices. • Increase monitoring of mercury use in the targeted areas. • Carry out campaigns to raise awareness of the consequences for the use of mercury and expand these to encompass other mining-related contaminants, such as cyanide. • Work with local, regional, and national government entities to develop a strategy to commercialize and market mercury-free gold. • Continue to support the formalization subcontracts and monitor their conditions to promote a fair distribution of the benefits of gold mining activity. PERCEPTIONS OF OL’S RELEVANCE, EFFECTIVENESS, AND IMPACT • Improve processes of information management and deepen descriptions of beneficiaries’ conditions to improve monitoring and measurement of interventions’ effectiveness. • Strengthen coordination among USAID, the implementing partner, the national government, and local government entities for better planning, complementarity of work, follow-up, and sustainability. • Strengthen the training and activities that promote improved environmental conditions in the different interventions of the Activity. • Continue and increase the dissemination of information on the benefits and changes in quality of life associated with legal mining and the value chain initiatives. 13 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV • Ensure continuity in the hiring of technical personnel from the targeted areas to mitigate security issues. Moreover, include ¿to who? in the training and in the spaces to address self-care protocols with vulnerable individuals and communities. • Broaden the gender focus in the Activity, increasing the participation of women. • Incorporate generational renewal strategies in value chain interventions to increase young people’s desire to stay in these areas, prevent their involvement in illicit activities, and contribute to the Activity’s sustainability. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 14 A. EVALUATION PURPOSE This evaluation of the Oro Legal activity (Legal Gold, or OL) aims to evaluate the final performance of the Activity on its various components. As part of this objective, the evaluation seeks to: i) Evaluate to what extent the achievements of the Activity are perceived as effective and as having an impact; ii) Identify the factors that affect the performance of the Activity’s components and sub-objectives and iii) Identify the contextual factors that could affect the sustainability of the Activity’s results. The evaluation analyzes the contributions of OL to achieving four main goals. These are: 1) Reducing illegal activity and the environmental devastation stemming from illegal mining, ii) Generating economic alternatives to illegal mining (beekeeping and the production of honey in Antioquia, and annatto in Chocó), iii) Rehabilitating highly degraded lands through planting of Acacia mangium in Antioquia and assisted natural regeneration in Chocó, and iv) Introducing technologies and practices to eliminate the use of mercury in ASGM. The evaluation methodology examines OL’s support to formalization of small mines; governance in the mining sector; promotion of alternative livelihoods; rehabilitation of degraded lands; and the reduction or elimination of mercury use in ASGM. This analysis centers on reviewing if the Activity presents sufficient incentives for artisan and small-scale miners to formalize their operations and improve their environmental practices. In addition, we analyzed the external and internal factors with positive and negative impacts on the progress made by OL toward strengthening governance of the mining sector, environmental governance, and associated policies. In addition, we explored beneficiaries’ perceptions of the impact of value chain projects (beekeeping and annatto) for the current and future wellbeing of their households outside of mining. It investigates the effectiveness and relevance, from environmental, social, and economic perspectives, of the different models developed in Antioquia and Chocó. The evaluation complements this investigation by examining what combination of instruments and approaches to reduce the use of mercury in ASGM has proved most effective. Finally, it identifies the principal perceptions of the relevance and efficacy of the different interventions supported by OL among actors directly and indirectly involved in the Activity. A central goal of the evaluation is to produce recommendations that will contribute to improving the implementation of programs like Oro Legal in priority zones. In this aspect, the evaluation seeks to present USAID decision-makers with concrete recommendations and practical options for implementation measures, enabling them to consider possible courses of action with regards to continuing, redesigning, or strengthening OL-type activities in the future. 15 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV B. BACKGROUND AND DESCRIPTION OF THE ACTIVITY16 Small-scale mining in Colombia presents a strong presence of illicit activity. A significant amount of this production is carried out by illegal and informal operators. It is estimated that close to 80 percent of gold produced in Colombia comes from illegal artisanal and small-scale miners. According to a study carried out by the Ministry of Mines and Energy (MME), between 2010 and 2011, 72 percent of Mining Production Units (MPUs) surveyed were small-scale mines, while 63 percent had no title, and 75.7 percent did not have any kind of environmental permit to operate. This situation persists despite programs of formalization and despite regulatory advances, such as Law 99 of 1993, which mandates that mining projects must obtain an environmental license to operate. ASGM is surrounded by illicit activities. The National Tax and Customs Directorate (DIAN) first sounded the alarm about money laundering in this sector in 2012, indicating at the time that illicit groups laundered at least $10 billion a year through the gold market. It bears noting that artisanal and small￾scale mining is centered in regions where armed groups control a host of illegal activities, including illicit extraction of minerals, mercury trafficking, cocaine production, illegal logging, and land grabbing. The use of mercury in the amalgamation of gold in artisanal mining is a serious issue in Colombia. Colombia is a signatory to the Minamata Convention on Mercury and issued Law 1658 of 2013, which set a deadline of July 2018 to eliminate this element from gold mining. Nonetheless, gaps remain throughout the chain of control, impeding full compliance with this goal. The consequences of artisanal mining practices in Colombia include environmental and socio-economic deterioration, as well as grave risks to human health. Illegal miners and small producers do not have appropriate training in extraction techniques, which causes deforestation and pollution, and consequent increases in environmental liabilities, threats to biodiversity, and risks to watersheds (e.g., Bajo Cauca in Antioquia, and throughout Chocó). Moreover, mining tends to discourage other rural income-generating activities, as residents move into mining projects, seeking higher returns for their work. The ultimate impact, however, is often to leave communities impoverished, with fewer legal economic alternatives, and under pressure from illegal armed groups. To address this situation, the MME developed a Policy for Mining Formalization in 2014, which was modified through the 2014-2018 National Development Plan. To bolster this policy, the Ministry of Environment and Sustainable Development (MADS) prepared an environmental guide, which laid out terms of reference for environmental impact assessments (EIA) required for the process of licensing small mining projects, and for the process of seeking a temporary environmental license for mine formalization. Oro Legal, in coordination with Swiss development cooperation, prepared and presented to the MME a white paper with suggested policy modifications. In this context, and in alignment with its strategic objectives in the country, USAID promoted and put into practice the OL Activity, with Chemonics International as the implementing partner. The Activity 16 The following data is relevant to the Activity: Contract No: AID-514-C-15-00003 Contracting Mechanism: Cost plus fixed fee contract. Period of Performance: September 2015 - April 2021 Total Estimated Cost (TEC): $22,122,365 Implementing Partner: Chemonics Inc. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 16 proposed to contribute to improving the capacity of the government, local communities, and the private sector to address informality in the ASGM sector and reduce its environmental impact through two strategic objectives: (i) improving the governance of the mining sector, and (ii) increasing the capacity of government entities, communities, and the private sector to manage the environmental impact of illegal mining. These strategic objectives were implemented across six results areas, as shown below: • Strengthening government capacity to govern gold mining activities: − ER 1.1. Strengthening government capacity to enforce compliance with the laws governing the mining sector. − ER 1.2. Increasing the participation of associations of artisanal mining associations, and of Afro￾Colombian and Indigenous communities in the formalization program. − ER 1.3. Increasing training and technical assistance for artisanal miners. • Strengthening the capacity of government entities, communities, and the private sector to confront the environmental impact of illegal mining: − ER 2.1. Rehabilitation of degraded areas. − ER 2.2. Generation of alternative livelihoods for communities involved in ASGM. − ER 2.3. Improving the quality of potable water in the areas of the intervention. OL worked with a diverse group of interested parties in 22 municipalities in the departments of Antioquia and Chocó (Table I). The actors involved included the national and regional governments, municipal administrations, Autonomous Regional Corporations, associations of miners and producers, private mining businesses, and other initiatives working in the area of the intervention. TABLE 1. MUNICIPALITIES TARGETED BY OL DEPARTMENT MUNICIPALITY Antioquia Barbosa Buriticá Cáceres Caucasia Don Matías El Bagre Nechí Remedios San Roque Segovia Tarazá Zaragoza Chocó Atrato 17 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Cértegui Condoto El Cantón del San Pablo Istmina Nóvita Quibdó Rio Quito Tadó Unión Panamericana Source: Prepared by Evaluation Team (2021) OL systematically incorporated gender equality and social inclusion, and the communication and diffusion of related methodologies and good practices across OL’s activities. In particular, the Activity prioritized supporting and expanding economic activities for women with high exposure to environmental health risks stemming from mining activities. OL included a total allocation of $22.1 million, and continued the mining component of BioREDD+, with a greater scope. This activity was also undertaken by USAID and implemented by Chemonics from July 2013 to May 2015. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 18 C. EVALUATION QUESTIONS AND METHDOLOGY USAID/Colombia designed this evaluation to respond to three objectives: i) to establish to what extent the achievements of the intervention are perceived as effective and as having an impact; ii) to identify the factors that affect the performance of the objectives and results areas; and iii) to determine the contextual factors that affect the sustainability of the Activity’s results. 3.1 EVALUATION QUESTIONS17 The evaluation responds to six (6) evaluation questions, each one of which has its own framework. The questions address the following thematic areas: A. Mining Formalization: To what extent do the application by Mining Production Units (MPU) of the formalization standards, improved operational efficiency, and access to legal markets gained through Oro Legal’s intervention provide sufficient incentives for miners involved in ASGM to remain legal/formal and improve the environmental performance of their mining operations? Context and guidelines to answer the question: This is a follow-on to question #3 from the mid-term evaluation, adjusted to capture the outcome of the contribution of mining formalization towards one of the main goals of the Activity. Given a national context marked by a steady increase in international gold prices, local pressure from illegal actors, and a widespread culture of unlawfulness, have formalized gold miners been able to change their behavior in ways that demonstrate USAID´s contribution in this sector to the Journey to Self-Reliance (J2SR)? If the benefits of formalization cited above have not provided sufficient incentives for ASGM miners to change mining and environmental behavior in most cases, what underlying factors are preventing uptake of formalization and what other approaches can be identified/suggested to increase interest/uptake and under what requisite conditions? B. Mining Governance and Policy: What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? Context and guidelines to answer the question: This is a follow-on to question #2 from the Mid-term Evaluation. OL’s Theory of Change (TOC) proposes that the Activity will strengthen mining governance and policy, including at the municipal level. However, mining policy is a national purview, which is a flaw in the TOC not identified during the Mid-term Evaluation. Municipal authorities are only responsible for the administration of the mining registry (RUCOM), have limited environmental authority, and lack the budget and means to enforce the mining and other laws against illicit armed groups linked to gold mining. Also, Antioquia is the only department with a Mining Secretariat with delegated mining authority by the Ministry of Energy (ME), whereas the department of Chocó does not enjoy similar status. Additionally, central government capacity to facilitate formalization is limited because of the sheer number of ASGM and other informal/illegal mining operations in Colombia, difficulties experienced in transferring operational responsibility for ASGM formalization from the ME to the National Mining Agency (ANM), and hesitant political will for effective mining policy reform. Given this context, it is 17 We based this section on the scope of work for the final performance evaluation of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal). 19 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV critical to understand how effective OL’s approaches (both formal and informal) have been to policy and institutional capacity strengthening and what factors supported or hindered the Activity’s effectiveness in this area. The evaluation will suggest or recommend other approaches where it is relevant to do so. C. Value Chain Development and Alternative Livelihoods: How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well-being outside the ASGM activities? Context and guidelines to answer the question: This is a follow-on to question from the Mid-term Evaluation. The economic (not simply financial) and/or environmental rationale for offering alternative livelihoods to ASGM families is to: (i) provide an alternative licit source of income; (ii) reduce the disproportionate environmental impact of ASGM in tropical ecosystems; (iii) reduce the pressure on vulnerable populations from illegal armed groups linked to illicit gold mining; and (iv) consolidate broader regional security. Economic reconversion includes a set of real and perceived values, benefits, and costs beyond investment in time and inputs and income generation from the economic activity per se, such as peace of mind, quality of life, more time to devote to the family (particularly relevant for single women heads of household), personal health and safety, higher productivity, participation in a legal marketplaces, etc., which are often hard to quantify, but are paramount for stakeholders and contribute to broader Government of Colombia (GOC) and USAID/Colombia development goals, objectives and priorities and, therefore, should be considered in answering this question. D. Rehabilitation of Areas Degraded by Illegal Mining: From the environmental, social, and economic point of view, how appropriate and effective have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? Context and guidelines to answer the question: Rehabilitation initiatives are one of the most visible and potentially valuable USAID contributions under Oro Legal that were not properly addressed during the Mid-term Evaluation. Large-scale degradation of land and water resources by unregulated alluvial mining and river dredging is one of the most significant environmental problems in Antioquia and Chocó, with particularly serious impacts to critical and fragile ecosystems along the Cauca and Atrato and Quito rivers and tributaries, respectively. Most rehabilitation or reclamation models are complex and costly, and not suitable or economically viable for remedying impacts in degraded areas considered “environmental orphans” by the state, communities, and landowners. The Oro Legal models were purposely designed to be low-cost, practical approaches that are appropriate for private landowners and collective communities, and particularly relevant for global and national initiatives in forest restoration and climate change mitigation. Are the models cost-effective in comparison to other approaches employed in Colombia and elsewhere? Are they good for each regional reality? How robust are the models in quickly reestablishing vegetative cover, reversing degradation, and recovering biodiversity? How committed are beneficiaries to future maintenance of the models over the medium to long term? What conditions are required and/or what changes could be made to the models to make them more effective and replicable? E. Elimination/Reduction of Mercury from the ASGM Supply Chain: What combination of instruments and approaches employed in the effort to reduce mercury usage in the ASGM have proven to be most effective and why? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 20 Context and guidelines to answer the question: Mercury has historically been used in ASGM, particularly by small-scale alluvial miners and river dredgers, as a cheap and effective amalgam to recover fine gold. Exposure to mercury is extremely harmful to humans and one of the major public health and environmental risks both in and downstream from areas where gold mining and processing are concentrated. Colombia is a signatory to the MINAMATA Convention on Mercury, a global treaty to protect human health and the environment from the adverse effects of mercury. Colombia also promulgated its own Mercury Law that bans all use of mercury in mining as of July 2018 and mandates financial support to mining operators for the transition to non-mercury processing, a provision of the law that has not been implemented. Through rigorous monitoring methods and documentation, Oro Legal beneficiary MPUs have eliminated or significantly reduced the amount of mercury used per gram of gold produced. How was this achieved and what were the motivations of beneficiary MPUs? What combination of instruments/approaches employed was most impactful? To what extent did external factors contribute to or hinder the Activity’s initiatives to reduce or eliminate the use of mercury? F. Perceived Relevance and Impact of the Activity: What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal’s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Context and guidelines to answer the question: The end of Oro Legal is an appropriate time for objective reflection and to seek honest feedback from all key stakeholders on the Activity’s contribution to ASGM, impact on the environment, and the socio-economic development in its two main geographies and in Colombia more generally. Stakeholders’ perceptions are influenced by three factors: (i) their personal circumstances and world view; (ii) their interests and expectations; and (iii) how well informed and knowledgeable they are about the Activity’s objectives, implementation, and outcomes. Assessment of key stakeholder opinions and feedback on specific components or activities, as well as incorporating broader cross-cutting considerations may uncover needs, expectations, and ideas that are otherwise hard to capture, which, in turn, could trigger awareness, adaption, recognition, innovation, and learning for improvement in the future. 3.2 METHDOLOGY18 To respond to the evaluation questions, we used a mixed methods approach, combining quantitative and qualitative instruments. This approach entailed collecting primary and secondary qualitative and quantitative data, with a focus on collaboration, learning, and adaptation (CLA) throughout the process. We also used Geographic Information Systems (GIS) to analyze and interpret spatial and geostatistical information to perform analysis, particularly data on rehabilitation of degraded areas. A. QUANTITATIVE METHOD The quantitative component of the methodology enabled a detailed inferential analysis of the data gathered and the relationships between data from different sources. This approach included multiple correspondence analysis, text mining, analysis of determinants, spatial econometrics, and other 18 A detailed explanation of the evaluation methodology can be found in Annex III, pp. 42 - 63 (work plan) and 64 - 100 (evaluation design). 21 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV instruments applied to information drawn from a survey of 669 beneficiaries (see Table 2) and from the available administrative registries.19 Source: Prepared by evaluation team (2021) To evaluate the mining formalization component of Oro Legal, we used a quantitative approach to explore the dynamics of formalization over time and across departments. We used primary and secondary information to develop econometric models to analyze the determinants of formalization. Concerning governance and mining policy, we analyzed strategic messages related to the adoption of good practices in mining to control, mitigate, and prevent environmental impacts. This was carried out through text mining analysis, which detected the most frequently used words and word associations. This analysis drew on the qualitative data from interview transcripts related with this area, as well as relevant official documents. For Value Chain Development and Alternative Livelihoods, we examined the relevant relationships and beneficiaries’ perceptions of effects for generating legal incomes and the sustainability of the alternative livelihoods created by the value chains. We carried out a descriptive analysis, complementing this with an analysis of variables and grouped observations to identify relationships between the sources of legal incomes, the sustainability of these alternatives, reduction in environmental impact, and the well-being and livelihoods of the beneficiaries. In addition, an econometric model was used to understand the perceived effect of the value chains and alternative livelihoods. For rehabilitation of degraded areas targeted by OL, we examined the state of the targeted areas, beneficiaries’ perception of relevance, and alternative strategies. This investigation was carried out through descriptive analysis that used primary and secondary information, examining a sample of polygons derived from satellite images taken from 2016 to 2020 in Antioquia and Chocó. Concerning mercury elimination, we confirmed differences among municipalities and the variables associated with greater or lesser use of mercury in ASGM. This part used descriptive analysis that detailed the use of mercury between 2017 and 2020 by all MPUs supported by the Activity. 19 The totals in Table 2 correspond to effective surveys conducted with beneficiaries. However, the sampling design provided for the calculation of expansion factors to expand the sample values to values of the reference universe. The totals reported in the descriptive and graphical analysis presented in this evaluation correspond to totals using the expansion factors. In the analyses where additional validation was necessary to confirm that the respondent appeared in the databases sent by USAID and confirm his/her participation in a specific activity through the survey, the expanded totals correspond to a: Mining=113, Beekeeping=281, Annatto=346, and Rehabilitation=318. In cases where it was necessary to estimate general indicators for the program independent of activity, the expanded totals correspond to: Mining=128, Beekeeping=297, Annatto=381, and Rehabilitation=377. These last totals reflect the fact that during fieldwork some beneficiaries indicated that they had been beneficiaries of activities other than those that had been reported by USAID. TABLE 2. SURVEYS CARRIED OUT FOR THE EVALUATION FRAMEWORK COMPONENT TYPE OF ACTOR MINING ANNATTO BEEKEEPING REHABILITATION TOTAL Effective Sample 97 212 159 201 669 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 22 Concerning the perceived relevance and impact of the Activity, we examined the perceptions of different actors involved with OL. Using descriptive analysis, we compared the perceptions of beneficiaries and institutional actors associated with the Activity and identified differences in how they rate the Activity for the same cluster of variables. B. QUALITATIVE METHOD The qualitative portion of the methodology combined an approach drawn from the phenomenology of perception with a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis. This approach enabled us to focus on the narratives and perspectives that beneficiaries used to understand OL, as well as examining the Activity’s bottlenecks, success factors, opportunities, and threats. Using a strict protocol for the protection of personal information, we carried out 120 semi-structured interviews with actors at the national (15), departmental (23), and municipal (82) levels. The evaluation also draws on secondary information from OL’s documentation, including strategic documents, diagnostics, and annual and quarterly reports. The qualitative analysis followed the principles of grounded theory, which aims to draw concepts from qualitative data. Once information was collected, it was processed and analyzed using the text analysis program NVivo. A variety of visualization tools and analyses were used, including word frequency analysis, text mining, comparison diagrams, maps, and cluster analysis (which was used to identify associations between analytical categories and thematic components of the Activity). TABLE 3. INTERVIEWS CARRIED OUT FOR THE EVALUATION FRAMEWORK ADMINISTRATIVE LEVEL OF THE ACTOR TYPE OF ACTOR NATIONAL DEPARTMENTAL MUNICIPAL TOTAL Institutional Actors 5 10 7 22 Implementer and Partners 10 7 17 Grantees, local partners, and direct beneficiaries 6 75 81 Total 15 23 82 120 Source: Prepared by the evaluation team (2021) C. TRIANGULATION OF FINDINGS Finally, we carried out a systematic triangulation of the methods, sources, and actors in the evaluation design matrix to validate, interrogate, or broaden the findings of the evaluation. This approach contrasted different perspectives and information sources, which helped reduce bias and increase the validity of results obtained from different methods. We used comparison across methods to establish if the quantitative results are consistent, and why this might be the case. Triangulation of actors, meanwhile, served to contrast the visions of different actors involved in the Activity, and triangulation among sources reinforced the results and conclusions of the evaluation. 23 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV D. LIMITATIONS OF THE EVALUATION The most important limitation to this evaluation was the restrictions imposed because of the COVID-19 pandemic. Restrictions on movement and on gatherings of people impeded us from carrying out full in￾person collection of primary data. To mitigate the risks of contagion for participants in interviews and focus groups, we carried out early reports and desk research from Bogotá, revisiting the evolution of the public health situation on a weekly basis. Considering this situation, we carried out meetings and interviews using video chat tools, including Google Meets, Microsoft Teams, Skype, and Zoom. In a few cases when video chat did not prove feasible, we conducted meetings and interviews via telephone. Another salient limitation was the difficulty with establishing contact with OL beneficiaries. There were difficulties reaching beneficiaries because in many cases (66 percent) the database either did not contain contact information or contained erroneous or duplicated telephone numbers. This meant that fieldwork carried out for the evaluation by the National Consulting Center (CNC) was able to reach 95 percent of the originally planned sample. Moreover, during fieldwork, we found that the effective universe of the mining formalization component was smaller than initially contemplated, which impacted our sample estimates. The collection of environmental information also posed challenges. Persistent cloud cover limited the use of satellite images to examine the rehabilitation of targeted zones that the mining activity had degraded, especially in areas corresponding to passive rehabilitation in Chocó. We used GIS tools to address this problem and reviewed the program’s effectiveness on this component for a sample of 13 polygons corresponding to 2,108 Ha of 17,028 Ha targeted by OL. The evaluation faced some obstacles for data analysis. The databases accessed had a structure that facilitated tracking of program indicators, but not statistical analysis, and the monitoring and evaluation system presented some challenges with understanding the information20. The number of beneficiaries fluctuated considerably over the life of the program, placing additional demands on the analysis. The evaluation also addressed some limitations associated with the methodology applied. While the beneficiary survey employed a sample of 159 beneficiaries from the beekeeping project and 212 from the annatto project, responses to the questions about income were voluntary and a significant portion of the beneficiaries surveyed did not respond to these questions. This meant that the analysis of incomes only included a lower sample than anticipated (98 beneficiaries from the beekeeping project and 57 from the annatto project). Moreover, few of the UPAs involved in the annatto project reported income, as the plantations had not yet reached a point of maturity that would allow for regular income generation. In addition, while the econometric models allowed for the isolation of the contribution of observable factors in the response variables, the models did not have the objective of finding causal effects, although the models do give a sense of associations and relationships among the variables of interest. Finally, given that the Activity was in its closing stages, we had little interaction with the implementing partner. Nevertheless, we held meetings and a workshop with the implementing partner to discuss the progress of the evaluation, as well as to discuss questions, bottlenecks, and follow-up steps. 20 In the database submitted by OL for analysis in this evaluation, the field (type report) was highlighted, which included information on baseline and follow-ups, where inconsistencies were found (e.g., follow-up dates placed prior to the baseline dates). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 24 E. FINDINGS AND CONCLUSIONS This section presents findings and conclusions, responding specifically to the six evaluation questions. The quantitative and qualitative evidence that supports these findings draws on the performance data of Oro Legal collected by USAID Colombia, and statistics and figures from official government databases. Primary information comes from a survey carried out by the evaluation team and from a review of official documents produced during the execution of OL activities, as well as conversations and interviews with actors involved in OL. The findings and conclusions related to the area of formalization and governance of the mining sector make up the first sub-section of this section, and respond to the following questions: 1. To what extent does formalization provide sufficient incentives for miners involved in ASGM to remain legal/formal and improve the environmental performance of their mining operations? 2. What main internal and external factors have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance? Given the relationship between incentives to pursue formalization and governance in the mining sector, the responses to these questions and the findings and conclusions drawn from them are combined in this section. The next four parts of this section lay out the findings and conclusions associated with the other program interventions, which focus on the following aspects: iii) beneficiaries’ perceptions of the impact of alternative livelihoods on their present and future well-being; iv) the relevance and effectiveness of different models of rehabilitation; v) what instruments and approaches proved most effective in reducing the use of mercury in ASGM; and, finally, vi) different actors’ perception of OL’s relevance and effectiveness. 5.1 INCENTIVES FOR ASGM MINERS TO REMAIN LEGAL/FORMAL AND TO IMPROVE THE ENVIRONMENTAL PERFORMANCE OF THEIR OPERATIONS, AND (II) FACTORS (INTERNAL OR EXTERNAL) THAT CONTRIBUTED TO OR HINDERED STRENGTHENING MINING AND ENVIRONMENTAL GOVERNANCE Evaluation Question 1 To what extent does the application by Mining Production Units (MPU) of the formalization standards, improved operational efficiency, and access to legal markets gained through Oro Legal’s intervention provide sufficient incentives for miners involved in ASGM to remain legal/formal and improve the environmental performance of their mining operations? Evaluation Question 2 What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? 25 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Since 2014 the Ministry of Mines and Energy (MME) has promoted a mindset shift about legalization/ formalization of mining operations, aiming to modernize the ASGM sector. Following the passage of Law 20 of 1969, the focus of the GOC had been exclusively on mines titling. The recent conceptual shift toward formalization has sought to develop mining activity in compliance with mining, environmental, social, labor, and tax regulations. The sectoral focus thus has shifted from a narrow business compliance approach to one with a broader and more entrepreneurial bent. In line with this vision of mining formalization, OL contacted more than 640 MPUs; of these, it supported 334 and formalized 146 (53 in Antioquia and 93 in Chocó) reaching an overall score of 108 percent in its performance target.21 The MPUs involved in OL represented a total gold production of around 194 million dollars and contributed over $14 million to the state in royalties, taxes, and payments into social security for their employees. OL provided support on regulatory and governance issues in the mining sector. Oro Legal supported preparation of regulations to implement articles 12, 22, and 326 of Law 1955 of 2019 (National Development Plan). OL also provided support to various local authorities to prepare their own municipal development plans, as they related to ASGM. According to one implementer: “Seven or eight articles were proposed for their National Development Plan that sought to moderate a bit the chaos of mining formalization.” In the same vein, a high-placed official in the GOC expressed that “OL allowed us to identify different barriers to continue advancing based on that experience” and “OL helped us to identify conditions that could allow us to improve the regulations and the application of the regulations” (KII with IP). Among Oro Legal’s main contributions to strengthening mining governance was to create links among miners, local communities, and mining and environmental authorities. These efforts were enhanced by OL staff’s participation in over 200 forums, congresses, talks, meetings, and other events related to mining activity. In addition, OL provided significant support to mining formalization through an E￾Learning platform for miners who were interested in formalizing their operations and for government officials. A. WITH 72 PERCENT OF BENEFICIARIES REPORTING THAT THEY ARE SATISFIED OR VERY SATISFIED WITH OL, A SIGNIFICANT CONTRIBUTION OF THE PROGRAM IS IN OVERCOMING ARTISAN AND SMALL-SCALE MINERS’ SKEPTICISM ABOUT THE MINING FORMALIZATION PROCESS. 21 MELA used the final report of Oro Legal for the total number of MPUs intervened, formalized, and their respective departmental breakdown (see "Final Report Artisanal Gold Mining - Environmental Impact Reduction Activity (Oro Legal" July 2021. https://land-links.org/wp-content/uploads/2021/07/USAID-Colombia_Oro-Legal_Final-Report.pdf). Unfortunately, the numbers included here were not available during this evaluation's data collection and analysis period. Therefore, the evaluation team could not triangulate this information with the other sources of data used. However, MELA's evaluation component included these numbers in the report following a requirement made by the COR of the Activity. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 26 Exhibit 1: Level of satisfaction among mining formalization participants (n=128)22 SOURCE: BENEFICIARY SURVEY OL helped to strengthen the desire of miners to remain legal/formal and maintain environmental performance by strengthening their sense of dignity. Dignification of ASGM included a growing sense of pride among miners in working within legal confines and a greater sense of belonging, which was associated with legal certainty and predictability. It also encompassed reduced production costs, use of environmentally responsible practices, and enrollment in social security and formal health systems. Mining and environmental authorities recognized OL’s work as fundamental to mining formalization because of the way it empowered miners and offered a new self-conception: one in which miners were not considered criminals or rejected by society. One beneficiary from Antioquia indicated that: “They [OL] saw that I wanted to do things in the right way, and I did it, because they were the ones that taught me to mine responsibly, I did what they told me to become formalized” (KII with beneficiary, Antioquia). OL demonstrated to beneficiaries the positive aspects of formalization while helping them to overcome the multiple disincentives of the formalization process. The training and advice that the Activity provided demonstrated how formalization could be achievable for small miners. Beneficiaries declared that they learned modernized mining practices that were in line with and abided by the different regulations. Following these practices in turn reduced enforcement actions from authorities and contributed to the implementation of more environmentally friendly ways of working. With regard to OL support to overcoming disincentives to formalization, one OL staff member said that: “The Environment Secretariat launched an arbitrary operation that destroyed machinery and captured some small miners supported by OL, who were in compliance with every one of the legal requirements […] we had to go [to the police precinct] and show the documents [attesting that the miners were in compliance] for them to be freed” (KII with OL staff). The econometric analysis of mining formalization23 showed that MPUs with mechanized operations scored 14.71 points higher on average on the formalization indicator than MPUs with traditional exploitation practices. Also, MPUs that had access to any public utility (water, sewerage, energy, gas, or internet) scored on average 6.48 points higher than those without access to public services. Finally, we found that MPUs gained an additional 0.85 points for every point they had at the outset, reflecting the 22 “n” refers to the size of the sample. 23 See Annex I. Analysis and complementary data – Quantitative Annex Formalization and mining governance - Econometric model of determinants of mining formalization, pp. 3-6). 27 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV importance of the baseline scoring for MPUs embarking in the process of formalization. These findings highlight the disincentives to mining formalization faced by the MPUs that began the formalization process in unfavorable conditions (Table 4). The good business practices promoted by OL enabled beneficiaries to make progress toward improved job security, respect for the rights of workers and more ethical practices. The MPUs formalized with the support of OL, reported greater commitments to the health of their workers and to their constitutional right to physical integrity, a shift considered difficult for small-scale mines. GOC authorities recognized this change in mentality among this group of beneficiaries, noting that: “They reached a level where they saw themselves as entrepreneurs, not [just] as miners, but rather with a broader vision of the mining business” (KII with GOC official). Twenty-eight percent of miners surveyed identified protection from work-related risks as one of the main benefits of formalization. The econometric analysis of formalization 24 found that MPUs that saved receipts (from sales and purchases of mining inputs) scored on average 10.4 points higher on the indicators of formalization than MPUs that did not. Moreover, MPUs that prepared a PTO scored, 8.2 points higher on the formalization indicators (Table 4). Beneficiaries that obtained their mining title and environmental license improved their earnings. Thirty￾nine MPUs (34.5 percent) out of 113 beneficiary units surveyed,25 declared that they had obtained their mining title and environmental license and that they had increased their incomes. Average monthly earnings after discounting production costs of formalized beneficiaries were $2,377, while this figure was $401 for respondents who did not achieve formalization. Miners could benefit from better prices for the gold sold and a reduced risk of law enforcement action once they were able to certify the origin of the gold exploited, by entries in the Unique Registry of Mineral Sellers (RUCOM). Legal mines can obtain higher prices for their gold on the legal market than informal mines can sell to illicit actors, who take advantage of the clandestine nature of these operations and the miners’ consequent fear of the authorities to extract a lower price. Until they received their title and environmental license, miners could not sell their products to sustain themselves during the process. In Antioquia, one beneficiary said, “the price of gold is high everywhere, and this has helped us a lot economically.” The Activity contributed to increasing the incomes of small miners, which was very important for families from a socioeconomic standpoint. One government official from a targeted municipality in Antioquia said that: “Now we are seeing increased recovery of mineral, generating more incomes, and of course, we are formalized, we can sell the mineral legally. This is considerably increasing incomes, and so socioeconomic conditions are going to improve” (KII with GOC official, Antioquia). B. THE GREATEST ACHIEVEMENTS OF ORO LEGAL IN MINING FORMALIZATION AND GOVERNANCE WERE BRINGING EFFORT AND RESOURCES TO BEAR ON THE FORMALIZATION SUBCONTRACTS AND THE SPECIAL RESERVE AREAS (ARES); ADAPTING TO THE NEEDS, 24 See Annex I. Analysis and complementary data – See Quantitative Annex Formalization and mining governance - Econometric model of determinants of mining formalization Table 1, pp. 3-6). 25 These figures correspond to the data obtained from the beneficiary survey, which found that 39 out of 113 MPUs had obtained mineral title and the environmental license. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 28 IDIOSYNCRASIES, AND ADMINISTRATIVE PARTICULARITIES OF THE TARGETED AREAS; EMPOWERING BENEFICIARIES THROUGH MEASURES TO DIGNIFY THEIR WORK; AND PROVIDING CONTINUOUS SUPPORT TO MINERS IN THEIR INTERACTIONS WITH AUTHORITIES. OL pursued an approach of working with groups of miners on formalization subcontracts and special reserve areas (ARE), which facilitated the implementation of the Activity. OL worked with clusters of beneficiaries on gold mining titles, either privately or communally held. This allowed technical and logistical staff to focus their efforts in a more efficient way on these clusters, rather than on individual miners. In addition, it allowed OL staff to prepare works and construction plans (PTOs) and environmental impact assessments (EIAs) for groups of beneficiaries. A GOC public servant, noted in an interview that: “Many of the traditional mining communities have gotten to the point where they understand where the mining formalization policy can get you and the different rules established to be able to access the benefits established within the framework of the policy, with different formalization and regularization figures existing for the special reserve areas” (KII with GOC official). The econometric models for the determinants of formalization found significant evidence of a spatial lag for the formalization indicator. That is to say, the behavior of an MPU influences individual behavior within a 1km radius. Intuitively, this explains the behavior of community councils, which joined together to form MPU conglomerates and being part of an MPU conglomerate increases the probability that an individual will pursue formalization (Exhibit 2). Exhibit 2: Map 1 Location of MPUs included in analysis 29 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV SOURCE: PREPARED BY EVALUATION TEAM BASED ON BENEFICIARY SURVEY, INFORMATION FROM PIRS26, AND USAID’S MONITORING AND EVALUATION PLAN. To evaluate the consistency of the estimations, we calculated three models for the same set of variables. We estimated Model 1 using OLS and included all variables (25 in total). Model 2 restricted the number of variables to seven and excluded the variable for time in OL.27 Model 3 used the restricted data set28 of seven variables and was estimated using a spatial lag model.29 The results of the last model had the best estimations of the three in terms of bias reduction, because including the spatial effect accounts for possible omitted variable bias.30 In Model 3, significant evidence appeared for a spatial lag effect on the formalization indicator (W*Yi), implying that the behavior of an MPU has an influence on the formalization of individuals within a one km radius. This analysis found that mechanized operations scored on average 14.7 points higher on the formalization index than MPUs using traditional techniques. MPUs that had any access to public utilities (water, sewerage, energy, gas, or internet) scored 6.48 points higher on average than those without any access to utilities. MPUs that saved their receipts and had a PTO scored 10.4 and 8.2 points higher than MPUs that did not do this. Beneficiaries with lower levels of education were more likely to formalize than those with a higher level of education. MPUs with a direct beneficiary with no formal education, or with a basic level of education, scored 11.98 points higher on average than MPUs with beneficiaries with higher levels of education. Finally, the model shows the importance of the starting points of individual MPUs; each point that an MPU scored on the entry index was associated with an additional 0.85 points on the formalization index. TABLE 4. Results of the econometric model of determinants of mining formalization MODEL 1 MODEL 2 MODEL 3 RESPONSE VARIABLE (Y1: FORMALIZATION INDEX) OLS - ALL VARIABLES OLS - RESTRICTED VARIABLES SPATIAL LAG MODEL – RESTRICTED VARIABLES MPU variables and attributes (X1i) Entry Index 0.67* 0.65* 0.85* Underground operation (dummy) -2.58 Family Labor (percentage) 0.00 Association (dummy) -1.82 Mechanized (dummy) 8.81* 14.62*** 14.71*** Labor -0.02 26 Performance Indicator Reference Sheets. 27 This variable corresponds to P206 in the questionnaire: How long ago did operations begin at this MPU? 28 Wooldridge, J. M. (2013). Introductory Econometrics - A modern approach. Pp. 145 29 Anselin, L., and A. Bera (1998). Spatial dependence in linear regression models with an introduction to spatial econometrics. In A. Ullah and D.E.A. Giles (eds.) Handbook of Applied Economic Statistics, pp. 237–89. New York: Marcel Dekker. 30 See Annex I. Analysis and complementary data – Quantitative Annex Formalization and mining governance - Econometric model of determinants of mining formalization, pp. 4-8. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 30 TABLE 4. Results of the econometric model of determinants of mining formalization MODEL 1 MODEL 2 MODEL 3 RESPONSE VARIABLE (Y1: FORMALIZATION INDEX) OLS - ALL VARIABLES OLS - RESTRICTED VARIABLES SPATIAL LAG MODEL – RESTRICTED VARIABLES Installed Capacity (1-100) -0.06 Days of Operations (1-30) 0.08 Mine Age (years) 0.02 Municipality different for sale (dummy) -0.83 Public services (dummy) 4.38 12.64*** 6.48* W*Yi (Spatial Lag) 0.56*** Variables from the formalization strategy (X2i) Time in the program (quarters) 7.85*** Training (dummy) 0.55 Bills (dummy) 2.73 12.65*** 10.44*** PTO (dummy) 3.54 10.61*** 8.19*** PTO Technologies (dummy) -0.25 EG (dummy) 0.83 Variables from the beneficiary (X3i) Age -0.08 Educ 2.39~ 10.76** 11.98*** Experience -0.03 Woman 5.76 Debts -0.79 Continue with a decrease of 25% in income -5.72 Continue with a decrease of 50% in income 3.30 Continue with a decrease of 75% in income -0.19 31 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV TABLE 4. Results of the econometric model of determinants of mining formalization MODEL 1 MODEL 2 MODEL 3 RESPONSE VARIABLE (Y1: FORMALIZATION INDEX) OLS - ALL VARIABLES OLS - RESTRICTED VARIABLES SPATIAL LAG MODEL – RESTRICTED VARIABLES Model parameters (Intercept) -36.77* 5.6 -25.68* R-squared 0.8702 0.6133 AIC 572.29 n 75 75 75 Note: The symbols “ * ” or " ~ " refer to the level of significance of the coefficients in the regression models. Significance is represented as follows: 0 percent “ *** ”, 1 percent “ ** “, 5 percent “ * “ and 10 percent " ~ ". Source: Prepared by evaluation team using data from the beneficiary survey, information from the PIRS31 and the USAID monitoring and evaluation plan. Contracting local residents of the zone where an intervention took place facilitated relationship-building and knowledge transfer between OL and mining communities; it also contributed to disseminating the knowledge about regulations and functions of the institutions in the territory. Recruiting technical staff from within the territory generated trust and facilitated the interactions of the Activity with the communities and local and regional GOC representatives. Multiple local authorities, however, lamented a lack of interaction with OL staff. Miners, communities, and authorities expressed that they were pleased that OL recognized their region by hiring local residents. Some actors interviewed by the evaluation team recognized that: “The strength of the program is that we know the capabilities of OL staff for the formalization process, [because] they are people with experience in the territory and in mining territories” (KII with beneficiary). OL served as an effective bridge between small-scale miners and authorities. Although national, regional, and local authorities (in the mining sector, the environmental sector, prosecutors, and police) were familiar with OL, miners were most familiar with Oro Legal because of OL’s work to connect miners with GOC authorities. Some mining leaders felt that their communities had not reached the level of technical, and communications sophistication required to have productive dialogues with the authorities overseeing their sector and lamented that OL was reaching an end. National GOC institutions recognized the capacity of OL to work with miners and connect them with government authorities; for instance, a GOC official said: 31 Note that the unit of analysis for the formalization component is the MPU, not the individual miner. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 32 “OL’s work in the territories is very valuable, because it means working with and holding the hand of the miners and bringing them into compliance with everything, which the state does not have the capacity to do” (KII with GOC Official). The greatest recognition of OL’s impact comes from miners. This evaluation found that miners may identify the objectives of the OL program with greater clarity than national, regional, and local authorities could; this is explained by the close work of Oro Legal with miners to bridge the gap between them and the authorities. While state actors and institutions regarded OL as a legitimate interlocutor, this did not mean that OL was able to shape contextual regulatory and institutional conditions, and GOC officials had a higher identification of the USAID name than for Oro Legal. C. OL ADAPTED TO IMPORTANT REGULATORY CHANGES RELATED TO ASGM DURING THE INTERVENTION PERIOD. OL constructed synergistic relationships with beneficiaries and with authorities in the mining and environmental sectors. The relationships that OL established allowed the Activity to support and advise beneficiaries on how to meet their legal obligations and file the associated paperwork. These relationships were also helpful for OL in promoting processes and mechanisms to stimulate formalization, in particular formalization subcontracts and the AREs. The Activity provided beneficiaries with technical, regulatory, environmental, labor, social, and economic training. These activities all contributed to dignifying ASGM and changing the negative self-perception that many miners had due to social stigma. OL was also able to offer government authorities a more positive perspective on ASGM. Nevertheless, in some cases, the obligations imposed by the formalization subcontracts could seem to be an obstacle for the formalization process. Some interviewees rejected or renounced their formalization contracts because they felt that they included exorbitant demands. Beneficiaries also mentioned the use of this instrument to maintain title without exploiting the mineral or as a way of keeping control over the land. Other actors reportedly leveraged this tool as a business, charging up to 20 percent of gross earnings to sign the subcontract. Some beneficiaries explained that they did not continue with the formalization subcontract because “… the [proposed] payment to the miners was very low, it was very unbalanced.” Beneficiaries took advantage from the institutional relationships constructed by OL. The difficulties many beneficiaries faced with required paperwork meant that OL needed to provide continuous support, both to reach the objective and to make sure beneficiaries did not give up on their intention to formalize. This level of support was recognized by miners and authorities as a significant achievement of the Activity. One beneficiary from Chocó explained that: “Mining regulations in Colombia are so complex, so difficult to comply with, and they generate so many legal gaps, that it seems almost impossible that small miners could continue this process by themselves” (KII with beneficiary, Chocó). D. LEGAL STATUS WITH MINING AND ENVIRONMENTAL AUTHORITIES, TOGETHER WITH DIGNIFICATION OF ASGM, WERE THE PRINCIPAL INCENTIVES TO REMAIN FORMAL. THIS SUGGESTS THAT ACTIVITIES LIKE OL SHOULD BE MORE PROACTIVE IN SUPPORTING GOVERNMENT AUTHORITIES IN EVALUATING WHERE THEIR OWN ADMINISTRATIVE 33 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV PROCESSES FALL SHORT, RATHER THAN PLACING ALL RESPONSIBILITY FOR MINING FORMALIZATION ON THE MINERS THEMSELVES. The main incentive for miners to formalize their operations, and to continue to operate in a formal manner, was to obtain their title and their environmental license. The ability to legally bring minerals to market is conditional on the provenance of the mineral title with an environmental license, and miners without it are unable to sell their gold on the formal market. For this reason, beneficiaries who, despite support from OL, were unable to obtain a mining title and environmental license expressed fear at the program’s closure. Having attained the title and license, on the other hand, strengthened miners’ feelings of dignity and pride in being part of the legal economy. Exhibit 332 synthesizes the principal incentives to formalize, to maintain that status, and to improve environmental practices in mining operations. Exhibit 3: Incentives to pursue formalization and improve environmental practices in mining operations (n=60) SOURCE: INTERVIEWS CARRIED OUT WITH BENEFICIARIES Program beneficiaries valued greatly the recognition as legitimate actors in the mining industry that they received through OL. Their expectations of being seen as actors in the legal mineral market, and not to feel that they were classed as criminals, motivated small-scale miners to continue their journeys toward formalization. Moreover, this recognition as legitimate actors in the mining industry helped them to feel that they would not have problems with law enforcement authorities in the future. One representative of a community council in Chocó expressed this sentiment in an interview: “I feel satisfied with the step forward that has been taken, not on a personal level but on a collective level, because results are being seen, [and] these are accomplishments that we did not have [before] and we are achieving these with the organization of OL” (KII with community council representative, Chocó). OL support to miners to obtain the mining title and environmental license is evident, but regulatory and procedural complexity reduces the likelihood of comprehensive formalization. While 36 percent of 32 This exhibit represents a qualitative compendium of nominal variables that classify (rather than score) incentives for mining formalization, based on the subjective importance assigned to each variable by interviewees. The longer the bar, the greater the importance of the incentive, measured by the number of mentions in interviews. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 34 participants surveyed asserted that the process of formalization is complicated (see Figure 4), miners highlighted OL's work to mediate between them and the authorities. Some mining leaders felt that their communities had not reached the level of technical and communications sophistication required for productive dialogues with the authorities overseeing their sector and lamented that Oro Legal was ending. Moreover, many interviewees talked about the processes of titling and licensing as though they constituted the entirety of the formalization process. The complexity of obtaining the mining title and the environmental license effectively made these administrative processes into the highest goal for miners and relegated other objectives to secondary status. In Chocó, for example, one beneficiary said: “I need, for God’s sake, before Oro Legal leaves, that we get the PTO and the license approved.” One beneficiary in Antioquia said: “I don’t know why they left so quickly, I thought that Oro Legal was going to wait until we got our title.” Speaking of the complexity of the processes, one beneficiary said: “We started the formalization process 18 years ago,” while a staff member of the implementing partner said, “mining regulations in Colombia are so complex, so difficult to comply with, and they generate so many legal gaps, that it seems almost impossible that small miners could continue this process by themselves” (KII with beneficiary, Antioquia). While miners are obliged to meet strict deadlines, the authorities fail to comply with the requirements established by law. While the regulation for small-scale mining establishes deadlines that both miners and authorities should follow during legal procedures to pursue formalization, it is only the miners who face penalties for not meeting them. The authorities recognize delays in responding to paperwork filed by miners but argue that this is due to a lack of personnel and logistical capacity. An OL staff member expressed worry over these delays: “And this is an international cooperation program […] I can’t imagine what it would be like to be a small-scale miner carrying out this process with these authorities, without a direct line to the director of formalization, the Secretary of titling, the director of audits” (KII with OL staff). The immediate costs required to meet the technical standards for formalization work against the formalization process. Regulatory requirements demand considerable investment, which means a long period during which miners cannot sell their product legally. While miners must meet strict deadlines to submit costly technical documents, government authorities may take years to issue a response. As a result of the rigorous technical, economic, and environmental requirements they face, miners fear formalization, whereas illegal operations offer immediate gains. This can be seen in Figure 3, where 52 percent of beneficiaries identify the complicated process as one of the principal barriers to formalization. Other key barriers are financial, including high costs (30 percent), the obligation to pay taxes (26 percent), and insufficient profitability (15 percent). At the same time, 13 percent of interviewees indicated that a lack of understanding of the process was an important obstacle for formalization. 35 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 4: Principal obstacles to formalization (n=113, multiple responses allowed) SOURCE: BENEFICIARY SURVEY Without the commitment of all the authorities involved, mining informality will continue to prevail. The difference in criteria among the different government actors involved is itself an important obstacle to achieving formalization, including the differences between multiple environmental agencies that use different frameworks. Miners argued that when they reach an agreement with one agency (e.g., the ANM), they often find that another issue has arisen with another agency (e.g., the Ministry of the Environment). A member of another government entity said: “They themselves invent rules as though they were pulling them out of a hat and since they are the authorities, it’s difficult to interpret these things; there’s no consistency in the guidelines or in the fine print for the requirements that each [agency] has. So, this can become arbitrary, it can lead to inconsistency between one region and another, these are big obstacles” (KII with GOC official). The evaluation did not find sufficient evidence on the effectiveness of memoranda of understanding (MOUs) to gauge their contribution to the program objectives. Turnover within agencies and political cycles for elected officials meant that counterparts who initially signed these agreements often rotated or moved to a new position. OL’s counterparts did not accord these agreements the same importance that OL did and were in general unaware of their contents. A staff member from the implementing partner explained that “with the change in mayors [in elections], we lost the agreements that were signed with the mayor’s office.” While the MOUs carried a degree of official weight, they did not form a foundation for formal interaction between OL and government institutions. An individual working for the IP explained that: “The Vice ministry of Mines did ask us questions, but in an informal way; we always hoped that there would be a more formal scenario, in which we could explain the documents” (KII with IP). The structural weaknesses identified in the formalization process were not formally presented to government authorities. OL participated, together with other actors in the mining sector, in presentations to the MME with proposals for the National Development Plan. However, there were fewer formal proposals for solutions to the different structural difficulties that emerged during implementation. The implementing partner may have been inhibited from submitting such proposals by not directly representing USAID in that situation. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 36 Knowledge transfer was made more difficult by the level of education of the miners. The technical requirements of the existing regulations demanded a high degree of general and specific knowledge. In sharp contrast to these demands, many of the miners did not know how to read or write. This posed difficulty for some trainings, as 67 percent of beneficiaries had only a basic education, and 13.6 percent indicated that they had no education at all. One GOC official put it this way: “We are talking about a population that is completely vulnerable in terms of unmet basic needs, access to education, and as such, any obligation at all is going to present a significant challenge for whoever decides to take this on” (KII with GOC official). In addition to these problems, government officials tended to have an image of miners and of ASGM that was not based in reality. In the public imagination, ASGM is populated by people without skills or abilities, who don’t care about the law, and in need of paternalistic intervention. Moreover, these are imagined to be people with highly profitable mines that can easily cover the costly process of formalization. Yet, a GOC official said that: “This is a completely vulnerable population in terms of unmet basic needs, in terms of mechanisms for education, and so obviously any obligation stemming from an alternative, well, that is going to be a pretty big challenge for anyone who decides to tackle it…The miners lack knowledge of the mechanisms for subsistence mining, how to sign up for the Genesis tool [a platform that allows subsistence miners to register with the government], and so, although the mechanisms exist, what is needed is that the miners really take ownership of this knowledge and fully comply with their obligations” (KII with GOC official). The MPUs and local actors are unaware of, or have outdated information about, mining regulations. The high turnover of government officials and frequent issuance of new rules make it difficult for many actors to stay up to date in terms of obligations and mechanisms. As one official explained, “environmental officials don’t know about the process of mining formalization – if mining officials don’t know it, environmental officials even less.” Another interviewee, an academic, elaborated on this theme, saying that: “In the ministries there are officials who are social communication specialists, economists, who have other professions and who don’t understand the specific issue of mining formalization” (KII with academic researcher). E. THE MINISTERIAL STATUS OF MINING FORMALIZATION AND THE STIGMATIZATION OF ASGM ARE THE MAIN OBSTACLES TO FORMALIZATION, WHILE THE PROCESS OF FORMALIZATION ITSELF SHOWS WEAKNESSES WITH REGARD TO GOVERNANCE. 37 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 5: Synthesis of the main obstacles and enablers of mining formalization (n=60) SOURCE: INTERVIEWS CARRIED OUT WITH BENEFICIARIES Mining formalization is a ministerial initiative, not a whole-of-government policy with enabling law obliging different institutions of the GOC to work together on formalization in the mining sector (Exhibit 533). This situation impedes interinstitutional coordination for the governance of the sector, as the authorities do not feel legally committed to the policy of formalization. The environmental authorities need to better understand the process if they are to support it from an environmental perspective. As one government official put it “the weakness was that there was no understanding among the ministries so that Oro Legal, the municipalities, or the department could carry out the programs of formalization.” Mining formalization is optional since a beneficiary can withdraw at any time. The process of formalization faces multiple disincentives, such as the years of effort required for mining titling and environmental licensing; the fact that selling the mineral is prohibited during the formalization process; and the huge investments needed to formalize. If these disincentives become too great, miners can withdraw from the process and return to informal status at any point without consequences. This situation is well understood by the different institutional actors. As one official put it “to hold a contract in Colombia isn’t for everyone, unfortunately.” Officials are unaware of or misinterpret regulations, to the disadvantage of the small miners, while authorities are unaware of the reality and particularities of ASGM. Clear guidelines for the interpretation of regulations do not exist, and each functionary can make their own decisions based on their own professional experiences. Decisions made by poorly trained officials acquire the status of an administrative procedure act and can introduce disincentives for the formalization processes promoted by an activity such as OL. National authorities acknowledge that there is great ignorance of the 33 This figure represents a qualitative compendium of nominal variables that classify (rather than score) obstacles and enablers of mining formalization, based on the subjective importance assigned to each variable by interviewees. The longer the bar, the greater the importance of an obstacle or enabler, measured by the number of mentions in interviews. The dotted line around the red bars separates obstacles to the process from enablers. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 38 particularities of artisanal mining, which makes formalization processes very difficult. Small-scale mining has been stigmatized and is frequently confused with illicit mineral extractors. According to a GOC official: “Ownership on the part of the authorities is a bit absent, in this case regulatory authorities, there’s a lack of knowledge of social, economic, productive realities and others that have grown up around mining activity in Colombia” (KII with GOC official). Areas that are protected by law, including forest reserve areas covered by Law 2 of 1959, and protected areas in general, have restrictions on mining and on mining formalization, and it is highly unlikely for miners to obtain an exemption from these protections. In addition to the considerable technical and legal difficulties of obtaining an exemption, a program like OL cannot include miners working in these zones. Moreover, the cost of obtaining this kind of permit is higher than the PTO and the EIA, which reduces the likelihood that a small miner would want to request an exemption. F. IT IS NECESSARY TO PRIORITIZE INTERVENTIONS FOCUSED ON GREATER CURRENT AND FUTURE INCOME, SINCE SUCCESSFUL FORMALIZATION DEPENDS ON SUCCESSFUL COMMERCIALIZATION; WITHOUT THIS CONDITION, MINERS WILL NOT SEE BENEFITS IN WORKING WITHIN THE LAW AND WILL RETURN TO THEIR PRIOR LIVELIHOODS. A greater emphasis by OL on gold commercialization could have improved the prospects for beneficiaries continuing to be formal/legal. Economic motives represent the major motivation for formalization (Exhibit 6), which means that an approach focused on better prices, more ethical buyers, and greater financial stability has contributed significantly to keeping beneficiaries engaged and interested in remaining formal, even after the conclusion of OL. An international gold trader noted that “when mining becomes legal the miners can participate in fair markets and receive a better price.” Another trader noted that “There’s an expectation of getting a better price for your gold, and that is why today you still see dissatisfaction.” Licit incomes only improve, however, once miners have their mining title and environmental license; without these instruments, sales are not legal. This fact has presented a considerable challenge for OL, as it effectively demands that miners have significant economic reserves to comply with the prior requirements (mining title and environmental license) needed to legally sell the gold that they produce. For miners to continue to seek formalization, they will have to see that economic benefits materialize. The motivation of miners is to earn a higher income. Consequently, difficulties with the administrative procedures to obtain a mining title and environmental license represent critical obstacles to the formalization process. Exhibit 6 shows that the principal perceived advantages of formalization include a sense of personal safety, access to more fair markets, access to better practices (e.g., that permit greater efficiency), more fair prices for materials and inputs, access to credit, and the ability to obtain machinery. All these factors are related to economic benefits. 39 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 6: Beneficiaries’ perceptions of the advantages of formalization. (n=113, multiple responses allowed) SOURCE: BENEFICIARY SURVEY G. SUPPORT FOR THE JOURNEY TO SELF-RELIANCE (J2SR) CAN BE SEEN IN AREAS SUCH AS REGULATORY COMPLIANCE, SYSTEMATIC MANAGEMENT OF ENVIRONMENTAL IMPACTS, SUPPORT FOR ENVIRONMENTAL PERFORMANCE, LEAVING INFORMALITY, AND SUPPORT FOR INSTITUTIONS; BUT THIS REQUIRES LONGER-TERM SUPPORT TO GUARANTEE LASTING STRUCTURAL CHANGE IN MINERS’ BEHAVIOR. The trainings offered by OL improved compliance with environmental regulations and the systematic management of environmental impacts. OL built capacity in ASGM to reduce environmental impacts through the implementation of cleaner and more efficient practices. These trainings have contributed to formalization in mining areas. For example, most beneficiaries surveyed stated that they agree or strongly agree that trainings helped reduce the use of mercury (88 percent) and that new practices have improved the environment (91 percent). A commercial partner in Antioquia noted that: “Those beneficiaries could come to trading for the long term because they have already been formalized and have an important foundation: Oro Legal left them with their training so they could have a long-term future and continue on the path of mining correctly carried out” (KII with a commercial partner, Antioquia). OL contributed to notable improvements in environmental performance through the impact studies required for the environmental license. This was recognized by environmental authorities, who consider the studies to be critical guidelines to protect the environment. Similarly, OL helped implement cleaner and more efficient gold processing techniques. A high-ranking GOC official, familiar with the trajectory of OL, noted that: “The environment has won from this. From what we have seen on the ground and from what the miners have told us, we only have words of appreciation.” In the same sense, a beneficiary in USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 40 Antioquia indicated that: “They have supported us in everything; how to manage the environment, the oils, the area of the river, they advised us on everything.” The formalization promoted by a program like OL still faces disadvantages in competing with illegality. Due to strong pressures from the deadlines, costs, and technical requirements established in mining and environmental regulations, miners have great incentives to turn to illegality. These incentives include financing, supplies, sales without registration, and avoidance of the payment of taxes and royalties. Miners must meet strict deadlines for the delivery of expensive technical documents, but authorities can take several years to issue a ruling on them. The technical, economic, and environmental requirements cause miners to fear the formalization process, while illegality offers immediate incentives. Lack of coordination between training entities was evident, which generated redundancies in services and training contents. This overloads the miners, taking them away from their work in their mines to attend the different sessions offered. Sometimes this produced excessive training in some topics or duplicated content from different actors (including OL, the National Learning Service (SENA), academics, NGOs, contractors, businesses). This contributed to disincentivizing miners to seek training, particularly for those who had to close their mine for a day to attend workshops. Two mayoral officials from different municipalities in Antioquia concurred on this. One said: “There was no communication in those ministries to know everything that they wanted to achieve, what they wanted to achieve, let's say on the issue of legal formalization.” Another said that “There was no understanding between the ministries so that OL, the municipalities or the governorates could specify [what would happen in] the formalization programs” (KII with Mayoral Official, Antioquia). We found low coordination between the entities relevant for formalization. Institutions recognized their lack of coordination but did not propose solutions. The mining and environmental authorities did not feel legally linked to the formalization process. Environmental authorities require a greater understanding of the mining process to become supporters of formalization and the idea that miners could move towards environmental protection. One staff member of the IP put it this way: “Sometimes we find a solution with the mining [Ministry] side, but the environmental [side] won’t allow it (…) The linkages with institutions, that is to say with mining authorities and environmental authorities, is whether the mining and environmental authorities can sit down responsibly and determine that the mining law effectively meshes with the environmental [law]” (KII with IP). Additionally, we found low operational capacity of the entities involved (mining and environmental). The authorities recognize the lack of human, economic, and logistical resources to devote to mining formalization. This lack of resources compromised OL's efforts by delaying processes, limiting results, and discouraging beneficiaries. A. THE INTERRUPTION OF THE ACTIVITY WITH SOME OF MPUS WITHOUT THEIR MINING TITLE OR ENVIRONMENTAL LICENSE COULD CAUSE MINERS TO BACKSLIDE ON THEIR INTENTIONS TO FORMALIZE THEIR OPERATIONS. 41 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV The future of OL’s beneficiaries is highly uncertain, as they did not foresee the conditions under which the Activity closed. Considerable fear exists among the beneficiaries, especially those who did not graduate to formalization by Activity close. One such beneficiary said: “Now that the program is in its closing phase they should, before they go, leave us with the PTO prepared and the license approved so that we can make partnerships with businesses that will support us with the economic capacity to carry out mining” (KII with beneficiary). Some miners compared the end of OL with the end of BioREDD+ and said that beneficiaries felt a sense of abandonment. Some interviewees considered the Corcrecer initiative as a continuation of OL; however, the objectives of Corcrecer are different from those of OL. Beneficiaries expressed their concern in interviews. One beneficiary in Chocó said: “For God’s sake, before going they should leave us with the PTO and [environmental] license.” Another beneficiary in Chocó put it this way: “My worry is that they are leaving, but we haven’t managed to get a title. If Oro Legal is not there to help advise and interact with the national government or with the Agency, there is no social muscle [influence]” (KII with beneficiary, Chocó). A beneficiary in Antioquia said that “Without the title in my hand, we can’t say that we are legalized (...) who is going to be our godfather to help us comply with the requirements?” Short-lived projects create skepticism and instability in miners. Some previous interventions have left miners feeling that these programs always remain unfinished. At the end of these activities, the miners return to a reality where they feel excluded. Some cited BioREDD+ as an example of an intervention that was well executed, but whose effectiveness decreased after its anticipated closure. One academic explained it this way: “From one moment to another the resources ran out, they promised us this and it turns out that they left us halfway. So, you have to be very careful with these assistance projects, so if the miners spend time and see that the project is serious, they will be on their way” (KII with academic researcher). The structural modification of traditional gold exploitation requires longer-term support. The internalization of more environmentally friendly business habits and practices, regulatory compliance, and passage through complex and costly administrative procedures highlight the need for a longer-term OL-type support activity. The time needed to cover costs, adopt technical standards, or simply await responses from authorities is measured in years. Some beneficiaries expressed their uncertainty in the following terms: “Oro Legal entered with USAID and they are already closing, and then, who is left with that, how is that going to continue? How can we continue with the confidence and peace of mind that the state will continue with the processes” (Interviews with beneficiaries)? A. VALUE CHAIN BENEFICIARIES’ PERCEPTIONS OF THE IMPACT OF ALTERNATIVE LIVELIHOODS FOR THEIR PRESENT AND FUTURE WELL￾BEING USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 42 Evaluation question 3 How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well-being outside the ASGM activities? OL promoted the expansion and strengthening of productive bases for beekeeping in the Bajo Cauca region of Antioquia. This value chain benefitted 335 families in the region. The families received 11,360 beehives, which were set up with nucleus colonies to populate them and begin production of honey. OL also provided advisory and technical assistance for beneficiaries to develop technical capacity to manage their apiaries. OL concluded 13 grant agreements, representing a total of $802,000, were signed with three non-profit organizations, seven producer associations, one community council, one Indigenous resguardo, two mayoral offices, and one business in the area of agriculture and forestry. In addition, OL proposed and financed the creation of COLTAPICOLA, an association of local specialists who provided assistance to the new beekeepers and who could continue to provide technical assistance after the conclusion of the Activity. Finally, OL set up contacts with potential buyers for the honey produced by the Activity’s beneficiaries and facilitated an agreement for commercialization of this product through a regional business. OL helped develop the value chain for annatto in Chocó. The Activity developed an improved annatto production system that incorporated 661 Ha. These properties stretched across six communities and producer organizations. OL signed subsidy agreements with the communities to provide producers with technical assistance. In five of the six communities, an equal number of collection and threshing centers were created and equipped, considering ease of access for harvesting and primary seed processing. To complement these efforts, OL encouraged and facilitated the constitution of the company Achiotes y Agros del Chocó SAS (A&ACH). Five community councils and a private business with experience in agroindustry partnered with A&ACH. OL also financed the overhaul and equipment of the A&ACH processing plant, where it supported annatto processing for the industrial dye market. OL managed the development of a commercial agreement with a leading company in the colorants industry in Colombia, which remained in the final negotiation phase at the end of the Activity. Finally, to give continuity to the entire process, OL structured a business model proposal. This proposal was presented to Acumen, a patient capital investment fund, to obtain financing and business support after the completion of OL. A. THE VALUE CHAINS SET UP BY OL ARE PERCEIVED AS ALTERNATIVE INCOME SOURCES THAT CAN BE VIABLE IF CHAINS OF PRODUCTION AND COMMERCIALIZATION ARE CONSOLIDATED. Beneficiaries’ perceptions of the two value chains differed based on two associated factors: the incomes that they gained, and the prospects for commercialization of their products. Of beneficiaries involved in beekeeping, 77 percent indicated that they were satisfied or very satisfied with OL, while 40 percent of annatto producers indicated that level of satisfaction (see Exhibit 7). The beekeeping value chain is embedded in a national market, in which Antioquia was the second-most productive department in Colombia between 2010 and 2019.34 Moreover, in the past decade, this value chain has been supported by USAID. In contrast, annatto in Chocó has historically been an orchard crop of cultural significance and incipient market structure. The value chain is not technically advanced, and it enjoys little access to 34 Dirección de cadenas pecuarias, pesqueras y acuícolas del Ministerio de Agricultura y Desarrollo Rural, 2020 43 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV agro-industrial markets. OL is the first USAID Activity to pilot an annatto value chain initiative. The joint category diagram from the econometric analysis shows that annatto producers’ dissatisfaction is related to the fact that it has proved impossible for them to sell a considerable proportion of their harvests, and the consequent lack of income from this activity. At the same time, this does not mean that annatto will not prove viable in the future; respondents who had spent less than a year cultivating annatto were those who expressed the greatest dissatisfaction.35 The market conditions of the honey market in the region, on the other hand, meant that beekeepers in general were able to find buyers for their products. Despite price fluctuations and delayed payments, they tended to see their activities turn into viable sources of household income. Beneficiaries with more than one year in the beekeeping value chain also tended to see this activity as more viable than those with less than a year working in beekeeping. Exhibit 7: Beneficiaries’ level of satisfaction with the value chains (n annatto=381, n beekeeping=297) SOURCE: PREPARED BY EVALUATION TEAM BASED ON BENEFICIARY SURVEY Income from beekeeping and annatto complemented household incomes, and if consolidated as value chains, would become more attractive and viable for beneficiaries. The beneficiary survey found that 76.4 percent of annatto producers and 44 percent of beekeepers carried out activities related to gold mining before OL. Moreover, it emerged in interviews that several of the beneficiary families continued to carry out gold mining activities due to the insufficient income obtained from the value chain projects. However, the survey showed that 68.5 percent of annatto producers and 92 percent of beekeepers consider these alternative activities more viable than gold mining. According to the results of the joint category diagram obtained from the econometric analysis, high satisfaction of beekeeping beneficiaries was associated with perceived income.36 Moreover, yield (kg/hive) is one of the variables that best explains an Agricultural Production Unit’s (UPA) level of income, as an increase of one kg of yield is associated with a 4.5 percent increase in total income reported by beneficiaries. Each additional point of capacity installed and used37 is associated with a 1.1 percent increase in total income for the beekeeping UPAs. The beekeeping UPAs that received business training had on average 4.94 percent more income 35 See Annex I. Complementary Data and Analysis – Quantitative Annex Value Chains and Alternative Livelihoods – Descriptive analysis, variable association, and variable grouping – Figure 4, Pp. 6. 36 See Annex 1 – Quantitative Annex, Value Chains and Alternative Livelihoods – Descriptive Analysis, variable associations and grouping of observations – Figure 5, pp. 7. 37 Producers surveyed for the evaluation estimated the percentage that they utilized of the capacity that was installed in their UPA. Utilization of installed capacity is the potential production or maximum volume of production for a business, unit, department, or section, over a set period of time, and taking into account available resources, including equipment, installations, human resources, technology, experience, and knowledge. An example of the measurement of this indicator is the Business Opinion Survey of the think tank Fedesarrollo, which tracks this indicator for a monthly national-level sample of businessmen. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 44 than those that did not receive this kind of training. For beneficiaries in the annatto component38, each additional point of capacity installed and used was associated with 1.1 percent higher income. Each additional worker in the annatto UPAs increased total income by 2.9 percent. The annatto UPAs that received technical assistance or production equipment registered 1.65 percent and 1.05 percent more income respectively. B. OL ACHIEVED NOTEWORTHY STRENGTHENING OF PRODUCTION IN BOTH VALUE CHAINS, PROVIDING BENEFICIARIES WITH INPUTS AND TECHNICAL CAPACITIES, AND CONSOLIDATED ENCLAVES OF PRODUCTION THAT REPRESENT A SIGNIFICANT PERCENTAGE OF PRODUCTION IN COLOMBIA. OL strengthened the capacities of the beneficiaries, increased their productive potential, and improved yields and quality thanks to the investments and technical assistance provided by the Activity. In the case of beekeeping, a 2016 study in Bajo Cauca in 201639 found that prior to the entry of OL, the average yield per beehive was 8.4 kgs/year. Beekeeping beneficiaries surveyed indicated an average yield of 14.2 kgs/year for 2020. The results of the 2020 survey were heterogenous; while 35.3 percent of beekeeping beneficiaries had not surpassed the 2016, 31.1 percent were producing over 20 kgs/year and 8.4 percent got over 30 kgs/year, which was the ideal goal proposed by the Activity (Exhibit 8). For annatto, information on yields prior to OL is less available, as this was a traditional crop, but one estimate40 found average annual yields of less than1.5 ton/Ha in Chocó. While OL annatto beneficiaries have not yet obtained complete harvests, one community leader declared in an interview that “We were planting three or four tons/Ha, and now there are crops between five and five and a half tons/Ha.” This indicates that investments in property, together with OL’s technical assistance, led annatto producers to adopt better practices to increase their productivity. Exhibit 8: Distribution of beekeepers by yield of their hives (n=262) SOURCE: BENEFICIARY SURVEY 38 For more details, see Annex I. Analysis and Complementary Data – Quantitative Annex, Value Chains and Alternative Livelihoods – Econometric Model of the Determinants of Incomes for Beekeeping and Annatto Production, pp. 8-9. 39 Diagnóstico de la actividad apícola en el Bajo Cauca Antioqueño, Corporación nacional de investigación y fomento forestal – CONIF para OL, 2016. 40 Evaluaciones Agropecuarias Municipales para el Ministerio de Agricultura y Desarrollo Rural (2007-2018). 45 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV OL established broad productive bases of beekeeping and annatto that could be the foundation for the consolidation of productive conglomerates in the targeted areas. In beekeeping OL established 11,360 hives. In 2019 a total of 135,117 hives were reported in Colombia41, meaning that OL’s contribution represents around 8.4 percent of the national total. Between 2016 and 2019 there was nationwide growth equivalent to 34,236 hives42, suggesting that OL supported 33 percent of this growth nationwide over the last four years. With current yields as calculated from the figures from the beneficiary survey, the productive potential of the hives established by OL stands at 161,312 kgs/year. For annatto, the figures from the EVA report indicated that in 2017 there were 664 Ha nationwide, and beginning in the same year, OL set up 661 Ha, making this the largest agro-industrial annatto establishment in the country. The productive potential of annatto was calculated by A&ACH at 3,000 tons/year. OL developed local technical skills and capacity. However, the proposed scheme does not ensure the continuity of technical assistance to producers. Technical assistance services for both beekeeping and annatto were provided by local professionals and technicians whose skills were strengthened by the Activity, leaving a knowledge base in the territories. For annatto the specialists trained were part of the community councils and organizations that received support. However, plans for continuity of assistance were not considered and once the Activity ended, these specialists did not continue to provide service. In the case of beekeeping, the creation of and support to COLTAPICOLA, bringing together technical specialists who were part of the OL team, was an innovative proposal to ensure continuity of services. However, these specialists do not have sufficient connections with the associations and producers, and the level of service provision is very low, so producers in practice receive little technical assistance to help them improve their yields. C. IN THE AREA OF BEEKEEPING, OL FOCUSED MAINLY ON STRENGTHENING UPAS AND PUT LESS EMPHASIS ON CONSOLIDATING LOCAL CHAINS TO INCREASE COMPETITIVENESS. Exhibit 9: Beneficiaries’ perceptions of the income gained from annatto and beekeeping (n annatto =346, n beekeeping=281) SOURCE: BENEFICIARY SURVEYS Beekeepers recognized the improvement in their incomes due to the increase in their productive capacity and saw prospects for future growth in this area. Of the beekeepers surveyed, 87.3 percent 41 Dirección de cadenas pecuarias, pesqueras y acuícolas del Ministerio de Agricultura y Desarrollo Rural. 42 Ibid. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 46 said their household income improved because of beekeeping (Figure 8). Some producers were on their own initiative investing to increase the hives in their apiary. According to OL's final reports, an average of 34 hives were delivered per family. Of the beekeepers surveyed, 31.4 percent said they had increased their number of hives after initial delivery; 16.1 percent said they had 45, which is the number of hives defined by OL as the desirable size of a Beekeeping Productive Unit. According to the model of determinants of beekeeping beneficiary incomes, each additional point of installed capacity corresponded to a 1.1 percent increase in total income.43 In beekeeping, the consolidation of an agro-chain within the territory was not promoted, nor was a unified strategy with a socio-business approach to support it. Although primary honey production was notably strengthened, there was no consistent work with the other links in the regional chain, such as collection, stockpiling, primary processing, packaging, and sale. The approach for work carried out under the 13 agreements depended on the organization with which the agreement was signed. The Activity did not directly strengthen the associations that participated, nor did it propose an associative work model to improve beekeeping competitiveness in the region. It is worth highlighting the case of the agreement that was made with a private forestry and agricultural company in the region. The company incorporated the beehives owned by Activity beneficiaries into the company's property as part of its production process, in a joint accounts arrangement with an association that was created to receive 25 percent of the revenue brought in by these hives. As a non-profit organization it could not distribute these resources among its members. Exhibit 10: Distribution of beekeepers by the number of hives in their apiaries (n=262) SOURCE: BENEFICIARY SURVEYS The opportunity to form a regional commercial alliance as a regional link in the value chain did not arise, and because this was included only at the end of the Activity this chain was not consolidated. While OL approached a regional honey trading company at the beginning of the Activity, the preferred course of action was to manage commercial contacts with external buyers and distributors or directly with large agro-industries. Due to a lack of local capacity for collection, primary processing, and quality assurance capacities, it was not possible to complete commercial agreements with these large buyers. At the end of 2019, OL had reestablished contact with the regional distributor and reached a commercialization agreement. However, there was not enough time to consolidate the relationships between this partner 43 Annex I. Complementary Data and Analysis – Quantitative Annex Value Chains and Alternative Livelihoods – Descriptive analysis, variable association, and variable grouping – Table 2, Pp. 8). 47 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV and the associations and producers. Moreover, the distributor did not have sufficient capacity or resources to absorb the new supply. D. OL WAS ABLE TO PROMOTE AN AGRO-INDUSTRIAL VALUE CHAIN FOR ANNATTO THAT, IF CONSOLIDATED, COULD BE A BENCHMARK FOR DEVELOPMENT IN CHOCÓ. Starting with a traditional crop with an incipient market structure, OL identified a variety of annatto suitable for an improved production system in Chocó. Prior to OL, an improved production system for annatto suitable for Chocó did not exist. The Activity started by analyzing the native variety of annatto and found useful benefits of this cultivar. An academic who participated in the evaluation described these benefits: “The seeds that come from there have a high production of bixin44 in comparison with other areas, other locations outside Chocó; really the difference is overwhelming” (KII with academic researcher). With this initial input, the Activity defined good practices for sowing and caring for the crop. These practices substantially improved yields over traditional practices, and reduced the time needed to reach the productive stage, from 18 to 12 months. This system was transferred to and used by specialists and producers in the area. This achievement alone is a significant contribution to the development of the annatto industry in Chocó. OL established the foundations of an agro-industrial value chain for annatto in Chocó. However, some critical issues remain to be resolved, which could pose serious risks for the sustainability of this achievement. In contrast to beekeeping in Bajo Cauca, with the annatto value chain OL strengthened three key links within the region: primary production by small producers; gathering and threshing 45 by community councils; and processing to extract bixin paste with the community business A&ACH SAS. A technical partnership was formed with the agro-industrial business Naranja Madura SAS, also a partner of A&ACH, to guarantee the needed technical assistance. While OL advanced a commercial partnership with a leading business in the colorant industry, the agreement that will be signed, according to information submitted by these actors, will only absorb a third of the supply provided by the cultivations supported by OL. This situation of mismatched supply and demand could lead to some producers abandoning the business. As one annatto producer noted: “We have to make an enormous effort to expand the business, to expand sales, after having won better clients and optimized all these processes (…) to keep the producers motivated and ensure that cultivation continues and that there is no dropout [from the Activity]” (KII with beneficiary). E. BUSINESS MODELS WERE CREATED FOR THE UPAS, BUT COMPREHENSIVE BUSINESS MODELS FOR THE CHAINS WERE NOT; THIS LED TO COMMERCIAL, FINANCIAL, AND TECHNICAL 44 Bixin is a natural ingredient, extracted from annatto seeds, which is used as a coloring additive in the food industry. 45 The first step in processing, in which the annatto seeds are separated from the capsule containing them. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 48 DIFFICULTIES THAT STILL FACE THE VALUE CHAINS TODAY AND, IN THE CASE OF ANNATTO, POSE A SIGNIFICANT RISK TO THE VALUE CHAIN’S SUSTAINABILITY. Producers of annatto felt dissatisfied with the incomes they received, and disappointed that the initial expectation of a guaranteed market for their products was not met. The annatto producers surveyed who gave an income report stated that they received an average of COP$430,545 per month for this activity during 2020 (a figure equal to about 49 percent of Colombia’s national minimum wage). Only 43.3 percent of them agreed with the statement that their household income improved with this activity (see Exhibit 9). Although OL paid wages to all annatto beneficiaries as a temporary subsidy to encourage their participation, only 55.6 percent of those surveyed stated that part of the income they received came from the sale of annatto. This evaluation found evidence of dissatisfaction and even disappointment; as one interviewee said: “Because at one point they said that now we had a buyer for all of the production, that we didn’t have to worry, that this was now settled. Of course, it wasn’t that way, it wasn’t that way then and it isn’t that way now” (KII with beneficiary). For beneficiaries involved in beekeeping, the fact that the associations and the regional commercial partner do not pay cash on delivery for the honey is an issue and leads to beneficiaries selling a portion of their honey to intermediaries and opportunistic traders. The current high production in the area and the growing demand for honey creates a favorable situation for intermediaries who pay cash and buy directly from the producer, as evidenced by the 39.6 percent of beekeepers surveyed who said they sell half or more of their production individually. This makes it difficult, however, to consolidate value-adding processes in the territory. Both the associations and the regional distributor have shown that they do not have enough working capital to buy cash for the high volumes of honey that are currently produced in the area. One producer noted that “people were complaining that they weren’t getting paid for the honey and okay then, I don’t know, what they need is to have resources to pay people cash.” This measure, if adopted, however, could lead to a drop in prices during production spikes. While the commercial partner has managed a credit of COP$250 million (US$67,843), permitting the company to adapt to new production, this is still not sufficient to guarantee cash payments for the entirety of current production. The absence of comprehensive business models capable of accounting for the impacts of investments led to overproduction of annatto and a shortage of capital in the beekeeping component. The program defined in detail the productive and financial models of the UPAs, but the goals for number of hives and Ha of annatto were not calculated from a business model that started with demand and evaluated the logistical requirements and financial resources for the commercialization of these new productions. Given the complexities of the dye agribusiness, bringing in a new customer for annatto requires a process of adjusting product specifications that typically takes more than six months. Only a third of the production can be sold to the annatto commercial partner, so there is a very high risk of beneficiary desertion. Some producers have already lost up to three harvests. In the beekeeping chain, even though the regional commercial partner secured increased working capital, this is still not enough to guarantee cash purchases to all producers. F. PROJECT PARTICIPANTS EXPRESSED A PREFERENCE FOR PARTICIPATING IN THE VALUE CHAINS INSTEAD OF IN ILLEGAL MINING BECAUSE OF THE EFFECTS ON THEIR QUALITY OF LIFE, FINDING THEM VIABLE AS LONG AS THEY CAN GENERATE INCOMES TO SUSTAIN THEIR LIVELIHOODS. 49 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV OL had positive effects for beneficiaries’ quality of life in both the beekeeping and the annatto components. Of beekeeping beneficiaries surveyed, 85 percent said that OL had contributed to improving quality of life for themselves and their families. For annatto, this percentage stood at 51 percent (Figure 10). The joint category diagram derived from the econometric analysis46 shows that these differences in perception are related to whether the value chains generate incomes. This confirms the relationship between perceived increases in quality of life and increased household income. The highest perceived quality of life was reflected in other aspects that beneficiaries valued, such as connections with their family and community, access to technical knowledge, and vocational growth, and the chance to engage in activities that were less physically demanding than mining. Exhibit 11: Beneficiaries’ perceptions of the impact of value chains on their quality of life (n annatto=346, n beekeeping=281) SOURCE: BENEFICIARY SURVEY Female participation was significant in the first productive step of both value chains, which generated positive effects for their autonomy, recognition, and belonging in the family and community. Female participation in value chains stood at 56 percent for beekeeping, and 28 percent for annatto. This participation created opportunities for them to be involved in remunerated work that complemented the unremunerated care work that women performed inside the home. This was beneficial for women’s sense of autonomy and recognition, as stated by one beneficiary: “This work has been something that has connected me with my three children. Even though I’m the one who is doing the work, they pay attention to what I’m doing. For example, this boy here will say, ‘look, now mama went to give the bees their food, mama went to visit the hives” (KII with beneficiary). In the case of annatto, this sense of recognition manifested in community terms, as expressed by one beneficiary thus: “women from other places come to visit us, for example women from Cocomacia, from Atrato, came here to see how we were working.” Women’s participation was evident mostly in the first link of the value chains, and less so in later stages. The annatto component as a regional agro-industrial chain is gaining recognition for enabling the inhabitants of Chocó to advance their own territorial development. From the community councils to the beneficiaries who worked in the annatto chain, a variety of actors repeatedly expressed the importance of this regional process. One beneficiary, for example, said that: 46 See Annex I. Analysis and Complementary Data - Quantitative Annex, Value chains and alternative livelihoods - Descriptive analysis, variable association and observation grouping – Figures 4 and 5, pp. 6-7). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 50 “Generating value chains around the community is always a goal, a dream for us. Why? Because this department deserves these possibilities. We have an impoverished countryside; now the program offers the possibility of reaching the dream and the goal that we have as an organized community” (KII with beneficiary). When they heard about this activity, other community councils expressed interest in participating in the value chain. The advances and results that OL achieved are unprecedented in the region and are valued by the community councils, who are now part of this agro-industrial, entrepreneurial process. G. THE PROGRESS MADE BY THE VALUE CHAINS DID NOT HAVE A DIRECT IMPACT ON SECURITY OR PUBLIC ORDER, BUT IT SUPPORTED HOUSEHOLD RESILIENCE, INCREASING PARTICIPANTS’ PERCEPTION OF SECURITY AND SENSE OF HAVING ALTERNATIVES FOR THE FUTURE IN THEIR HOME TERRITORIES. Although the security situation continues to threaten value chain beneficiaries, having viable alternatives for income generation strengthens the household resilience by reducing economic vulnerability, strengthening social networks at the family and community level. Beneficiaries expressed perceptions of greater security and support for institutions. The percentage of beneficiaries who felt that value chain activities contributed to improving safety was 64.6 percent for annatto and 74.7 percent for beekeeping. Likewise, the beneficiaries expressed that the productive projects contribute to expectations for future prosperity in the territory. Indeed, a statistical analysis of variable association47 showed that beneficiaries engaged in beekeeping considered the activity to be viable and indicated they would continue to pursue that activity even in a hypothetical situation where income gained from beekeeping decreased by as much as 75 percent. Beneficiaries perceived the value chain activities as a way for people to be productively engaged, thus helping to reduce the risk of recruitment by armed actors, especially for young people. Beneficiaries appreciated that the value chains generated alternatives for young people to make productive use of their time and offered outlets for them to contribute their energy and ideas. They recognized this as a protective factor against recruitment by armed groups. This is supported by the findings of Colombia’s Human Rights Ombudsman (Defensoría del Pueblo), which identified the absence of support networks for the families of children, adolescents, and young people as one of the main risk factors for recruitment by armed groups.48 Offering alternative sources of income, then, improves families’ abilities to provide safe spaces in the face of external threats, including forced recruitment. Beneficiaries and other actors perceived the presence of armed groups as the most significant threat to the further development of value chains. Among the beekeeping beneficiaries, 41 percent identified security as an obstacle to development in the region. Moreover, among the potential obstacles from a commercial, technical, or productive standpoint, security issues were mentioned by most beneficiaries, with an incidence 18 percentage points higher than a lack of commercial agreements and 23 points higher than a lack of financing. 47 Annex I. Complementary Data and Analysis – Quantitative Annex Value Chains and Alternative Livelihoods – Descriptive analysis, variable association, and variable grouping – Figure 5, Pp. 7. 48 Informe Defensorial: Prevención del reclutamiento de niños, niñas y adolescentes, Análisis de la política pública con enfoque étnico. Defensoría del Pueblo, marzo de 2014. 51 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV H. BENEFICIARIES RECOGNIZED THE POSITIVE ENVIRONMENTAL EFFECTS OF ANNATTO AND APICULTURE PROJECTS IN TERMS OF RECOVERY IN DEGRADED AREAS AND THE PREVENTION OF FURTHER ILLEGAL EXTRACTION. OL has improved environmental quality through the value chain activities and avoided greater environmental degradation, reducing illegal extraction in the areas targeted by the intervention. Beekeepers and annatto producers surveyed tended to agree that these activities had slowed environmental damage in their area, with 78.8 percent and 68 percent affirming this, respectively. Producers see reasons to actively discourage illicit mining in the areas of their value chain activities. An annatto producer explained it this way: “Here it was possible to suspend mining activity, even though it had been very important, because here there was a kind of fortress […] with this crop that we have here, so that we have stopped mining” (KII with beneficiary). Beneficiaries also said that their value chain activities had also helped rehabilitate areas that had been degraded. Interdependence between beekeeping and the rehabilitation model using Acacia Mangium strengthened the sustainability of both components. From the beginning, OL established an objective of linking the planting of Acacia mangium with beekeeping and envisioned the production of honey from this tree to differentiate the product. However, the honey produced by bees from Acacia mangium has a darker color, so producers had to include multi-flower honey in the mix to improve market acceptance. Still, beekeepers positively perceive the value of acacia trees on the productivity of their hives and are committed to caring for the areas planted with these trees. 5.3 RELEVANCE AND EFFECTIVENESS (ENVIRONMENTAL, SOCIAL, AND ECONOMIC) OF THE DIFFERENT MODELS OF REHABILITATION IN DEGRADED AREAS IN ANTIOQUIA AND CHOCÓ Evaluation question 4 From the environmental, social, and economic point of view, how appropriate and effective have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? The areas in which OL was active are areas of considerable environmental importance, meaning that rehabilitation efforts in these areas contributed to achieving key national and international environmental goals. Colombia has identified 40 percent of its continental surface as being subject to some degree of degradation, one of the most important types of soil damage. The country has also suffered biodiversity losses due to changes in land use and deforestation associated with mining.49 OL sought to address the negative environmental effects of illegal mining effectively.50 In prioritizing this project, USAID took on an important goal, but a challenging one, owing to the state of development of regulations in the mining 49 IPBES,http://www.humboldt.org.co/images/documentos/pdf/oportunidades/capitulo-4-motores-directos-de-transformacion-y￾perdida-de-biodiversidad.pdf 50 Environmental assessment, USAID, 2017 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 52 sector, the presence of illegal groups, various environmental and mining authorities at the national, regional, and local levels, and the characteristics of local communities. A. THE SELECTION OF TARGETED AREAS WAS APPROPRIATE GIVEN THEIR RICHNESS IN NATURAL RESOURCES AND BIODIVERSITY. The Chocó bioregion is considered a biodiversity hotspot because of its high number of endemic species and the presence of pristine forests. It is a mega-diverse forest region, whose forests, mangrove swamps, and estuaries are home to 9,000 species of vascular plants, 200 species of mammals, 600 bird species, 100 reptile species, and 120 amphibian species, of which 25 percent are endemic.51 It is an important area because of its geographical location at the confluence of the Pacific, Caribbean, and Andean regions.52 Bajo Cauca, Antioquia, is 80 percent covered by humid tropical forest and premontane forest ecosystems, and the zone is rich in surface and underground water. Its main rivers are the Man, the Cauca, the Cacerí, and the Nechí, with hundreds of small tributaries. Its forest ecosystems are home to many animal and plant species, which have been over-exploited, generating considerable ecological impacts.53 The areas where OL worked are zones of high importance from an environmental standpoint. The polygons targeted by OL overlap with forest reserve zones by 96 percent in Chocó and 33 percent in Antioquia; 36 percent of the focus areas in Chocó and 33 percent in Antioquia are located in prioritized conservation areas as defined by national policy.54 In Antioquia, eight percent of the targeted zones are in priority areas for restoration, and six percent are in priority areas for rehabilitation and recovery under the Unique Registry of Ecosystems and Environmental Areas (Exhibit 12).55 51 Evaluación Nacional de Biodiversidad y Servicios Ecosistémicos. Instituto Humboldt. 2019. 52 Ibid. 53 Bajo Cauca: Desarrollo Regional, una tarea común universidad-región, Instituto de Estudios Regionales (INER) Dirección de Regionalización. Universidad de Antioquia, 2000. ISBN 9586557456. 54 Documento Conpes 3680 de 2010. 55 The estimation of these areas was carried out using cartographic methods and using information from the Colombian Environmental Information System (SIAC). 53 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 12: Cross analysis of areas targeted by OL with areas of environmental importance. The values presented here correspond to Ha rehabilitated by OL with respect to environmentally important areas in each department. SOURCE: PREPARED BY THE EVALUATION TEAM USING ENVIRONMENTAL INFORMATION FROM SIAC. B. OL WORKED IN AREAS WITH SIGNIFICANT LEVELS OF DEGRADATION AND HIGH RISK FOR COMMUNITIES DUE TO LACK OF ECOSYSTEM SERVICES (FOOD AND WATER SECURITY), AND THE POSSIBILITY OF NATURAL DISASTERS. As a result of mining extraction activities, the two departments have been affected by biodiversity loss, to the detriment of resources such as soil and water. The mining activity carried out for decades in the area has caused biodiversity loss and negatively affected the provision of environmental goods and services, such as water, air, soils suitable for cultivation and climate regulation. These areas are subject to deforestation, soil degradation, and increased vulnerability to the negative effects of natural disaster risks and climate change. The municipalities of San Juan, Istmina, Unión Panamericana, Cantón de San Pablo, and Río Quito in the department of Chocó were repeatedly identified in early alerts for deforestation,56 with the adaptation of land for mining and coca cultivation the main two causes. In Antioquia, these alerts detected the active core of deforestation in the Serranía of San Lucas in the north-eastern part of the department.57 The areas targeted by OL overlapped with medium to high vulnerability areas. The degree of vulnerability varied by department. In Chocó, as shown in Exhibit 13, almost 13,000 Ha have a medium level of vulnerability, while 2000 Ha are located in high vulnerability zones. In Antioquia, most of the areas where OL worked had high (1,514 Ha) and very high (155 Ha) levels of vulnerability. 56 Caracterización de las principales causas y agentes de la deforestación a nivel nacional período 2005-2015. González, J. Cubillos, A., Chadid, M., Cubillos, A., Arias, M., Zúñiga, E., Joubert, F. Pérez, I, Berrío, V. Instituto de Hidrología, Meteorología y Estudios Ambientales – IDEAM-. Ministerio de Ambiente y Desarrollo Sostenible. Programa ONU-REDD Colombia. Bogotá, 2018.IDEAM, 2017 57 Décimo Boletín de Alerta Temprana de Deforestación. IDEAM, 2017. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 54 Exhibit 13: Cross analysis of areas targeted by OL with environmentally vulnerable areas. The figure shows Ha that cross between OL rehabilitation zones with respect to vulnerable areas. For the department of Chocó 12,980 Ha (of the 15,049 targeted by OL) are in medium-vulnerability zones. For Antioquia, the majority of the polygons are in high-vulnerability zones. SOURCE: PREPARED BY EVALUATION TEAM USING INFORMATION FROM THE COLOMBIAN ENVIRONMENTAL INFORMATION SYSTEM (SIAC) Orphan mining environmental liabilities are systems with anthropogenic disturbances where the production of ecosystem services is compromised. Among the environmental impacts of an abandoned mine are physically altered landscapes, debris, water, soil and air pollution, loss of vegetation, open shafts, acid seepage, metal washing, and increased sediment. Gold extraction affects the first layer of the soil, altering the physical and chemical properties and leading to loss of edaphic structure, causing instability that leads to desertification, sterilization, or increased erosion.58 This affects the landscape of the area, generating a domino effect on its fauna and flora due to the fractionation or transformation of the biome. Mining liabilities in Chocó have led to severe environmental impacts, threateningdifferent species. Extreme weather events have affected public infrastructure and housing. Of the 5,976 plants species found in the department, around 10 percent face some threat.59 Extreme weather (hydro-climatic) events have occurred, including avalanches, mudslides, fires, floods, unusually heavy rains, droughts, electrical storms, and gales, which have damaged homes and the basic infrastructure of municipalities. Studies of ecosystem vulnerability60 have found that the greatest vulnerability is related to floods and droughts, which affect traditional agriculture practices that local communities rely on to meet their needs. In Antioquia, OL beneficiaries recognized mining’s impact by changes in the landscape and effects on local flora and fauna. The beneficiaries recognized changes in their territory caused by mining activities 58 Contaminación y remediación de suelos en Colombia: aplicación a la minería de oro. Martínez Sepúlveda, José Alejandro. Miguel Reinaldo Casallas. 1a edición / Bogotá: Universidad EAN, 2018. 112 páginas. 9789587565836 59 Vulnerability of native forests in the Colombian Chocó: mining and biodiversity conservation. Valois Hamlet y Martínez Carolina. 2016. BOSQUE 37(2): 295-305, 2016 DOI: 10.4067/S0717-92002016000200008. 60 Studies carried out to inform the Plan Integral de Cambio Climático del Chocó. 55 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV and described the presence of holes, gullies, and eroded soils. There was a shortage of land to develop livestock or agricultural activities. In interviews, beneficiaries highlighted salient aspects, such as: “The region of Bajo Cauca is badly hit by deforestation caused by mining, so our intention was to reforest” (Interviews with beneficiaries). Another beneficiary in Antioquia said that: “Everything here was pure sand, pure stone, this was what there was throughout the whole zone. And for people who lived around here, it was very hard, because it had grown a lot, so there was the problem of water” (KII with beneficiary, Antioquia). OL beneficiaries recognized the need to reverse environmental damages and valued the recovery of the environment thanks to the implementation of this component. Fifty-one percent of program beneficiaries participated in the program out of a motivation to protect the environment, and 18 percent to recover areas degraded by mining, while 14 percent highlighted that they wanted a future with food security (Exhibit 14). In addition, in the interviews, interest in the trees that were part of the rehabilitation component came up frequently, as did planting projects and a desire to work to improve living conditions (Exhibit 15). Exhibit 14: Motivations of beneficiaries to participate in the rehabilitation component of OL (n=318). SOURCE: BENEFICIARY SURVEY USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 56 Exhibit 15: Word cloud for the rehabilitation component SOURCE: DRAWN FROM INTERVIEWS AROUND THEMES OF REHABILITATION, REFORESTATION, ENVIRONMENTAL POLICY, AND ENVIRONMENTAL MOTIVATIONS C. OL DREW UP A STRATEGIC BUDGET TO ADDRESS THE AMBITIOUS GOAL OF MANAGING 16,000 HECTARES OF ENVIRONMENTAL LIABILITIES. OL filled funding gaps, in the absence of public investment in rehabilitation to manage orphaned environmental liabilities in priority areas due to the impacts of these liabilities. The absence of a legal definition of environmental liabilities in Colombia creates difficulties for the different entities to allocate resources to the rehabilitation of areas degraded by mining and incorporate the activity into institutional planning instruments. Although there is a common idea of what a mining environmental liability is, its definition should be legal, and there should be the guidance for managing these liabilities. One study defined this as “deterioration or harm to natural resources as a consequence of activities that generate pollution or detriments to the environment.”61 Nevertheless, individuals working in public institutions indicated that they do not have a clear definition for orphaned liabilities, and that activities in this area are not included in institutional action plans, making it non-viable to make investments to respond to this problem. An official in a local institution in Antioquia explained that: “To start with, at the national level there is no regulation around this issue, so it is very difficult for us to define an orphaned liability. To the extent that from our action plan we can access resources for recovery of areas, we do it, but we do not have a budget line for it” (KII with GOC official). With an estimated cost of $1,700/Ha, the OL model has lower estimated average costs than similar programs. When comparing the estimated costs of similar programs, OL has a lower average cost per hectare. The National Restoration Plan adopted by MADS, with a rehabilitation goal of one million hectares, has an estimated investment of COP$6.2 billion pesos, which yields an average cost of $1,820/Ha. The World Wildlife Fund (WWF) indicates that “Agreement does not exist on how much it costs 61 Estudio sobre lineamientos, incentivos y regulación para el manejo de los pasivos ambientales mineros (PAM), incluyendo cierre de faenas mineras. Bolivia (Estado Plurinacional de), Chile, Colombia y el Perú. Oblasser, A. 2016. Serie Medio Ambiente y Desarrollo, N° 163 (LC/L.4208), Santiago, Comisión Económica para América Latina y el Caribe (CEPAL). 57 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV to recovery a hectare degraded by mining, with costs varying between COP $1,300,000 to COP $90,000,000.” As a counterpart to OL, however, WWF invested $2,000/Ha rehabilitated.62 OL's model allowed the program to manage an ambitious extension of the available resources. Rehabilitation projects have been pursued in Colombia since 1993, promoted by the Botanical Garden of Bogotá, National Parks, and various NGOs. In total this includes 102 projects covering 87,870 Ha. Of these projects, 40.7 percent have coverage of less than 10 Ha, and only 3 percent are large-scale projects, defined as covering between 1,000 and 10,000 Ha. OL pursued a different set of strategies: it combined active and passive rehabilitation strategies, made use of different species, and provided temporary labor support to local inhabitants to carry out the sowing and care of seedlings in nurseries, watering and sowing. As well as relying on municipalities to advance soil management, it allowed managing a total area of 17,028 Ha, of which 13,421 corresponded to passive rehabilitation. The passive rehabilitation strategy has a low cost, especially for some tropical ecosystems. These processes, however, can be delayed and can be affected by the isolation of the source ecosystems. D. FROM AN ENVIRONMENTAL AND SOCIAL PERSPECTIVE, IT WAS APPROPRIATE TO IMPLEMENT DIFFERENTIATED MODELS IN ANTIOQUIA AND CHOCÓ, TAKING INTO ACCOUNT THEIR BIOLOGICAL, CLIMATIC, AND SOCIAL CHARACTERISTICS. Acacia mangium has long been used and recognized in the department of Antioquia. In 1995 the Central Antioquia Regional Autonomous Corporation (Corantioquia) and the National University had tested Acacia mangium in Bajo Cauca to rehabilitate soils degraded by alluvial mining. The species is suitable for timber, and, given its adaptability, has been used in rehabilitation programs in various countries of the tropics.63 Its use in rehabilitation programs stems from its ability to fix nitrogen in the soil, which improves soil productivity. Acacia mangium owes this ability to its symbiotic relationships with mycorrhizal fungi. In this relationship, the tree’s root system is host to the fungi, which provide considerable benefits, including enabling greater extension of the root system, compensating for soil infertility, increasing the species’ tolerance to drought-related stress, and contributing to the creation of soil.64 This utility in the rehabilitation of degraded lands is complemented by the species’ use as a source of timber. Over the past decade in Colombia, Acacia mangium has been exploited for its timber potential, as it can be cut into beams, which can be further used for the construction of housing and furniture, tannins that are useful for painting, biocarbon for energy, and pulped into fibers for paper.65 Nevertheless, there is also evidence of risks related to the use of acacia mangium. Given the extensive planting of this species, several reports mention that, over time, Acacia mangium will naturally fall, probably because the trees are shallowly rooted. Thisdoes not occur with native species, which are 62 Dejando un territorio vivo a nuestros renacientes. Lecciones aprendidas y recomendaciones para procesos de recuperación de áreas degradadas por minería en el territorio colectivo de Cocomacia. Vásquez, C., Murillo, W., Córdoba, D. & J. Gamboa. 2019. Cocomacia – WWF. Cali – Colombia. 63 Taxonomy, distribution, biology and use as an exotic. In K. Awang & D. Taylor (Eds) Acacia mangium Growing and Utilization. Pinyopusarerk, K., B. L. Sim & B.V. Gunn (1993). Winrock International & FAO, Bangkok, Thailand. 1-19. 64 Diversidad de hongos micorrizógenos arbusculares de una crono-secuencia de suelos aluviales degradados por actividad minera en el Bajo Cauca Antioqueño. Medina Marisol, Orozco Francisco Márques María Helena. 2009. Rev.Fac.Nal.Agr.Medellín 62(1): 4749-4759. 65 Aspectos fisiológicos y de aprovechamiento de Acacia mangium Willd. Una revisión de la revista colombiana de Ciencias hortícolas. Reyes Giovanni, Carmona Sandra, Fernández María. 12(1): 244-253 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 58 more deeply rooted and can resist storms.66. While the species’ results for soil rehabilitation are favorable, the trees can also come to dominate primary forests. Therefore, some studies warn of this invasive behavior and recommend the use of native species.67 Other issues for the use of Acacia mangium include the species’ high consumption of water.68 The model of active rehabilitation with native species used in Chocó used seeds and seedlings from neighboring ecosystems. It took this ecosystem as a model, and used a landscape focus, using a methodology that promotes the entire ecosystem together.69 Some rehabilitation activities were assisted (active) where interventions were direct, such as enrichment and removal or transfer of plants. Other rehabilitation activities were spontaneous (passive), when agents causing degradation were removed and the area was left without other kinds of direct activities;70 in some cases, just closing off these areas proved sufficient. Beneficiaries considered OL’s model to be appropriate. Community council members in Chocó expressed that the rehabilitation model in the department was appropriate because it was carried out in concert with the community, who determined what could and could not be done. These beneficiaries highlighted certain aspects, such as: “It was appropriate because it was coordinated with the community, which is what decided to plant trees in degraded areas for the community to conserve them, so that the community would support this and not continue with mining (…) They also planted guava, guama, which are native species” (KII with community council member, Chocó). In Antioquia Acacia was recognized generically as a tree that would help the soil recover and therefore well-suited for this intervention. One official in a mayor’s office said that: “The acacia is the only thing to plant in these areas that are so degraded. The acacia will put down roots even in stone, and it produces plenty of falling leaves and organic material for soil recovery. We could say that we were going to plant something else for nostalgic reasons, [but] it would not have the same results” (KII with GOC official, Antioquia). E. IT PROVED IMPORTANT TO BASE SUPPORT FOR REHABILITATION IN THE SOCIAL INSTITUTIONS OF THE REGIONS, ESPECIALLY IN CHOCÓ, WHERE THE COMMUNITY COUNCILS PLAYED A KEY ROLE IN REHABILITATION ACTIONS. OL recognized the community councils as holding authority, recognized the legality of collective land tenure, and drew on ancestral local knowledge to support rehabilitation. OL drew on community 66 Ecological Assessment of Restored Subtropical Forests in Hong Kong. Zheng, Hailong. 2009. A Thesis Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy in Geography and Resource Management. The Chinese University of Hong Kong. 67 Acacia spp,: invasive Trees along the Brunei Coast, Borneo. Islam, Shafi Noor, Mohamad, Siti Mazidah & Azad, Abul Kalam. 2019. Springer Nature. https://doi.org/10.1007/978-3-319-91382-7_14 68 Simposio Caribeño sobre Acacia mangium, 2014 69 La restauración ecológica en Colombia: Tendencias, necesidades y oportunidades. Murcia C y Guariguata MR. 2014. Documentos Ocasionales 107. Bogor, Indonesia: CIFOR. 70 Restoration of degraded lands in the interior Columbia River basin: passive vs. active approaches. McIever y Starr, 2001. Elsevier. Forest Ecology and Management 153 (2001) 15-28. 59 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV knowledge, including appropriate times to look for seeds and seedlings. This activity united generations in the community, strengthened the social fabric and potentially contributed to the sense of ownership of land and natural resources by the participants. Beneficiaries in Chocó explained process as follows: “First we cleaned everything, pulled up the weeds and collected garbage that had been left around, like jars and cans. We filled in the ditches and holes with soil. And we took advantage of the inputs they [OL] gave us, such as chicken manure, coffee husks, and a powder. I don’t know what it was called. The truth is it was very good. They also brought us chemicals to fumigate, to kill the weeds, to kill the ants” (Interviews with beneficiaries, Chocó). In Bajo Cauca OL selected properties that had legal tenure and prepared contracts for forest use. Of the beneficiaries surveyed, 40 percent indicated that they had a contract to support their participation in planting of Acacia mangium. Appealing to economic motivations proved an appropriate strategy to incentivize community participation. OL also worked to combine the planting of this species with beekeeping activity, which promoted greater care for the trees. Interviews found that beneficiaries took ownership of the activity, saying that it was important that people knew the trees from the nursery. As one beneficiary from Antioquia put it: “We had nurseries around the area where we were going to carry out the project because we believe that it is very important for people to know the trees since they were babies, to start generating other sensitivities and other connections with the project and with what they were going to plant” (KII with beneficiary, Antioquia). These activities helped develop a greater connection between the beneficiaries and the trees from the beginning, which encouraged ownership of the project. F. OL’S REHABILITATION MODELS IN ANTIOQUIA AND CHOCÓ INCREASED AREAS COVERED BY VEGETATION AND REDUCED DEGRADED AREAS. To analyze changes in ground cover and deforestation, we drew on satellite images of a sample of the area targeted by OL, made up of 2,077 Ha (represented by 13 polygons) under active rehabilitation.71 This analysis found a four percent increase in vegetation cover and a 14 percent decrease in deforestation in the sample. Using ArcGIS software to interpret the images, we calculated variation in vegetation cover to measure rehabilitation. Comparing satellite images over the period from 2016 to 2020, we analyzed vegetation cover compared to bare ground, interpreted as degraded soil. We used the SAVI index for this analysis since it allowed us to identify different phenological states, making it possible to observe the vegetation in initial stages of development.72 Using this method, we determined the behavior of all the polygons in the sample. Exhibit 16 shows the results for the pilot farm. 71 Due to persistently cloudy conditions, it was not possible to carry out a similar analysis for areas undergoing passive rehabilitation. 72 Comparación del NDVI con el PVI y el SAVI como indicadores para la asignación de modelos de combustible para la estimación del riesgo de incendios en Andalucia. Tecnologías Geográficas para el Desarrollo Sostenible, 164-174. Sánchez Esperanza, Torres Martín, Palacios Arturo, Aguilar Mónica, Pino Isabel, Granado Laura. 2000. Universidad de Alcalá. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 60 Exhibit 16: Behavior of vegetation cover vs degraded areas in the shapefile from the pilot farm, in the municipality of Caucasia. The right side shows satellite images from the farm, with the upper image being from the year 2016 and the lower image from 2020. SOURCE: CALCULATIONS CARRIED OUT FOR THE EVALUATION USING SATELLITE IMAGES. The variation in vegetation coverage found in the sample from Antioquia was not homogenous. For some polygons, vegetation coverage increased over time, as in Vegas de Segovia, Fundación Mineros, Villa Grande, and Chilona (Table 5); while in others it decreased, as in Asogauca and Reforestadora Zaragoza.73 An analysis of the overall behavior of the samples found a four percent increase in vegetation cover and a 13 percent decrease in deforestation. 73 Annex I. Supplemental Analysis and Data – Rehabilitation component (pp. 10-18) shows a review of satellite imagery the behavior of the sum of vegetation over for all polygons in the sample. 74 The thematic accuracy obtained for 11 of the 13 analyzed polygons is greater than 85 percent. The polygons for Tropico Diverso and Cañada la Rica presented ranges of years for which the accuracy is lower, so these years were eliminated in the calculations. TABLE 5. CHANGE IN VEGETATION COVER ORO LEGAL’S TARGETED AREAS IN ANTIOQUIA74 SHAPEFILE ∆ VEGETATION COVER BEGINNING TO END Trópico Diverso 3% Villa Grande 6% Vegas de Segovia 10% Asogauca -17% Reforestadora Zaragoza -14% Asogauca P1 -1% Fundación Mineros P1 2% Fundación Mineros P2 8% 61 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Source: Calculations performed for the evaluation from satellite images In the case of active rehabilitation in Chocó, the sample showed a nine percent increase in vegetation cover and a 35 percent decrease in degraded areas (see Exhibit 17). The sample in this department was limited by persistently cloudy conditions, which decreased the visibility of the images. This meant that information could not be gathered for passive rehabilitation. The active rehabilitation zones, however, showed OL’s best performance in reducing land degradation, as well as a small, but gradual and sustained increase in vegetation coverage. Exhibit 17: Aggregate behavior of vegetation coverage vs degraded areas in Río Quito, Villaconto, Chocó. The right side of the figure shows satellite images from the farm. SOURCE: CALCULATIONS PERFORMED FOR THE EVALUATION FROM SATELLITE IMAGES. Beneficiaries had positive perceptions of the rehabilitation component results. In general, they indicated that the reforestation process was working, as, in addition to the trees that were planted, they observed the return of native animals, such as birds and monkeys. Beneficiaries also felt that the increase in shade helped moderate high temperatures and gave them spaces to walk or relax. One beneficiary in Chocó indicated that: “The reforestation has been really useful, because it didn’t just [plant trees] but rather now one can see a lot of monkeys, parrots, [an animal] that is called the mountain cat here, partridges….” (KII with beneficiary, Chocó). Local actors perceived positive results from the Activity, such as improvements in local microclimate, increased biodiversity, and soil recovery. Of the beneficiaries surveyed, 72 percent mentioned positive effects for water quality, 77 percent recognized positive changes in the landscape, and 72 percent thought that land degraded by mining had recovered (Exhibit 18). Beneficiaries also perceived positive effects of rehabilitation for their quality of life, with 81 percent of respondents indicating that it had positive effects for the community, including for health and food security. Respondents also mentioned Chilona 1% Afroclaver P1 48% Afroclaver P2 1% USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 62 improvements to the landscape, greater tranquility, and repairing relationships with the natural world. One beneficiary in Antioquia said that: “Now we don’t feel the heat like before, you don’t see the land so eroded, now you can see green fields, you see little iguanas, all the animals we have around here, the water is cooler. Then this makes you, yourself, change, you go out [into the fields or forest] and you say, how nice this looks, like it was before. So, this changes your life, because there’s more oxygen, there’s more shade” (KII with beneficiary, Antioquia). Exhibit 18: Beneficiaries’ perceptions of the results of the rehabilitation component of OL (n=318) SOURCE: BENEFICIARY SURVEY G. THE SUSTAINABILITY OF REHABILITATION PROMOTED BY OL IS NOT ASSURED, AND RISKS REMAIN TO THE EFFORT AND RESOURCES INVESTED THIS FAR, BECAUSE ACCORDING TO BENEFICIARIES, CONSERVATION OF REHABILITATED AREAS HAS WEAKENED OVER TIME. The Community Councils do not have the ability to cover the costs of continued care, monitoring, and control of the managed areas. Beneficiaries note that commitment to guarding and conserving the places where trees were planted has weakened over time. It has not been possible to perform timely maintenance of these plantations due to the lack of resources of the councils and communities that would enable them to contribute their time and work. Community members often do not have the time to go to the rehabilitation areas, which can entail trips of up to eight hours. Beneficiaries expressed that: “There are trees disappearing because the maintenance was not done when it had to be done. There we need the program to continue to give us a hand, to continue supporting us with working capital to be able to have personnel to help us restore things – to see if a tree died, to replace it, to check if the weeds are overrunning it, to be able to weed it” (Interviews with beneficiares). Local institutions were not sufficiently involved in supporting rehabilitation activities. This includes their participation from the establishment of the nursery, to caring for targeted areas, to promoting the positive effects of OL. The same local entities point out that activities were not carried out directly with 63 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV them and that there was no joint work to leave in place capacity, which has led to weakening the processes of community ownership after the end of the Activity and even to the loss of nursery infrastructure established by OL. Interviews carried out for the evaluation with GOC officials at different levels confirmed this. One national-level GOC official said that “The results that we have, well, they sent us in a fact sheet, otherwise we don’t have much information.” At the local level, an official in one mayor’s office mentioned that “No projects were executed directly with [the mayor’s office], so it doesn’t know about this matter.” Finally, OL did not promote sustainable exploitation of the acacia planting. Although expectations were raised and commitments were made to promote commercial use of this wood, this was not carried out, and beneficiaries did not receive training on how they might do so. Nor was there planning for gradual replacement of Acacia trees with native timber plants once the soil has been rehabilitated. In turn, the beneficiaries do not currently have the resources to maintain the management of the plantations. Thus, a risk exists to the sustainability of the models implemented by OL, a situation that could be overcome through projects that increase the income of the beneficiaries and support continued management in rehabilitated areas (Exhibit 19). Exhibit 19: Expectations of rehabilitation beneficiaries to obtain economic benefits from rehabilitation (n=318) SOURCE: BENEFICIARY SURVEY 5.4 EFFECTIVE COMBINATION OF INSTRUMENTS AND APPROACHES TO REDUCE THE USE OF MERCURY IN ASGM Evaluation question 5 What combination of instruments and approaches employed in the effort to reduce mercury usage in ASGM have proven to be most effective and why? OL's support for small-scale mining included modifications in gold mining techniques to eliminate the use of mercury. This activity involved significant efforts and investments by the Activity, particularly in terms of mercury-free processing equipment and technical assistance to improve mining environmental performance. Reduction in mercury use is gradual, and OL constantly monitored mercury to assess its results. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 64 A. LAW 1658 OF 2013 AND THE ENTRY INTO FORCE OF MERCURY PROHIBITION IN MID-2018 CONSTITUTES THE MOST IMPORTANT EXTERNAL FACTOR FOR THE REDUCTION OF MERCURY IN MINING PRACTICES. The GOC adopted Law 1658 on July 15, 2013,75 to protect human health and preserve renewable natural resources and the environment. This regulation applies throughout the national territory and limits the import, production, marketing, handling, transportation, storage, final disposal, and release into the environment of mercury for industrial activities. The Ministry of Mines and Energy (MME) was charged with promoting compliance with this obligation to eradicate mercury use in the sector in a maximum of five years. Enormous challenges persist, however, in the chain of control and state regulatory capacities. ASGM takes place in remote regions under the control of illegal armed groups and by small producers who are not trained in appropriate extractive techniques. OL benefited from the issuance of this regulation since it made mandatory an objective of OL, with a positive impact on Activity results associated with the use of mercury. B. SUPPORT FOR FORMALIZATION, TRAINING ON THE EFFECTIVENESS OF GOLD EXTRACTION USING CLEAN TECHNIQUES AND RAISING AWARENESS ABOUT THE IMPACT OF MERCURY ON HEALTH AND THE ENVIRONMENT WAS AN EFFECTIVE COMBINATION OF FACTORS TO ELIMINATE MERCURY. OL provided technical assistance for the preparation of Work and Construction Plans (PTO), as well as Environmental Impact Assessments (EIA) with the MPUs. Article 78 of the Mining Code mandates that PTOs should establish and determine the location of minerals, the exploitable quality and quantity, the technical viability of its extraction, and the environmental impact of implementing this plan (ANM, 2018). Preparing the PTO is a step toward appropriate mineral exploitation, Itprovides knowledge about mineral deposits and analyses of how to exploit these deposits can be registered in the PTO. OL provided important technical assistance to small miners, who otherwise would not have prepared these instruments (at least not with the required quality, nor with effective guidance for mercury-free practices). Ninety percent of MPU beneficiaries indicated that the PTO determined mercury-free mineral processing, and 88 percent recognized that OL helped make accessible the techniques and technologies for the PTO (Figure 19). This allowed the evaluation to conclude that the technical assistance provided by OL (carrying out a study of mineral deposits) guided best practices for mercury￾free extraction, and as such, that formalization was related to the elimination of mercury. Exhibit 20: Support provided by OL to MPUs for PTOs (n=113) SOURCE: BENEFICIARY SURVEY 75 This law responds to Colombia’s status as a signatory to the Minamata Convention on Mercury. 65 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV The training provided by OL for MPUs on formalization achieved their aims and small miners are aware of the current regulations regarding mercury. Miners recognize that mercury is banned and understand that viable alternatives to its use exist. In interviews, beneficiaries affirmed that they do not use mercury, either because they have never used it or because they have reduced its use. They recognized the health dangers and environmental impacts of mercury. One miner said, for example: “This question of mercury is very hard and very difficult, here we don’t use it, we mine without it, for clean, legal mining and for clean land. This mercury is really dangerous, I don’t know anything about it, and I’m a miner and I’m not familiar with it” (KII with a beneficiary). In the beneficiary surveys, beneficiaries of the formalization component mentioned the support and capacity building of the training provided by OL. More than 80 percent of the beneficiaries agreed or strongly agreed that OL helped reduce the use of mercury, that technological change helped reduce the use of mercury in gold processing, that new mercury-free practices yielded more processed gold, and that these practices were helping the environment. Exhibit 21: Level of agreement (4 and 5 on a scale from 1 to 5) with statements about OL’s support for reduction in the use of mercury in mining (n=113) SOURCE: BENEFICIARY SURVEY Beneficiaries who were subcontractors with large mining businesses were aware that they could not use mercury. This was corroborated by partners, who explained that they exercised this control based on the procedures required and support by OL: “The condition that they had for the miners consisted of not using mercury in any of their processes, and the best way to comply was that the ASGM miners delivered the concentrates from the sands, and they (as a legal mining company) processed them with their own technology” (KII with a local partner). Another beneficiary from Antioquia noted that beneficiaries who were subcontractors could now “grind the gold in an area of the mining business where they have the machinery that you need to not use mercury.” Training and, above all, assistance to access new techniques are key elements for beneficiaries in eliminating the use of mercury. Small and informal miners have social and economic conditions that make it difficult for them to decide to change their gold mining practices. Stopping the use of mercury is a long-term process that implies accepting behavioral changes in one’s daily work and recognizing that USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 66 the economic benefits are not immediate. Oro Legal made efforts to demonstrate the efficiency of new practices and to supply the inputs for the technological reconversion. This was one of the achievements of OL that was most recognized by the beneficiaries. One beneficiary from Antioquia affirmed that: “Mercury, to tell the truth, is the cheapest method to extract gold. The other methods are more expensive. So, the advising works. Of course, everyone is aware that they can’t use it [mercury], but that’s why they don’t do it. Many times, the way to survive is stronger than the need to take care of oneself” (KII with beneficiary, Antioquia). Awareness campaigns that highlight the impacts on human health and the environment have had good results. Officials interviewed from the mining and environmental ministries highlighted the development of outreach campaigns as their main strategy to reduce the use of mercury. They recognized the results of OL by reaching users directly and raising awareness of the risks of using mercury. More than 70 percent of the beneficiaries surveyed felt motivated to eliminate mercury from their practices because of health concerns (Exhibit 22). This exercise complements the actions taken by local institutions, which recognized that OL reached miners directly. Exhibit 22: Motivations to not use mercury in the MPUs (n=113, multiple responses allowed) SOURCE: BENEFICIARY SURVEY C. THERE IS NO CERTAINTY ABOUT FULL ELIMINATION OF MERCURY USE AMONG MINERS. OL’s activities contributed to increased compliance with the regulation and have achieved a reduction in the use of mercury. Calculations performed for the evaluation based on OL information show a decreasing trend in mercury use. Beneficiaries went from using 7 grams of mercury to produce 1 gram of gold in the second half of 2017 to 0 grams of mercury to produce 1 gram of gold in the first quarter of 2020. This reduction is more notable in alluvial and underground mining (Exhibit 23). Mercury is used in ASGM because of its ease, speed, and cheapness in recovering gold from amalgamation. The use of mercury depends on the mineralogical composition of the material from which the gold is being recovered, such as whether it contains sulfides, quartz, or other metallic elements.76 Studies carried out 76 Sinopsis nacional de la minería aurífera artesanal y de pequeña escala. Ministerio de Ambiente y Desarrollo Sostenible. 2012. Proyecto Contribución a la construcción de un plan estratégico regional para la reducción del uso de mercurio en la minería aurífera artesanal y de pequeña escala. 67 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV by the Energy Mining Planning Unit77 indicate that underground mining uses between 15 and 35 grams of mercury per gram 1 of gold, and alluvial mining uses an average of 11.8 grams of mercury per 1 gram of gold; this can be reduced with mini-dredgers, and through closed circuits. Exhibit 23: Trends in mercury use among OL beneficiaries SOURCE: CALCULATED BY EVALUATION TEAM FROM INFORMATION FROM ORO LEGAL While mercury use has decreased, it has not been possible to eradicate it completely. According to beneficiary surveys, 1.1 percent of beneficiaries still use mercury (Exhibit 24). This reflects the discrepancies between different sources regarding the results of mercury reduction efforts, and it suggests that, although there is a significant reduction in mercury use, there are still beneficiaries of OL that use it in mining operations. Beneficiaries vehemently affirm their non-use of mercury, but government authorities and researchers on the subject are aware that mercury continues to be used in gold mining. Local authorities in different municipalities expressed as much during interviews. One said: “You look here, and people have spread further to more distant areas to look for gold. They use mercury, but they come to sell it at the same place [that they buy it] after they burn it and make the amalgam at home [at the mining site].” Another local official said that “You will not see reduction, unless in 2025, 2028, there is actually a decided policy - but as long as there is mercury and people can use it, they will continue to use it” (KII with GOC official). Exhibit 24: Mercury use by OL beneficiaries (n=113) SOURCE: BENEFICIARY SURVEY 77 Indicadores de la minería en Colombia. Unidad de Planeación Minero Energética, 2014. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 68 D. THE PROHIBITION OF MERCURY IS A COMPLEX CHALLENGE, INVOLVING THE CONTROL OF ILLEGAL TRAFFICKING, WHICH REMAINS ACTIVE IN THE REGION According to Law 1658, the state had five years to prepare the mining sector for mercury eradication, but it began the required work late. The MME published its strategic plan to eliminate mercury in 2016, three years after the passage of the law. Faced with this reality, local institutions expressed that this work should have begun when the law was adopted, and that awareness raising should have proceeded in parallel with the development of the strategic plan. During an interview for this evaluation, an official of the national government affirmed that: “The regulatory framework on mercury is clear. Now, is this practical and suited to the needs and realities of the field? Well, no, because a law simply does not eliminate the practice. And the Ministry had the power to technically support the miners, and it started late, long after the law went into effect” (KII with GOC official). Assuming that a small percentage of ASGM miners do continue to use mercury, the challenge lies in controlling its trafficking and access to it. Various actors affirmed that the purchase of the mineral occurs in municipalities close to those inhabited by OL beneficiaries, and in gold shops. The miners who continue to use mercury in the extraction of gold are usually located in areas far from other miners and urban centers. One official in Chocó said that: “I think that, without mentioning any particular organization, there is corruption, and this is the reason that mercury continues to make its way to the regions. So, attacking this corruption is important to reduce the influx of mercury. And this raises again the issue of knowledge and [awareness] campaigns” (KII with GOC official). In this sense, activities like OL can complement institutional activities by guiding the formulation of a comprehensive policy, using the knowledge generated in the implementation of local actions. E. FORMALIZATION SUBCONTRACTS WORKED TO CONTROL MERCURY USE. The links formed with large mining companies through formalization subcontracts were an effective way to control the use of mercury. As discussed in section 5.1, OL used its management and logistical capabilities to organize groups of miners around formalization subcontracts. This led to recognition by various actors that formalization subcontracts work very well to control the use of mercury, as long as the process is transparent, and one party is not allowed to have an advantage over the other. In other words, honest dialogue and transparent agreements can be promoted between mining companies and small miners. However, 57 percent of MPUs still sell their product in gold shops, and only six percent sell to a large-scale mining company (Exhibit 25). 69 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 25: Buyers to whom MPUs sell their product (n=113) SOURCE: BENEFICIARY SURVEY 5.5 DIRECT AND INDIRECT ACTORS’ PERCEPTIONS OF OL’S RELEVANCE AND EFFECTIVENESS Evaluation Question 6 What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal’s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Given the extent of the Activity and the variety of actors involved, it is important to establish how these actors perceived the relevance and effectiveness of OL. Inquiring about these perceptions can provide a comprehensive view of an activity like OL, contrasting different opinions and points of view, and yielding a look at both the success factors and the bottlenecks of the intervention. Given that this section cuts across all components of the Activity, some findings overlap with what was stated in the previous sections. For this reason, we reference evidence already discussed in previous sections of the document as a complement to the analysis presented in this section. The main conclusions regarding the relevance and effectiveness of the intervention are presented below. A. OL IS PERCEIVED AS RELEVANT AND APPROPRIATE TO THE CONTEXT OF THE REGIONS IN WHICH IT IS ACTIVE, AND BENEFICIARIES HIGHLIGHT ITS MULTIPLE STRENGTHS; CONTINUED FOLLOW-UP IS CONSIDERED IMPORTANT TO GUARANTEE ITS CONTINUED SUSTAINABILITY. In general, as discussed in section 5.1, beneficiaries are highly satisfied with the Activity and highlight its multiple strengths. Beneficiary surveys found that 73 percent of beneficiaries in mining formalization, 77 percent in beekeeping, 59 percent in rehabilitation, and 40 percent in annatto, respectively, report their satisfaction with OL as a four or five (on a scale of one to five, with five being the highest) (Exhibit 26). Moreover, interviews highlighted the relevance of the Activity, as well as its effects for the environment, mining activity, economic development, quality of life, and the participation of women and young people. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 70 Exhibit 26: Satisfaction with OL and perception of its effectiveness (n formalization=128, n beekeeping=297, n annatto=381, n rehabilitation=377) SOURCE: BENEFICIARY SURVEYS AND INTERVIEWS 78 The Activity is also relevant in all its components to the context and needs of the targeted regions and municipalities. The evidence presented in section 5.3 shows that the targeted municipalities are areas affected by high environmental degradation due to mining activity, thus highlighting the importance of (for example) the OL rehabilitation component. Along the same lines, sections 5.1 and 5.2 clearly show the importance of mining formalization and the promotion of alternative economic activities in the targeted areas. The inputs, machinery, training, and assistance provided by OL are highly valued by beneficiaries; the importance of follow-up is again highlighted as a requirement for sustainability. One beneficiary from Antioquia noted: “Oro Legal accomplished for us what I expected it would; they offered us a lot of advice and a lot of help, and they fulfilled it. I am very grateful to Oro Legal for everything it did” (KII with beneficiary). Nonetheless, some beneficiaries worried about the continuity of OL after the Activity closed. One beneficiary from Chocó said: “It was a very good thing, but there was also a lack of follow-up with the program, they completely forgot about us, it’s done and goodbye” (KII with beneficiary). This is a clear threat to OL’s achievements and could pose a challenge due to time and contractual obligations specific to the Activity. Finally, the evaluation identified direct beneficiaries who were unaware of OL’s objectives and activities or reported having received no support from the Activity. This group is small, as just five out of 52 78 Perceptions of OL’s effectiveness are measured by the number of mentions in interviews. 71 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV interviewees (9.7 percent) and 51 of 669 survey respondents (7.6 percent) were unaware of the Activity. One interviewee said: “There was no support from Oro Legal. I participated in the occasional meeting, but there was no technical assistance. I am still waiting for support or supplies to arrive. But okay, at least Oro Legal is an entity that has the will and is ready to help” (KII with beneficiary). B. OL PROVIDED KNOWLEDGE ON FORMALIZATION AND REDUCING MERCURY USE, CONTRIBUTED TO DIGNIFYING ARTISANAL MINING AND INCREASED BELIEF IN SUSTAINABLE MINING AND COORDINATION WITH MINING AND ENVIRONMENTAL AUTHORITIES. HOWEVER, INSTITUTIONAL WEAKNESS CONTINUES TO AFFECT TRUST AND EFFECTIVENESS. OL contributed to creating a positive perception that it is possible to carry out mining operations in an environmentally, socially, and economically sustainable way. It also facilitated linkages between MPUs and the mining and environmental authorities in the formalization process, although trust and perceptions of effectiveness are affected by the weak capacity of the relevant institutions. Section 5.1 explores the supporting evidence in full. This section indicates that OL contributed to reversing the skepticism of artisanal and small-scale miners, while acting as a bridge between small-scale miners and the authorities. Quantitative evidence indicates that 32 percent of respondents consider that one of the main advantages of mining formalization is access to best practices, while only two percent of respondents indicate that there is no advantage to formalization. These linkages do not always work smoothly, as while miners must comply with strict deadlines to carry out formalization, the authorities often do not themselves comply with these deadlines. The Activity filled in knowledge gaps among the miners, providing support and tools for the formalization process and for reduction in mercury use. Beneficiaries noted that, before the legal prohibition on the use of mercury, OL began to share knowledge on what would need to be done to comply with this ban. One beneficiary from Antioquia said: “When the government banned mercury, many people here were left adrift, because the government just said: “this is over and that’s that” and sent in the police. In this case, Oro Legal filled this hole that was there (…)” (KII with beneficiary, Antioquia). The Activity offered tools and knowledge that the miners would not have otherwise obtained. Another beneficiary from Antioquia affirmed that OL “taught things that otherwise would have been inaccessible, because to get an environmental advisor costs money, all that costs money and Oro Legal gave us that advice.” Environmental advise activities seem to have had an impact on mercury use. Quantitative evidence (section 5.4) indicates that a major motivation for 49 percent of beneficiaries surveyed to not use mercury was to protect water sources, and 42 percent did not use mercury to protect air quality. C. THERE IS A POSITIVE PERCEPTION OF RECOVERY OF NATIVE SPECIES, REGULATION OF TEMPERATURE (LOCAL MICROCLIMATE), AND THE AWARENESS RAISING AMONG COMMUNITIES THAT CREATES LOCAL OWNERSHIP IN THESE ZONES, WHILE OTHER RESULTS, SUCH AS REHABILITATION, HAVE TO BE CONSIDERED AS MORE LONG TERM. Beneficiaries had positive perceptions of the recovery of native species and changes in the landscape, and highly valued the rehabilitation interventions in degraded areas are highly valued. This finding is discussed USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 72 in section 5.3, which fully presents the evidence on increases in vegetation cover, and the decrease in degraded areas achieved by OL’s rehabilitation models in Antioquia and Chocó. Beneficiaries expected that some effects of the rehabilitation intervention would be by their nature visible in the future and over the long term. Different actors involved in OL agreed that reforestation and restoration of degraded areas would take place over the long term, even if they have yielded positive results in the immediate term. In the words of one beneficiary from Antioquia: “Although the issue of reforestation and exploitation is long-term, you can see the economic issues there, which is in exploitation [of forest products] in the long-term and let's say the issue of beekeeping in the shorter term” (KII with beneficiary, Antioquia). D. THE VALUE CHAIN ACTIVITIES CONTRIBUTED TO IMPROVING THE ENVIRONMENT THROUGH BEEKEEPING AND ANNATTO AND THROUGH TRAININGS ON HOW TO MANAGE WASTE GENERATED BY THE INTERVENTION. Beneficiaries valued the beekeeping and annatto value chain activities for their environmental contributions. This is supported by the findings in section 5.2, which provides more extensive evidence on this point. The entirety of the qualitative and quantitative data indicates that OL has improved environmental conditions, averted land degradation, and discouraged illegal mining in the areas covered by the intervention. E. THE TRAINING HAD A POSITIVE INFLUENCE ON THE ADOPTION OF ENVIRONMENTALLY FRIENDLY PRACTICES AND THE APPROPRIATE MANAGEMENT OF WASTE GENERATED BY VALUE CHAIN ACTIVITIES. IT IS WORTH NOTING THAT BOTH VALUE CHAINS (ANNATTO AND BEEKEEPING) TAUGHT BENEFICIARIES HOW TO CORRECTLY MANAGE THESE WASTES. ONE BENEFICIARY WORKING IN THE AREAS OF ANNATTO AND REHABILITATION NOTED THAT “THE SHELL OF THE ANNATTO, FOR EXAMPLE, THEY TOLD US COULD BE USED AS FERTILIZER ON OUR PLANTS, SO WE DID THAT.” A BEEKEEPING BENEFICIARY TOLD INTERVIEWERS THAT INSTEAD OF THROWING THE BAGS CONTAINING BEE FOOD ON THE GROUND, “WE PUT THEM IN OUR POCKETS AND TAKE THEM HOME TO THROW IN THE TRASH THERE.” THE VALUE CHAINS REPRESENT ECONOMIC ALTERNATIVES THAT GENERATE CONSIDERABLE ADDITIONAL INCOME THROUGH TECHNICAL ASSISTANCE AND THE PROVISION OF INPUTS, AS WELL AS OPPORTUNITIES FOR FORMAL EMPLOYMENT. Value chain activities represent an economic alternative for families, and the technical assistance and inputs provided have contributed to increasing incomes. This is supported by section 5.2, which shows in depth how beneficiaries consider value chains to be viable alternatives for income generation if they are able to consolidate the productive and commercial chains of which they are a part. This section also explains how the income garnered from the value chains complements household income and positively impacts beneficiaries’ quality of life. OL generated opportunities for formal employment. The beneficiary survey found that 38 percent of respondents are affiliated with health and pension systems, which is a proxy for formal employment. The highest percentages are found among beneficiaries of the mining formalization component (51 percent), annatto (49 percent), and beekeeping (39 percent). For those working on rehabilitation (20 percent), the Activity did not have an objective to generate formal work or to establish an alternative livelihood 73 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV (Exhibit 27). This finding accords with the evidence presented in section 5.1, which shows that the good business practices imparted by OL enabled advances in labor rights and security. Interviewees also confirmed that they had found formal work with contracts that included benefits. Exhibit 27: Beneficiaries affiliated to health and pension systems (n formalization=128, n beekeeping=297, n annatto=381, n rehabilitation=377) SOURCE: SURVEY OF BENEFICIARIES F. OL IMPROVED QUALITY OF LIFE IN THE COMMUNITIES (RECOGNITION, COMMUNITY WORK, KNOWLEDGE, AND PURCHASING POWER) WHILE LEGALITY GENERATED MULTIPLE ADDITIONAL BENEFITS (SOCIAL AND JURIDICAL SECURITY, SELF-ESTEEM, AND DECREASED RISKS). The value chain activities improved quality of life by incentivizing collaborative work in the family and in communities, promising greater knowledge, employment, and incomes. Eighty-five percent of beekeeping beneficiaries and 51 percent of annatto beneficiaries considered that the support from OL had contributed to improving the quality of life of their families (Exhibit 28). This follows from section 5.2, which lays out the evidence that the value chains had a positive effect on beneficiaries’ quality of life, as represented by the generation of employment, greater purchasing power, and knowledge of technical terms and language. Exhibit 28: Perception of OL support for quality of life (n annatto=346, n beekeeping=281 SOURCE: SURVEY OF BENEFICIARIES USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 74 The value chains generated recognition and self-recognition among local communities, propelling territorial development and offering a benchmark for other regions, groups, and associations. It is worth noting that the annatto component has pushed forward a regional agro-industrial chain that demonstrates the capacity of the residents of Chocó to steer their own territorial development. This component represents a project without precedent in the department. Legalization of mining operations generated a variety of benefits associated with security before the law, peace of mind, reduced personal risk, higher self-esteem, a sense of belonging, lower costs, increased sales, and the absence of worry about fines and law enforcement operations. In general, the Activity contributed to greater legality, which improved beneficiaries’ quality of life through benefits like access to social insurance in case of accidents and a sense of security from knowing that law enforcement authorities would not impede miners’ work. This is accompanied by an increase in recognition and self￾esteem, as it breaks with the common paradigm that miners have no intention of being legal. This point is deepened in section 5.1. G. DESPITE PERCEPTIONS OF GREATER PEACEFULNESS IN THE TARGETED ZONES, THE SECURITY SITUATION IS STILL COMPLEX, AND THIS HAD AN IMPACT ON OL’S EFFECTIVENESS. Beneficiaries perceived a greater sense of security in areas where OL was active. One beneficiary explained that “this is the first time that the institutions have come to these regions, you can at least feel valued and that you exist, and this has implications for security in the area.” These shifts also contributed to a reduction in illicit activity in these areas, in favor of the establishment of alternative livelihoods. This point is further developed in section 5.2, which demonstrates how these alternatives (among other benefits) offer options for young peoples’ use of time, reducing vulnerability to recruitment by armed actors. Nevertheless, the security situation in the areas where OL operated continues to be complex and these complexities have negative implications for the effectiveness of the Activity. Threats from armed groups were detected, which constitute the chief obstacle or potential threat to the development of the beekeeping value chain, as discussed in section 5.2. H. THE ACTIVITY GENERATED WORK OPPORTUNITIES THAT IMPROVED THE QUALITY OF LIFE FOR WOMEN AND THEIR FAMILIES, DUE TO INCREASED INCOME, EMPOWERMENT, AND COMMUNITY TIES. FOR YOUNG PEOPLE, THERE ARE FEW WORK OPPORTUNITIES IN THE AREAS TARGETED BY OL, AND THIS IS COMPOUNDED BY LOW INTEREST IN ACTIVITIES ASSOCIATED WITH THE COUNTRYSIDE. OL generated work opportunities for women, improved their incomes, promoted their empowerment, and generated community linkages that improved their quality of life and that of their families. As shown in Figure 27, 37 percent of direct beneficiaries were women, participating mostly in the beekeeping (50 percent), rehabilitation (44 percent), and annatto (29 percent) components. Section 5.2 explores women’s participation in depth, finding that it was significant in the first link of the value chain, generating positive effects for autonomy, recognition, and family connections. In the mining formalization component, while mining is a historically male-dominated activity, 14 percent of participants were women. 75 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Exhibit 29: Distribution of direct beneficiaries by sex (n formalization=128, n beekeeping=297, n annatto=381, n rehabilitation=377) SOURCE: DIRECT BENEFICIARY DATABASE Beneficiaries perceived that it was important to involve young people in the value chain activities to connect generations and to keep youth from getting involved in illicit activities. One beneficiary said, “involving young people depends on the parents or on producers, they should lead by example (…).” Section 5.2 examines this point in greater detail, finding that the value chains offer young people an alternative way to use their time, which can help keep them from being recruited by armed groups. There are few opportunities in the labor market for young people, and sometimes they are not very interested in activities associated with the countryside. Actors interviewed indicated that there are few opportunities for young people to enter the labor market. In the words of one beekeeping beneficiary, “a threat in the countryside is that young people want to improve their lives, and the armed groups take advantage of that with false promises to lure them away (…).” This is also complicated when young people are not interested in rural activities and seek to migrate to the cities. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 76 F. RECOMMENDATIONS The evaluation team developed 43 recommendations, which largely seek to suggest potential courses of action for future USAID Activities related with ASGM. These recommendations are organized as follows: eight are related to mining formalization and governance, eight are related to value chains and alternative livelihoods, eleven address the theme of rehabilitation in areas degraded by mining activities, seven are associated with the elimination or reduction of mercury in the ASGM supply chain, and six are related to the perceived relevance, impact, and effectiveness of the Activity. Each subsection presents the recommendation with related implementation measures and timeframe for carrying them out. 6.1 INCENTIVES FOR ASGM MINERS TO CONTINUE BEING LEGAL/FORMAL AND IMPROVE THE ENVIRONMENTAL PERFORMANCE OF THEIR OPERATIONS AND (II) FACTORS (INTERNAL OR EXTERNAL) THAT HAVE CONTRIBUTED TO OR HINDERED STRENGTHENING OF MINING AND ENVIRONMENTAL GOVERNANCE RECOMMENDATION 1 CONTINUE TO EMPHASIZE TO MINERS AND AUTHORITIES, IN ALL ACTIVITIES AND COMPONENTS OF ACTIVITIES LIKE OL, THAT ENVIRONMENTAL, SOCIAL, AND ECONOMIC PERFORMANCE CAN ONLY BE IMPROVED THROUGH PROCESSES OF FORMALIZATION AND DIGNIFICATION OF ASGM. Implementation measures: i) Incorporate measures to combat stigmatization of ASGM in all components for activities of this type; ii) Hire technical staff from the targeted areas to facilitate communication with miners, local communities, and the authorities; iii) Identify the most useful business practices for ASGM and develop trainings specifically for this sector; iv) Prepare documents, booklets, Point of Purchase displays, and dissemination tools emphasizing the benefits of formalization. The responsible parties are USAID, the implementing partner, and the MME. This is a short-term measure (up to two years). RECOMMENDATION 2 MAINTAIN AND/OR REINFORCE THE ADAPTIVE APPROACH OF ACTIVITIES LIKE OL WORKING ON ASGM WITH REGULATORY CHANGES AND TURNOVER OF OFFICIALS IN GOVERNMENT ENTITIES. Implementation measures: i) Pay constant attention to the issuance of regulations that affect (negatively or positively) the program’s work; ii) Design, through the implementing partner and in coordination and partnership with the relevant institutional actors, specific guidelines for dialogue and direct connection with government officials, including objectives, scope, key actors, and local commitments. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 3 INCREASE USAID’S PARTICIPATION IN ESTABLISHING THE RELATIONSHIP BETWEEN THE IMPLEMENTING PARTNER AND GOVERNMENT INSTITUTIONS, IN ORDER TO LEND OFFICIAL STATUS TO THE IMPLEMENTER’S PROPOSALS AND REQUESTS. 77 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Implementation measures: i) Formalize meetings with the authorities by taking minutes that state what is presented, what has been agreed upon, commitments, and responsible parties. If possible, engage USAID or other credible actors in these institutional meetings; ii) Partner with research institutions to prepare analytical documents on the economic, social, and environmental impacts (and solutions) associated with current processes and the results of the formalization policy, and communicate these findings. The experience of OL and of BioREDD+ provide useful insights for improving these processes; iii) Design short courses, aimed at officials in the mining and environmental authorities, that present the recent experiences of USAID and other national and international organizations. The responsible parties are USAID and the implementing partner. This is a medium-term measure (up to two years). RECOMMENDATION 4 STRENGTHEN THE TECHNICAL ENTRY CRITERIA AND THE MINIMUM STAGE THAT BENEFICIARIES MUST REACH WITHIN THE FORMALIZATION PROCESS TO GUARANTEE THAT THE BENEFICIARIES COMPLY WITH THE INTERVENTION CYCLE AND DEFINE WHAT HAPPENS FOR BENEFICIARIES WHO DO NOT REACH THE MINIMUM STAGE SET BY THE PROGRAM. Implementation measures: i) Define the minimum duration of OL-type activities to be the length of time required to obtain the mining title and the environmental license; ii) Establish a plan for closing activities that includes preparing beneficiaries to continue, guaranteeing their formal status, after work concludes. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 5 ENSURE THAT THE DESIGN OF OL-TYPE PROGRAMS INCLUDES, FROM THE BEGINNING, ACTIONS TO CREATE NEW CHANNELS FOR THE COMMERCIALIZATION OF GOLD AND ASSOCIATED MINERALS AND TO ESTABLISH PARTNERSHIPS WITH ORGANIZATIONS THAT PROMOTE FAIR TRADE, ETHICAL MARKETS, AND PRICE DIFFERENTIALS THAT SUPPORT ENVIRONMENTAL AND SOCIAL RESPONSIBILITY. Implementation measures: i) Set as a specific objective for OL-type activities the opening of new channels of formalization through the identification of potential partners from the beginning of the activity, to involve them in the process, and sign formal entry agreements with the beneficiaries under the environmental or social requirements established by this counterpart. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 6 VERIFY THE ACHIEVEMENTS OF BIOREDD+ AND OL, TO DETERMINE THE MOST APPROPRIATE DURATION OF TIME FOR THESE PROGRAMS TO SATISFY THE EXPECTATIONS OF USAID. THIS INCLUDES COORDINATING ACTIVITIES WITH OTHER TRAINING PROVIDERS TO STREAMLINE, MAXIMIZE RESOURCES, AND REDUCE PRESSURE ON MINERS. Implementation measures: i) Coordinate the timing of OL-type activities with those focused on support to small miners to achieve structural changes in terms of formalization and behavior; ii) Establish ongoing dialogue and partnerships with training organizations, such as SENA, academic institutions, NGOs, state contractors, governors’ offices, extractive businesses, and mining and environmental USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 78 authorities, with the goal of rationalizing and increasing the effectiveness of trainings for ASGM. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 7 MORE ACTIVELY USE THE POSITION OF USAID TO PROMOTE OBLIGATORY MINING FORMALIZATION AS AN OFFICIAL STATE POLICY, DRAWING ON THE EXPERIENCES OF OL. Implementation measures: i) Document and socialize more broadly USAID’s experiences with activities like OL to promote mining formalization as a comprehensive state policy; ii) Examine the benefits and drawbacks of making formalization obligatory, in order to present a proposal to the government. The responsible parties are USAID, the implementing partner, and the MME. This is a medium-term recommendation (up to two years). RECOMMENDATION 8 CONTINUE THE PRACTICE OF HIRING TECHNICAL PERSONNEL FROM THE TARGETED REGIONS. Implementation measures: i) Identify (inventory) and draw on the pool of professionals in the areas that are included in the program. The responsible parties are USAID and the implementing partner. This is a medium-term recommendation (up to two years). 6.2 VALUE CHAIN BENEFICIARIES’ PERCEPTION OF THE IMPACT OF ALTERNATIVE LIVELIHOODS FOR THEIR PRESENT AND FUTURE WELL-BEING RECOMMENDATION 9 CONTINUE TO INCENTIVIZE THE DEVELOPMENT OF AGRICULTURAL VALUE CHAINS, SELECTED THROUGH A COMPREHENSIVE ANALYSIS OF VOCATIONS AND SUITABILITY ACROSS THE SELECTED TERRITORY, TO ADVANCE GRADUAL TRANSITIONS OF ACTIVITY ACROSS MINING AREAS. Implementation measures: i) Incorporate into the Activity a comprehensive evaluation of suitability (environmental, agroecological, logistical, cultural, and with regard to land use planning) for each value chain to be promoted; ii) Work with local communities and regional government entities to select the value chains; iii) Conclude agreements among producers and government institutions to enable a gradual transition from mining into new activities according to the levels of development of the value chains in each area. The responsible parties are governors’ offices, mayoral offices, USAID, departmental and municipal secretaries of agriculture, autonomous regional corporations, the Rural Agriculture Planning Unit (UPRA), Agrosavia, and ICA. This is a medium-term measure (up to two years). RECOMMENDATION 10 CONTINUE TO DEVELOP TECHNICAL CAPACITY IN THE TARGETED TERRITORIES AND TO DESIGN STRATEGIES THAT PROMOTE THE SUSTAINABILITY OF TECHNICAL ASSISTANCE OVER TIME. 79 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Implementation measures: i) Involve technical and professional staff from the targeted areas in the provision of technical assistance, training them in agricultural extension approaches, and unifying the criteria for productive systems to transfer capacity to local producers; ii) Design technical assistance models in the territories to assure their financing from different sources: integration with PDEAs,79 support from the private sector or civil society, and self-financing through the agricultural business itself. The responsible parties are governors’ offices, mayoral offices, USAID, the implementing partner, departmental and municipal secretaries of agriculture, SENA, ADR, and technical and commercial partners. This is a medium-term measure (up to two years). RECOMMENDATION 11 MANAGE SUPPORT FOR THE CONTINUITY OF TECHNICAL ASSISTANCE FROM COLTAPICOLA AND THE TECHNICAL SPECIALISTS FROM THE COMMUNITY COUNCILS TO THE PRODUCERS THAT PARTICIPATED IN OL. Implementation measures: i) Provide advising to COLTAPICOLA and the community councils in order to determine, with the producers and the organizations, financing options for provision of technical assistance, such as: direct financing with the business, working with PDEA, or securing support from a third party; ii) Manage the integration of the service provision model of COLTAPICOLA and the community councils for the beekeeping and annatto projects, including already-existing and new projects. The responsible parties are USAID, COLTAPICOLA, community councils, beekeeping associations, Campo Dulce, technical and commercial partners for the two value chains, departmental and municipal agriculture secretaries, and ADR. This is a short-term measure (less than six months). RECOMMENDATION 12 FOCUS INTERVENTIONS TOWARDS THE DEVELOPMENT OF LOCAL AND REGIONAL PRODUCTION CHAINS THAT ADD VALUE, CONNECT WITH MARKETS, AND ARE COMPETITIVE. Implementation measures: i) Identify and define, in the design phase, the links of the value chain to be promoted by the project; this exercise should analyze its viability, including the capacity present in the territory, current connections with markets, current and potential demand, available resources, and the time required for execution; ii) Incorporate in the new interventions any previous processes of developing agrobusinesses- in the region, giving preference in the value chain to actors that add value in the targeted area; iii) Implement monitoring mechanisms that guarantee the chain approach in execution, reconciling the requirements, possibilities, and agreements of each link in the chains from primary production to commercialization. The responsible parties are USAID and the implementing partner. This is a medium-term measure (up to two years). RECOMMENDATION 13 GUARANTEE THAT THE INCENTIVES FOR ESTABLISHMENTS WITH SMALL PRODUCERS, ESPECIALLY ON A LARGE SCALE, ARE SUPPORTED BY A COMMERCIAL CHAIN OF THE 79 Departmental Agricultural Extensions Plans are supported in their planning, monitoring, and evaluation by the Rural Development Agency. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 80 VOLUMES TO BE PRODUCED AND THE DESIGN OF AN ASSOCIATIVE BUSINESS MODEL THAT ENSURES ITS VIABILITY. Implementation measures: i) Identify in the planning phase the market or markets that will be linked to in order to identify the technical specifications, commercial conditions, and volumes of products that will be required, involving commercial partners from the beginning of execution; ii) Incorporate from the start of execution the design of a business model with a value chain approach that brings together possibilities, time required, costs and prices for primary production, processes for aggregating value, logistics, and pre-agreements for demand, and distribute resources across the different links of the chain as needed; iii) Include in planning processes the size of the productive establishments in line with the business model and, if necessary, define the phases of establishment according to the advances envisioned in production and commercialization. The responsible parties are USAID and the implementing partner. This is a medium-term measure (up to two years). RECOMMENDATION 14 ENSURE THE CONTINUITY OF FINANCIAL, TECHNICAL, AND BUSINESS SUPPORT IN ORDER TO COMPLETE THE CONSOLIDATION OF THE ANNATTO VALUE CHAIN IN CHOCÓ. Implementation measures: i) Guarantee support for A&ACH to complete the process with Acumen and, if financing is not secured, support the enterprise in finding alternative mechanisms. The responsible parties are USAID and A&ACH. This is a short-term measure (less than six months). RECOMMENDATION 15 STRENGTHEN INCENTIVES FOR THE PARTICIPATION OF WOMEN AND YOUNG PEOPLE IN AGRICULTURAL VALUE CHAINS AND FOR THE ACTIVE INVOLVEMENT OF FAMILIES AND THEIR WAY OF LIFE IN THE PRODUCTIVE ACTIVITIES. Implementation measures: i) Incorporate family and community participation into agricultural extension activities to encourage women’s leadership and the involvement of young people in the productive units; ii) Determine a strategy that guarantees the participation of women in decision making processes and in all the links of the value chain targeted by the project. The responsible parties are USAID and the implementing partner. This is a medium-term measure (up to two years). RECOMMENDATION 16 STRENGTHEN STRATEGIES FOR CULTURAL AND COMMUNITY OWNERSHIP OF PROJECTS, FACILITATING THE EMPOWERMENT OF LOCAL AGRIBUSINESSES AND THE PARTICIPATION OF YOUTH LEADERSHIP IN THE COMMUNITY. Implementation measures: i) Identify and involve community organizations and representative leadership in activities from the beginning of execution and at different links of the value chains; ii) During execution, implement leadership training strategies aimed at young people from producer families so that they participate in the activities of adding value, innovation, and technology; iii) Implement training and support plans to strengthen the capacity of organizations and leaders to gradually take over the operation of the agrobusinesses- created in the execution of the projects. The responsible 81 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV parties are USAID, the implementing partner, mayors’ offices, SENA, and chambers of commerce. This is a long-term measure (more than two years). RECOMMENDATION 17 CONTINUE TO PROMOTE VALUE CHAINS THAT INCORPORATE THE AGROECOLOGICAL SYSTEMS OF THE TARGETED AREAS. Implementation measures: i) Incorporate into the activities design analysis and studies to identify the productive systems that are most viable and adaptable, with the lowest environmental impact; ii) Implement monitoring actions during activity execution to ensure that productive establishments and agribusinesses do not move the boundaries of cultivated and uncultivated land, use water rationally, and manage waste well; iii) Implement communications activities to reach out to producers and publicize the synergies between good environmental management and the productivity of the establishments. The responsible parties are USAID, mayors’ offices, the implementing partner, autonomous regional corporations, and UPRA. This is a medium-term measure (up to two years). 6.3 RELEVANCE AND EFFECTIVENESS (ENVIRONMENTAL, SOCIAL, AND ECONOMIC) OF THE DIFFERENT MODELS FOR REHABILITATION OF DEGRADED LANDS IN ANTIOQUIA AND CHOCÓ RECOMMENDATION 18 INCORPORATE ENVIRONMENTAL CRITERIA INTO THE DESIGN OF ACTIVITIES LIKE OL, TO ALLOW TARGET AREAS TO BE SELECTED IN A WAY THAT IS ALIGNED WITH THE GOALS OF NATIONAL AND LOCAL POLICY AND THAT INCREASES THE IMPACT OF THE REHABILITATION ACTIVITY. IMPLEMENTATION MEASURES: i) Reinforce the targeting of areas using information available from the Institute of Hydrology, Meteorology and Environmental Studies (IDEAM) and local entities, such as satellite images, information from the forest and carbon monitoring system, early warnings of deforestation, and identification of active nuclei of deforestation; ii) Construct a baseline for environmental liabilities by carrying out social mapping exercises; this offers a participatory methodology for research in the territory, and would contribute to raising awareness of environmental degradation. The responsible parties are USAID, MADS, IDEAM, regional autonomous corporations, IIAP, mayors’ offices, and local communities. This is a medium-term measure (up to two years). RECOMMENDATION 19 INCLUDE SOIL ANALYSIS, MEASURING POLLUTION AND DEGRADATION, IN THE DESIGN OF ACTIVITIES LIKE OL. IMPLEMENTATION MEASURES: i) Formulate a soil action plan to determine the state of soil pollution and degradation, and to promote its recovery through bioremediation actions that reduce the risks of contamination (e.g., mercury, cyanide, fuels, oils) and increase soil fertility. The responsible parties are USAID, governors’ offices, mayors’ offices, autonomous regional corporations, IIAP, and academic institutions. This is a short-term measure (six months). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 82 RECOMMENDATION 20 ACTIVITIES LIKE OL SHOULD WIDELY DISSEMINATE THEIR RESULTS AND THE EFFECTS OF REHABILITATION FOR QUALITY OF LIFE AND SOCIETY AT LARGE TO GENERATE COMMUNITY OWNERSHIP AND GREATER CARE OF THE REHABILITATED ZONES BY LOCAL POPULATIONS. IMPLEMENTATION MEASURES: i) Implement monitoring and evaluation mechanisms for variables on health, food security, and other social benefits of rehabilitation; ii) Implement publicity campaigns on the impacts of rehabilitation for quality of life, with an emphasis on health and food security, and making use of testimonies and results from OL. The responsible parties are USAID, governors’ offices, mayors’ offices, autonomous regional corporations, and NGOs. This is a short-term (six months) and medium￾term (up to two years) measure. RECOMMENDATION 21 CONNECT REHABILITATION COMPONENTS OF ACTIVITIES LIKE OL WITH THE ETHNIC DEVELOPMENT PLANS OF THE COMMUNITY COUNCILS, TO STRENGTHEN ENVIRONMENTAL GOVERNANCE AMONG AFRO-COLOMBIAN COMMUNITIES. IMPLEMENTATION MEASURES: i) Coordinate with the community councils to connect rehabilitation with the chapter on environmental sustainability in the ethnic development plan; ii) Transfer knowledge to the community councils about conservation incentives with the goal of encouraging commitment to rehabilitation actions. The responsible parties are USAID and the community councils in targeted areas. This is a short-term measure (six months). RECOMMENDATION 22 CONNECT REHABILITATION COMPONENTS OF ACTIVITIES LIKE OL WITH LOCAL ENVIRONMENTAL MOVEMENTS AND LEADERS TO WORK JOINTLY WITH THEM AND ENCOURAGE THEIR WORK IN THESE AREAS. IMPLEMENTATION MEASURES: i) Create a mapping of social movements, environmental movements, and environmental leaders as part of the design process for rehabilitation activities; ii) Undertake new rehabilitation activities with local movements and leaders and publicize these activities through communications work by the Activity and USAID. The responsible parties are USAID, NGOs, regional autonomous corporations, and SENA. This is a short-term measure (six months). RECOMMENDATION 23 PROMOTE PAYMENT FOR ENVIRONMENTAL SERVICES AND/OR THE SUSTAINABLE EXPLOITATION OF ACACIA WOOD. IMPLEMENTATION MEASURES: i) Sign conservation agreements; and ii) Develop a strategy for the sustainable exploitation of Acacia wood, with this measure connecting to the recommendation on ecological succession. The responsible parties are USAID, MADS, MADR, autonomous regional corporations, IIAP, and the business community. This is a medium-term measure (up to two years). 83 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV RECOMMENDATION 24 IN PROGRAMS LIKE OL, MANAGE ACACIA MANGIUM PLANTING IN A WAY THAT MOVES TOWARD ECOLOGICAL SUCCESSION, MAKING USE OF ENDEMIC SPECIES THAT CONTRIBUTE TO REHABILITATION OF THE SOIL (APPLIES TO ANTIOQUIA). IMPLEMENTATION MEASURES: i) Design and implement a progressive substitution plan that encourages sustainable exploitation of forest resources, including through the identification of native species that allow for the replacement of Acacia mangium, establishing a reference ecosystem for the component, including synergies with value chain activities (e.g., beekeeping), upkeep of nurseries, and the planting of endemic species. The responsible parties are NGOs, regional autonomous corporations, IIAP, mayors’ offices, and USAID. This is a short-term measure (six months). RECOMMENDATION 25 IN ACTIVITIES LIKE OL, WHICH INCLUDE PASSIVE REHABILITATION COMPONENTS, ESTABLISH ALLIANCES WITH LOCAL AUTHORITIES AND GRASSROOTS COMMUNITIES TO INCREASE LOCAL OWNERSHIP AND ENSURE THE RESULTS OF REHABILITATION (APPLIES TO CHOCÓ). IMPLEMENTATION MEASURES: i) Design and implement a control strategy for areas under passive rehabilitation, identifying factors that could limit primary growth; ii) Bring together work with community councils and local government institutions, such as mayors’ offices, CAR, and IIAP) to strengthen local ownership; iii) Manage resources in order to finance working capital and incentives for community members to take charge of the care and maintenance of the zones of passive rehabilitation, especially when these are far from where community members live and work, implying extra effort to take care of them; iv) Involve local entities (mayors’ offices, CAR, and IIAP) in the selection of planting sites and encourage counterparts to commit inputs, whether funding, technical assistance, or land. The responsible parties are USAID, NGOs, autonomous regional corporations, IIAP, mayors’ offices, and local communities. This is a short-term (six months) and medium-term measure (up to two years). RECOMMENDATION 26 DOCUMENT THE COSTS OF REHABILITATION TO CONTRIBUTE TO THE PLANNING OF SUBSEQUENT ACTIVITIES WITH A SIMILAR SCOPE. IMPLEMENTATION MEASURES: i) Prepare a detailed costing for rehabilitation components, with differentiation by type of territory being managed (Type I, II, and III as defined in the implementation of OL) and by type of rehabilitation (whether active or passive), based on information from the implementer and local operators. Activities should also facilitate access to that information by the responsible institutions. Responsible parties are USAID, the implementing partner (regional coordinators), and local staff. This is a short-term measure (six months). RECOMMENDATION 27 ACTIVITIES THAT INCLUDE PASSIVE REHABILITATION WITHIN COLLECTIVE TERRITORIES SHOULD INCLUDE THE DEVELOPMENT OF A TERRITORIAL CONTROL STRATEGY, IN CONCERT WITH COMMUNITY COUNCILS TO CONTRIBUTE TO THE MAINTENANCE OF THE MANAGED AREAS. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 84 Implementation measures: i) Advance conflict resolution trainings in order to strengthen the role of community councils, respecting the traditional customs of the community; ii) Support the community councils in activities that encourage their ethnic-cultural identity and promote management and conservation of natural resources; iii) Connect the actions of the Activity with the national round table on restoration in order to develop the work of the institutions in the territory; iv) Improve linkages with local institutions so that they can exercise control and prevent the entry of activities that cause environmental damage in areas where the Activity is working. The responsible parties are USAID, community councils, educational institutions, and NGOs. This is a short-term measure (six months). RECOMMENDATION 28 SUPPORT THE ALIGNMENT OF PLANTATIONS ON PRIVATE LANDS TO EXERCISE BETTER CONTROL OF THE PLANTED AREAS AND NOT ALLOW THE ACTIONS PROMOTED BY ACTIVITIES SUCH AS OL TO BE LOST THROUGH DEGRADATION AS NEW PRESSURE FACTORS ENTER THE ECOSYSTEM. Implementation measures: i) Review alignment of the properties that are managed, together with the community councils (continuity of social cartography); ii) Support the communities and property owners in aligning their approaches. It could be possible to promote the use of living fences. The responsible parties include autonomous regional corporations, NGOs, IGAC, and USAID. This is a medium-term measure (up to two years). RECOMMENDATION 29 IMPLEMENT PLANS TO CONNECT MAYORS’ OFFICES, AUTONOMOUS REGIONAL CORPORATIONS, AND IIAP TO ENSURE CONTINUITY OF THE ACTIVITIES CARRIED OUT BY THESE INSTITUTIONS (E.G., KNOWLEDGE TRANSFER). Implementation measures: i) Conclude agreements with local institutions and the community councils to support the maintenance of nurseries; ii) Incorporate donations of seedlings from local institutions; iii) Advance entrepreneurial agricultural activities with local populations to sustainably exploit the wood and non-wood products of rehabilitated areas. Beneficiaries should also receive training on how to make use of these resources (with the environmental and agricultural authorities) and on how to prepare carbon credit projects; iv) Incorporate studies to determine the viability of applying to the GOC’s incentive program for planting and to design a financing strategy for verification of greenhouse gas reduction. These exercises would allow the program to set expectations within the community, based on the viability of specific courses of action; v) Establish partnerships with businesspeople who are applying for carbon tax exemptions. The responsible parties are USAID, MADS, MADR, autonomous regional corporations, IIAP, and the business community. This is a medium-term measure (up to two years). 6.4 THE MOST EFFECTIVE COMBINATION OF INSTRUMENTS AND APPROACHES TO REDUCE THE USE OF MERCURY IN ASGM RECOMMENDATION 30 CONTINUE TO DELIVER TRAINING TO MINERS ON ISSUES RELATED TO THE PROHIBITION ON MERCURY. 85 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV Implementation measures: i) Consult with local communities on the most appropriate methods of communication to generate impact with miners; ii) Train personnel in the targeted areas to replicate the trainings for ASGM miners; iii) Coordinate with municipal mining secretariats to prepare a training program on mining and environmental regulations. The responsible parties are USAID, NGOs, local mining organizations, and mayors’ offices. This is a short-term measure (six months). RECOMMENDATION 31 INCLUDE IN THE DESIGN OF ACTIVITIES LIKE OL A COMPONENT SUPPORTING GOVERNMENT ENTITIES TO ADVANCE GEOLOGICAL STUDIES TO LOOK FOR MINERAL DEPOSITS. Implementation measures: i) Carry out geological investigations of mineral deposits to assist ASGM beneficiaries in preparing their PTOs. The responsible parties are USAID and the Colombian Geological Service. This is a medium-term measure (up to two years). RECOMMENDATION 32 CARRY OUT A TECHNOLOGICAL RECONVERSION PROGRAM FOR ASGM MINERS WHO HAVE RECEIVED TRAINING AND KNOW HOW TO MAKE USE OF THE NEW PRACTICES. Implementation measures: i) Develop financing strategies for reconversion of technology for small miners; ii) Promote support for reconversion for small miners among businesses and NGOs; iii) Carry out geological investigations to search for mineral deposits, supporting ASGM beneficiaries in preparing their PTOs. The responsible parties are USAID, MME, mayors’ offices, mining businesses, and NGOs. This is a medium-term measure (up to two years). RECOMMENDATION 33 INCREASE MONITORING OF MERCURY USE IN THE TARGETED AREAS. Implementation measures: i) Formulate a strategy for financing visits by mining and environmental authorities to carry out the corresponding controls and monitoring; ii) In activities like OL, support implementation of the sectoral environmental action plan, increasing air and water quality studies. The responsible parties are local organizations, mayors’ offices, autonomous regional corporations, and governors’ offices. This is a short-term (six months) and medium-term measure (up to two years). RECOMMENDATION 34 CARRY OUT CAMPAIGNS TO RAISE AWARENESS OF THE CONSEQUENCES OF THE USE OF MERCURY AND EXPAND THESE TO ENCOMPASS OTHER MINING-RELATED CONTAMINANTS, SUCH AS CYANIDE. Implementation measures: i) Broaden the information included in awareness-raising campaigns on the contaminants used in gold processing, drawing on research carried out in academic institutions; ii) Work with relevant NGOs to develop diverse types of advertisements to make clear the impacts of contaminants in mining activities. The responsible parties are USAID, academic institutions, and NGOs. This is a short-term (six months) and medium-term measure (up to two years). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 86 RECOMMENDATION 35 WORK WITH LOCAL, REGIONAL, AND NATIONAL GOVERNMENT ENTITIES TO DEVELOP A STRATEGY TO COMMERCIALIZE AND MARKET MERCURY-FREE GOLD. Implementation measures: i) Activities like OL should include a strategy to sell gold to international buyers, specialized jewelry companies, and ethical buyers. The responsible parties are USAID, MME, MADS, MCIT, and governors’ offices. This is a short-term (six months) and medium-term measure (up to two years). RECOMMENDATION 36 CONTINUE TO SUPPORT THE FORMALIZATION SUBCONTRACTS AND MONITOR THEIR CONDITIONS TO PROMOTE A FAIR DISTRIBUTION OF THE BENEFITS OF GOLD MINING ACTIVITY. Implementation measures: i) Support current OL beneficiaries to advance in signing subcontracts with large mining companies; ii) Develop interest on the part of large mining enterprises to work with small miners; iii) Work with the mining authorities to develop incentives for mining enterprises to contribute to processes of mercury-free gold processing in ASGM; iv) include a strategy to sell gold to international buyers, specialized jewelry companies, and ethical buyers. The responsible parties are USAID, mining enterprises, and MME. This is a short-term measure (six months). 6.5 DIRECT AND INDIRECT ACTORS’ PERCEPTIONS OF OL’S RELEVANCE AND EFFECTIVENESS RECOMMENDATION 37 IMPROVE PROCESSES OF INFORMATION MANAGEMENT AND DEEPEN DESCRIPTIONS OF BENEFICIARIES’ CONDITIONS TO IMPROVE MONITORING AND MEASUREMENT OF INTERVENTIONS’ EFFECTIVENESS. Implementation measures: i) Define protocols and standards for traceability and management of information, including structuring databases; ii) Define the variables and instruments that characterize different aspects of the intervention, including incomes; iii) Apply instruments that can characterize the beginning, middle, and end of the intervention (baseline, mid-line, and final). The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 38 STRENGTHEN COORDINATION AMONG USAID, THE IMPLEMENTING PARTNER, THE NATIONAL GOVERNMENT, AND LOCAL GOVERNMENT ENTITIES FOR BETTER PLANNING, COMPLEMENTARITY OF WORK, FOLLOW-UP, AND SUSTAINABILITY OF INTERVENTIONS. Implementation measures: In any possible continuation of OL, i) Establish a coordinating body with the participation of USAID, the implementing partner, MME, ANM, MADS, MADR, and local government entities to define a framework for the interventions, establish lines of complementarity and concurrency, define a support plan and carry out monitoring activities with periodic reports (including results indicators); ii) implement knowledge transfer sessions between the implementing partner and 87 | ORO LEGAL EVALUATION FINAL REPORT USAID.GOV government officials with the aim of improving knowledge of realities in the targeted areas and discussing opportunities to improve the procedures or support defined by the authorities. The responsible parties are USAID, the implementing partner, and national and local government entities. This is a short-term measure (six months). RECOMMENDATION 39 STRENGTHEN TRAINING COMPONENTS AND ACTIVITIES THAT PROMOTE IMPROVED ENVIRONMENTAL CONDITIONS IN THE ACTIVITY. Implementation measures: i) Broaden the scope of training in best environmental practices for activities like OL (waste management, recycling, careful use of water, alternative energy, multimodal transport, etc.); ii) Strengthen partnerships with local and departmental institutions (regional autonomous corporations, mayors’ offices, NGOs, and private groups) and seek to coordinate investments. The responsible parties are USAID, the implementing partner, and national and local government entities. This is a medium-term measure (up to two years). RECOMMENDATION 40 CONTINUE AND INCREASE THE DISSEMINATION OF INFORMATION ON THE BENEFITS AND CHANGES IN QUALITY OF LIFE ASSOCIATED WITH LEGAL MINING AND THE VALUE CHAIN INITIATIVES. Implementation measures: i) Design and implement strategies and instruments for the dissemination of benefits through community associations and community councils (using visual and didactic materials and participatory meetings); ii) Strengthen pedagogical activities and activities that explain the benefits of OL-type interventions, drawing on testimonies and the experience of participants; iii) Develop strategies for sharing information on benefits and incentives targeted for partners in the interventions. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 41 ENSURE CONTINUITY IN THE HIRING OF TECHNICAL PERSONNEL FROM THE TARGETED AREAS TO MITIGATE SECURITY ISSUES. MOREOVER, INCLUDE IN THE TRAINING AND IN THE SPACES OF THE INTERVENTION DISCUSSIONS OF PROTOCOLS FOR SELF-CARE FOR VULNERABLE INDIVIDUALS AND COMMUNITIES. Implementation measures: i) Hire local people to facilitate access to the targeted areas and instill greater trust to the beneficiaries; ii) Identify the protocols, manuals, and booklets defined by the National Protection Unit (e.g., the Manual for Self-Security and Self-Protection); iii) Identify relevant socio-emotional tools, designed by different entities and NGOs, that local people who are promoting reconciliation can use to develop leadership abilities and promote peace; iv) Disseminate this material through the program to the relevant beneficiaries. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 42 BROADEN THE GENDER FOCUS TO INCREASE THE PARTICIPATION OF WOMEN, WHICH SHOWED GOOD RESULTS IN OL. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT | 88 Implementation measures: i) Include a minimum percentage of women participants in the selection criteria for beneficiaries; ii) Continue initiatives that strengthen women’s empowerment and disseminate information on the benefits of women’s participation in the Activity; iii) Create spaces where women beneficiaries can share and exchange experiences; iv) In discussions with partners, emphasize the importance and benefits of participating in initiatives that also include women’s participation. The responsible parties are USAID and the implementing partner. This is a short-term measure (six months). RECOMMENDATION 43 REFLECT ON THE DEFINITION OF GENERATIONAL RENEWAL STRATEGIES IN VALUE CHAIN INTERVENTIONS TO INCREASE YOUNG PEOPLE’S DESIRE TO STAY IN TARGETED AREAS, PREVENT THEIR INVOLVEMENT IN ILLICIT ACTIVITIES, AND CONTRIBUTE TO THE SUSTAINABILITY OF THE ACTIVITY. Implementation measures: i) Define a strategy to raise awareness of the importance and benefits of generational renewal in the value chains; ii) Include in the value chain initiatives a component for knowledge transfer from adults to young people in the community (e.g., from fathers to sons); iii) Create partnerships with education secretariats and educational institutions through social service programs to promote training and participation of young people in the value chains. The responsible parties are USAID, the implementing partner, local governments, and educational institutions. This is a medium-term measure (up to two years). 89 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ANNEX 1: METHDOLOGICAL DESIGN APPLIED TO ANSWER THE EVALUATION QUESTIONS In this section, we briefly present how we approached the development of the methodological design to answer the six evaluation questions. In the first part, we summarize the process for the consolidation of the design that we implemented. In the second, we point out the different topics (sub-questions) we analyzed and triangulated the information. Finally, we present the connection between the findings we produced and the subjects (sub-questions) to broadly answer the six evaluation questions. PROCESS FOR THE CONSOLIDATION OF THE EVALUATION DESIGN We took the six evaluation questions established in the SoW and their respective context paragraphs and guidelines as a starting point to carry out the evaluation. To consolidate the first deliverable of the evaluation process – evaluation design and work plan -, we concentrated on understanding the questions considering the operational and conceptual elements of Oro Legal (OL). We made a first documentary review focused on section C of the contract, annual reports (2016-2019), and annual work plans (2015- 2019). This first documentary review allowed us to identify the main lines of work, the type of initiatives and strategies used by Oro Legal to reach the territories, and how Oro Legal linked with their beneficiaries. Because of this first documentary review, a series of questions of a conceptual and operational nature arose about elements that were not explicit within the revised documentation and that it was necessary to clarify to design the methodological approach. To answer these questions of a conceptual and operational nature, we asked MEL Activity, USAID, and Oro Legal the need to have thematic meetings with the Activity's management team and their regional leaders to respond to the concerns that arose. Given that this evaluation planning moment coincided with the last months of OL's operation, characterized by many administrative tasks for the implementing partner, we received the proposal to hold a single workshop with the entire implementing partner's team. This workshop had parallel thematic sections led by the evaluation team. We sought to collect information that would allow us to clear up operational and conceptual doubts and approximate the six evaluation questions from a perspective consistent with what had been the conceptualization and implementation of the Activity. With the different inputs of the preliminary documentary review, as well as the material derived from the workshop with the implementing partner, we built the design matrix80 that identified for each evaluation question: (1) the main performance indicators, (2) the key actors at the national, departmental and municipal levels, (3) the existing information sources (documentary, monitoring and administrative data of public access) and other primary sources that we explored in the framework of this evaluation. (4) For the latter, we built qualitative collection instruments (interview guides) and quantitative ones (survey). In a follow-up meeting, USAID requested us, based on the learning from the mid-term evaluation of Oro Legal, to survey only direct beneficiaries of the Activity (individuals identified as such by the official databases provided by OL) and avoid collecting information with indirect beneficiaries. In line with this strategy, all data collection instruments (see Annex VI) had an initial pre￾completion (survey) section (or knowledge and relationship with the Activity (interviews)) to ensure that the interviewee/respondent knew the Activity and, therefore, could provide accurate information 80 The design matrix in Annex V contains the details of the elements referenced here. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 90 about their conceptualization of OL's implementation. On the other hand, the instruments included specific sections to each component of OL covered by the evaluation questions. In addition, we designed the interview guides and the survey to guarantee that the responses of both approaches could be spun in the most straightforward way to facilitate triangulation and the production of robust evidence around the findings. (5) Finally, we included some proposals for an analytical approach to the information collected by each type of source, drafted in a generic way to allow for emerging data with the expectation of the quality and richness of the information that would become evident after its collection. EVALUATION QUESTIONS, TOPICS (SUB-QUESTIONS) AND FINDINGS To build a set of valuable findings, conclusions, and recommendations for USAID and the Implementing Partner, during the design phase, we identified within each of the six evaluation questions and their respective context and guidelines paragraphs, a series of topics (sub-questions) around which we organized the analysis and triangulation of information. To answer each of the six evaluation questions, we focused on providing findings conceived around each topic (sub-question) and thus covered the general question's breadth. The topics (sub-questions) that we identified around each of the six evaluation questions are detailed below: Mining Formalization: G. Formalization standards H. Access to legal markets I. Incentives to remain legal J. Incentives to improve the environmental performance of mining operations K. Behavior change L. Factors preventing the uptake of formalization Mining Governance and Policy: G. Factors contributing to the strengthening of Colombian mining and environmental governance H. Factors hindering the strengthening of Colombian mining and environmental governance Value Chain Development and Alternative Livelihoods H. Perception of impact in economic outcomes (financial and non-financial) I. Perception of impact in non-economic factors (environment, recognition, security, and pressure from illegal armed groups) Rehabilitation of Areas Degraded by Illegal Mining: H. Appropriateness and effectiveness of the differentiated departmental approach I. Cost-effectiveness J. Progress made K. Local commitment to maintenance and monitoring Elimination/Reduction of Mercury from the ASGM Supply Chain: I. How was it done, and what were the motivations J. Impact of instruments combinations 91 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV K. External factors contributing L. External factors hindering Perceived Relevance and Impact of the Activity: B. Main perceptions from different stakeholders Rehabilitation of Areas Degraded by Illegal Mining: F. Appropriateness and effectiveness of the differentiated departmental approach G. Cost-effectiveness H. Progress made I. Local commitment to maintenance and monitoring Elimination/Reduction of Mercury from the ASGM Supply Chain: I. How was it done, and what were the motivations J. Impact of instruments combinations K. External factors contributing L. External factors hindering Perceived Relevance and Impact of the Activity: D. Main perceptions from different stakeholders E. The findings included in the final report are related to at least one of the topics (Sub-questions), covering all the question's edges to provide answers that cover the evaluation questions' breadth and complexity. The tables below present the connection between the findings and different topics (Sub-questions)81: 81 Since question six focuses on the perceptions of the different stakeholders, the answer takes elements developed in the five previous questions. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 92 TABLE 6: MINING FORMALIZATION TOPICS (SUB-QUESTIONS) AND FINDINGS A. FORMALIZATION STANDARDS B. ACCESS TO LEGAL MARKETS C. INCENTIVES TO REMAIN LEGAL D. INCENTIVES TO IMPROVE THE ENVIRONMENTAL PERFORMANCE OF MINING OPERATIONS E. BEHAVIOR CHANGE F. FACTORS PREVENTING THE UPTAKE OF FORMALIZATION Since 2014 the Ministry of Mines and Energy (MME) has promoted a shift in mindset about legalization/formalization of mining operations, aiming to modernize the ASGM sector X In line with this vision of mining formalization, OL contacted more than 640 MPUs; of these, it provided support to 334 and formalized 146 (53 in Antioquia and 93 in Chocó) X X X X OL helped to strengthen miners' desire to remain legal/formal and maintain environmental performance through dignifying ASGM X X OL showed beneficiaries the benefits of formalization while helping them to overcome the multiple disincentives to formalization X X The good business practices promoted by OL enabled advances in workers' rights and more ethical practices. X Beneficiaries that obtained their mining title and their environmental license improved their earnings X X X OL's logistical and management support helped push forward the approach of working with groups of miners around formalization subcontracts and special reserve areas (ARE). X X 93 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 6: MINING FORMALIZATION TOPICS (SUB-QUESTIONS) AND FINDINGS A. FORMALIZATION STANDARDS B. ACCESS TO LEGAL MARKETS C. INCENTIVES TO REMAIN LEGAL D. INCENTIVES TO IMPROVE THE ENVIRONMENTAL PERFORMANCE OF MINING OPERATIONS E. BEHAVIOR CHANGE F. FACTORS PREVENTING THE UPTAKE OF FORMALIZATION Contracting local residents of the zone in which an intervention was carried out facilitated relationship-building and knowledge transfer between OL and mining communities, and contributed to disseminating knowledge of regulations and functions of the institutions in the territory X The greatest recognition of OL's impact comes from miners. X Nevertheless, in some cases the obligations imposed by the formalization subcontracts could seem to be an obstacle for the formalization process X Beneficiaries benefitted from the institutional relationships constructed by OL X The main incentive for miners to formalize their operations, and to continue to operate in a formal manner, was to obtain their title and their environmental license X X X OL support to miners to obtain the mining title and environmental license is evident, but regulatory and procedural complexity reduces the likelihood of comprehensive formalization X X While miners are obliged to meet strict deadlines, the authorities fail to comply with the requirements established by law X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 94 TABLE 6: MINING FORMALIZATION TOPICS (SUB-QUESTIONS) AND FINDINGS A. FORMALIZATION STANDARDS B. ACCESS TO LEGAL MARKETS C. INCENTIVES TO REMAIN LEGAL D. INCENTIVES TO IMPROVE THE ENVIRONMENTAL PERFORMANCE OF MINING OPERATIONS E. BEHAVIOR CHANGE F. FACTORS PREVENTING THE UPTAKE OF FORMALIZATION The costs immediately demanded to meet the technical standards for formalization work against the formalization process X X The evaluation did not find sufficient evidence on the effectiveness of memoranda of understanding (MOUs) to gauge their contribution to the program objectives X Knowledge transfer was made more difficult by the level of education of the miners. X X In addition to these problems, government officials tended to have an image of miners and of ASGM that was not based in reality X The MPUs and local actors are unaware of, or have outdated information about, mining regulations X X Officials are unaware of or misinterpret regulations, to the disadvantage of the small miners, while authorities are unaware of the reality and particularities of ASGM X Areas that are protected by law, including forest reserve areas covered by Law 2 of 1959, are subject to restrictions on formalization, and it is highly unlikely for miners to obtain an exemption from these protections X X A greater emphasis by OL on gold commercialization could have improved the prospects for beneficiaries continuing to be formal/legal X 95 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 6: MINING FORMALIZATION TOPICS (SUB-QUESTIONS) AND FINDINGS A. FORMALIZATION STANDARDS B. ACCESS TO LEGAL MARKETS C. INCENTIVES TO REMAIN LEGAL D. INCENTIVES TO IMPROVE THE ENVIRONMENTAL PERFORMANCE OF MINING OPERATIONS E. BEHAVIOR CHANGE F. FACTORS PREVENTING THE UPTAKE OF FORMALIZATION Licit incomes only improve, however, once miners have their mining title and environmental license; without these instruments, sales are not legal X X X X For miners to continue to seek formalization, they will have to see that economic benefits materialize X The trainings offered by OL improved compliance with environmental regulations and the systematic management of environmental impacts X X OL contributed to notable improvements in environmental performance through the impact studies required for the environmental license X The formalization promoted by a program like OL still faces disadvantages in competing with illegality X The future of OL's beneficiaries is highly uncertain, as they did not foresee the conditions under which the Activity closed X Short-lived projects create skepticism and instability in miners X The structural modification of traditional gold exploitation requires longer-term support X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 96 TABLE 7: MINING GOVERNANCE AND POLICY: TOPICS (SUB-QUESTIONS) AND FINDINGS A. FACTORS CONTRIBUTING TO THE STRENGTHENING OF COLOMBIAN MINING AND ENVIRONMENTAL GOVERNANCE B. FACTORS HINDERING THE STRENGTHENING OF COLOMBIAN MINING AND ENVIRONMENTAL GOVERNANCE OL provided support on regulatory and governance issues in the mining sector X OL's work to link miners, local communities, and mining and environmental authorities is one of its main contributions to mining governance X OL served as an effective bridge between small-scale miners and authorities X OL constructed synergistic relationships with beneficiaries and with authorities in the mining and environmental sectors X Beneficiaries benefitted from the institutional relationships constructed by OL X Program beneficiaries valued greatly the recognition as legitimate actors in the mining industry that they received through OL X Without the commitment of all the authorities involved, mining informality will continue to prevail X The more structural weaknesses identified in the formalization process were not formally presented to government authorities X Mining formalization is a ministerial initiative, not a national policy, and it does not have the force of law X Mining formalization is optional since a beneficiary can withdraw at any time X Officials are unaware of or misinterpret regulations, to the disadvantage of the small miners, while authorities are unaware of the reality and particularities of ASGM X Lack of coordination between training entities was evident, which generated redundancies in services and training contents X We found low coordination between the entities related to formalization X 97 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 8: VALUE CHAIN DEVELOPMENT AND ALTERNATIVE LIVELIHOODS: TOPICS (SUB-QUESTIONS) AND FINDINGS A. PERCEPTION OF IMPACT IN ECONOMIC OUTCOMES (FINANCIAL AND NON- FINANCIAL) B. PERCEPTION OF IMPACT IN NON- ECONOMIC FACTORS (ENVIRONMENT, RECOGNITION, SECURITY, AND PRESSURE FROM ILLEGAL ARMED GROUPS) OL promoted the expansion and strengthening of productive bases for beekeeping in the Bajo Cauca region of Antioquia. X OL helped develop the value chain for annatto in Chocó X Beneficiaries' perceptions of the two value chains differed based on two associated factors: the incomes that they gained, and the prospects for commercialization of their products X Income from beekeeping and annatto complemented household incomes, and if consolidated as value chains, would become more attractive and viable for beneficiaries X OL strengthened the capacities of the beneficiaries, increased their productive potential, and improved yields and quality thanks to the investments and technical support provided by the Activity X OL established broad productive bases of beekeeping and annatto that could be the foundation for the consolidation of productive conglomerates in the targeted areas X OL developed local technical skills and capacity. However, the proposed scheme does not ensure the continuity of technical assistance to producers X Beekeepers recognized the improvement in their incomes thanks to the increase in their productive capacity and saw prospects for future growth in this area X In beekeeping, the consolidation of an agro-chain within the territory was not promoted, nor was a unified strategy with a socio-business approach to support it. X The opportunity to form a regional commercial alliance as a regional link in the value chain did not arise, and because this was included only at the end of the Activity this chain was not consolidated X Starting with a traditional crop, OL identified a variety of annatto suitable for an improved production system in Chocó X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 98 OL established the foundations of an agro-industrial value chain for annatto in Chocó. However, some critical issues remain to be resolved, which could pose serious risks for the sustainability of this achievement X Producers of annatto felt dissatisfied with the incomes they received, and disappointed that the initial promise of a guaranteed market for their products was not kept X For beneficiaries involved in beekeeping, the fact that the associations and the regional commercial partner do not pay cash on delivery for the honey is an issue, and leads to beneficiaries selling a portion of their production to intermediaries and opportunistic traders X The absence of comprehensive business models capable of accounting for the impacts of investments led to overproduction of annatto and a shortage of capital in the beekeeping component X OL had positive effects for beneficiaries' quality of life in both the beekeeping and the annatto components X Female participation was significant in the first productive step of both value chains, which generated positive effects for their autonomy, recognition, and belonging in the family and community X The annatto component as a regional agro-industrial chain is generating recognition and self-recognition of the ability of the inhabitants of Chocó to propel their own territorial development X Although the security situation continues to threaten value chain beneficiaries, having viable alternatives for income generation strengthens the household resilience by reducing economic vulnerability, strengthening social networks at the family and community level X X The alternative activities were perceived as offering a way for people to spend their time that helped reduce the risk of recruitment by armed actors, especially for young people X Beneficiaries and other actors perceived the presence of armed groups as the most significant threat to the further development of value chains X OL has improved environmental quality through the value chain approach and avoided greater environmental degradation, reducing illegal extraction in the areas targeted by the intervention X Interdependence between beekeeping and the rehabilitation model using Acacia Mangium strengthened the sustainability of both components X X 99 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 9: REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING: TOPICS (SUB-QUESTIONS) AND FINDINGS A. APPROPRIATENESS AND EFFECTIVENESS OF DIFFERENTIATED DEPARTMENTAL APPROACH B. COST EFFECTIVENESS C. PROGRESS MADE D. LOCAL COMMITMENT TO MAINTENANCE AND MONITORING The areas in which OL was active are areas of considerable environmental importance, meaning that rehabilitation efforts in these areas contributed to achieving important national and international environmental goals X The Chocó bioregion is considered a biodiversity hotspot because of its high number of endemic species and the presence of pristine forest. X Bajo Cauca, Antioquia, is 80 percent covered by tropical humid forest and premontane forest ecosystems, and the zone is rich in surface and underground water X The areas where OL worked are zones of high importance from an environmental standpoint X As a result of mining extraction activities, the two departments have been affected by biodiversity loss, to the detriment of resources such as soil and water. X The areas of work by OL overlapped with areas of medium to high vulnerability X Orphan mining environmental liabilities are systems with anthropogenic disturbances where the production of ecosystem services is compromised X As a consequence, in Chocó, there are severe environmental impacts; different species are threatened and extreme weather events have had impacts on public infrastructure and housing X In Antioquia, OL beneficiaries recognized mining's impact by changes in the landscape and effects on local flora and fauna X OL beneficiaries recognized the need to reverse environmental damages and valued the recovery of the environment thanks to the implementation of this component X X OL filled funding gaps, in the absence of public investment in rehabilitation, to manage orphaned environmental liabilities in priority areas due to the impacts of these liabilities. X X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 100 TABLE 9: REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING: TOPICS (SUB-QUESTIONS) AND FINDINGS A. APPROPRIATENESS AND EFFECTIVENESS OF DIFFERENTIATED DEPARTMENTAL APPROACH B. COST EFFECTIVENESS C. PROGRESS MADE D. LOCAL COMMITMENT TO MAINTENANCE AND MONITORING With an estimated cost of $1,700/Ha, the OL model has lower estimated average costs than similar programs X OL's model allowed the program to manage an ambitious extension of the available resources X X Acacia mangium has long been used and recognized in the department of Antioquia. X Nevertheless, there is also evidence of risks related to the use of acacia mangium X The model of active rehabilitation with native species used in Chocó used seeds and seedlings from neighboring ecosystems. X X Beneficiaries considered OL's model to be appropriate X OL recognized the community councils as holding authority, recognized the legality of collective land tenure, and drew on ancestral local knowledge to support rehabilitation. X X In Bajo Cauca OL selected properties that had legal tenure and prepared contracts for forest use X X To analyze changes in ground cover and deforestation, we drew on satellite images of a sample of the area targeted by OL, made up of 2,077 Ha (represented by 13 polygons) under active rehabilitation. This analysis found a four percent increase in vegetation cover and a 14 percent decrease in deforestation in the sample X The variation in vegetation coverage found in the sample from Antioquia was not homogenous. X 101 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 9: REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING: TOPICS (SUB-QUESTIONS) AND FINDINGS A. APPROPRIATENESS AND EFFECTIVENESS OF DIFFERENTIATED DEPARTMENTAL APPROACH B. COST EFFECTIVENESS C. PROGRESS MADE D. LOCAL COMMITMENT TO MAINTENANCE AND MONITORING In the case of active rehabilitation in Chocó, the sample showed a nine percent increase in vegetation cover and a 35 percent decrease in degraded lands X Beneficiaries perceived rehabilitation component results positively X Local actors perceived positive results from the Activity, such as improvements in the microclimate, increased biodiversity, and soil recovery. X The Activity could have implemented a strategy so that local communities have the resources to leverage continuity of OL's work X The Community Councils do not have the ability to cover the costs of continued care, monitoring, and control of the managed areas X Local institutions were not sufficiently involved in supporting rehabilitation activities X The opportunity to promote conservation financial incentive projects was not considered X X Finally, OL did not promote sustainable exploitation of the acacia plantings. X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 102 TABLE 10: ELIMINATION/REDUCTION OF MERCURY FROM THE ASGM SUPPLY CHAIN: TOPICS (SUB-QUESTIONS) AND FINDINGS A. HOW WAS DONE AND WHAT WERE THE MOTIVATIONS B. IMPACT OF INSTRUMENTS COMBINATION C. EXTERNAL FACTORS CONTRIBUTING D. EXTERNAL FACTORS HINDERING OL's support for small-scale mining included modifications in gold mining techniques to eliminate the use of mercury X The GOC adopted Law 1658 on July 15, 2013, with the goal of protecting human health and preserving renewable natural resources and the environment X OL provided technical support for the preparation of Work and Construction Plans (PTO), as well as Environmental Impact Assessments (EIA) with the MPUs. X OL provided important technical assistance to small miners, who otherwise would not have been able to prepare these instruments (at least not with the required quality, nor with effective guidance for mercury-free practices). X The trainings provided by OL for MPUs on formalization achieved their aims and small miners are aware of the current regulations regarding mercury. X Beneficiaries who were subcontractors with large mining businesses were aware that they could not use mercury X Training and, above all, support to access new techniques are key elements for beneficiaries in eliminating the use of mercury X 103 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 10: ELIMINATION/REDUCTION OF MERCURY FROM THE ASGM SUPPLY CHAIN: TOPICS (SUB-QUESTIONS) AND FINDINGS A. HOW WAS DONE AND WHAT WERE THE MOTIVATIONS B. IMPACT OF INSTRUMENTS COMBINATION C. EXTERNAL FACTORS CONTRIBUTING D. EXTERNAL FACTORS HINDERING Awareness campaigns that highlight the impacts on human health and the environment have had good results X OL's activities contributed to increased compliance with the regulation and have achieved a reduction in the use of mercury. X While mercury use has decreased, it has not been possible to completely eradicate it. X According to Law 1658, the state had five years to prepare the mining sector for mercury eradication, but it began the required work late X Assuming that a small percentage of ASGM miners do continue to use mercury, the challenge lies in controlling its trafficking and access to it X The links formed with large mining companies through formalization subcontracts was an effective way to control the use of mercury X USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 104 ANNEX II: QUANTITATIVE DATA COLLECTION METHODS AND CHALLENGES In the quantitative data collection, were performed 670 surveys. The participants answered survey questions as follows: 120 on mining, 185 on rehabilitation, 183 on annatto, and 182 on beekeeping (See Exhibit 28). Component Antioquia Chocó Total Mining 26 94 120 Beekeeping 182 0 182 Annatto 0 183 183 Rehabilitation 177 8 185 Exhibit 30: Total Surveys Performed SOURCE: DATA PROVIDED BY ORO LEGAL, 2020 SAMPLE DESIGN A. TARGET POPULATION The target population belongs to three groups: 1. Miners participating in mining formalization activities. 2. Individuals working in agricultural value chains, specifically the beekeeping and annatto chains. 3. Individuals participating in rehabilitation activities of areas degraded by mining activities. According to the data provided by Oro Legal, there were a total of 1,247 direct beneficiaries in 21 municipalities of Antioquia and Chocó. Direct beneficiaries corresponded to the following categories in the departments of Antioquia and Chocó, as follows: Component Antioquia Chocó Total Mining 81 86 167 Beekeeping 345 0 345 Annatto 0 320 320 Rehabilitation 412 3 415 Exhibit 31: Number of Beneficiaries per Category According to Department SOURCE: DATA PROVIDED BY ORO LEGAL 105 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV B. SAMPLING FRAME The sampling frame was one of the most relevant components in need of defining82. The following sampling frames were used for this research: • Miners participating in mining formalization activities: This database of direct mining beneficiaries has 167 participants. • Individuals participating in beekeeping and annatto activities: These databases are comprised of 345 direct beneficiaries of annatto and 320 direct beneficiaries of beekeeping. • Individuals participating in land rehabilitation activities of areas degraded by mining activities: This database has 415 direct beneficiaries: 412 from Antioquia and three from Chocó. We selected survey participants from all the direct beneficiaries registered in the database in each category to guarantee probabilistic sampling. Additionally, through associations, community councils, local organizations, mining projects, and other groups, we obtained the contact information of selected direct beneficiaries whose contact information was unavailable. Of the 1263 beneficiaries in the direct beneficiary’s database, only 667 beneficiaries had phone numbers; we obtained the contact information of 92 beneficiaries; thus, a total of 759 registered beneficiaries had contact phone numbers. In each category, a simple random sampling was implemented as follows: 𝑛𝑛ℎ = 𝑛𝑛0 1 + 𝑛𝑛0 𝑁𝑁ℎ Where 𝑛𝑛0 = 𝑧𝑧 1−𝛼𝛼 2 2 ∗ 𝑆𝑆ℎ 2 𝑒𝑒2 𝑦𝑦 𝑆𝑆ℎ 2 = 𝑃𝑃ℎ(1 − 𝑃𝑃ℎ) Value 𝑁𝑁ℎ is the number of direct beneficiaries in category h. For sample size calculation purposes, a value of 𝑃𝑃ℎ = 0.5 was considered to ensure that the sample size enabled the estimation of any proportion within the precision parameters. Value 𝑒𝑒 corresponds to the maximum margin of error, set at 5% for each component. The value of 𝑧𝑧1−𝛼𝛼/2 is the percentile (1 − 𝛼𝛼/2)100% in a normal distribution and corresponds to the value 1.96 to reach a 95% confidence level. Moreover, taking the background experience of this type of data collection into consideration, oversampling of 30% was set to mitigate the risks due to the lack of survey responses. To ensure dispersion the sample we proposed, included observations from all the municipalities where the program was present. We created a proportional allocation for each group of interest to distribute the sample in all municipalities. 82 Stukel, D., & Friedman, G. (2016). Sampling Guide for Beneficiary-Based Surveys in Support of Data Collection for Selected Feed the Future Agricultural Annual Monitoring Indicators. Washington, DC: Food and Nutrition Technical Assistance Project, FHI 360, page 23. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 106 We used the randomized real value negative selection algorithm described by Gutiérrez83 to select the beneficiaries sampling in each component. With 𝑁𝑁𝑐𝑐ℎ the number of beneficiaries 𝑐𝑐 in the municipality ℎ, to select a simple random sample of 𝑛𝑛𝑐𝑐ℎ, the following steps of the algorithm were followed: 3. Generate 𝑁𝑁𝑐𝑐ℎ realizations of a random variable with uniform distribution 𝜁𝜁𝑘𝑘 ∼ 𝑈𝑈(0,1): 𝜁𝜁1, … , 𝜁𝜁𝑁𝑁𝑐𝑐ℎ. 4. Assign 𝜁𝜁𝑘𝑘 to the Kth element of the population of component c in the municipality ℎ. 5. Sort elements in ascending order with respect to 𝜁𝜁𝑘𝑘. 6. Mark the order of eligibility from number 1 to 𝑁𝑁𝑐𝑐ℎ. 7. Select the first 𝑛𝑛𝑐𝑐ℎ elements according to the sample size in the municipality and component. The above selection algorithm allowed the marking of eligibility order of beneficiaries, so it is possible to have a replacement mechanism that does not affect the initial randomization when the sample and the oversample are not sufficient to achieve the minimum sample. MINING FORMALIZATION For this component we proposed a minimum sample of 117 beneficiaries to have a maximum error margin of 5% for estimates with proportions close to 50%. According to the oversample of 30%, 152 beneficiaries were selected to reach the minimum sample. The sample distribution by municipality including the oversample, is presented below. Direct Beneficiaries of Mining Formalization Sample Size Allocation Department Municipality Direct Beneficiaries Antioquia Barbosa 3 3 Buriticá 10 9 Cáceres 23 21 Don Matías 5 5 Remedios 11 10 San Roque 11 10 Segovia 1 1 Tarazá 16 15 Zaragoza 1 1 83 Gutiérrez, A. (2016). Estrategias de muestreo: diseño de encuestas y estimación de parámetros. Bogotá: Universidad Santo Tomás. Page 61 107 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Direct Beneficiaries of Mining Formalization Sample Size Allocation Department Municipality Direct Beneficiaries Chocó Certegui 11 10 Condoto 5 5 Tadó 41 36 Unión Panamericana 29 26 Total General 167 152 Exhibit 32: Distribution Sample with Oversample for the Mining Component SOURCE: OWN CALCULATIONS BASED ON DATA PROVIDED BY ORO LEGAL BEEKEEPING AND ANNATTO To guarantee a margin of error of less than 5% with a confidence level of 95%, we proposed a selection of 237 beneficiaries in annatto and 228 beneficiaries in beekeeping to achieve the minimum sample. Department Municipality Direct Beneficiaries of Annato Sample Allocation Chocó Atrato (Yuto) 111 76 Cantón San Pablo 74 51 Certegui 2 1 Quibdó 20 14 Rio Quito 127 87 Unión Panamericana 11 8 Total 345 237 Exhibit 33: Distribution of the Sample with Oversample for Annatto SOURCE: OWN CALCULATIONS BASED ON DATA PROVIDED BY ORO LEGAL Department Municipality Direct Beneficiaries of Beekeeping Sample Allocation USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 108 Antioquia Caucasia 103 73 El Bagre 72 52 Nechí 57 41 Tarazá 29 21 Zaragoza 57 41 Total 318 228 Exhibit 34: Distribution of the Sample with Oversample for Beekeeping SOURCE: OWN CALCULATIONS BASED ON DATA PROVIDED BY ORO LEGAL REHABILITATION The rehabilitation component was comprised of 412 direct beneficiaries in Antioquia and three in Chocó. We proposed a stratified random sampling, where subpopulations corresponded to two departments to achieve a maximum margin of error of 5% and a confidence level of 95%. The sample size with an oversample was comprised of 262 direct beneficiaries. Department Municipality Direct Beneficiaries of Rehabilitation Sample Allocation of Direct Beneficiaries Antioquia Cáceres 24 15 Caucasia 30 19 El Bagre 315 198 Zaragoza 43 27 Total Antioquia 412 259 Chocó Quibdó 1 1 Rio Quito 2 2 Total Chocó 3 3 Grand Total 415 262 Exhibit 35: Distribution of the Sample with Oversample for Rehabilitation SOURCE: OWN CALCULATIONS BASED ON DATA PROVIDED BY ORO LEGAL 109 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV C. EXPANSION FACTORS AND ADJUSTMENTS FOR NON-RESPONSES In general, surveys require three steps to reach the final weight for each individual. The first step is to calculate the base expansion factors of the sample. The second step is the adjustment for partial or non￾responses, based on the items with no response. The final step is to create additional factor adjustments to correct selection biases created by lack of coverage or responses. The goal is to adjust the resulting weighted sample estimates of the known population values for some key variables. All selected items have an expansion factor that is then adjusted according to the response or coverage rate84. As a simple random sampling, in the case of the different components, the expansion factor of the Kth beneficiary is determined by the inverse of the first-order inclusion probability: 𝑑𝑑0𝑘𝑘 = 1 𝜋𝜋𝑘𝑘 ℎ = 𝑁𝑁𝑐𝑐ℎ 𝑛𝑛𝑐𝑐ℎ Where 𝜋𝜋𝑘𝑘 ℎ is the probability of inclusion of the Kth direct beneficiary of component c within the municipality h, 𝑁𝑁𝑐𝑐ℎ y 𝑛𝑛𝑐𝑐ℎ are the number of direct beneficiaries, and the corresponding sample size in component c in the municipality h. The procedure established was conducted to adjust the expansion factors, where changes or incidences were registered at the end of each data collection85. An adjustment for unknown eligibility followed this 𝑎𝑎1,𝑐𝑐 and an adjustment for non-ignorable non-response 𝑎𝑎2,𝑐𝑐, where subscript 𝑐𝑐 represents the component of interest (mining, land rehabilitation, annatto or beekeeping beneficiaries); thus, the adjusted expansion factor for the surveyed beneficiary Kth of the population or component 𝑐𝑐 was the following: 𝑑𝑑𝑐𝑐 = 𝑎𝑎1,𝑐𝑐𝑎𝑎2,𝑐𝑐𝑑𝑑0k. D. SAMPLE SIZE ADJUSTMENT We identified coverage problems of the sampling frame during the fieldwork; that is, those outside of the target population and those who were not included in the sample frame but stated to have participated in some program components. The former were respondents who were part of the database but stated not to be program beneficiaries. Therefore, they were not included when calculating the expansion factor. Among reasons provided by this group of respondents were the following: • Attended an Oro Legal informational meeting because someone registered them but was not program beneficiaries as they never received any sponsorship. • Performed mining activities but have never been sponsored by the program. 84 Kalton, G., & Flores-Cervantes, I. (2003). Weighting Methods. Journal of official statistics, 19(2), Pages 81--97. 85 Valliant, R., Dever, J., & Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples. Pittsburgh, PA, USA: Springer. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 110 • Participated at the beginning of Oro Legal, but after the tree planting rehabilitation activity, they ceased to continue in the program; their information, however, remained in the database. Additionally, during the face-to-face fieldwork, beneficiaries who did not belong to the sample frame at the beginning of the study stated that they had participated in Oro Legal. Consequently, the inference universe underwent some adjustments. Thus, 35 records that were identified as out-of-universe were eliminated; this resulted in a population decrease from 167 to 132 for the mining component. In this situation, a minimum of 99 surveys were required to achieve a maximum margin of error of 5%, although the total number of conducted surveys was 120.27 surveys participants responded they did not correspond to any category: 21 from Antioquia and six from Chocó. Therefore, we did not use those observations to calculate the expansion factor equation. Universe Size Sample Size Component Initial Final Initial Final Mining 167 132 118 120 Annatto 345 345 183 183 Beekeeping 320 320 176 182 Rehabilitation 415 417 202 185 Total 1247 1214 679 670 Exhibit 36: Sample Size Adjustment SOURCE: DATA PROVIDED BY ORO LEGAL We implemented adjustments for non-responses and sample frame flaws following the methodology proposed by Valliant86. When calculating the design-induced errors after adjusting the base expansion factor, the resulting errors did not exceed 5%. 86 Valliant, R., Dever, J., & Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples. Pittsburgh, PA, USA: Springer 111 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 37: Sampling Error SOURCE: MEL ACTIVITY DATA COLLECTION METHODOLOGY We collected in all components through a mixed collection scheme, starting with telephone calls to those beneficiaries who had a contact number and, simultaneously, with MEL Activity and Oro Legal, we contacted associations, community councils, local organizations, and other groups that could provide contact information. Finally, at the same time, logistics for face-to-face visits for collecting in-person information for the remaining beneficiaries who did not have contact numbers took place. If other beneficiaries selected in the sample remained in the municipality and did not respond to the telephone survey, we approached them in person. Before beginning the fieldwork, we consolidated the study's sampling frame. we trained the enumerators on January 21, 2021, and conducted the pilot test on January 23, 2021. This pilot test made it possible to measure the performance of enumerators, the database quality, and any requirements needed for the fieldwork. Once the pollster training was completed on January 21, the fieldwork began on January 26 and was completed on March 9, 2021. The general steps to perform the fieldwork were the following: 1. First, to conduct surveys via telephone to those beneficiaries with updated data. 2. The logistical coordination team contacted community leaders who helped to assemble beneficiaries at a specific location. Annatto Beekeeping Mining Land Rehabilitation USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 112 3. Appointments of face-to-face visits were scheduled through community leaders in all the cases where beneficiaries could not be contacted via telephone. During the quantitative field operation, 67087 surveys were conducted (102% of the original plan). CHALLENGES A. SAMPLE LOSS FACTORS The problems identified regarding telephone contacts were specifically related to the quality of contact data in the database. Of the 1,263 direct beneficiaries in the database, only 667 included phone number, we obtained data from another 92 contacts; in total, 759 contacts had had numbers. An additional setback experienced during the fieldwork was related to the quality of the contact information. The reasons for unsuccessful contacts were as follows: 58% of the contacts did not answer the call, or the call was sent to voicemail, and 19% corresponded to out-of-service numbers. However, 50% of the answered calls ended in effective surveys. 5% were rejected at the beginning of the survey; 28% corresponded to calls that did not result in contact with the beneficiary; 1% of calls were abandoned; 7% of contacts stated that they did not know the beneficiary; 2% stated that they did not participate in the program, and 1% of the contacts had passed away. In addition, 4% of calls were canceled by field supervision because of inconsistencies in the information. SITUATION QUANTITY PERCENTAGE Successful Interview 164 50% Rejected 16 5% Drop-Out 4 1% Passed Away 1 1% Did not Participate in Program 14 4% Unknown Respondents 24 7% Annulled 14 4% Interview not Started 95 28% TOTAL 332 100% Exhibit 38: List of Calls that Resulted in Contact with Respondents SOURCE: DATA PROVIDED BY ORO LEGAL, 2021 It should be noted that considering these indicators, it was necessary to perform fieldwork to collect as much information as possible and to reach the required sample. For this purpose, community leaders were contacted in the municipalities who aided in assembling the beneficiaries in a specific location to conduct the survey. Additionally, respondent referrals were used. Finally, we searched the Internet for names of some MPUs to contact managers and beneficiaries who currently work there, according to Oro Legal databases. 87 It is important to mention that this data was used to calculate the expansion factor for each component. 113 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV B. DIFFICULTIES IN THE FIELDWORK A series of technical, logistical, and operational difficulties encountered during the fieldwork and the corresponding actions to solve them are listed below: TABLE 11: DIFFICULTIES DIFFICULTY DESCRIPTION The sampling frame with beneficiaries did not appear in the database provided by the client In El Bagre (Antioquia), five surveys were conducted with beneficiaries who did not appear in the database and were in the rural district of Puerto Claver. This information was provided by Jesús Tate, legal representative of the Afroclaver Community Council, who was contacted through the database. Individuals who stated they belonged to more than one component but only appeared in one database provided by the client. In Antioquia and Chocó, surveys were conducted with beneficiaries who responded to two components. Beneficiaries who stated that they have not participated in the program 27 records did not belong to any component; 21 were from Antioquia and six from Chocó. These were respondents registered in the database but stated they were not beneficiaries of the program. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 114 ANNEX III: SUPPLEMENTAL ANALYSIS AND DATA 1. QUANTITATIVE ANNEX The methodological design of this evaluation uses six distinct methodological approaches to carry out a quantitative analysis of the Oro Legal (OL). These approaches seek to respond to the six guiding questions of the evaluation; the degree to which this was possible depended on the quality of the primary information collected during the evaluation process. MINING FORMALIZATION AND GOVERNANCE (EVALUATION QUESTION 1) DESCRIPTIVE ANALYSIS, Z PROPORTION TEST This analysis draws on disaggregated data from OL’s monitor system, with a cutoff at the third quarter. That data covers 215 MPUs that had at least one follow-up at the cut-off point. An analysis of OL’s formalization index shows that the variables affecting formalization can be grouped into two groups: 60 percentage points of the influence on formalization are grouped at the level of the program, that is variables related to OL itself (in red in Exhibit 30). Meanwhile, 40 points can be attributed to variables related to the characteristics of the targeted mining production units (MPUs); these factors are in blue in Exhibit 30). Average (Baseline) 1. Legalization (20 points) 20 2. PTO (10 points) 3 3. EIA (15 points) 3 Level of formalization (indicator 0-100) 4. Environmental guida (10 poitns) 2 5. SST (10 points) 0.8 6. Mining security (10 points) 1.8 7. Economic factors (!0 points) 1.2 8. Environmental factors (10 points) 2.4 Exhibit 39: Components of the OL formalization index SOURCE: PREPARED BY EVALUATION TEAM (2021) The program-level variables tended to show notable increases in the three follow-ups after the baseline. The MPU-level variables, on the other hand, showed a steady trend of improvement after the baseline. The overall trend for the formalization indicator shows growth from the baseline, with the most improvement registering at the first follow-up (Exhibits 31 and 32). 115 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 40: Trends for program-level indicators of formalization (n baseline=215) SOURCE: PREPARED BY EVALUATION TEAM (2021). Exhibit 41: Trends in MPU-level indicators of formalization (n baseline=215) SOURCE: PREPARED BY EVALUATION TEAM (2021). ECONOMETRIC MODEL OF THE DETERMINANTS OF MINING FORMALIZATION As proposed in the evaluation design, we estimate here a model for the determinants of mining formalization. To do so, we triangulated information from the beneficiary survey, the methodological sheets of the performance indicators (PIRS), and the Monitoring, Evaluation, and Learning Plan (AMELP) of the Oro Legal Activity. While the beneficiary survey had a sample of 97 MPUs that participated in the mining formalization component, when we triangulated this information with USAID and validated the quality of the information (e.g., checking the geographic coordinates of MPUs), it proved necessary to trim the sample for analysis to 75 MPUs. 0 5 10 15 20 25 Average score Legalization PTO EIA Environmental Guide 0 1 2 3 4 5 6 7 8 9 10 Average score SST Mining security Economic factors Environmental factors USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 116 Exhibit 42 shows a mapping of the coordinates of the 75 MPUs supported by OL. It is interesting to note that OL’s formalization indicator presents patterns of spatial clustering, so that some zones appear to be characterized by clusters of MPUs with high or low levels of formalization. This type of phenomenon led us to consider spatial effects in the econometric specifications, and in the results below a spatial model has been included to measure this type of formalization behavior. Exhibit 42: Location of MPUs included in analysis SOURCE: PREPARED BY EVALUATION TEAM (2021). The model takes Yi (the indicator for formalization in OL) as the dependent variable,88 and takes the vector X1i for the independent variables (attributes of the MPUs), X2i (variables related to the strategy of formalization) and X3i (variables related to the person who was a direct beneficiary). In order to evaluate the consistency of the estimations, we estimated three models using the same set of variables: (i) the first model contained all the variables and was estimated using Ordinary Least Squares (OLS); (ii) 88 We considered the possibility of estimating the second equation envisioned in the evaluation design, which referred to a binary response model (Logit or Probit). Nevertheless, only 15 MPUs in the set of variables for the model achieved the threshold for formalization (>75 points). This meant that there would be more independent variables in the model than cases of formalization, and that the exercise would be infeasible. 117 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV the second model restricted the number of variables and excluded the variable time in the program); and (iii) the third model used a restricted variable set and was estimated using a (Spatial Lag Model). The results of the three models are presented in Exhibit 43 below. Model 1 Model 2 Model 3 Response variable (Y1: Formalization Index) OLS - All variables OLS - Restricted variables Spatial Lag Model – Restricted variables MPU’s variables and attributes (X1i) Entry Index 0.67* 0.65* 0.85* Underground operation (dummy) -2.58 Family Labor (percentage) 0.00 Association (dummy) -1.82 Mechanized (dummy) 8.81* 14.62*** 14.71*** Labor -0.02 Installed Capacity (1-100) -0.06 Days of Operations (1-30) 0.08 Mine Age (years) 0.02 Municipality different for sale (dummy) -0.83 Public services (dummy) 4.38 12.64*** 6.48* W*Yi (Spatial Lag) 0.56*** Variables from the formalization strategy (X2i) Time in the program (quarters) 7.85*** Training (dummy) 0.55 Bills (dummy) 2.73 12.65*** 10.44*** PTO (dummy) 3.54 10.61*** 8.19*** PTO Technologies (dummy) -0.25 EG(dummy) 0.83 Variables from the beneficiary (X3i) Age -0.08 Educ 2.39~ 10.76** 11.98*** Experience -0.03 Woman 5.76 Debts -0.79 Continue with a decrease of 25% in income -5.72 Continue with a decrease of 50% in income 3.30 Continue with a decrease of 75% in income -0.19 Model parameters (Intercept) -36.77* 5.6 -25.68* R-squared 0.8702 0.6133 AIC 572.29 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 118 n 75 75 75 Signif. Codes 0 " *** ", 0.001 " ** ", 0.05 " * ", 0.1 " ~ " Exhibit 43: Results for econometric model of the determinants of mining formalization SOURCE: PREPARED BY EVALUATION TEAM (2021) he results of the three econometric specifications reveal some interesting findings. These include: • The model using all the variables (Model 1) shows a strong association between (time in the program) and the formalization indicator. Each additional quarter in the program is associated with an increase of 7.82 points on average. At the same time, the results on the final formalization indicator depended on the starting point of MPUs at entry, as each point on the formalization indicator at entry was associated with 0.67 points on the final score. In this model, the variable for mechanized operations and beneficiaries’ education level were positive and significant. Nevertheless, an interpretation is presented in the next two models, which showed greater consistency with the estimation. • The second model evaluated a restricted group of variables, keeping variables with greater statistical significance and excluding the variable for time in the program. This allowed us to contrast important associations, which the (time in the program) variable, being highly correlated with other regressors, would have otherwise obscured. The results of this model showed that access to public utilities, the practice of saving bills and receipts, have a mining work and operation plan (PTO), and beneficiaries’ levels of education were determinants of formalization. • The third model evaluated the same group of variables as the second model, but additionally incorporated a spatial lag effect. The results of this model provided the best estimations of the three models, as the inclusion of spatial effects corrected for possible omitted variable bias. This model found evidence for a significant spatial effect on the formalization indicator (W*Yi): the behavior of other MPUs in a 1km radius influenced the formalization behavior of individual MPUs. Additionally, it found that MPUs with mechanized operations scored 14.71 points higher on the formalization indicator compared to MPUs with traditional operations. MPUs with access to a public utility (water, sewerage, electricity, gas, or internet) scored on average 6.48 points higher than MPUs without any access to these services. MPUs that saved receipts and bills and had a PTO tended to score 10.44 and 8.19 points higher, respectively, than MPUs that did not. Having a direct beneficiary with no education, or only basic primary education, was associated with a formalization indicator score 11.98 points higher in comparison with MPUs with beneficiaries with higher levels of education. Finally, this model found that each point that an MPU scored at entry was associated with an additional0.85 points on the final formalization indicator, reflecting the importance of the starting point for each MPU’s journey toward formalization. VALUE CHAINS AND ALTERNATIVE LIVELIHOODS (EVALUATION QUESTION 3 DESCRIPTIVE ANALYSIS, VARIABLE ASSOCIATION AND OBSERVATION GROUPING To evaluate possible associations in the responses of beekeeping and annatto beneficiaries we elaborated a multivariate analysis, specifically multiple correspondence analysis, for a set of ten variables related to both value chains. This analysis drew on a total of 371 surveys of value chain beneficiaries. During the processing and analysis of the databases, 42 surveys with missing variables were filtered out. 119 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV This analysis yielded a joint category plot (Figure 4), which allowed us to represent the options for response categories as points on a two-dimensional plane. On this diagram, closeness to, or greater distance from, these points indicate greater or lesser association between beneficiaries’ responses. The results of this analysis are shown in Exhibits 44 and 45 below. Exhibit 44: Joint Category Plot – Annatto SOURCE: PREPARED BY EVALUATION TEAM (2021) In the annatto chain, we identified relevant associations between activities that were carried out and the expected objectives of OL: As multiple correspondence analysis allows for the reduction of dimensionality of the original data, the axes of Figure 6 represent the scores of the two main components that can summarize the 10 variables analyzed. These dimensions are latent variables obtained from the analysis and help to summarize the information. Dimension 1 is characterized by summarizing variables related to the viability of OL, barriers to the Activity, and receiving incomes. Dimension 2 is characterized by motivations, acquired commitments, and the frequency of receipt of incomes. The attributes associated with different colors are as follows: • Blue box: Annatto beneficiaries were dedicated to mining prior to OL. Their principal motivation was to improve their incomes and they saw the lack of stability in commercial agreements as the primary obstacle. • Red box: Beneficiaries were dissatisfied with the lack of or in frequency of income. This is associated with not considering the annatto value chain activities to be viable for the future. Beneficiaries with less than one year participating in this component manifest the greatest dissatisfaction. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 120 Exhibit 45: Joint Category Plot – Beekeeping SOURCE: PREPARED BY EVALUATION TEAM (2021) Applying this analysis to the beekeeping value chain also yielded a set of relevant associations. These associations, however, differed to some extent from the results for the annatto value chains, particularly with respect to viability. • Green box: Beneficiaries were less involved with mining before OL. They manifest greater satisfaction with the frequency of received incomes and consider security to be the main obstacle to their productive activities. • Purple Box: Beneficiaries consider beekeeping to be a viable activity and would continue with it even if they experienced a drop in income. Beneficiaries with over a year working on this activity tend to consider it to be viable. Their main motivations for continuing with this activity is associated with developing activities that are less risky than mining, and where they can commercialize and sell their products more easily. ECONOMETRIC MODEL OF THE DETERMINANTS OF INCOMES IN BEEKEEPING AND ANNATTO PRODUCTION Model 1 Response variable (Y1: Log Income received by the productive chain) OLS Microeconomic determinants (X1i) Size of the UPA -0.015 Yields of the UPA 0.045*** Use of installed capacity 0.011** Number of people -0.010 Experience in production -0.007 121 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Legal advice (dummy) -0.165 Environmental technical assistance (dummy) 0.341 Technical advice on bee (dummy) -0.464 Technical assistance in handling equipment and tools (dummy) -0.026 Beekeeping technical training (dummy) -0.127 Work safety training (dummy) 0.124 Business training (dummy) 0.494~ Environmental management training (dummy) -0.042 Hives and bees (dummy) -0.095 Protection equipment (dummy) -0.281 Agreement for commercialization (dummy) -0.354 Territorial determinants (Zi) Geographic isolation (dummy) -0.232 Transportation means by cars or motorcycles (dummy) -0.040 Model parameters (Intercept) 13.8813*** R-squared 0.3794 n 98 Signif. Codes 0 " *** ", 0.001 " ** ", 0.05 " * ", 0.1 " ~ " Exhibit 46: Results for income determinants model – Beekeeping SOURCE: PREPARED BY EVALUATION TEAM (2021) Model 2 Response variable (Y1: Log Income received by the productive chain) OLS Microeconomic determinants (X1i) Size of the UPA 0.098 Use of installed capacity 0.011~ Number of people 0.029* Experience in production -0.004 Legal advice (dummy) -0.254 Environmental technical assistance (dummy) -0.688 Technical advice on Annatto issues (dummy) 0.656 Technical assistance (dummy) 1.659~ Work safety training (dummy) 0.490 Business training (dummy) 0.014 Environmental management training (dummy) -0.547 Inputs (dummy) -1.315 Equipment (dummy) 1.051~ Agreement for commercialization (dummy) -0.080 Territorial determinants (Zi) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 122 Geographic isolation (dummy) 0.105 Transportation means by cars or motorcycles (dummy) 0.000 Model parameters (Intercept) 11.868 R-squared 0.339 n 57 Signif. Codes 0 " *** ", 0.001 " ** ", 0.05 " * ", 0.1 " ~ " Exhibit 47: Results for income determinants model – Annatto SOURCE: PREPARED BY EVALUATION TEAM (2021) In general terms, the results show that only four of the 18 variables evaluated had explanatory power for the incomes generated by the Agricultural Production Units (UPAs). We draw two main conclusions about the value chains from these results. • In the case of beekeeping (Table 2), we found that yield of honey (kg/hive) is one of the factors that most explains beneficiary incomes. An increase of one additional kg in yield is associated with an increase of 4.5 percent in the total income reported by beneficiaries. Each additional point of utilized capacity is associated with a 1.1 percent increase in total income. Beneficiaries who received business training had on average 4.94 percent higher incomes than those who did not receive this type of training. • In the case of annatto (Table 3), we found that each additional point of utilized capacity was associated with a 1.1 percent increase in incomes. Moreover, each additional employee increased the total income of the UPA by 2.9 percent. UPAs that received technical assistance or equipment support registered income increases of 1.65 and1.05 percent respectively. REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING (EVALUATION QUESTION 4) DESCRIPTIVE ANALYSIS – REHABILITATION OF DEGRADED AREAS In the same way that we did for the value chain components, we carried out a multiple correspondence analysis for eight questions for beneficiaries of the Activity component for rehabilitation of degraded areas. This analysis drew on 201 observations from surveys with rehabilitation component beneficiaries. In the processing and analysis of this information, 35 observations with missing information were discarded from the set. The resultant joint category plots incorporate variables on two dimensions. Dimension1 is characterized by variables related to time in OL and the type of participation in OL, and related commitments. Dimension 2 is characterized by motivations to take part in OL and the expectations of beneficiaries. 123 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 48: Joint Plot of Category Points SOURCE: PREPARED BY EVALUATION TEAM (2021) Analysis of these associations generated some key findings: • Red box: Beneficiaries with less than one year in the activity do not expect payment for environmental services. This is the group with the highest level of dissatisfaction with respect to their expectations for rehabilitation. • Blue box: Beneficiaries who have been in the program for a greater length of time (more than two years), and who have contributed land or labor to the nurseries, have the highest expectations for payment for environmental services. TEXT MINING ANALYSIS Text analysis reinforced understanding and perceptions of the program related to the rehabilitation component. We analyzed 36 pages (21,301 words) of interview transcriptions grouped around key components (rehabilitation, reforestation, policy, and environmental motivations), USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 124 Exhibit 49: Word cloud from interview transcriptions Source: Prepared by evaluation team (2021) Exhibit 50: Top ten most mentioned words SOURCE: PREPARED BY EVALUATION TEAM (2021) An analysis of word associations allowed us to summarize the messages that most frequently emerged from the interviews (Exhibit 41). These included: 1) Mining-only community, 2) Good work planting trees, 3) People from the program and people from the process, and 4) Nurseries to plant trees later. 100 57 50 49 48 46 44 44 43 0 20 40 60 80 100 120 125 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 51: Word association dendrogram SOURCE: PREPARED BY EVALUATION TEAM (2021) 2. REHABILITATION COMPONENT – REVIEW OF SATELLITE IMAGES An analysis of word associations allowed us to summarize the messages that most frequently emerged from the interviews (Exhibit 51). These included: 1) Mining-only community, 2) Good work planting trees, 3) People from the program and people from the process, and 4) Nurseries to plant trees later. Exhibit 52 shows the results obtained from these satellite images, covering the period 2016-2020 with the changes in vegetation cover and degraded areas shown first for the entirety of the sample, and then disaggregated by department. GLOBAL Beginning Final Real increase Vegetation cover (ha.) 1,618.1 1,684.5 4% Degraded area (ha.) 458.8 393 -14% ANTIOQUIA Beginning Final Real increase Vegetation cover (ha.) 1533.8 1592.9 4% Degraded area (ha.) 428.5 373.2 -13% Total 1962.3 1996.2 CHOCÓ Beginning Final Real increase Vegetation cover (ha.) 84.3 91.6 9% USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 126 Degraded area (ha.) 30.3 19.8 -35% Total 114.6 111.3 Exhibit 52: Results of satellite image analysis SOURCE: PREPARED BY EVALUATION TEAM (2021) In total, we observed 13 polygons, using these to calculate SAVI (Soil Adjusted Vegetation Index), which is useful for identifying vegetation at initial stages of development and in areas of low vegetation cover density. Exhibits 53-66 below show the results of this analysis. Exhibit 53: Pilot Farm SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 54: Polygon for Trópico Diverso SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 55: Polygon for Villa Grande: SOURCE: PREPARED BY EVALUATION TEAM (2021) Rehabilitation year 2016 2017 2018 2019 2020 Vegetation Cover (ha.) 0,262 7,234 10,147 10,793 10,316 Degraded area (ha.) 11,193 4,221 1,309 0,662 1,139 Total area (ha.) 11,455 127 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 56: Polygon for Villaconto (Rio Quito) – Active Rehabilitation SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 57: Polygon for Villaconto (Rio Quito) – Passive Rehabilitation SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 58: Polygon for Vegas de Segovia SOURCE: PREPARED BY EVALUATION TEAM (2021) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 128 Exhibit 59: Polygon for Refrestadora Zaragoza SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 60: Polygon for Asogauca P1 SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 61: Polygon for Asogauca P2 SOURCE: PREPARED BY EVALUATION TEAM (2021) 129 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 62: Polygon for Fundación Mineros P1 SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 63: Polygon for Fundación Mineros P2 SOURCE: PREPARED BY EVALUATION TEAM (2021) Exhibit 64: Polygon for Chilona SOURCE: PREPARED BY EVALUATION TEAM (2021) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 130 Exhibit 65: Polygon for Afroclaver, Cañada la Rica SOURCE: PREPARED BY EVALUATION TEAM (2021) Note: In Cañada la Rica (Exhibit 65), the years 2016 and 2017 (shown in white in the table) had a thematic exactitude lower than 85 percent. Therefore, we do not include values for these years in our calculations of vegetation cover. Exhibit 66: Afroclaver Quebraada Mellizos SOURCE: PREPARE BY EVALUATION TEAM (2021) 3. GIS TEAM INPUTS An annex with PowerPoint and Excel files can be found in Google Drive at the following link: https://drive.google.com/drive/folders/1PwsvRqENdcCV4IUrkYUyLSBlDHNCSGdY?usp=sharing 4. VISUALIZATION FOR QUALITATIVE ANALYSIS A folder with qualitative visualizations by component can be accessed at the following Google Drive link: https://drive.google.com/drive/folders/188L2nSyKvacDuvYGQY7ZLPi3jIaMQmEY?usp=sharing 131 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ANNEX III: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX89 The following is the matrix of evaluation findings, conclusions, and recommendations. We divided the matrix by each component and each evaluation question. We also divided each table into an A and B part. Part A presents the findings, conclusions, and recommendations, while Part B presents the implementation measures, responsible parties, and timeframe for the recommendations. MINING FORMALIZATION (QUESTION 1) AND MINING GOVERNANCE AND POLICY (QUESTION 2) QUESTION 1 - TO WHAT EXTENT DO THE APPLICATION BY MINING PRODUCTION UNITS (MPU) OF THE FORMALIZATION STANDARDS, IMPROVED OPERATIONAL EFFICIENCY, AND ACCESS TO LEGAL MARKETS GAINED THROUGH ORO LEGAL’S INTERVENTION PROVIDE SUFFICIENT INCENTIVES FOR MINERS INVOLVED IN ASSM (ARTISANAL AND SMALL GOLD MINING) TO REMAIN LEGAL/FORMAL AND IMPROVE THE ENVIRONMENTAL PERFORMANCE OF THEIR MINING OPERATIONS? QUESTION 2 - WHAT HAVE BEEN THE MAIN FACTORS (INTERNAL TO ORO LEGAL OR EXTERNAL ARISING FROM THE SECTOR CONTEXT, STAKEHOLDER ATTITUDES/BEHAVIORS, OR EXISTING GOVERNMENT POLICY AND CAPACITY) THAT HAVE CONTRIBUTED TO OR HINDERED OL’S PROGRESS IN STRENGTHENING COLOMBIAN MINING AND ENVIRONMENTAL GOVERNANCE AND POLICIES AT NATIONAL, REGIONAL, AND MUNICIPAL LEVELS? 89 This matrix presents findings, conclusions, and recommendations drafted for the April 30th, 2021 version of this report. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 132 TABLE 12A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 1 Q1 - By dignifying ASSM, AGM promoted a will to pursue formalization and environmentally sound practices. Q2 - AGM dignified ASSM and, by doing so, strengthened it. Q1 - AGM showed beneficiaries the benefits of formalization and helped them overcome the process’s many disincentives. Q1/2 - The good environmental practices disseminated by AGM increased job security, improved respect for worker’s rights, and promoted more ethical methods. Q1 - Miners that obtained a mining claim and an environmental license improved their income. Q1 - AGM’s intervention helped improve the environmental performance of beneficiaries. Q2/1 - AGM was an efficient intermediary between small miners and mining authorities. a. More than 71.9% of beneficiaries declared a high degree of satisfaction with AGM's implementation; this proves the program's success in overturning the artisanal and small-scale miners' distrust in formalization processes. 1. In all AGM-type programs' activities and components, continue to underline to miners and authorities that environmental, social, and economic performance only improves through ASSM (artisanal and small-scale mining) formalization and dignification processes. 2 Q1 - Focusing on small miners’ formalization subcontracts and special reserve areas (ARE) improved AGM’s management and logistic performance. Q1 - The duties imposed on formalization subcontracts might become an obstacle to the process. Q1 - AGM created synergies with beneficiaries and mining and environmental authorities. Q1 - Beneficiaries profited from the institutional relationships created by AGM. Q1 - The costs and haste inherent to the formalizing process’s technical standards can hinder formalization. Q1 - Beneficiaries that obtained a mining claim and an environmental license improved their income. b. During its implementation period, AGM adapted to the changes in regulation related to ASSM. 2. Further and reinforce the ASSM businesses adaptative approach in AGM￾type interventions to face changes in regulations and public servants. 3 Q1 - The main incentives to commence and follow through with the formalization process are the mining claim and an environmental license. Q1 - By dignifying ASSM, AGM strengthened the desire to comply with environmental and mining regulations. Q1/2 - Authorities are not aware of the specific conditions that define ASSM. c. Dignifying the ASSM and complying with mining and environmental regulations are the main incentives for beneficiaries to consider formalization. That is why a program like AGM must be more adamant in its support to public agencies to help them assess and mend their shortcomings regarding administrative procedures and 3.Increase USAID's involvement in the IP's (Implementing Partner) institutional relations to bestow the submitted requests and proposals with formal status. 133 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 12A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS Q2 - USAID’s renown and selecting local miners eased the program’s interactions with public agencies and communities. Q2 - Internal processes to comply with procedures are complex and take a long time, affecting the formalization program. Q2 - There are constraints to the formalization process in forestry reserve areas defined by Law 2 of 1959; these limitations apply to most protected areas. Q2 - Mining Production Units (MPUs) and local stakeholders are not up to date regarding mining regulations. Q2/1 - AGM did not explicitly divulge the most significant weaknesses identified in the formalization process. Q2/1 - AGM is mostly known among miners. Q2/2 - Stakeholders recognize AGM as an active agent in mining formalization. Q2/2 - Focusing on small miners’ formalization subcontracts and special reserve areas (ARE) improved AGM’s management and logistic performance. Q2/2 - The research showed a significant capability to adapt to regional nuances and regulation changes. Q2/2 - The process of dignifying beneficiaries empowered them, increasing the odds of their future commitment to the mining formalization process. Q2/2 - The research showed support to the miners in the long and cumbersome procedures required by public agencies. Q1/2 - Miners must comply with strict terms, but authorities disregard the ones that apply to them. Q1/2 - Public servants are unaware of regulations or misapply them, which goes against the small miners. Q1/2 - Without the commitment of all the agencies involved, formalization will not be possible. Q2 - Formalization is not a national policy, and it does not have the strength of a law or a bill. Q2 - Miners do not have to formalize their activities; this means they can drop the process and return to operate outside of the law like they have been doing since 1969 when the government nationalized the mines. Q2/1 - Memorandums of understanding proved ineffective in reaching the set goals. avoid blaming formalization mistakes on small miners. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 134 TABLE 12A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS Q2/1 - AGM has a more significant impact on miners and communities than on public agencies. Q2/2 - The research showed limited capabilities of public agencies (environmental and mining). Q2/2 - The lack of academic education of small miners hampers training activities. Q2/2 - Short-term initiatives generate skepticism and restlessness in miners. Q2 - Public servants have a preconception of miners and ASSM that, in most cases, is far from reality. Q2 - It is unlikely that a small miner can succeed in claiming an area included in a forestry reserve defined by Law 2 of 1959. Q2/1 - The complexity of the paperwork required to obtain a mining claim and an environmental license deter the possibility of a comprehensive formalization process. 4 Q1 - The future of AGM’s beneficiaries is highly uncertain as there is no clear exit strategy for the program. Q1 - The licit income can only improve after obtaining a mining claim and an environmental license; without these documents, we can’t talk of lawful sales. d. The end of the program, leaving most beneficiaries without administrative achievements (mining claim and environmental license), can cause setbacks in the formalization process. 4. Strengthen technical criteria for admission, establish the minimum stage that beneficiaries should achieve in the formalization process to guarantee compliance with the intervention cycle, and define the fate of those that fail to reach the program's required level. 5 Q1 - AGM could have underscored more the importance of gold marketing; this would have improved the beneficiaries’ disposition to insist on formalization. Q1 - Licit income only improves with the obtention of mining claims and environmental licenses; there are no licit sales without these tools. Q1 - By dignifying ASSM, AGM promoted a will to pursue formalization and environmentally sound practices. Q1/2 - Seeing the economic improvements associated with formalization guarantees the change towards legality. Q1/2 - AGM has to increment the emphasis on gold marketing to improve the sustainability of formalization efforts. e. It is important to have an approach inclined towards actions that entail current and future profits: the formalization process's success depends on marketing prosperity. Without this last condition, the miners will not find benefits in lawful alternatives and return to their former methods. 5. For new AGM-type programs, contemplate from the beginning efforts meant to open new marketing channels and forge alliances with organizations that encourage fair-trade and ethical markets that allow surcharges for environmental and social production costs. 135 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 12A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 6 Q1/1 - Any project requires a longer life span if trying to produce in beneficiaries (1) a profound change of their behaviors and habits (learned through generations), (2) the adoption of business methods, (3) the implementation of clean practices, (4) the acceptance to comply with regulations, expensive procedures, and due diligence. Q1/1 - The costs, terms, and technical demands of formalization (defined in mining regulation) usually overshadow the upside of lawful behavior. Q1/1 - Training offered by AGM favored compliance with environmental regulations and the implementation of measures to mitigate impacts. Q1 - AGM’s intervention helped improve the environmental performance of beneficiaries. Q1 - The lack of a coordinated effort from training organizations (AGM, SENA, universities, NGOs, public contractors, agents from the governors’ offices, mining companies, and public agencies) usually overwhelms the beneficiaries with demanding agendas that drive them away from their livelihoods. Q2/2 - The lack of an exit strategy to guide beneficiaries after the program’s departure will alter AGM’s effectiveness in the short term. Q2/2 - Organizations involved in formalization do not coordinate actions among themselves. Q2/2 - Illicit alternatives tend to offer higher revenues to miners. f. The research found different contributions to the J2SR, but it is necessary to have longer-lasting support to guarantee profound and enduring changes in the miner's operation. 6. Review the results obtained by the Bioredd+ and AGM programs to determine the most appropriate duration needed for similar initiatives to comply with USAID's expectations. Coordinate activities with other trainers to optimize and maximize resources and reduce the burden on miners. 7 Q2 - Formalization is not a national policy, and it does not have the strength of a law or a bill. Q2 - Mining formalization is voluntary; a beneficiary can withdraw at any time. Q2 - Formalization procedures are long, cumbersome, and inefficient. g. The current state of affairs of the formalization process within the Ministries and the stigmatization of ASSM are the main barriers to the initiative's success. Even more, the formalization process shows weaknesses regarding governance. g. The current state of affairs of the formalization process within the Ministries and the stigmatization of ASSM are the main barriers to the initiative's success. Even more, the formalization process shows weaknesses regarding governance. 8 Q2/1 - Focusing on small miners’ formalization subcontracts and special reserve areas (ARE) improved AGM’s management and logistic performance. Q2 - Hiring local personnel made relationships easier and helped AGM train mining communities; it also played a part when divulging regulation and explaining the role of agencies in regional dynamics. Q2/1 - Selection of professional personnel increased trust and eased the interaction among the program, public agencies, and communities. h. Some of AGM's most significant achievements are: concentrating resources and efforts using subcontracts and special reserve areas; adapting to the needs, habits, and administrative specificities in targeted areas; empowering beneficiaries by dignifying their activity; and offering ongoing support to miners in their interactions with public agencies 8. Maintain the habit of hiring local technical personnel in the focused regions. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 136 TABLE 12B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 1 • Include measures to fight the stigmatization of ASSM in all the components of similar programs. • Hire local technical personnel in all the focused regions to expedite communication with miners, communities, and authorities. • Identify the most helpful ASSM business practices to prepare training sessions for members of this sector. • * Draft and produce POP (Point of Purchase) documents and booklets, and other dissemination tools that promote formalizing mining businesses’ benefits. USAID, IP (Implementing Partner), MME (Ministry of Mines and Energy) Medium 2 • Regularly monitor the issuance of regulations that have a positive or negative impact on AGM's management. • * Through the Implementing Partner and in agreement with institutional stakeholders, design specific guidelines (containing objectives, scope, key-actors, and local commitments) to communicate and interact with public servants. USAID, IP Short 3 • * Formalize meetings with authorities by taking minutes that reflect discussions, agreements, commitments, and responsible parties. If possible, involve USAID or other reliable stakeholders in the institutional meetings. • * Draft documents analyzing the economic, social, and environmental impacts and solutions related to current procedures and the formalization policies, engaging media for dissemination activities. • * Devise short courses aimed at public servants of mining and environmental agencies; the topic should be ASSM experiences from recent USAID and national and international organizations' actions. USAID, IP Medium 4 • Stipulate the time needed to obtain the mining title and the environmental license as the shortest term for AGM-type programs. • Incorporate in the program's exit strategy the necessary actions to guarantee that beneficiaries continue their formalization efforts after activities conclude USAID, IP Short 5 • Specifically, request opening new formalization channels as an essential objective for AGM-type programs. To this end, identify potential allies from the get-go to make them part of the process and sign formal agreements with them to accept beneficiaries that comply with their environmental or social requirements. USAID, IP Short 137 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV VALUE CHAIN DEVELOPMENT AND ALTERNATIVE LIVELIHOODS (QUESTION 3) HOW DO BENEFICIARIES INVOLVED IN OL’S VALUE CHAIN ACTIVITIES PERCEIVE THE IMPACT OF ALTERNATIVE LIVELIHOODS ON THEIR PRESENT AND FUTURE WELL-BEING OUTSIDE THE ASSM ACTIVITIES? AS A LAWFUL ALTERNATIVE INCOME SOURCE. TABLE 13A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 9 The two value chains’ beneficiaries have different perceptions due to two associated factors: received income and commercialization perspectives for each product. The beekeeping and annatto businesses supplement the miner’s domestic income and, in case they become a regular revenue, they would allow beneficiaries to transition to more attractive and viable activities. a. Beneficiaries perceive the agribusinesses promoted by AGM as viable alternatives to generate revenue in mining areas, provided the consolidation of the respective agricultural and commercial chains 1. Continue to incentive the development of productive agricultural opportunities based on a comprehensive regional vocation and aptitude analysis allowing for a progressive transition of activity in mining areas. 10 AGM strengthened the beneficiaries' capacities, increased their productive potential, and improved their yields and product quality due to the investments and technical support provided. AGM created a solid production base for beekeeping and annatto that may be the cornerstones for future productive clusters in the targeted regions. AGM improved the professional skills of local technicians and left installed technical capacity. However, the proposed scheme does not guarantee continuity of the technical assistance to the farmers. b. AGM achieved remarkable productive improvements in both value chains, endowed the productive units of the beneficiaries with assets and technical knowledge, and consolidated national and regional representative production hubs. 2. Continue to foster regional technical capacities and devise strategies that increase the sustainability and duration of technical assistance. 3. Arrange support for the continuity of technical assistance from COLTAPICOLA and from Community Council technicians to AGM beneficiaries. 6 • Reconcile AGM-type programs' length with efforts to support small miners to ameliorate structural changes that improve formalization and performance. • Open a permanent communication line and forge alliances with training organizations and agencies like SENA, universities, NGOs, government contractors, governors' offices, mining companies, and mining and environmental authorities to improve ASSM training efforts' effectiveness. USAID, IP Short 7 • Document and openly divulge USAID's results with AGM-type programs to promote mining formalization as a comprehensive national policy. • Weigh the costs and benefits of a mandatory mining formalization policy to submit a proposal to the national government. USAID, IP, MME Medium 8 • * Identify and systematize available professionals in the respective regions for eventual job opportunities. USAID, IP Medium USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 138 TABLE 13A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 11 Beekeepers acknowledge an increase in their income due to an upsurge in their productive capacity and foresee eventual growth in this production line. In the case of beekeeping, the program did not foster a regional chain or a unified strategy to strengthen farmers organizations with a business-oriented approach. AGM did not offer Campo Dulce a chance to continue its consolidation process as a regional link in the beekeeping value chain; as they only came into play at the end of the program, there was not enough time to strengthen the business relation. Starting with a traditional crop, AGM identified and validated an annatto variety with a technical production system suitable for Chocó. AGM defined the basis for Chocó's annatto agribusiness production chain. However, there are still some critical unresolved issues that jeopardize its sustainability. c. In the case of beekeeping, AGM focused on strengthening the farmers productive units and gave less emphasis to the consolidation of the chains within the regions to increase their competitivity. d. In the case of annatto, AGM promoted a regional agribusiness chain that, in case it succeeds, it will become a benchmark for development for Choco. 4. Steer interventions towards promoting local and regional productive chains that add value, increase competitiveness, and connect available markets. 12 Annatto growers are unhappy with the business revenue and the unfulfilled initial promise of having a guaranteed buyer. The beekeepers have an issue with their associations and Campo Dulce that cannot buy all their production for cash, so they sell part of it to informal brokers and occasional buyers. The lack of comprehensive business models capable of quantifying investments' repercussions caused an overproduction of annatto and a shortage of working capital for beekeepers. e. The absence of comprehensive business models and a planned production created marketing, financial, and technical complications that burden the chains today and, in the case of annatto, bring about significant sustainability risks. 5. Guarantee that an actual commercial demand backs the incentives for small productive ventures (especially when they reach large scales), considering estimated production volumes and an associative business model that ensures sales. 6. Guarantee continuity of financial, technical, and entrepreneurial support to complete the annatto value chain's consolidation in Chocó. TABLE 13B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 9 • Incorporate a comprehensive aptitude assessment (environmental, agricultural, logistic, commercial, cultural, and zoning) for each agricultural production chain to promote. • * Reach an understanding with the community and the regional government on the agricultural production chain to choose. Governors' Offices, Mayors' Offices, USAID, Departmental and Municipal Secretaries of Agriculture, Autonomous Regional Corporations (CARs), UPRA, Agrosavia, ICA Medium 139 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 13B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) • * Define agreements between producers and public institutions regarding a gradual transition from mining to new activities that respond to development levels in local value chains. 10 • " Bring together regional professionals and technicians, training them in agricultural extension and unifying concepts regarding productive activities to contribute to local producers' technical assistance. • Design technical assistance regional models looking to procure the necessary funding from the potential sources: Agricultural Extension Departmental Plan (PDEA), private sector, civil society, and self-funding from the business operation." • " Advise COLTAPICOLA, Community Councils, and organizations to find technical assistance funding alternatives for producers, including self-funding by own agribusiness income, the regional PDEA, or soliciting resources from third parties. • Arrange the consolidation of COLTAPICOLA's and the Community Councils' technical assistance model with new or existing regional beekeeping and annatto projects." "Governors' Offices, Mayors' Offices, USAID, Program Operators, Departmental and Municipal Secretaries of Agriculture, SENA, ADR, Technical and Commercial Allies USAID, COLTAPICOLA, Community Councils, Beekeepers' Associations, Campo Dulce, Technical and Commercial Allies (of both value chains), Departmental and Municipal Secretaries of Agriculture, ADR Long Short 11 • Identify and define the project’s design phase the productive chain links that it will develop. This process will stem from a viability analysis that considers regional capacities, connections to current markets, actual and potential demand, available resources, and implementation time. • Include in new regional interventions previous agribusiness development projects, giving preference to participants in the productive chain that add local value. • Implement follow-up mechanisms that guarantee a systemic chain approach, acknowledging requirements, possibilities, and agreements in each step of the process, from production to marketing. USAID, Program Operators Medium 12 • "In the planning phase, identify and approach the market or markets that will absorb the production to establish technical specifications, commercial conditions, and required volumes, involving commercial allies from the beginning. • From the start, devise a business model with an integrated approach to account for contingencies, production times, costs, and prices for primary production, added value processes, logistics, and basic commercial agreements. This process should serve to allocate resources among the different stages in the productive chain. USAID, Program Operators Medium Medium USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 140 TABLE 13B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) • In the planning stages, estimate the number and size of the productive units according to the business model and define development phases, if necessary, to accommodate production and marketing." • Assure support for A&ACH to complete the process with Acumen and, in case the funding falls through, help them find alternative sources. USAID, A&ACH Short HOW DO BENEFICIARIES INVOLVED IN OL’S VALUE CHAIN ACTIVITIES PERCEIVE THE IMPACT OF ALTERNATIVE LIVELIHOODS ON THEIR PRESENT AND FUTURE WELL-BEING OUTSIDE THE ASSM ACTIVITIES? AS A MEANS TO IMPROVE QUALITY OF LIFE. TABLE 14A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 13 AGM positively affected the quality of life of beekeeping and annatto beneficiaries. Female participation was substantial in the first productive link of the two value chains, which increased their autonomy, recognition, and community and family cohesion. Annatto as a regional agribusiness increases recognition and self-awareness of Chocó natives’ capabilities to boost their own regional development. f. Farmers prefer agricultural chains over illegal gold mining activity due to the effects on their quality of life; they consider these alternatives as viable as long as they are profitable. 7.Strengthen incentives to promote women and youth's participation in agribusinesses; increase families' active involvement in productive chores. TABLE 14B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 13 • In agricultural technical training strategies, accommodate family and social workshops that cultivate female leadership and youth participation in productive units. • Design a plan that guarantees women's engagement in decision-making activities throughout all the project's business chain links. USAID, Program Operators Medium 141 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV HOW DO BENEFICIARIES INVOLVED IN OL’S VALUE CHAIN ACTIVITIES PERCEIVE THE IMPACT OF ALTERNATIVE LIVELIHOODS ON THEIR PRESENT AND FUTURE WELL-BEING OUTSIDE THE ASSM ACTIVITIES? AS A WAY TO REDUCE PRESSURE FROM ILLICIT GROUPS AND CONSOLIDATE REGIONAL SAFETY TABLE 15A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 14 AGM's productive alternatives strengthen the beneficiaries' resilience in the face of uncertain security conditions and increase hope towards future regional perspectives. Beneficiaries perceive that the agricultural alternatives offer ways to use time that reduce the odds of being recruited by armed groups, especially for young people. Beekeeping beneficiaries and stakeholders perceive that armed groups' proximity is the main obstacle and greatest threat for the business and its future development. g. Promoting agribusinesses does not directly affect regional security or public order, but it does contribute to households’ resiliency, safety perceptions, and future regional expectations. 8. Strengthen cultural and social appropriation strategies in projects, fostering empowerment of agribusinesses by local stakeholders and young community leaders' participation. TABLE 15B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 14 • Identify and include, from the beginning, community representative organizations and leaders with the potential to manage the value chain's different links. • Implement strategies to train young members of the producers’ families as leaders; training should include participating in innovation and value-adding activities. • Implement training and experiential learning plans to strengthen organizational and its leaders' capacities to take over agribusiness operation gradually during project execution. USAID, Program Operators, Mayors' Offices, SENA, Chambers of Commerce Long USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 142 HOW DO BENEFICIARIES INVOLVED IN OL’S VALUE CHAIN ACTIVITIES PERCEIVE THE IMPACT OF ALTERNATIVE LIVELIHOODS ON THEIR PRESENT AND FUTURE WELL-BEING OUTSIDE THE ASSM ACTIVITIES? AS A MEANS TO REDUCE ENVIRONMENTAL IMPACTS TABLE 16A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 15 AGM improved environmental conditions through the value chains, avoiding more significant degradation and slowing illegal mining in the targeted areas. The interdependence between beekeeping and the rehab model designed around Acacia Magnium strengthened both components' sustainability odds. h. Beneficiaries widely recognize beekeeping and annatto's positive environmental effects, as they help restore degraded areas and prevent the expansion of illegal mining. 9. Continue to foster agricultural products that blend friendly with the regional agroecological systems. TABLE 16B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 15 • Enrich the design with analysis and studies that identify the most viable and less damaging production systems suitable for the local environment. • Carry out monitoring tasks during the implementation to guarantee that productive venues and agribusinesses are not crossing the agricultural boundary, are rationally using water, and properly dispose of waste. • Carry out dissemination activities aimed at producers to make them aware of synergies between productivity and environmental management. USAID, Mayors' Offices, Program Operators, CARs, UPRA Medium REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING (QUESTION 4) FROM THE ENVIRONMENTAL, SOCIAL, AND ECONOMIC POINT OF VIEW, HOW APPROPRIATE AND EFFECTIVE HAVE BEEN THE DIFFERENTIATED REHABILITATION MODELS DEVELOPED BY ORO LEGAL IN ANTIOQUIA AND CHOCÓ IN AREAS PREVIOUSLY DEGRADED BY ILLEGAL MINING? 143 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 17A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 16 AGM chose the right areas for the implementation from an environmental standpoint as they are widely known for their ecological value; rehabilitation in those places contributes to national and international public policy goals. AGM to carry out territorial analysis with geographic information systems to select the areas to be rehabilitated was appropriate. a. The program selected worthy areas with valuable environmental resources and wealth. 1. In the design of AGM-type programs, to increase the impact of activities, include environmental considerations to choose areas aligned with national and local policy goals. 17 AGM worked in areas with high degradation levels posing significant risks to the communities due to the lack of ecosystemic services (provision of food and water) and the increased possibility of natural disasters; these issues make the election relevant from a social standpoint. Beneficiaries acknowledge the need to revert the damages done by mining; that is why environmental rehabilitation is the program's justifies from a social point of view. b. AGM worked in sites with high degradation levels posing significant risks to the communities due to the lack of ecosystemic services (provision of food and water) and the increased possibility of natural disasters. 2. Consider soil analysis in the design of the rehab component of AGM-type programs; these should serve to measure resource contamination and degradation. 18 From the beneficiaries' perspective, the rehabilitation reduce food insecurity and improve the quality of life of the local population. c. Beneficiaries recognize positive effects on their livelihoods due to environmental rehabilitation. 3. In AGM-type programs, widely divulge the results and consequences of rehabilitation actions on the quality of life; this should increase the beneficiaries and local population appropriation and care of restored areas. 19 AGM developed the communities' awareness and improved their knowledge and technical capabilities; these changes eased the degraded areas' rehabilitation process. d. AGM's support to rehab through local communities proved to be a success; especially in Chocó, the Community Councils played a relevant role in implementing actions. 4. Coordinate AGM-type programs' rehab activities with the local Ethnic Development Plans from the Community Councils to enhance Afro-Colombian communities' environmental governance. 5. Coordinate AGM-type program's rehab initiatives with local environmental leaders and movements. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 144 TABLE 17B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 16 • Emphasize area selection processes using available evidence from the IDEAM and local agencies: satellite images, information from the woods and carbon M&E system, early deforestation warnings, and identification of deforestation nuclei. • Build an environmental liabilities baseline through social mapping exercises (should work as a collective method to do regional land research). These maps improve the relationship between the local community with the territory and increase awareness about environmental degradation. USAID, Ministry of Environment and Social Development (MADS), IDEAM, CARs, IIAP, Mayors' Offices, Community Medium/Long 17 • Draft an action plan to determine the soil's degree of deterioration and degradation and promote restoration through bio-remedies that diminish contamination risks (due to mercury, cyanide, fuel, and oil, among others) and increase fertility. USAID, Universities, UMATA, Mayors' Offices, CARs, IIAP Short 18 • Put into practice M&E mechanisms of outcome variables like health, food safety, and other social profits of rehab. • Using AGM results, carry out campaigns to inform about the benefits of rehabilitation on life quality, emphasizing health and food safety. USAID, Mayors' Offices, Think Tanks Short/Medium 19 • " Concert with the Community Council Assemblies and determine the steps needed to harmonize rehab actions with the environmental chapter of the Ethnic Development Plan. • Train Community Councils on ways to apply conservation incentives to augment commitment towards rehab actions." • Map social and environmental organizations, trends, and leaders and integrate the result in the design of rehab actions. • Instill new rehab actions in leading local movements and disseminate them through the program and USAID. Community Councils, USAID USAID, IP, SENA, NGOs, CARs Short Short 145 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ARE THE REHAB MODELS SUITED FOR REGIONAL CONDITIONS? TABLE 18A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 20 It was the right decision to implement different models for each region. In Antioquia, the Acacia magnum has a long history of use and is very well known. AGM's rehabilitation model in Chocó agrees with the National Restoration Plan and the IIAP's guidelines (as a local research institute). e. It was very appropriate to have differentiated approaches in the two departments, recognizing biological, environmental, and social differences. 6. In AGM-type programs, offer technical assistance on the use and substitution of Acacia to advance ecological succession (Antioquia). 7. In AGM-type programs that encompass passive rehabilitation, increase territorial control to enhance the outcome of environmental mining liabilities rehab (Chocó). TABLE 18B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 20 • Design and implement a gradual substitution plan to achieve sustainable forestry exploitation; this plan should identify local species that allow a gradual substitution of Acacia magnium, establish a given ecosystem as a goal, contemplate synergies with productive chains (e.g., beekeeping), manage nurseries, and plant endemic species. • Design and implement a control strategy covering passive rehab areas, identifying stressing factors that can hinder primary production. • Integrate actions between Community Councils and local agencies (mayor's offices, CARs, and IIAP -Pacific Coast Environmental Investigation Institution-) to bolster territorial control. • Procure funds as working capital or incentives to allow community members to maintain and preserve rehab areas, especially when they are far from their homes or workplaces. • Involve local agencies (mayor's offices, CARs, and IIAP) in the seedling selection and obtain leverage or matching funds from local governments (money, technical support, or land)." USAID, IP, IIAP, CARs, Mayors' Offices USAID, IP, Community, IIAP, CARs, Mayors' Offices Short/Medium Short/Medium USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 146 ARE THE MODELS COST-EFFECTIVE IN COMPARISON TO OTHER APPROACHES EMPLOYED IN COLOMBIA AND ELSEWHERE? TABLE 19A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 21 AGM filled the funding void left by the government to rehabilitate neglected environmental liabilities. AGM's resources allowed to obtain an ambitious extension. f. AGM devised a strategic plan to accomplish an ambitious goal. 8. Determine and document the rehab costs to plan future projects of the same magnitude and scope. TABLE 19B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 21 • Perform a detailed exercise to determine the rehab activity costs, discriminating by type of managed area (Type I, II, and III defined by AGM) and type of rehab (active or passive); this should use information from the IP and local operators. The resulting data should be available to related institutions. USAID, Local Operators, IP Short HOW ROBUST ARE THE MODELS TO REESTABLISH VEGETATIVE COVER QUICKLY, REVERSING DEGRADATION, AND RESTORING BIODIVERSITY? TABLE 20A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 22 In general, the vegetation cover in analyzed areas increased. However, in some of those shapes, the research found depletion. On average, vegetation cover increased 4% in Antioquia, and degraded areas decreased by 13%. On average, vegetation cover increased 9% in Chocó, and degraded areas decreased by 35%. g. AGM's rehab models in Antioquia and Chocó increased the area with vegetation cover and reduced degraded areas. 9. Devise a regional control strategy with Community Councils to maintain passive rehab in targeted areas, in programs that encompass passive rehabilitation within collectively owned land. 10. Support efforts to define private plots' boundaries; this should improve regional control, bypass new stress factors to the ecosystem, and avoid losing benefits from initiatives promoted by AGM-type programs. 147 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 20B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 22 • Observing social or ethnic traditions carry out conflict resolution training sessions to strengthen the Community Council's role. • Support the Community Councils when implementing activities that promote ethnic and cultural identity and encourage natural resources preservation. • Coordinate the program's initiatives with the National Advisory Restoration board to improve the institutions' regional management. • Improve relations of local agencies to better their overseeing role and avoid new human activities that deplete AGM's areas." • Check the limits of the targeted plots and the Community Councils (extension of the social mapping exercise). • Support the community and the plot owners when defining land boundaries; natural fences are encouraged." USAID, IP, Community Councils, Universities, NGOs USAID, IP, NGOs, CARs, Regional IGAC Short Medium IN THE MEDIUM TO LONG TERM, HOW COMMITTED ARE BENEFICIARIES TO FUTURE MAINTENANCE OF THE MODELS? TABLE 21A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 23 Maintenance for AGM's rehabilitated areas is at risk due to (1) the Community Councils' limited capacity to obtain funding and (2) coordination gaps with public agencies. The sustainability of AGM's models depends on projects that increase the beneficiaries' income and help extend the management of rehabilitated areas. h. Coordination among local organizations is lacking, affecting the sustainability of the implemented actions. 11. Implement plans to coordinate efforts with the mayor's offices, the CARs, and the IIAP to guarantee the AGM's continuity in these institutions and a proper transfer of knowledge. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 148 TABLE 21B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 23 • Sign agreements with local institutions and Community Councils to extend support to nurseries. • Develop seedlings' donations from local institutions to advance with scheduled plantations. • Undertake agricultural endeavors with the local dwellers to harvest wood-related products (and others) in rehabilitation areas. Also, train beneficiaries on the procedures and necessary paperwork required by government agencies to exploit resources; also show them how to draft and submit carbon-credit projects. • Carry out inquiries to determine the feasibility of Forestry Incentive Certificate programs and design a funding strategy to verify reductions in greenhouse effect gases' emissions; this information should feed realistic projections and adjust the community's expectations. • Facilitate agreements among businesspeople looking to apply to the carbon tax exemption (Decree 926/2017). MADS, CARs, USAID, Local Mining Entrepreneurs, MADR Medium ELIMINATION/REDUCTION OF MERCURY FROM THE ASSM SUPPLY CHAIN (QUESTION 5) WHAT COMBINATION OF INSTRUMENTS AND APPROACHES EMPLOYED TO REDUCE MERCURY USAGE IN THE ASSM HAVE PROVEN TO BE MOST EFFECTIVE, AND WHY? TO WHAT EXTENT DID EXTERNAL FACTORS CONTRIBUTE TO OR HINDER THE ACTIVITY’S INITIATIVES TO REDUCE OR ELIMINATE THE USE OF MERCURY? WHAT COMBINATION OF INSTRUMENTS/APPROACHES EMPLOYED WAS MOST IMPACTFUL? HOW WAS THIS ACHIEVED, AND WHAT WERE THE MOTIVATIONS OF BENEFICIARY MPUS? TABLE 22A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 24 The mercury ban entry into force (Law 1658) is the main external reason for reducing the use of this metal in the MPUs. a. Regulations are the primary external reason that explains the miners' interest in reducing mercury use; they want to avoid penalties and other legal consequences. 1. Continue training miners on mercury use and its prohibition. 149 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 22A: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDINGS CONCLUSIONS RECOMMENDATIONS 25 The best-combined approach to reduce mercury use involves supporting miners' formalization, training beneficiaries, promoting the use of new processing technologies, and carrying out awareness campaigns. AGM's technical assistance (deposit exploration) within the PTOs (Work and Implementation Plans) guides the best practices no-mercury mining. As a consequence, exists a correlation between formalization and the use mercury. AGM's initiatives helped beneficiaries comply with regulations as they systematically reduced the use of mercury in the region. b. Support the formalization process, train beneficiaries emphasizing gold extraction efficiency with clean methods, and raise awareness on the impact’s mercury produces on human health and the environment is a powerful combination to reduce its use. 2. When designing AGM-type programs, include resources to support government agencies carrying out geological research in mining sites. 3. Create a program for trained miners to move towards technological reconversion. 26 The research found that the most significant reduction in the use of mercury (per gram of gold) occurred in alluvial mining (-9.3 Hg/t), with similar results in Antioquia and Chocó. Avoiding adverse health and environmental impacts are some of the most common motivations for miners. Also relevant is evading legal complications when working in unlawful conditions. The research found differences between the mercury reduction numbers reported by the beneficiaries and the ones stated by the authorities. c. There is no certainty regarding miners' reduction of mercury in the gold mining process. 4. Increase regional mercury monitoring. 5. Conceive communication campaigns informing about the use of mercury and other pollutants typically used in mining (like cyanide). 27 The mercury ban has a tough challenge when trying to control its illegal traffic, which is very active in the region. Close to 1% of the beneficiaries still use it. AGM fulfilled institutional gaps after the entry in force of Law 1658 (banning the use of mercury) and contributed to adopting new practices to process mined gold (offering training). d. The mercury ban faces a tough challenge when trying to control its illegal traffic, which is very active in the region. 6. Discuss with local, regional, and national agencies a strategy to market mercury-free gold. 28 The formalization subcontracts help control the use of mercury. The commercialization of gold in informal markets diminishes control on the use of clean technologies. e. The formalization subcontracts help prevent the use of mercury. 7. Continue to support and oversee subcontracts (related to formalization) to promote a fair benefits distribution of the gold mining activity. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 150 TABLE 22B: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS), MEDIUM (NO MORE THAN 2 YEARS, LONG (MORE THAN 2 YEARS) 24 • Ask the local communities about the most effective communication channels to reach miners. • Train local people to teach ASSM miners about mercury use and its prohibitions. • Agree with mining secretaries (in mayors' offices) on a training program project about environmental and mining regulations. USAID, NGOs, Local Organizations, Mining Board, Mayors' Offices Short 25 • Carry out geological studies in ASSM miners' mining sites to contribute to their Work Program (PTO). • Develop funding strategies for ASSM miners' technological renewals. • Commit support from companies and NGOs to ASSM miners' technical advancements. USAID, Colombian Geological Service MME, Mayors' Offices, Mining Businesses, NGOs, USAID Medium Medium 26 • Contrive a funding strategy to allow mining and environmental authorities to visit mines to oversee activities. • In AGM-type programs, support implementing the sector's Mercury Environmental Action Plan through air and water quality studies. • Take advantage of university research to increase the scope of previous campaigns about different pollutants used in gold mining. • Agree with NGOs on new ways to raise awareness about gold-mining pollutants and their impact on the environment. Local Organizations, Mayors' Offices , CARs, Governors' Offices, USAID USAID, Universities, NGOs Short/Medium Short/Medium 27 • Include in AGM-type programs a plan to market gold with international buyers, specialized jewelers, and ethical consumers. MME, MT, MADS, MCIT, Governors' Offices, USAID Short/Medium 28 • Support the program's current beneficiaries to sign subcontracts with major mining companies. • Inculcate interest in mining companies to take in small miners. • In agreement with the mining authorities, create incentives for companies that contribute to processing mercury-free gold in ASSM mines. Mining Businesses, USAID Short 151 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV PERCEIVED RELEVANCE AND IMPACT OF THE ACTIVITY (QUESTION 6) WHAT ARE THE MAIN PERCEPTIONS OF THE RELEVANCE AND EFFECTIVENESS OF THE USAID - ORO LEGAL’S INTERVENTION AMONG DIRECT AND INDIRECT STAKEHOLDERS - ASSM OPERATORS, GOLD MINING PRIVATE SECTOR, ACADEMIA, INTERNATIONAL DONORS, USG AGENCIES, GOC AGENCIES, ETC.? TABLE 23A. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDNIGS CONCLUSIONS RECOMMENDATIONS 29 In general, beneficiaries are highly satisfied with the program and highlight OL strengths based on the effects observed. All AGMs components are perceived as relevant and pertinent according to the context and regional and local needs. Beneficiaries value AGM's supplies, machinery, training, and technical support as positive; however, the follow-up and ex-post assistance are considered vital to the sustainability of the results. Some indirect beneficiaries are unaware of AGM's actions and goals and declare not receiving any support. a. In general, beneficiaries perceive the intervention as relevant and pertinent to the regional conditions and underline its strengths; however, there are opportunities to improve the follow-up to guarantee sustainability after AGM's departure. 1. Improve information management processes and deepen the characterization of beneficiaries to improve monitoring and measurement of the effectiveness of interventions. 30 AGM contributed to change the way miners think, helping them embrace the possibility of sustainable mining (from an environmental, social, and economic perspective). AGM filled a knowledge gap in miners and offered them support and tools for the formalization and mercury reduction processes. AGM opened lines of communication between the MPUs and mining and environmental authorities to ease the formalization process. However, the insufficient institutional capacity still hinders the perception of trust and effectiveness. b. AGM offered information regarding the formalization process and ways to reduce mercury use and contributed to dignify ASSM. Also, it strengthened the credibility of sustainable mining and increased coordination between miners and environmental and mining agencies. However, the frailty of these organizations still hinders trust and effectiveness. 2. Strengthen coordination efforts among USAID, the IP, the national government, and targeted regional agencies to improve the interventions' planning, complementarity, follow-up, and sustainability. 31 The research found signs indicating the recovery of native species and improvements in the landscape; data also shows a positive assessment of rehab interventions in areas exhausted by mining. People are aware that some of the effects expected from the intervention are not yet visible due to their long-term nature. Beneficiaries appreciate beekeeping as an alternative source of income and as an environmentally friendly activity. Training had a positive bearing in adopting clean practices to manage waste from the value chains. c. Although some rehab results are long-term, there is a positive perception regarding environmental effects like the restoration of native species, improvement of micro- climates, and increased community awareness that produces local appropriation. d. The promotion of agribusinesses contributes to improving environmental conditions due to training in beekeeping practices and 3. Strengthen training and other initiatives that bolster environmental conditions in all the program's interventions. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 152 TABLE 23A. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. FINDNIGS CONCLUSIONS RECOMMENDATIONS systems to treat waste coming from the intervention. 32 Agribusinesses prove to be a fruitful alternative for Agribusinesses prove to be a fruitful alternative for families; technical assistance and help with supplies have increased the families' income. AGM created legal job opportunities. The chosen value chains improve the beneficiaries' livelihoods; they incentivize collective work (family and community) and are alternatives to work, learn, and increase income. The value chains promote community recognition and self-awareness, propel regional development, and are benchmarks for other regions and associations. Lawful activities create benefits regarding legal peace of mind, a reduction of personal risks, higher self-esteem, a sense of belonging, lower costs, bigger sales, and no fines or law-enforcement interventions. e. Agribusinesses are a viable economic option that creates additional income (when beneficiaries receive technical training and proper supplies) and offers new job opportunities. f. AGM improves targeted communities' livelihood (recognition, community work, technical knowledge, and purchasing power), and lawfulness creates multiple benefits (legal and social peace of mind, self-esteem, and risk reduction). 4. Give continuity and increase communication efforts to inform about benefits and improvements in life quality following mining legalization and value chains initiatives. 33 Beneficiaries and local stakeholders have an increased sense of security (peace of mind) in targeted areas with AGM activities. However, the security conditions in AGM areas are still uncertain and negatively affect the program's success. g. Albeit beneficiaries' increased sense of security (peace of mind) in targeted areas, the security conditions in AGM areas are still uncertain and negatively affect the program's success. 5. Continue to hire technical staff in targeted areas to mitigate security barriers. In addition, include in the training sessions for beneficiaries, the dissemination of self-care protocols for vulnerable people and communities. 34 AGM created job opportunities for women, increased their income, fostered their empowerment, and strengthened community relations; all of this improved their families' livelihoods and well-being. The research found it crucial to involve youth in the value chains to create a generational replacement and avoid the risks of getting involved with illicit activities. Young people have few job opportunities and, sometimes, little interest in getting involved with agricultural activities. h. AGM created job opportunities for women, increased their income, fostered their empowerment, and strengthened community relations; all of this improved their families' livelihoods and well-being. Young people have few job opportunities and, sometimes, little interest in getting involved with agricultural activities. The research found it crucial to encourage a generational replacement to avoid young people's risks of being lured to illicit activities. 6. Augment the gender perspective in of the program's interventions, increasing female participation, which has shown good results. 7. Consider outlining strategies to stimulate generational renewal in the productive value chains initiatives to increase young people’s permanence in the regions, prevent their involvement with illicit activities, and further the intervention’s sustainability. 153 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 23B. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS MATRIX NO. IMPLEMENTATION MEASURES RESPONSIBLE PARTIES TERM - SHORT (6 MONTHS) MEDIUM (NO MORE THAN 2 YEARS) LONG (MORE THAN 2 YEARS) 29 • Define protocols and standards, including a database structure, to trace and manage information. • Choose variables and tools to measure the types of interventions (consider the income variable). • Carry out measurements at the beginning, middle, and end of the program (baseline, half￾way, and end line). USAID, IP Short 30 • Considering a scenario where AGM would continue, create a coordination board to bring together different stakeholders (USAID, the IP, Ministry of Mining, National Mining Agency, Ministry of Environment, Ministry of Agriculture, and targeted regional agencies) so they can: define the scope of the interventions, foster complementarity among institutions, create follow-up scheme, and agree on periodic reports. • Implement knowledge transfer sessions between the implementing partner and government officials aimed at improving knowledge of the reality of the territory and discuss opportunities to improve the procedures or support defined by the authorities. USAID, IP, Local and Regional Government Agencies Short 31 • Increase the scope of training actions in better environmental practices in AGM-type projects (residue management, recycling, water use, alternative energies, multimodal transportation, etc.). • Stimulate alliances with regional and local institutions (CARs, mayors' offices, NGOs, and businesses) and look for converging investments. USAID, IP, Local and Regional Government Agencies Medium 32 • Through beneficiary associations and Community Councils, design and implement an outreach strategy with the respective tools (visual media, educational material, and gathering venues). • Take advantage of beneficiaries' testimonies and experiences to improve educational actions and exemplify the benefits. • Define strategies to inform allies about benefits and incentives. USAID IP 33 • Hire local personnel for the program's local interventions to facilitate access to targeted areas and increase beneficiaries' confidence. • Identify the National Protection Unit's protocols, manuals, and booklets (e.g., Self- protection and Security Manual). • Identify social and emotional tools created by agencies and NGOs, supporting people endorsing local reconciliation and fostering leadership and peacebuilding skills. USAID, IP Short USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 154 • Disseminate this material in the program among relevant beneficiaries. 34 • When determining criteria to select beneficiaries, define the minimum female participation percentage. • Continue with initiatives that empower women and disseminate the benefits of female participation in the program. • Create opportunities for women to exchange their experiences. • In meetings with the program's allies, emphasize the importance of initiatives that include women and explain the benefits. • "Define a sensibilization strategy to inform about the importance of generational renewal in the value chains. • Include in the productive chain initiatives a component of knowledge transfer from adults to young people in the community (e.g., parents to children). • * Through social service programs, forge alliances with the Secretaries of Education and schools to promote youth education and participation in the value chains. USAID, IP USAID, IP, Local Governments, Educational Institutions Short Medium 155 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ANNEX IV: DETAILED METHDOLOGY WORK PLAN INTRODUCTION90 Colombia’s gold mining sector is severely affected by illegal and informal activities fostering widespread deforestation, riverbanks sedimentation, and mercury contamination. Illegal and informal gold mining is a significant challenge growing faster than the institutional capacity to regulate and control mining activities. This situation poses a threat to public health, significantly limits the government’s tax revenues, and creates growing environmental liabilities. Alongside illicit economies such as coca cultivation, land appropriation, logging, trafficking in wildlife, extortion, and money laundering, illegal mining is part of the criminal structures that threaten Colombia’s peace consolidation. Given the critical challenges posed by uncontrolled gold mining, USAID has responded with an ambitious artisanal gold mining activity aiming to help Colombia build sufficient governance capacity for gold mining activities, including enforcement of legislation and addressing illegal mining’s environmental impacts. The Artisanal Gold Mining - Environmental Impact Reduction Activity - Oro Legal – is implemented by Chemonics International. Since 2015 it supports Colombian efforts to reduce the social and environmental impacts of artisanal and small gold mining (ASGM) in twenty-two municipalities within the departments of Antioquia and Chocó. Oro Legal works through the legalization and formalization of illegal mining, capacity building, and strengthening, the endorsement of more appropriate technology and improved practices, development of new legal routes to market, and the promotion of sustainable alternatives to mining. Oro Legal works closely with the Government of Colombia (GOC), non￾governmental entities (NGOs), communities, and the private sector to achieve a set of indicator targets. As part of the Activity, and after approximately five (5) years of implementation, USAID is conducting a final performance evaluation to assess: if Oro Legal has achieved its deliverables and objectives, to what extent the achievements at the different sub-objectives of the intervention are perceived effective and impactful, and which factors may be playing a role to modulate the performance toward those sub￾objectives, the Activity’s main goal, and USAID’s strategic development goals. The evaluation will support USAID/Colombia and Oro Legal Implementing Partner (IP) accountability among stakeholders. The evaluation will also provide lessons learned and insightful recommendations to help USAID/Colombia make programming decisions for future similar activities. It will also share knowledge gained with relevant stakeholders in the Government of Colombia (GOC), Mining Sector, International Donors, and Communities. This document describes the work plan for the final performance evaluation for the Oro Legal Activity. The plan has seven sections; the first one contains a timetable and logistical arrangements for this endeavor. The second section presents the evaluation team; the third lists the evaluation milestones; the fourth and fifth detail a schedule for the team’s data collection efforts, locations, and dates to pilot data 90 We based this section on the project’s available documentation and the Scope of Work for the final Performance Evaluation of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 156 collection; the sixth describes the evaluation’s general methodology; and the seventh contains an outline of the evaluation report. CONTEXT91 ORO LEGAL GENERAL INFORMATION The Activity has two main objectives. The first is to build effective governance capacity for gold mining activities via: (i) strengthening Colombian government capacity to enforce gold mining legislation; (ii) enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs and; (iii) providing training and technical assistance (TA) to artisanal miners. The second objective is to increase the capacity of Colombian government agencies, communities, and the private sector to address the environmental impact caused by illegal mining through: (i) rehabilitating degraded areas; (ii) generating diversified sources of income/alternative livelihoods for communities that cannot or should not be involved in gold mining and; (iii) improving drinking water quality in mining areas. TABLE 24: INFORMATION SUMMARY ACTIVITY NAME: Artisanal Gold Mining Environmental Impact Reduction - Oro Legal USAID’S OFFICE: USAID/Colombia – Environmental Office IMPLEMENTING PARTNER: Chemonics CONTRACT NUMBER: AID-514-C-15-00003 TOTAL ESTIMATED COST: $22,122,365 LIFE OF PROJECT: 2015/09/23 – 2021/03/31 ACTIVE GEOGRAPHIC REGIONS: Oro Legal Activity is implemented in 22 municipalities in the departments of Antioquia and Chocó: Barbosa, Buriticá, Cáceres, Caucasia, Don Matías, El Bagre, Nechí, Remedios, San Roque, Segovia, Tarazá, Zaragoza (Antioquia); Atrato (Yuto), Cértegui, Condoto, El Cantón San Pablo, Istmina, Nóvita, Quibdó, Rio Quito, Tadó, Unión Panamericana (Chocó). DEVELOPMENT OBJECTIVE: DO 4: “Environmental resilience and low carbon development strengthened”. Intermediate Results: IR 4.1 “Natural resource management strengthened”. Sub-IR 4.1.1 “Legality, rehabilitation, and reduction of mercury use in small, artisanal mining operations improved.” EXTERNAL OR INTERNAL EVALUATION: External BACKGROUND Colombia’s gold mining sector suffers from pervasive illegality. Most of the yearly average 50 metric tons of gold produced in Colombia are done by illegal and informal actors, driving widespread deforestation, riverbanks sedimentation, and mercury contamination. This reality poses a threat to public health and 91 We based this section on the Scope of Work for the final Performance Evaluation of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal) and the Activity’s MEL Plan. 157 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV costs the government tens of millions of dollars in lost tax revenues and environmental liabilities left behind. As a result, addressing illegal mining is a significant concern in Colombia’s post-conflict scenario. Gold mining in Colombia has increased by over 300% since 2006, and illegal and informal gold mining is a growing challenge. The scale of mining has grown faster than the institutional capacity to regulate and control mining activities adequately. Illegal mining is part of illicit economies that include coca cultivation, land appropriation, logging, trafficking in wildlife, trafficking in persons, and extortion and money laundering by the criminal organizations involved in these activities. According to the GOC, illegal gold mining replaced drug trafficking as the primary source of income for organized crime, armed groups, and illicit actors. This situation is fueling violence as there is a strong link between the criminal organizations responsible for coca crops and those involved in trading the gold from illegal mining. At the national level, 43 percent of Colombia’s area affected by illegal mining is also affected by illicit coca cultivation, and Antioquia and Chocó bear the heaviest lift of the phenomenon. Uncontrolled gold mining is a major stimulus for Colombia’s illicit economy and a significant driver of environmental degradation, causing large-scale deforestation, riverbanks destruction, and mercury contamination. It also scars the landscape with a legacy of highly degraded and contaminated abandoned mining sites. In 2019, alluvial gold exploitation degraded an estimated 98,000 hectares of land at the national level, and the departments of Antioquia and Chocó concentrated 78% of this area. The resulting biodiversity loss in Chocó is of particular concern as this is one of the most biodiverse regions in the world. Besides, these highly degraded sites are so devastated that they cease to be part of the productive economy, no longer providing viable livelihood options unless and until they receive significant investments for remediation and restoration. Due to mercury to amalgamate gold in artisanal gold mining, Colombia is the world’s largest mercury polluter per capita. As a result, some Antioquia towns are among the highest urban per capita air￾mercury polluted in the world. The country has passed legislation banning mercury in mining; it went into effect in July 2018. But full implementation of this ambitious goal presents yet significant challenges along the chain of control. USAID’s strategic development goals have two levels: 1) An Agency’s overarching objective known as “Journey to Self-Reliance” -J2SR- which looks for (a) helping countries to build capacity and (b) to impact behaviors triggering positive changes; so the ultimate goal of foreign assistance is ceasing to exist in the future. 2) A USAID/Colombian Mission’s Environmental Development Objective of improving the environmental resiliency by strengthening the management of natural resources [Intermediate Result 4.1 / CDCS 2014-2020]. USAID supports the GOC’s efforts to create a durable and inclusive peace in the wake of the country’s 50-year armed conflict. Tackling illegal activities, including illegal mining, and providing alternative incomes through the licit economy, is paramount for sustainable and participatory peace. Given the important challenges posed by uncontrolled gold mining, USAID has responded with the Agency’s largest and most ambitious artisanal gold mining activity to build effective governance capacity for gold mining activities, aimed to strengthen Colombian Government capacity to enforce gold mining legislation and address the environmental impact caused by illegal mining. Oro Legal promotes responsible legal mining, alternative livelihoods, and environmental rehabilitation. The main goal of Oro Legal Activity is to help the GOC’s mining sector and the communities to address the informality of the ASGM to reduce its environmental liabilities through: USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 158 1. Building effective governance capacity for gold mining activities via: a. Strengthening Colombian government capacity to enforce gold mining legislation. b. Enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs. c. Providing training and technical assistance to artisanal miners. 2. Increasing the capacity of GOC entities to address the environmental impact caused by illegal mining through: a. Restoration of degraded areas. b. Generating alternative livelihoods for communities that cannot or should not be involved in gold mining. c. Improve drinking water quality in mining areas. This Oro Legal Activity contract totaling almost $22.1 million follows the mining component of the previous USAID/Colombia BioREDD+ activity implemented by Chemonics from July 2013 to May 2015. DESCRIPTION OF THE THEORY OF CHANGE AND RESULTS FRAMEWORK The Theory of Change (TOC) underlying the Oro Legal Activity is that if the program strengthens government capacity to manage gold mining and address environmental damages effectively, that will improve the social, environmental, and economic performance of gold mining operations, thereby supporting the GOC to create sustainable and inclusive peace. Each of the two objectives of Oro Legal has a TOC. The Oro Legal Activity has two Objectives: the first objective focuses on supporting mining sector institutions and organizations to improve and/or better enforce regulations covering gold mining in target regions, thus reducing the negative environmental and social impacts often implied by these activities. The second objective seeks to increase the capacities of the GOC, communities, and the private sector to address the environmental degradation caused by unauthorized gold mining. The results framework (see Exhibit 67) is the guide for Oro Legal to implement USAID’s vision for the Activity and links USAID/Colombia’s Strategy at CDCS 2014-2020. It incorporates crosscutting responses to tasks, including the differentiated approach to gender inequity and vulnerable populations. 159 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 67: The Oro Legal Results Framework PURPOSE The purpose of this final performance evaluation is to conduct a systematic analysis of the performance of the Oro Legal Activity to assess to what extent the achievements at the different sub-objectives of the intervention are perceived effective and impactful, and which factors may be playing a role to modulate the performance toward those sub-objectives, the Activity´s main goal, and USAID´s strategic development goals. We will assess Objective 1 and Objective 2 concerning the following five expected results:92 Expected result 1.1: Capacity in government entities to enforce gold mining legislation strengthened Expected result 1.2: Participation of ASGM organizations and indigenous and Afro-Colombian communities in formalization programs strengthened 92 We will not assess Expected result 2.3 (Drinking water quality improved) in this evaluation. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 160 Expected result 1.3: Environmental impacts reduced through technical assistance and training Expected result 2.1: Reforestation NAMA supported in areas degraded by unauthorized ASGM operations Expected result 2.2: Alternative livelihoods for ASGM communities promoted To this end, the methodological approach considers the support provided by Oro Legal (OL) to the following targets: 1. Reduce illegal activity and environmental devastation caused by illegal mining. 2. Generate economic alternatives to illegal mining (honey production in Antioquia and annatto production in Chocó). 3. Rehabilitate highly degraded lands using reforestation with Acacia maginum in Antioquia and assisted natural regeneration in Chocó. 4. Introduce technologies and practices capable of eliminating the use of mercury. In the evaluation we also contemplate contextual information that may affect the sustainability of results and identifies the main perceptions regarding OL intervention’s relevance and effectiveness among direct and indirect stakeholders. EVALUATION QUESTIONS93 Considering the Oro Legal Activity purpose stated above, we will conduct a systematic analysis of the Activity’s performance and progress towards its main goals to answer six (6) main questions. Each of the questions will examine several contextual elements to guide the answers. The questions are the following: 1. Mining Formalization: To what extent do the application by Mining Production Units (MPU) of the formalization standards, improved operational efficiency, and access to legal markets gained through Oro Legal’s intervention provide sufficient incentives for miners involved in ASGM to remain legal/formal and improve the environmental performance of their mining operations? Context and guidelines to answer the question: This is a follow-on to question #3 from the Mid￾term Evaluation, adjusted to capture the outcome of the contribution of mining formalization towards one of the main goals of the Activity. Given a national context marked by a steady increase in international gold prices, local pressure from illegal actors, and a widespread culture of unlawfulness, have formalized gold miners been able to change their behavior in ways that demonstrate USAID´s contribution in this sector to the J2SR? If the benefits of formalization cited above have not provided sufficient incentives for ASGM miners to change mining and environmental behavior in a majority of 93 We based this section on the Scope of Work for the final Performance Evaluation of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal). 161 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV cases, what underlying factors are preventing uptake of formalization and what other approaches can be identified/suggested to increase interest/uptake and under what requisite conditions? 2. Mining Governance and Policy: What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? Context and guidelines to answer the question: This is a follow-on to question #2 from the Mid￾term Evaluation. OL’s Theory of Change (ToC) proposes that the Activity will strengthen mining governance and policy, including at the municipal level. However, mining policy is a national purview, which is a flaw in the ToC not identified during the Mid-term Evaluation. Municipal authorities are only responsible for administration of the mining registry (RUCOM), have limited environmental authority, and lack the budget and means to enforce the mining and other laws against illicit armed groups linked to gold mining. Also, Antioquia is the only department with a Mining Secretariat with delegated mining authority by the Ministry of Energy (ME), whereas the department of Chocó does not enjoy similar status. Additionally, central government capacity to facilitate formalization is limited because of the sheer number of ASGM and other informal/illegal mining operations in Colombia, difficulties experienced in transferring operational responsibility for ASGM formalization from the ME to the National Mining Agency (ANM), and hesitant political will for effective mining policy reform. Given this context, it is critical to understand how effective OL’s approaches (both formal and informal) have been to policy and institutional capacity strengthening and what factors supported or hindered the Activity’s effectiveness in this area. The evaluation will suggest or recommend other approaches where it is relevant to do so. 3. Value Chain Development and Alternative Livelihoods: How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well￾being outside the ASGM activities? Context and guidelines to answer the question: This is a follow-on to question from the Mid￾term Evaluation. The economic (not simply financial) and/or environmental rationale for offering alternative livelihoods to ASGM families is to: (i) provide an alternative licit source of income; (ii) reduce the disproportionate environmental impact of ASGM in tropical ecosystems; (iii) reduce the pressure on vulnerable populations from illegal armed groups linked to illicit gold mining; and (iv) consolidate broader regional security. Economic reconversion includes a set of real and perceived values, benefits, and costs beyond investment in time and inputs and income generation from the economic activity per se; e.g., peace of mind, quality of life, more time to devote to the family (particularly relevant for single women heads of household), personal health and safety, higher productivity, participation in a legal marketplaces, etc., which are often hard to quantify, but are paramount for stakeholders and contribute to broader Government of Colombia (GOC and USAID/Colombia development goals, objectives and priorities and, therefore, should be considered in answering this question. 4. Rehabilitation of Areas Degraded by Illegal Mining: From the environmental, social, and economic point of view, how appropriate and effective have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 162 Context and guidelines to answer the question: Rehabilitation initiatives are one of the most visible and potentially valuable USAID contributions under Oro Legal that were not properly addressed during the Mid-term Evaluation. Large-scale degradation of land and water resources by unregulated alluvial mining and river dredging is one of the most significant environmental problems in Antioquia and Chocó, with particularly serious impacts to critical and fragile ecosystems along the Cauca and Atrato and Quito rivers and tributaries, respectively. Most rehabilitation or reclamation models are complex and costly, and not suitable or economically viable for remedying impacts in degraded areas considered “environmental orphans” by the state, communities, and landowners. The Oro Legal models were purposely designed to be low-cost, practical approaches that are appropriate for private landowners and collective communities, and particularly relevant for global and national initiatives in forest restoration and climate change mitigation. Are the models cost-effective in comparison to other approaches employed in Colombia and elsewhere? Are they good for each regional reality? How robust are the models in quickly reestablishing vegetative cover, reversing degradation, and recovering biodiversity? How committed are beneficiaries to future maintenance of the models over the medium to long term? What conditions are required and/or what changes could be made to the models to make them more effective and replicable? 5. Elimination/Reduction of Mercury from the ASGM Supply Chain: What combination of instruments and approaches employed in the effort to reduce mercury usage in the ASGM have proven to be most effective and why? Context and guidelines to answer the question: Mercury has historically been used in ASGM, particularly by small-scale alluvial miners and river dredgers, as a cheap and effective amalgam to recover fine gold. Exposure to mercury is extremely harmful to humans and one of the major public health and environmental risks both in and downstream from areas where gold mining and processing are concentrated. Colombia is a signatory to the MINAMATA Convention on Mercury, a global treaty to protect human health and the environment from the adverse effects of mercury. Colombia also promulgated its own Mercury Law that bans all use of mercury in mining as of July 2018 and mandates financial support to mining operators for the transition to non-mercury processing, a provision of the law that has not been implemented. Through rigorous monitoring methods and documentation, Oro Legal beneficiary MPUs have eliminated or significantly reduced the amount of mercury used per gram of gold produced. How was this achieved and what were the motivations of beneficiary MPUs? What combination of instruments/approaches employed was most impactful? To what extent did external factors contribute to or hinder the Activity’s initiatives to reduce or eliminate the use of mercury? 6. Perceived Relevance and Impact of the Activity: What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal’s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Context and guidelines to answer the question: The end of Oro Legal is an appropriate time for objective reflection and to seek honest feedback from all key stakeholders on the Activity’s contribution to ASGM, impact on the environment, and the socio-economic development in its two main geographies and in Colombia more generally. Stakeholders’ perceptions are influenced by three factors: (i) their personal circumstances and world view; (ii) their interests and expectations; and (iii) how well informed and knowledgeable they are about the Activity’s objectives, implementation, and outcomes. Assessment of key stakeholder opinions and feedback on specific components or activities, as well as incorporating 163 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV broader cross-cutting considerations may uncover needs, expectations, and ideas that are otherwise hard to capture, which, in turn, could trigger awareness, adaption, recognition, innovation, and learning for improvement in the future. DATA SCHEDULE AND LOGISTICAL ARRANGEMENTS The evaluation approach has four phases; the first one is planning, where we define the work plan and the evaluation methodology. The second one is the fieldwork, where we collect, compile, and tabulate the data for further analysis. The third one is analyzing the data and producing the report and presentations. The final phase is divulging the information, where dissemination and final adjustments take place. The steps for each phase are the following: 1. Planning and trust-building between teams and the CLA approach a. Producing the work plan b. Defining the evaluation schedule and responsibilities c. Reviewing the information and scheduling meetings with the Implementing Partner d. Developing the evaluation design e. Presenting the methodology to the COR team f. Defining the requirements to hire the firm that collects the quantitative information g. Preparing and translating the work plan document: the CLA methodology will provide an opportunity to present the material and help identify critical messages h. Writing and translating the methodological document 2. Fieldwork a. Carrying out national-level interviews b. Presenting the Early Report to the COR team c. Hiring the firm that collects the quantitative and qualitative information from direct beneficiaries d. Developing, testing, adjusting, validating, and preparing quantitative and qualitative instruments for fieldwork e. Gathering qualitative information f. Gathering quantitative information 3. Information analysis and report production (report writing) a. Structuring the qualitative codebook b. Coding of qualitative data (NVIVO) c. Processing quantitative data (Stata) d. Analyzing information: the CLA team will help identify preliminary findings and unintended outcomes and consequences and triangulate evidence and reach conclusions e. Structuring the presentation with findings and conclusions f. Structuring the presentation of recommendations g. Preparing an introductory workshop to discuss the recommendations and implementation measures: the CLA approach helps to guide this workshop h. Drafting the preliminary assessment report i. Writing the final evaluation report j. Submitting databases and documentation 4. Socialization and final adjustments a. Implementing the dissemination plan: the CLA approach will help identify and engage stakeholders to increase the use of results and recommendations and package and share evaluation findings in accessible formats USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 164 b. Giving the final presentations c. Submitting the final report to the DEC Exhibit 68 shows an estimated schedule with each activity’s duration and the delivery dates for the products. The evaluation should take no longer than seven months. After submitting the work plan and the evaluation design reports, we will begin with fieldwork. For this purpose, an external consulting team will collect quantitative and qualitative information. We will inform USAID about any delays caused by external sources and constraints related to, for example, the COVID-19 pandemic or security conditions on the field. We will seek USAID’s approval if there is any need to reschedule activities and products. We may alter, move, or change the agenda in case some circumstances postpone actions. We can comply with the activities with fixed dates as long as none of the previous tasks/activities change. If any activity suffers a delay for any reason, we will postpone the scheduled dates by the same number of days of the delay. 165 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 166 Exhibit 68: Timetable 167 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV MEMBERS OF THE EVALUATION TEAM The people, positions, and responsibilities listed hereafter and contained in Exhibit 69 comprise the team assembled for this evaluation. In Annex: Members of the Evaluation Team there is a more detailed description of the evaluation team. 1. Evaluation Team Leader (Manuel Fernando Castro): • Lead the performance evaluation design, including the work plan, data collection and analysis methods, the sampling strategy, data collection tools, data analysis plan, and report outline. • Serve as the primary contact with USAID counterparts and the Implementing Partner. • Lead and manage the evaluation team operation. • Implement the methodology, the work plan, and the data analysis. • Lead the writing of the main technical deliverables and technical communications, including the Performance Evaluation. • Oversee the quality and timeliness of the data collection implemented by a local firm. • Oversee the anonymization process of qualitative and quantitative data. • Lead debriefs and presentations to USAID and the Implementing Partner. • Facilitate participatory evaluation methods with the Implementing Partner and other key stakeholders to promote collaboration, learning, and adapting. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. 2. Mining affairs Specialist (Jairo Herrera): • Support the methodology and instruments’ design to assess the mining affairs in the targeted municipalities receiving interventions. • Review of primary and secondary sources of information related to mining issues. • Contribute to the analysis of data for the performance evaluation, with ownership of the mining affairs issues analysis. • Support the Evaluation Leader in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 3. Environmental Specialist (María Andrea Rueda): • Support the methodology and instruments’ design to assess the environmental issues in the targeted municipalities receiving interventions. • Review of primary and secondary sources of information related to environmental issues. • Contribute to the analysis of data for the performance evaluation, with ownership of environmental issues analysis. • Support the Evaluation Leader in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 4. Value Chain Specialist (Luis Fernando Monroy): • Support the methodology and instruments’ design to assess the value chain issues in the targeted municipalities receiving interventions. • Review of primary and secondary sources of information related to value chain issues. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 168 • Contribute to the analysis of data for the performance evaluation, with ownership of the value chain issues analysis. • Support the Evaluation Leader in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 5. Social Specialist (Juliana Moreno): • Support the methodology and instruments’ design to assess the social issues in the targeted municipalities receiving interventions. • Review of primary and secondary sources of information related to social issues. • Contribute to the analysis of data for the performance evaluation, with ownership of the social aspect analysis. • Support the Evaluation Leader in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 6. Quantitative methods specialists (Oscar Quiroz): • Support the Evaluation Leader designing the Work Plan methodology, including a sampling methodology, designing the instruments for quantitative data collection, analysis, and drafting the final quantitative report. • Help oversee and ensure the quality of data collection for the performance evaluation. • Lead the analysis of quantitative data for the evaluation. • Support the triangulation of sources of information for the performance evaluation. • Support the Evaluation Director in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 7. Evaluation Specialist (Mauricio Aguilar): • Lead the evaluation’s design and methodology, sampling, and technical analyses. • Guide and coordinate the qualitative and quantitative data collection efforts with the data collection firm and the MEL Activity’s staff. • Lead planning in each stage of the evaluation, including a detailed list of activities, inputs needed, and a schedule. • Coordinate the deliverables and reporting, including the data, reports, and presentations. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 8. Qualitative methods specialist (Camila Chavarría): • Support the Evaluation Leader designing the Work Plan methodology, including a sampling methodology, designing the instruments for quantitative/qualitative data collection, analysis, and final report drafting. • Help oversee and ensure the quality of data collection for the performance evaluation. • Support the analysis of quantitative/qualitative data for the evaluation. • Support the triangulation of sources of information for the performance evaluation. 169 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV • Support the Evaluation Director in preparing reports and presentations for USAID and the Implementing Partner. • Perform the application of qualitative instruments in fieldwork for the qualitative data collection. • Oversee the anonymization process of qualitative and quantitative data. 9. MEL’s Lead Evaluation Expert (Orlando Gracia): • Serve as a liaison between the Evaluation Team and MEL’s COP and USAID’s COR. • Responsible for staffing, SoW development, financial management and reporting, staff direction, and oversight. 10. MEL’s Evaluation Specialist (Juan Guillermo Bedoya): • Support the Lead Evaluation Expert when liaising with MEL’s COP and USAID’s COR. • Give support to evaluation management. • Analyze indicators and Monitor data. 11. MEL’s Quantitative Specialist (Luisa Fernanda Cardona): • Support the Lead Evaluation Expert when liaising with MEL’s COP and USAID’s COR. • Give support to evaluation management. 12. MEL’s Qualitative analyst Specialist (Javier Rodríguez): • Refine the evaluation’s coding system based on the methodological design. • Sort and classify fieldwork information. • Encode and integrate interviews and other information. • Analyze qualitative data. 13. MEL’s GIS Specialist: • Provide technical expertise on spatial data collection, map series creation, compile available geospatial data sets, conduct spatial analysis and data visualization, and interpret aerial photos or satellite images. 14. MEL’s CLA Specialist: • Lead the tasks and efforts to ensure the evaluation findings and recommendations are transparent and disseminated to relevant stakeholders. • Lead the actions to ensure the evaluation is designed and implemented with a collaborative approach. 15. MEL’s Monitoring Specialist: • Analyze indicators and Monitor data. • Carry out an initial assessment of Monitor indicators for the evaluation. 16. Data collection firm (TBD): • Quantitative data collection. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 170 Exhibit 69: Evaluation Team94 EVALUATION MILESTONES As milestones for the present evaluation, the SoW specifies some essential deliverables, documents, and meetings; we will deliver the products in the sequence described below, taking all the necessary steps. The proposed milestones are: 1. Evaluation Work Plan. It will include: i. Draft schedule and logistical arrangements ii. List of members of the evaluation team, describing their roles and responsibilities iii. Evaluation milestones iv. Anticipated schedule of efforts to collect evaluation team data v. Locations and dates for piloting data collection efforts (if applicable) vi. Proposed evaluation methodology including selection criteria for comparison groups, if applicable; and vii. Evaluation Report outline (if different from the attached template) 2. Evaluation Design. We will deliver this document at the same time as the Work Plan. It should contain: i. A detailed evaluation design matrix ii. Draft questionnaires and other data collection instruments iii. A list of potential interviewees, the sites to be visited, and the proposed selection criteria with a justification iv. The limitations of the design v. The dissemination plans 94 Own source. 171 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 3. Briefing and Interim Meetings. We should hold a mid-term presentation with the COR, the MEL Specialists, and the implementing partner to report the evaluation’s progress, potential challenges, and emerging opportunities. We will also provide the COR/AOR periodic briefings and feedback on the team’s findings. 4. Workshop for Recommendations discussion. This workshop aims to discuss the summary of findings and conclusions with USAID and draft any requested recommendations collaboratively. 5. Draft Evaluation Report. This delivery should address each of the questions identified and follow the guidance provided in the SoW. 6. Final Presentations. We should hold a final presentation to discuss the summary of findings and conclusions (and recommendations, if applicable) with USAID. 7. Final Evaluation Report. The final evaluation report will respond and incorporate USAID’s comments to the final draft evaluation report and the presentations. 8. Submission of Dataset to the Development Data Library. The contractor must submit to the COR and the Development Data Library (DDL) a copy of any dataset created or obtained during or for this award. 9. Submission of Final Evaluation report. The contractor must submit the final evaluation report and its summary to the Development Experience Clearinghouse (DEC) within three months of USAID’s final approval. ANTICIPATED SCHEDULE OF TEAM DATA COLLECTION EFFORTS The data collection schedule has two phases. The first is related to the analysis of secondary sources that include, at least, the following documents: 1. The activity, monitoring, evaluation, and learning plan – AMELP for OL 2. Award documents, including the modification to include humanitarian assistance 3. The Quarterly Reports 4. The Annual Reports 5. The Annual Work Plans 6. Baseline Reports 7. The background information on OL components 8. Other secondary information provided by OL The second phase involves obtaining primary information. To this purpose, the evaluation team, through the Panagora counterpart, will carry out the due diligence and hire a firm to collect quantitative and qualitative data from direct beneficiaries. Also, the evaluation team will design, pilot qualitative instruments to make corresponding adjustments, and will train the firm interviewers Later, the evaluation team will gather the field information at different levels; in other words, it will conduct interviews, and other instruments to national and regional institutions, like government agencies, private companies, Afro-Colombian and indigenous organizations, as well as direct Oro Legal MPU. All the field information collected by the firm will be transcribed, coded, and analyzed. LOCATIONS AND DATES TO PILOT THE DATA COLLECTION EFFORTS Between November 9 and 20 (two weeks), we will carry out a pilot exercise to collect data; this activity will target stakeholders such as OL’s regional members, representatives from the Antioquia’s Governor’s office and several municipalities, national government agencies, and Afro-Colombian and USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 172 indigenous associations participating in the program. Concerning OL’s members, we will conduct a CLA workshop to address the evaluation questions. These interviews will provide inputs to fine-tune the data collection instruments and prepare an Early Report that we will draft in the second week of December. The Early Report will contain institutional interviews at the national level and others at the departmental (Antioquia) and local levels in some municipalities of Antioquia and Chocó For the following steps, it is relevant to mention that a data collection firm hired for this purpose will pilot the survey instruments with mining production units – MPU (MPU surveys) and direct beneficiaries (surveys and interviews) in 2021. PROPOSED EVALUATION METHODOLOGY The evaluation focuses on five of the six expected results of Oro Legal Activity, namely: Objective 1: Building effective governance capacity for gold mining activities • Capacity in government entities to enforce gold mining legislation strengthened (O1 - R1) • Participation of ASM organizations and indigenous and Afro-Colombian communities in formalization programs strengthened (O1 - R2) • Environmental impacts reduced through technical assistance and training (O1 - R3) Objective 2: Increasing the capacity of the Colombian Government (GOC) entities to address the environmental impact caused by illegal mining This evaluation comprises four stages to achieve its goals. The first stage is designing and planning; it involves reviewing and collecting available information and developing the evaluation matrix, the stakeholder map, the information collection instruments, and presenting the methodology. The second stage is carrying out the fieldwork; this begins with a pilot test of qualitative information collection and an early report. In this stage, we prepare and carry out the fieldwork. The third stage, called analysis and report, comprises coding, processing, and analyzing collected data. These actions should result in a workshop to present and discuss preliminary recommendations and prepare the final report based on their validation. The fourth and final stage is adjusting and communicating results. In this phase, we will produce final observations and adjustments and complete the dissemination plan. It is worth clarifying that Oro Legal works with a complex context, dealing with stakeholders from different sectors, nature, and hierarchy, and this evaluation should consider these nuances. An accurate contextual assessment will help us understand how OL achieves results, complies with deliverables, and satisfies programmatic objectives. Additionally, we recognize that the strategy’s actions are not independent of their context. In other words, Oro Legal acts in regions that have dynamics of their own, thus demanding us to evaluate and determine if the activities are relevant to the local context. We will make use of mixed methods (qualitative and quantitative) to achieve our objectives, and we will receive support from both MEL’s CLA (Communications, Learning and Adapting) and GIS (Geographic Information Systems) teams. Quantitative and qualitative tools are complementary, compensate limitations of each used separately, and allow for further triangulation of sources; primary and secondary information sources will supply both tools. 173 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Quantitative methods allow for detailed and inferential data analysis to record and interpret data and their relationships. As a result, information reported will allow characterization and interpretation, leading to a descriptive analysis covering different variables, such as socio-demographic characteristics. We will complement this analysis with econometric variations analysis, associated factor models, and quasi-experimental methods. To that purpose, we will use primary information such as beneficiaries’ surveys, and administrative records. Qualitative methods focus on narratives and perceptions that beneficiaries and stakeholders may have of the intervention, its significance, and the perceived influence from the environment. These will also help identify the strengths, opportunities, weaknesses, and threats of the intervention. Moreover, we will collect primary information through semi-structured interviews at national, regional, and local levels, while secondary information will consist of strategic documents, diagnostic documents, and annual performance reports. Furthermore, we will manage all the information under the highest data quality control standards. Also, the CLA approach will inform the whole process, using confidence-peer-review strategies. Consequently, we expect to learn from the fieldwork and put together relevant recommendations for decision-makers designing an upcoming activity to build on OL’s actions. Finally, we will use GIS tools to interpret available spatial and geostatistical information for a georeferenced analysis. EVALUATION REPORT SUGGESTED OUTLINE Abstract Executive Summary 1. Evaluation Purpose 2. Activity Description and Background 3. Evaluation Questions and Methodology 4. Evaluation Findings and Conclusions 5. Evaluation Recommendations 6. Annexes I. Recommendations Matrix II. Supplemental Analysis and Data III. Findings, Conclusions, and Recommendations Table IV. Detailed Methodology V. Instruments VI. Sources of Information VII. Evaluation SOW VIII. Summary Information about Evaluation Team Members IX. Signed Disclosures of Conflicts of Interest from Evaluation Team Members. X. Statement of Differences (when applicable) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 174 MEMBERS OF THE EVALUATION TEAM Exhibit: 70: Evaluation Team 175 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV EVALUATION DESIGN INTRODUCTION The Artisanal Gold Mining - Environmental Impact Reduction Activity - Oro Legal – is financed by the United States Agency for International Development (USAID) and implemented by Chemonics International. It supports Colombian efforts to reduce the social and environmental impacts of artisanal and small gold mining (ASGM) in twenty-two municipalities within the departments of Antioquia and Chocó. Oro Legal works through the legalization and formalization of illegal mining, capacity building and strengthening, the endorsement of more appropriate technology and improved practices, development of new legal routes to market, and the promotion of sustainable alternatives to mining. Oro Legal works closely with the Government of Colombia (GOC), non-governmental entities (NGOs), communities, and the private sector to achieve a set of indicator targets. As part of the Activity, and after approximately five (5) years of implementation, USAID is conducting a final performance evaluation to assess: if Oro Legal has achieved its deliverables and objectives, to what extent the achievements at the different sub-objectives of the intervention are perceived effective and impactful, and which factors may be playing a role to modulate the performance toward those sub￾objectives, the Activity’s main goal, and USAID’s strategic development goals. The evaluation will support USAID/Colombia and Oro Legal Implementing Partner (IP) accountability among stakeholders. The evaluation will also provide lessons learned and insightful recommendations to help USAID/Colombia make programming decisions for future similar activities. It will also share knowledge gained with relevant stakeholders in the Government of Colombia (GOC), Mining Sector, International Donors, and Communities. This document describes the designed methodology for the Activity’s performance evaluation. The report has seven sections. 1. The Evaluation Matrix includes a synthesis of the questions to be answered. It also lists the indicators, agents involved, information sources, and the instruments for the evaluation. 2. The Quantitative Approach 3. The Qualitative Approach 4. The Triangulation Strategy 5. The Spatial-analysis Approach 6. The Evaluation’s Limitations 7. The Dissemination Plan BACKGROUND95 Before implementing Oro Legal, gold production in the country had a growth of 327% between 2007 and 2012, going from 15.5 tons to 66.2 tons due to the increase in international prices of this mineral. 95 We based this section on the revised MEL Plan of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 176 This disproportionate increase overwhelmed the authorities’ institutional capacity to manage and regulate the mining resource exploitation. The Departmental Mining Census, produced between 2010 and 2011 by the Ministry of Mines and Energy (MME), showed that 72% of the Mining Productive Units (MPU) surveyed were small-scale, and 63% did not have a mining title, despite the different mining legalization programs carried out between 1969 and 2013. On the other hand, Law 99 of 1993, which created the National Environmental System - SINA-, established the obligation to obtain an environmental license for projects that may damage natural resources; however, this mining census reported that 75.7% of the MPUs did not have any environmental authorization. Governance flaws are evident in mining regions, where mining or environmental authorities do not oversee mining practices. This lack of supervision and high business informality created an opportunity for different illegal actors to hijack vulnerable gold exploitations. This situation led to a significant environmental and socio-economic deterioration in the unlawful mining regions, which adds to pre￾existing risks to human health generated by artisanal mining practices. Along with the above, the most recent report on Evidence of Alluvial Gold Exploitation –EVOA-, carried out by the United Nations Office on Drugs and Crime –UNODC- (2019), highlights that 57% of the exploitation of alluvial gold is outside the law. This exploitation increased in the targeted areas, going from 83,620 hectares in 2016 to 98,028 hectares in 2019; 78% of these areas are in the departments of Chocó and Antioquia (UNODC, 2020). Additionally, the first alert on money laundering through illegal mineral extraction was reported by the National Tax and Customs Directorate -DIAN- in 2012, indicating that illicit groups launder at least ten billion dollars per year using the gold market, which to date represented 3% of the country’s GDP. According to official production records, about 80% of the gold comes from artisanal and small-scale miners through illicit extractors; however, by the beginning of the 2010 decade, there were no technical or legal tools to differentiate these two actors. Two investigations laid the foundations to develop these tools: the first, published by the United Nations Environment Program -UNEP- and the Ministry of the Environment in 201296; and the second published by the Inter-American Development Bank -BID- and Fedesarrollo, between 2012 and 2014 (Sabogal, 201297; Goñi, Sabogal and Asmat98, 2014). The results presented in these reports set the foundations for the Mining Formalization Policy issued by the MME in 201499, modified in 2016100 due to the National Development Plan 2014-2018 (Law 1753 of June 9, 2015 ) regarding the need to characterize artisanal and small-scale gold mining (ASGM) and to differentiate it from so-called illegal mining. 96 PNUM and MADS (2012). National synopsis of artisanal and small-scale gold mining. Bogotá 97 Sabogal, A. (2012). Creating a baseline for illegal gold mining in Colombia: methodological report. Economic and Social Research Center – Fedesarrollo, Bogotá. 98 Goñi, E., Sabogal, A., and Asmat, R. (2014). Informal gold mining in Colombia: main results creating a baseline. Bogotá, D.C., Fedesarrollo and IADB. 99 MME, (2014). Resolution 90719, July 8, 2014. 100 MME, (2016). Resolution 40391, April 20, 2016. 177 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV In support of the Mining Formalization Policy, the Ministry of Environment and Sustainable Development (MADS) produced an environmental guide101 for traditional mining activities. This guide was recently modified by Resolutions 447 and 448 of May 2020, which issue the terms of reference to prepare the environmental impact study –EIA- required in the global or definitive licensing process for small mining projects and the environmental temporary license process for mining formalization. As stated by the BID and Fedesarrollo (Goñi, Sabogal, and Asmat, 2014), ASGM takes place fundamentally in regions with armed groups that control illegal economies such as illicit mineral extraction, mercury trafficking, cocaine production, logging, and land appropriation, among others. Regarding the use of mercury, according to studies carried out by the United Nations Industrial Development Organization -ONUDI-,102 the urban area of the municipality of Segovia in Antioquia showed the highest levels of mercury in air recorded in the world. This problem motivated Colombia to sign the Minamata Agreement and issue Law 1658 of 2013, which granted a non-extendable period of five years, completed in July 2018, to eliminate this chemical element from the gold extraction process. In addition to the significant governance conflicts in the country’s mining regions due to groups outside the law, there are also the environmental impacts generated by illegal extractors and small producers not trained in appropriate extractive techniques. Deforestation and contamination of water currents are two of the biggest concerns, especially in regions like Antioquia’s Bajo-Cauca basin and Chocó. Other significant environmental impacts are related to areas previously mined and now abandoned without the proper mitigation measures, named environmental liabilities.103 Finally, mining tends to discourage other rural activities; people in these areas tend to put aside other ventures to pursue an improved livelihood through mining projects, and many times this path does not lead to better conditions.104 This situation leaves behind an impoverished community with fewer licit economic alternatives and the pressure of illegal armed groups. Faced with this reality, USAID supports the Colombian government’s efforts to confront illegal activities, promote mining formalization, and provide alternative, sustainable, and participatory solutions to affected communities, promote responsible mining, and develop alternative economic activities and ecological restoration. This initiative complies with USAID’s strategic objectives that seek to contribute to capacity building and generate positive changes in behaviors with negative impacts and improve environmental resilience by strengthening the management of natural resources.105 The main goal of Oro Legal Activity is to help the GOC’s mining sector and the communities to address the informality of the ASGM to reduce its environmental liabilities through: 101 MADS, (2015). Resolution 1258, June 2015. 102 VEIGA, M. (2010). Antioquia, Colombia: the world’s most polluted place by mercury: impressions from two field trips. Report prepared by UNIDO (ONUDI). 103 Environmental liabilities refer to the environmental damages that group water, soil, and air pollution, and the destruction of resources and ecosystems caused by a business venture’s everyday work or sudden accidents (MADS). 104 Ayala, H. et al. (2019). Diagnosis of environmental and social information regarding mining activity and illegal extraction of minerals in the country. Scientific and sociological research document regarding the impacts of mining activity and the illicit exploitation of minerals in Colombia’s ecosystems. 105 Intermediate result 4.1 – CDCS 2014-2020. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 178 1. Building effective governance capacity for gold mining activities via: a. Strengthening Colombian government capacity to enforce gold mining legislation. b. Enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs. c. Providing training and technical assistance to artisanal miners. 2. Increasing the capacity of GOC entities to address the environmental impact caused by illegal mining through: a. Restoration of degraded areas. b. Generating alternative livelihoods for communities that cannot or should not be involved in gold mining. c. Improve drinking water quality in mining areas. This Oro Legal Activity contract totaling almost $22.1 million follows the mining component of the previous USAID/Colombia BioREDD+ activity implemented by Chemonics from July 2013 to May 2015. Oro Legal has two main objectives. The first is to build effective governance capacity for gold mining activities via: (i) strengthening Colombian government capacity to enforce gold mining legislation; (ii) enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs and; (iii) providing training and technical assistance (TA) to artisanal miners. The second objective is to increase the capacity of Colombian government agencies, communities, and the private sector to address the environmental impact caused by illegal mining through: (i) rehabilitating degraded areas; (ii) generating diversified sources of income/alternative livelihoods for communities that cannot or should not be involved in gold mining and; (iii) improving drinking water quality in mining areas. Oro Legal works with a diverse group of stakeholders, including national and regional governments, municipal administrations, Regional Autonomous Corporations (CAR), mining and producer associations, private mining companies, and other initiatives working in the regions related to Oro Legal’s activities. Oro Legal’s two main objectives group together a set of activities under the six expected results described below: ER1.1. Provide selective support to the GOC aimed at political and legal reform to create a strengthened framework to enforce mining laws and regulations more effectively. ER 1.2. Identify and support mining production units (MPUs), associations, and community councils (CCs) to strengthen their participation in mining formalization in compliance with the current GOC regulatory framework and strengthen their administrative and technical capacities. ER 1.3. Provide technical assistance (TA) and training for MPUs and associations and CCs to improve their environmental and social performance, especially to reduce and eventually eliminate the use of Mercury (Hg) in gold (Au) extraction. ER 2.1. Strengthen nationally appropriate mitigation activities (NAMAs) through the rehabilitation of areas degraded by mining via public-private partnerships under long-term agreements and benefit 179 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV distribution schemes involving vulnerable populations, reducing environmental liabilities, and increasing carbon capture forestry plantations. ER 2.2. Support a limited portfolio of value chains that can be taken to scale to provide diversified incomes for communities and families in target areas. ER 2.3. Support municipal agencies and other stakeholders to adopt sound integrated water resources management (IWRM) activities to improve water resources’ quality in targeted municipalities. The activities mentioned above will incorporate crosscutting elements related to gender equality, social inclusion, and the communication and dissemination of improved methodologies and practices. The implementation of activities will prioritize support for women with high exposure to environmental health risks resulting from their mining activities. Oro Legal will also develop strategies to promote income diversification/alternate livelihoods, particularly those that expand women’s economic opportunities. PURPOSE The purpose of this final performance evaluation is to conduct a systematic analysis of the performance of the Oro Legal Activity to assess to what extent the achievements at the different sub-objectives of the intervention are perceived effective and impactful, and which factors may be playing a role to modulate the performance toward those sub-objectives, the Activity´s main goal, and USAID´s strategic development goals. To this end, the methodological approach considers the support provided by Oro Legal (OL) to the following targets: 1. Reduce illegal activity and environmental devastation caused by illegal mining. 2. Generate economic alternatives to illegal mining (honey production in Antioquia and annatto production in Chocó). 3. Rehabilitate highly degraded lands using reforestation with Acacia maginum in Antioquia and assisted natural regeneration in Chocó. 4. Introduce technologies and practices capable of eliminating the use of mercury. In the evaluation, we also contemplate contextual information that may affect the sustainability of results and identifies the main perceptions regarding OL intervention’s relevance and effectiveness among direct and indirect stakeholders. EVALUATION QUESTIONS106 Considering the Oro Legal Activity purpose stated above, we will conduct a systematic analysis of the Activity’s performance and progress towards its main goals to answer six (6) main questions. Each of the 106 We based this section on the Scope of Work for the final Performance Evaluation of USAID/Colombia Artisanal Gold Mining Activity (Oro Legal). USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 180 questions will examine several contextual elements to guide the answers. The questions are the following: 1. Mining Formalization: To what extent do the application by Mining Production Units (MPU) of the formalization standards, improved operational efficiency, and access to legal markets gained through Oro Legal’s intervention provide sufficient incentives for miners involved in ASGM to remain legal/formal and improve the environmental performance of their mining operations? Context and guidelines to answer the question: This is a follow-on to question #3 from the Mid￾term Evaluation, adjusted to capture the outcome of the contribution of mining formalization towards one of the main goals of the Activity. Given a national context marked by a steady increase in international gold prices, local pressure from illegal actors, and a widespread culture of unlawfulness, have formalized gold miners been able to change their behavior in ways that demonstrate USAID´s contribution in this sector to the J2SR? If the benefits of formalization cited above have not provided sufficient incentives for ASGM miners to change mining and environmental behavior in a majority of cases, what underlying factors are preventing uptake of formalization and what other approaches can be identified/suggested to increase interest/uptake and under what requisite conditions? 2. Mining Governance and Policy: What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? Context and guidelines to answer the question: This is a follow-on to question #2 from the Mid￾term Evaluation. OL’s Theory of Change (TOC) proposes that the Activity will strengthen mining governance and policy, including at the municipal level. However, mining policy is a national purview, which is a flaw in the TOC not identified during the Mid-term Evaluation. Municipal authorities are only responsible for administration of the mining registry (RUCOM), have limited environmental authority, and lack the budget and means to enforce the mining and other laws against illicit armed groups linked to gold mining. Also, Antioquia is the only department with a Mining Secretariat with delegated mining authority by the Ministry of Energy (ME), whereas the department of Chocó does not enjoy similar status. Additionally, central government capacity to facilitate formalization is limited because of the sheer number of ASGM and other informal/illegal mining operations in Colombia, difficulties experienced in transferring operational responsibility for ASGM formalization from the ME to the National Mining Agency (ANM), and hesitant political will for effective mining policy reform. Given this context, it is critical to understand how effective OL’s approaches (both formal and informal) have been to policy and institutional capacity strengthening and what factors supported or hindered the Activity’s effectiveness in this area. The evaluation will suggest or recommend other approaches where it is relevant to do so. 3. Value Chain Development and Alternative Livelihoods: How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well￾being outside the ASGM activities? Context and guidelines to answer the question: This is a follow-on to question from the Mid￾term Evaluation. The economic (not simply financial) and/or environmental rationale for offering alternative livelihoods to ASGM families is to: (i) provide an alternative licit source of income; (ii) reduce the disproportionate environmental impact of ASGM in tropical ecosystems; (iii) reduce the pressure on 181 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV vulnerable populations from illegal armed groups linked to illicit gold mining; and (iv) consolidate broader regional security. Economic reconversion includes a set of real and perceived values, benefits, and costs beyond investment in time and inputs and income generation from the economic activity per se; e.g., peace of mind, quality of life, more time to devote to the family (particularly relevant for single women heads of household), personal health and safety, higher productivity, participation in a legal marketplaces, etc., which are often hard to quantify, but are paramount for stakeholders and contribute to broader Government of Colombia (GOC and USAID/Colombia development goals, objectives and priorities and, therefore, should be considered in answering this question. 4. Rehabilitation of Areas Degraded by Illegal Mining: From the environmental, social, and economic point of view, how appropriate and effective have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? Context and guidelines to answer the question: Rehabilitation initiatives are one of the most visible and potentially valuable USAID contributions under Oro Legal that were not properly addressed during the Mid-term Evaluation. Large-scale degradation of land and water resources by unregulated alluvial mining and river dredging is one of the most significant environmental problems in Antioquia and Chocó, with particularly serious impacts to critical and fragile ecosystems along the Cauca and Atrato and Quito rivers and tributaries, respectively. Most rehabilitation or reclamation models are complex and costly, and not suitable or economically viable for remedying impacts in degraded areas considered “environmental orphans” by the state, communities, and landowners. The Oro Legal models were purposely designed to be low-cost, practical approaches that are appropriate for private landowners and collective communities, and particularly relevant for global and national initiatives in forest restoration and climate change mitigation. Are the models cost-effective in comparison to other approaches employed in Colombia and elsewhere? Are they good for each regional reality? How robust are the models in quickly reestablishing vegetative cover, reversing degradation, and recovering biodiversity? How committed are beneficiaries to future maintenance of the models over the medium to long term? What conditions are required and/or what changes could be made to the models to make them more effective and replicable? 5. Elimination/Reduction of Mercury from the ASGM Supply Chain: What combination of instruments and approaches employed in the effort to reduce mercury usage in the ASGM have proven to be most effective and why? Context and guidelines to answer the question: Mercury has historically been used in ASGM, particularly by small-scale alluvial miners and river dredgers, as a cheap and effective amalgam to recover fine gold. Exposure to mercury is extremely harmful to humans and one of the major public health and environmental risks both in and downstream from areas where gold mining and processing are concentrated. Colombia is a signatory to the MINAMATA Convention on Mercury, a global treaty to protect human health and the environment from the adverse effects of mercury. Colombia also promulgated its own Mercury Law that bans all use of mercury in mining as of July 2018 and mandates financial support to mining operators for the transition to non-mercury processing, a provision of the law that has not been implemented. Through rigorous monitoring methods and documentation, Oro Legal beneficiary MPUs have eliminated or significantly reduced the amount of mercury used per gram of gold produced. How was this achieved and what were the motivations of beneficiary MPUs? What USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 182 combination of instruments/approaches employed was most impactful? To what extent did external factors contribute to or hinder the Activity’s initiatives to reduce or eliminate the use of mercury? 6. Perceived Relevance and Impact of the Activity: What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal’s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Context and guidelines to answer the question: The end of Oro Legal is an appropriate time for objective reflection and to seek honest feedback from all key stakeholders on the Activity’s contribution to ASGM, impact on the environment, and the socio-economic development in its two main geographies and in Colombia more generally. Stakeholders’ perceptions are influenced by three factors: (i) their personal circumstances and world view; (ii) their interests and expectations; and (iii) how well informed and knowledgeable they are about the Activity’s objectives, implementation, and outcomes. Assessment of key stakeholder opinions and feedback on specific components or activities, as well as incorporating broader cross-cutting considerations may uncover needs, expectations, and ideas that are otherwise hard to capture, which, in turn, could trigger awareness, adaption, recognition, innovation, and learning for improvement in the future. STRATEGY The evaluation focuses on five of the six expected results107 of Oro Legal Activity, namely: Objective 1: Building effective governance capacity for gold mining activities • Capacity in government entities to enforce gold mining legislation strengthened (O1 - R1) • Participation of ASM organizations and indigenous and Afro-Colombian communities in formalization programs strengthened (O1 - R2) • Environmental impacts reduced through technical assistance and training (O1 - R3) Objective 2: Increasing the capacity of the Colombian Government (GOC) entities to address the environmental impact caused by illegal mining • Reforestation NAMA supported in areas degraded by unauthorized ASM operators (O2 - R1) • Alternative livelihoods for ASM communities promoted (O2 - R2) This evaluation comprises four stages to achieve its goals. The first stage is designing and planning; it involves reviewing and collecting available information and developing the evaluation matrix, the stakeholder map, the information collection instruments, and presenting the methodology. The second stage is carrying out the fieldwork; this begins with a pilot test of qualitative information collection and an early report. In this stage, we prepare and carry out the fieldwork. The third stage of analysis and report comprises the coding, processing, and analysis of collected data. This stage includes a workshop to present and discuss preliminary recommendations and prepare the final report after validation. The 107 USAID/Colombia (2020). Scope of work (SOW) for the final performance evaluation of USAID/Colombia artisanal gold mining activity (Oro Legal). 183 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV fourth and final stage sets the final presentations and communicates the results, conclusions, and recommendations; this phase includes refining the dissemination plan. Several other elements are essential to clarify. First, it is vital to point out that Oro Legal deals with a complex context, encompassing different stakeholders. Thus, on the one hand, this evaluation must comprise the agencies at the national, departmental, and municipal levels; it must also involve all pertinent actors, including groups enabling the strategy and direct beneficiaries in the targeted regions. This work will help determine if OL is achieving results, complying with deliverables, and assessing the intervention’s added value. It is worth clarifying that Oro Legal works with a complex context, dealing with stakeholders from different sectors, nature, and hierarchy, and this evaluation should consider these nuances. An accurate contextual assessment will help us understand how OL achieves results, complies with deliverables, and satisfies programmatic objectives. Additionally, we recognize that the strategy’s actions are not independent of their context. In other words, Oro Legal acts in regions that have dynamics of their own, thus demanding us to evaluate and determine if the activities are relevant to the local context. We will make use of mixed methods (qualitative and quantitative) to achieve our objectives, and we will receive support from both MEL’s CLA (Communications, Learning and Adapting) and GIS (Geographic Information Systems) teams. Quantitative and qualitative tools are complementary, compensate limitations of each used separately, and allow for further triangulation of sources; primary and secondary information sources will supply both tools. Quantitative methods allow for detailed and inferential data analysis to record and interpret data and their relationships. As a result, information reported will allow characterization and interpretation, leading to a descriptive analysis covering different variables, such as socio-demographic characteristics. We will complement this analysis with econometric variations analysis, associated factor models, and quasi-experimental methods. For that purpose, we will use primary information such as beneficiaries’ surveys and administrative records. Qualitative methods focus on narratives and perceptions that beneficiaries and stakeholders may have of the intervention, its significance, and the perceived influence from the environment. These will also help identify the strengths, opportunities, weaknesses, and threats of the intervention. Moreover, we will collect primary information through semi-structured interviews at national, regional, and local levels, while secondary information will consist of strategic documents, diagnostic documents, and annual performance reports. Furthermore, we will manage all the information under the highest data quality control standards. Also, the CLA approach will inform the whole process, using confidence-peer-review strategies. Consequently, we expect to learn from the fieldwork and put together relevant recommendations for decision-makers designing an upcoming activity to build on OL’s actions. Finally, we will use GIS tools to interpret available spatial and geostatistical information for a georeferenced analysis. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 184 DETAILED EVALUATION DESIGN MATRIX We will use the evaluation design matrix (Table 18) to link the evaluation questions to the programmatic components of Oro Legal. It describes the variables, indicators, and tools proposed to measure, the stakeholders to interview, the main and ancillary data sources, and the quantitative and qualitative instruments intended to apply. The matrix summarizes/condenses the evaluation’s scope and objectives; it serves as a roadmap for the ET to address each question systematically. Once finished the technical peer-review of the matrix, the pilot test and fieldwork will take place. 185 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT 1 To what extent do the application by Mining Production Units of the formalization standards, improved operational efficiency, and access to legal markets (MPU) gained through OL’s intervention provide sufficient incentives for miners involved in artisanal and small gold mining (ASGM) to remain legal/formal and improve the environmental performance of their mining operations? Participation of ASM organizations in formalization programs -O1 R2 * Operational efficiency -Banking of MPUs and miners -Administrative and financial management -Mining technification -Reduction in operating costs * Access to legal markets -Mining titles and license -Occupational health and safety (Formal jobs by MPUs) * Environmental performance -Dumping management -Disposal of tailing (mercury and cyanide) -Technification of clean practices * Incentives for legality/formality -Price stability -Increase in the sale price -Perception of security -Type of buyer and new buyers -Increase in MPU production due to formalization Mixed National: *OL Staff * ANM * MME - Mining Formalization Coordination * MADS - Sectional Directorate * BGI-Better Gold Initiative * WWF Colombia * Anexpo SAS * Sun Valley – Mining formalization project Regional-Departmental: * OL Staff in Antioquia * OL Staff in Chocó * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Corantioquia * Cornare * Codechocó * IIAP: Pacific Environmental Research Institute Municipal: * Mining Secretaries (Mining/Planning/Environment/Development Secretaries) * Gramalote * MEL Plan * Monitor * SIME (OL's information system) * Data bases (O1 y O2) * Annual and quarterly reports * White Paper * MPU information (participants and non￾participants) * Institutional stakeholders' interviews * Beneficiarie’s interviews * Mining cadaster (ANNA MINERÍA) *Commitments/Agreements between MPUs and buyers *Monthly briefings *OL communication material *Surveys * Surveys and databases - Descriptive analysis, proportion z-test and t-student test - Econometric model of mining formalization determinants: * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis (context and meanings) * SWOT USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 186 TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Monthly income from MPUs and households -Profitability of MPU * Mineros S.A. * Mining Projects * Mining formalization/legalization beneficiaries 2 What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? Government capacity to enforce gold mining legislation strengthened - O1-R1 * Internal factors -MPUs linked to the program (Beneficiary linkage strategy) -Quality and scope of technical assistance -Relationship of OL’s MPUs with public agencies and external actors -Linking mining and environmental authorities to the OL formalization process -Internal organization and technical capacity of OL -Adaptability of OL to environmental conditions and needs of the territory (knowledge of beneficiaries and background) -Contribution to institutional and regulatory strengthening processes -Funders' perception of trust (USAID) * External factors -Clarity of the mining and environmental regulatory framework Qualitative National: * OL Staff * MME * MADS * Ministry of the Interior * ANM Regional-Departmental: * OL Staff in Antioquia * OL Staff in Chocó * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Corantioquia * Cornare * Codechocó * IIAP: Pacific Environmental Research Institute Municipal: * Municipal mayor's office * Mining projects * Institutional interviews * Interviews to participants and non-participants * Annual and quarterly reports * Monitor * SIME (OL's information system) * Data bases (O1 y O2) * Monthly briefings * OL communication material * White Paper * Secondary information - Text mining analysis * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis (context and meanings) * SWOT 187 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Mining and environmental regulatory knowledge (officials and miners) -Management processes and transparency and accountability framework -Technical and financial capacity of institutions and officials -Social and environmental perception of mining in the regions, before and after the intervention of OL. -Knowledge of the reality of the territory by officials -Trust in state institutions -Security and peace conditions in the OL municipalities * Implementers * OL Beneficiaries 3 How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well-being outside the ASGM activities? Alternative livelihoods of ASM communities promoted – O2- R2 * Alternative legal sources of income -New household income derived from value chains -Regular sales of associative productive initiatives -Size and productivity of the UPA (Agricultural Production Unit) -Technification of the productive activity -Working household members -Women's economic participation Mixed National: * Oro Legal * ACUMEN * Fundación IC * WWF Colombia * ICCO Regional-Departmental: * OL Staff in Antioquia * OL Staff in Chocó * MEL Plan * Monitor * SIME (OL's information system) * Data bases (O1 and O2) * Annual and quarterly reports Information from participants and non- participants * Interviews to participants and non-participants * Interviews to institutional stakeholders * Surveys and databases - Descriptive analysis, variable association and grouping observations -Econometric model of income determinants for beekeeping and annatto production * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis (context and meanings) * SWOT USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 188 TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT *Sustainability of legal production alternatives -Sustainability perception of the productive chain supported -Linkages with formal markets -Social and business skills of the beneficiaries and organizations -Access to technical assistance *Reduction of environmental impact -Perception of local environmental improvement -Perception of mining labor reconversion by productive alternatives of OL -Perception of socioeconomic stability associated to the productive activity of OL *Regional security consolidation due to OL -Security perception of the environment *Reduction of pressure on vulnerable populations by illegal armed groups linked to illegal gold mining -Perception of pressure from illegal armed groups on beneficiaries * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Fundación Mineros * Fundación Oleoductos de Colombia * Custodiar SA Municipal: * Mining Secretaries (Mining/Planning/Environment/Development Secretaries) * Annatto’s implementers * Beekeeper's implementers * Annatto’s beneficiaries * Beekeeper's beneficiaries Commercial allies: *Naranja Madura (annatto) *Color Química (annatto) *Campo Dulce (honey) *LOK (honey) * Baselines *Monthly briefings *OL communication material *Surveys 189 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Perception of trust and strengthening of social relations *Effects on well-being and quality of life -Perception of improvement in the household economy -Perception of improvement in time dedicated to the family (gender approach) -Perception of improvement in access to basic services -Perception about tranquility (social peace) -Perception of improvement in physical and personal safety (decrease of occupational risks) 4 How appropriate and effective since the environmental, social, and economic point of view have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? Rehabilitation of Areas Degraded by Illegal Mining. – O2 – R1 *Relevance of rehabilitation models -Adaptation to territorial realities -Model of assimilation by beneficiaries -Areas intervened by degradation of gold extraction -Alignment with sectoral policy *Environmental rehabilitation effectiveness Mixed National: * OL Staff * MADS Regional-Departmental: * OL Staff in Antioquia * OL Staff in Chocó * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Interviews to participants and non-participants * Interviews to institutional stakeholders * Annual and quarterly reports * Monitor * SIME (OL's information system) * Data bases (O1 y O2) *Monthly briefings *OL communication material * Surveys and databases - Descriptive analysis of rehabilitation of degraded areas. - Econometric model of rehabilitation of degraded areas effectiveness (optional). * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 190 TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Replicability of rehabilitation models -Rehabilitated areas due to gold extraction degradation -Reduced environmental degradation -Consolidated Acacia forest plantations (cost of plantations) -Biodiversity recovered (Birds, amphibians, insects, etc) -Natural regeneration (Chocó) (perceptions of inhabitants) *Sustainability of rehabilitation models -Commitment and ownership by the beneficiaries to maintain actions in the medium and long term (territorial approach) -Beehives productivity -Community monitoring (gender focus) -Sustainability incentives (REDD projects) -Perception of social benefits associated with rehabilitation (territorial approach) -Complexity of rehabilitation models * Corantioquia * Cornare * Codechocó * IIAP: Pacific Environmental Research Institute Municipal: * Mining Secretaries (Mining/Planning/Environment/Development Secretaries) * Rehabilitation implementers * Rehabilitation beneficiaries * Secondary information - Text mining analysis (context and meanings) * SWOT 191 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Costs of rehabilitation models -Linkage of private and public actors 5 What combination of instruments and approaches employed in the effort to reduce mercury usage in the ASGM have proven to be most effective and why? Environmental impacts reduced (mercury use) through technical assistance and training – O1- R3 *Effect of legal prohibition *Education and training *Variation in price of mercury *Awareness of health risks *Awareness to environmental risks *Fostering the use of new technologies *Adoption / appropriation of new technologies by beneficiaries *Miners linked to the Zero Mercury Project *Changes in perception and behavior regarding the use of mercury *Mercury elimination or reduction motivation *Zero mercury pilot effectiveness * Degree of satisfaction of the MPU with the production after the implementation of the cleaner production techniques indicated by OL * Degree of technical or economic difficulty in Mixed National: * OL Staff * ANM * MADS * MME * WWF Colombia * Anexpo SAS * BGI-Better Gold Iniciative * CIIGSA Regional-Departmental: * OL Staff in Antioquia * OL Staff in Chocó * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Corantioquia * Cornare * Codechocó * IIAP: Pacific Environmental Research Institute Municipal: * MEL Plan * Monitor * SIME (OL's information system) * Data bases (O1 y O2) * Annual and quarterly reports * Information of participants and non￾participants * Interviews to participants and non-participants * Interviews to institutional stakeholders * Baselines * White paper * Monthly briefings * OL communication material * Environmental Assessment * Surveys * Surveys and databases - Descriptive analysis of mercury reduction in the ASGM supply chain - Econometric model of the determinants of mercury use in ASGM (optional) * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis (context and meanings) * SWOT USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 192 TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT implementing the cleaner production techniques indicated by OL *Clearness of the mining and environmental regulatory framework *Mining and environmental regulatory knowledge (officials and miners) *Security and peace conditions in OL municipalities *Internal organization and technical capacity of OL * Mining Secretaries (Mining/Planning/Environment/Development Secretaries) * Gramalote * Mineros S.A * Mining projects * Mining formalization/legalization beneficiaries 6 What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal´s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Perceived relevance and impact of OL – Crosscutting elements O1 y O2 *Knowledge of the activity * Formalization and legalization of mining -Level of relevance (environmental, socio- economic, mining activity) -Level of effectiveness (environmental, socio￾economic, mining activity) *Governance and mining policies -Level of relevance (environmental, socio- economic, mining activity) -Level of effectiveness (environmental, socio￾economic, mining activity) *Value chain development and alternative livelihoods Mixed National: * OL Staff * ANM * MADS * MME Ministry of the Interior * WWF Colombia *ACUMEN *Fundación IC *ICCO *Anexpo SAS *BGI-Better Gold Iniciative *CIIGSA Regional-Departmental: * OL Staff in Antioquia Interviews to participants and non-participants * Interviews to institutional stakeholders *Annual and quarterly reports *Stakeholder mapping and relationship strategy *Monthly briefings *OL communication material *Surveys * Surveys - Descriptive analysis and factor analysis * Semi structured interviews with national, regional, and municipal agents * Semi structured interviews with direct and indirect beneficiaries * Perception analysis (context and meanings) * SWOT 193 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT -Level of relevance (environmental, socio￾economic) -Level of effectiveness (environmental, socio￾economic) *Rehabilitation of degraded areas -Level of relevance (environmental, socio￾economic, mining activity) -Level of effectiveness (environmental, socio￾economic, mining activity) *Mercury elimination / reduction -Level of relevance (environmental, socio￾economic, mining activity) -Level of effectiveness (environmental, socio￾economic, mining activity) * OL Staff in Chocó * Antioquia Governor's Office (Antioquia's Mining Secretary and Antioquia´s Environment Secretary) * Chocó Governor's Office (Chocó´s Development Secretary) * Corantioquia * Cornare * Codechocó * IIAP: Pacific Environmental Research Institute * Fundación Mineros * Fundación Oleoductos de Colombia * Custodiar SA Municipal: * Municipal Mayor’s Office (Mining/Planning/Environment/Development Secretaries) * Gramalote * Mineros S.A. * Mining projects * Beekeepers Executors * Annatto Executors * Rehabilitation areas Executors *Mining formalization/legalization beneficiaries * Beekeeping beneficiaries * Annatto beneficiaries USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 194 TABLE 25: Design Matrix NO. EVALUATIONS QUESTIONS LINES OF INTERVENTION (COMPONENT) MAIN EVALUATION INDICATORS METHOD AGENTS DATA SOURCE QUANTITATIVE INSTRUMENT QUALITATIVE INSTRUMENT * Rehabilitation areas beneficiaries Commercial allies: * Naranja Madura (annatto) * Color Química (annatto) * Campo Dulce (honey) * LOK (honey) 195 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV QUANTITATIVE APPROACH It is proposed six methods for the quantitative analysis: a) descriptive analysis, b) Proportion Z-test and T-Student test, c) econometric approaches, d) multivariate methods, and e) text mining analysis. The quantitative methodology should answer, entirely or partially, all the evaluation questions listed in the Purpose section. MINING FORMALIZATION (QUESTION1) DESCRIPTIVE ANALYSIS, PROPORTION Z-TEST AND T-STUDENT TEST According to OL’s reports for FY19, out of the 643 initially contacted MPUs, 215 MPUs (33.4%) decided to take part of the formalization process. Among these, 26.5% (57 MPUs) reached formalization and 156 MPUs make up the remaining current portfolio. Using this information as a starting point, we will do a descriptive analysis to explore the formalization rate patterns and dynamics over time and between departments. We will also calculate the proportion Z-test to evaluate significant differences in the formalization ratio between the previous analysis units. Based on Lopez (2017)108, to establish if there are significant differences between the percentages reported by the qualitative variables comparing two or more groups, the assessment will use the Z-test proportion comparison. This test uses the binomial distribution to calculate the value or proportion P of some event of interest. If the evaluation will proceed to compare this P proportion for two groups or categories (A and B), the result would be PA and PB. In this case, the null hypothesis Ho: 𝑃𝑃𝐴𝐴 = 𝑃𝑃𝐵𝐵 can be contrasted through the following test statistic: 𝑍𝑍 = 𝑃𝑃𝐴𝐴−𝑃𝑃𝐵𝐵 �𝑃𝑃𝐴𝐴×𝑄𝑄𝐴𝐴 𝑛𝑛𝐴𝐴 +𝑃𝑃𝐵𝐵×𝑄𝑄𝐵𝐵 𝑛𝑛𝐵𝐵 where 𝑄𝑄𝐴𝐴 = 1 − 𝑃𝑃𝐴𝐴 and 𝑄𝑄𝐵𝐵 = 1 − 𝑃𝑃𝐵𝐵. If the results reject the null hypothesis, there would be significant differences in the proportion of interest between the groups analyzed. Considering the primary and secondary information related to question Q1, we will develop T-Student tests to corroborate significant differences in the MPU’s average production (in metric tons) and income between 2016 and 2018 and between groups of beneficiaries by regions. Based on Lopez (2017)109, the T-Student test will be used for independent samples to establish if there are significant differences between the averages reported in the different information characterization analyses.110 Mathematically, these tests start from two samples n1 and n2 that come from populations with means 𝜇𝜇1 and 𝜇𝜇2 and variances 𝛿𝛿1 2 𝑎𝑎 𝛿𝛿2 2. The random variable 𝑍𝑍 = �𝑋𝑋1−𝑋𝑋2�−(𝜇𝜇1−𝜇𝜇2) �𝛿𝛿1 2 𝑛𝑛1+ 𝛿𝛿2 2 𝑛𝑛2 follows a standard normal distribution. 108 Lopez, L (2017). Bioestadística. Ed. Universidad Nacional de Colombia. Chapter 9. Page. 72 109 López, L (2017). Bioestadística. Ed. Universidad Nacional de Colombia. Chapter 9. Page. 70 110 We will perform statistical tests to check testable assumptions when required. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 196 Since it is unknown whether both samples have the same variance, a variation of the T-test statistic is used instead: 𝑇𝑇:𝑡𝑡 = �𝑋𝑋1−𝑋𝑋2�−𝑑𝑑0 𝑆𝑆𝑝𝑝� 1 𝑛𝑛1+ 1 𝑛𝑛2 where 𝑠𝑠𝑝𝑝 2 = 𝑠𝑠1 2(𝑛𝑛1−1)+𝑠𝑠2 2(𝑛𝑛2−1) 𝑛𝑛1+𝑛𝑛2−2 have the sample standard deviation 𝑠𝑠1 and 𝑠𝑠2. In this case, the null hypothesis can be described as Ho: 𝜇𝜇1 − 𝜇𝜇2 = 𝑑𝑑0, that is if 𝑑𝑑0 = 0, it would contrast averages with no significant differences. The null hypothesis is rejected when −𝑡𝑡𝛼𝛼 2,𝑛𝑛1+𝑛𝑛2−2 < 𝑡𝑡 < 𝑡𝑡𝛼𝛼 2,𝑛𝑛1+𝑛𝑛2−2 concluding that there are significant differences between the averages reported in both samples. ECONOMETRIC MODEL OF MINING FORMALIZATION DETERMINANTS Using primary information, monitoring and follow-up information from the Performance Information Reference Sheets (PIRS) and the M&E plan, and secondary information from official sources111, we propose to estimate the following econometric model to answer the evaluation’s Q1. The first econometric model uses observations at the MPU level; in this model the dependent variable would be 𝑌𝑌𝑖𝑖 (formalization and legalization levels indicators).112 The independent variables would be the following: • A vector made of the MPU’s variables and attributes, 𝑋𝑋1𝑖𝑖: type of mining (alluvial, underground, dredges), the length of the mining exploitation, number of miners, and entry-level of the formalization indicator, among others. • M&E variables from the formalization strategy, 𝑋𝑋2𝑖𝑖: technical assistance, mining training, and clean production training for processing mercury, among others. • Municipal and departmental control variables, external to the program, that can help explain regional formalization progress, 𝑋𝑋3𝑖𝑖: demographic variables, specific traits, illicit economy indicators, and CAR management indicators, among others113. In line with the above, we propose the following econometric specification: 𝑌𝑌𝑖𝑖 = 𝛾𝛾0 + 𝛾𝛾1𝑋𝑋1𝑖𝑖 + 𝛾𝛾2𝑋𝑋2𝑖𝑖 + 𝑋𝑋3𝑚𝑚 + 𝑢𝑢𝑖𝑖 (1) Where: • 𝑖𝑖: Mining Production Unit 111 (1) Alluvial Gold Exploitation – Evidence from remote perception 2018. UNODC. Nov. 2019”, (2) Due diligence in Colombian gold supply chain: gold mining in Antioquia; 2016, Organization for Economic Co-operation and Development (OECD). (3) Goñi, E., Sabogal, A., and Asmat, R., 2014; Informal gold mining in Colombia: main results creating a baseline. Bogotá, D.C., Fedesarrollo, and IADB. (4) Sabogal, A., 2012; Creating a baseline for illegal gold mining in Colombia: methodological report. Economic and Social Research Center – Fedesarrollo, Bogotá. 112 These indicators are calculated from PIRS and M&E information. The legalization indicator corresponds to 60 points of the 100 possible points of the monitoring and follow-up scheme; This indicator responds to the variables evaluated by OL’s technical staff. The formalization indicator corresponds to 40 points out of the 100 possible points and responds to the indicators associated with the miner’s commitment to the process. 113 Specific indicators are presented in ANNEX 1. 197 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV • 𝑚𝑚: Municipalities • 𝑢𝑢𝑖𝑖: Error The second econometric model will use MPU-level observations and shares the same explaining variable set valid for equation (1). The difference with that model has to do with its dependent variable 𝑌𝑌𝑖𝑖: dichotomic variable that can be 1 if the MPU graduated with 75% of the required points or 0 in any other case. Due to the binary nature of the dependent variable, we postulate equation (2) to model the probability of MPUs graduating.114 𝑃𝑃(𝑌𝑌𝑖𝑖 = 1) = 𝛬𝛬(𝛾𝛾0 + 𝛾𝛾1𝑋𝑋1𝑖𝑖 + 𝛾𝛾2𝑋𝑋2𝑖𝑖 + 𝑋𝑋3𝑚𝑚 + 𝑢𝑢𝑖𝑖 (2) It is important to limit the scope of the results and the possible limitations of these econometric models. As mentioned above, 26.5% of the project’s MPUs achieved 75 out of 100 points in the formalization scale. This situation means that the program’s use of direct beneficiaries may have selection bias115 and not represent the total universe of informal MPUs. Therefore, it is important to point out that these analyses’ results will be valid for MPUs accompanied by OL and cannot be extrapolated to the total universe. Likewise, although these models will establish conditional associations of variables, it is possible that the problem of omitted variables116 exists, which together would make these models not measure causal effects or quantitatively evaluate impacts of the intervention. The econometric specification of equations (1) and (2) are subject to adjustment depending on primary and secondary information availability. In any case, this type of model depends on the collection of primary information available on February 12th (2021), which may be at risk due to the current COVID￾19 pandemic and timing of data collection. The information that is subsequently collected will be included at the final written report, however it will not be included in the presentation of results117. In case the data changes significantly, an erratum will be made describing the adjustments that are necessary. MINING GOVERNANCE AND POLICY (QUESTION 2) TEXT MINING ANALYSIS Using as input the qualitative information from the transcription of interviews related to the development of Q2, we propose to prepare a textual data analysis called Word Cloud and Hierarchical Clustering. We will use these tests to detect the most frequent words and word associations, searching for strategic messages that summarize the information regarding adopting good practices in gold mining to control, mitigate, and prevent the negative environmental impacts. VALUE CHAIN DEVELOPMENT AND ALTERNATIVE LIVELIHOODS (QUESTION 3) 114 We will assess the Probit and Logit models; the model’s selection will depend on the best fit and fulfilling these two econometric specifications’ assumptions. 115 We had to modify the strategy and locate active miners within company properties or community council areas; this does not represent most small, informal miners. 116 One example is the formalization progress (graduation) as related to business ownership and community council areas. 117 In case the information collected is not enough to allow econometric estimations, descriptive analyses of survey information will be conducted making use of module V. Mining Formalization USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 198 DESCRIPTIVE ANALYSIS, VARIABLE ASSOCIATION AND GROUPING OBSERVATIONS This analysis starts with a descriptive study based on primary information from achiote and apiculture value chains’ direct beneficiaries. Then we use this survey to conduct multivariate analyzes known as multiple correspondence analysis and cluster analysis. This examination will help us find multiple relationships between qualitative attributes to summarize the information from the components and outline the conclusions regarding alternative licit income sources, sustainability of legal productive alternatives, reducing environmental impacts, and beneficiaries’ well-being and livelihoods. Following guidelines by Peña (2002) to analyze and represent possible associations in contingency tables, we will develop a multiple correspondence analysis to conclude about qualitative variables118. This analysis begins with an F matrix of relative frequencies of the form: ∑𝐼𝐼 𝑖𝑖=1 ∑𝐽𝐽 𝑗𝑗=1 𝑓𝑓𝑖𝑖 = 1, Which we use to construct a data matrix: 𝑍𝑍 = 𝐷𝐷𝑓𝑓 −1/2 𝐹𝐹𝐷𝐷𝐶𝐶 −1/2 Whose components are: 𝑧𝑧𝑖𝑖 = � 𝑓𝑓𝑖𝑖𝑖𝑖 �𝑓𝑓𝑖𝑖.𝑓𝑓. � Through matrix operations, it is possible to establish a joint projection of row profiles 𝐶𝐶𝑓𝑓 = 𝐷𝐷𝑓𝑓 −1/2 𝑍𝑍𝐴𝐴2 and column profiles 𝐶𝐶𝑐𝑐 = 𝐷𝐷𝑐𝑐 −1/2 𝑍𝑍𝐵𝐵2, where 𝐴𝐴2 contains the vectors of 𝑍𝑍′𝑍𝑍. The result of projecting these coordinates is a graph called a perceptual map that represents the association between the categories summarized in a contingency table. Additionally, with the same datasets, there is a possibility for the use of other multivariate analyzes such as the following methodologies: (i) K-means cluster, (ii) Two-step cluster, or (iii) Heretical Cluster. We will use these tools to identify clustering patterns among beneficiaries of the Oro Legal program’s different action lines. ECONOMETRIC MODEL OF INCOME DETERMINANTS FOR BEEKEEPING AND ANNATTO PRODUCTION: We propose the following econometric approach to analyze Q3, particularly regarding the perceived effect of alternative livelihoods regarding licit income generation and sustainability of productive options. To understand income from alternative licit sources, we will estimate an econometric model to measure the determinants of compensation received by achiote and apiculture value chains’ direct beneficiaries. We will record income as the average periodic value of production, sales, or revenue for 2020 from the 118 Among others, computed qualitative variables from the primary beneficiary’s survey include: • Families and/or individual members participation in OL annatto supported activities • Individual or family economic activity before the OL program • Motivation to participate in OL’s annatto activities • Individual/family commitments with OL while joining beekeeping/annatto activities • Obstacles to the development of beekeeping/annatto in the targeted regions 199 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV productive activity. This way, the model will have as the dependent variable 𝑌𝑌𝑖𝑖: income received by the productive chains.119 We will try to quantify the effect of different determinants from this proceeding through control variables in econometric models. Concerning the determinants of this income, we propose to control the model with microeconomic determinants present in the economic literature (𝑋𝑋𝑖𝑖: the size of the UPA, returns on the UPA, investment level, use of installed capacity, number of people dedicated to productive activity, seniority or experience in production, technical assistance, associativity, among others). Additionally, we propose to control the model through territorial determinants of income generation such as (𝑍𝑍𝑖𝑖: the distance between the place of production and the market place, access to demand, transportation costs, dispersion of UPAS, access to public goods and infrastructure120, among others)121. It is important to note that the MEL-GIS team will calculate the territorial determinants of income through productive activities and their level of access to tertiary roads, which will allow analyzing the distances and times involved to transport the products to the centers market. 122 Following this logic, we propose the following general econometric specification: 𝑌𝑌𝑖𝑖 = 𝛾𝛾0 + 𝛾𝛾1𝑋𝑋𝑖𝑖 + 𝛾𝛾2𝑍𝑍𝑖𝑖 + 𝑢𝑢𝑖𝑖 (3) Where: ● 𝑖𝑖: OL’s beekeepers and annatto farmers ● 𝑚𝑚: Municipalities ● 𝑢𝑢𝑖𝑖, : Error The model contained in equation (3) includes a yield definition by UPA understood as the ratio of annatto tons over hectares in equation (4) and honey kilograms over beehives in equation (5), as shown by the following equations: 𝑌𝑌 𝑌𝑌 𝐴𝐴 𝐴𝐴𝑖𝑖 = 𝑇𝑇 𝑖𝑖 𝐻𝐻 𝐻𝐻𝑒𝑒 𝑖𝑖 (4) 𝑌𝑌 𝑌𝑌 𝐴𝐴𝐴𝐴 𝑖𝑖 = 𝐻𝐻 𝐾𝐾 𝐾𝐾 𝑖𝑖 𝐵𝐵 ℎ𝑖𝑖𝑖𝑖 𝑖𝑖 (5) 119 The effect of the price is in the income received by the beneficiaries. Although the price is a determining factor in producers’ income, it is neither the only nor the main one. The productive projects with small producers try to manage variables to improve their income; these include yield, the quality of their products, the efficiencies in their production, and logistics costs. Price behavior is taken into account to define, together with the expected yields and estimated costs, the size of the minimum productive unit (number of hives or number of hectares) such that it guarantees a sufficient income to the producer and that they can overcome foreseen price changes based on estimated market behavior. 120 Secondary information at the municipal level will be explored to approximate the variable. 121 Specific indicators are presented in ANNEX 2. 122 We will consider the number of operational units (active beehives or hectares of achiote in the productive phase) and the current development stage to estimate the yields and evaluate the income, which is part of the information to collect with OL. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 200 The econometric specifications of equation (3) are subject to adjustment depending on primary and secondary information availability. In any case, this type of model depends on the collection of primary information available on February 12th (2021), which may be at risk due to the current COVID-19 pandemic. The information that is subsequently collected will be included as an annex of descriptive tables, however it will not be included in the final presentation of results. REHABILITATION OF AREAS DEGRADED BY ILLEGAL MINING (QUESTION 4) DESCRIPTIVE ANALYSIS – REHABILITATION OF DEGRADED AREAS The analysis of the rehabilitation of degraded areas will combine secondary and primary information. Using information from satellite images between 2016 and 2020, we will prepare a descriptive analysis to analyze the mining areas targeted by OL for rehabilitation in the departments of Antioquia and Chocó. We can use this to analyze the mining areas intervened by OL’s and examine if there is noticeable regrowth and identify mining areas not benefited from OL and examine vegetation cover. On the other hand, we will review OL’s documents on the proposed process to restore the two areas and conclude about the pertinence of the implemented strategy. We will also review secondary information on alternative interventions to diagnose and rehabilitate mining affected areas, define environmental liabilities, identify and classify environmental impacts on natural elements, erosion, and determine the effects on the ecosystem and the landscape, among others that can provide valuable references for the analysis. We will also collect primary information from direct beneficiaries of the rehabilitation projects in Antioquia and Chocó. This research will include inquiring about the people’s perception of the targeted areas regarding the improvements, including landscape and ecosystem restoration, the accompaniment of the public institutions, the intervention’s length, the feasibility of replication and its sustainability in the future. Similarly, based on the qualitative sampling and analysis we will gather and wage opinions from different organizations: public agencies, universities, and NGOs. TEXT MINING ANALYSIS Using as input the qualitative information from the transcription of interviews related to the development of Q4, we propose to prepare a textual data analysis called Word Cloud and Hierarchical Clustering. We will use these tests to detect the most frequent words and word associations, searching for strategic messages that summarize the information regarding adopting good practices in gold mining to control, mitigate, and prevent the negative environmental impacts. We will review documents prepared with primary information and the “fichas ambientales” for OL’s sub-activities recorded on the Monitor System to determine the effectiveness of the environmental intervention before and during the project’s execution. We will also consider in the analysis the available parameters from existing tests and samples regarding water, biological resources and the abiotic component. The quality of the information that we receive will determine our environmental component analysis; if the information is insufficient, we will consider additional fieldwork to validate the data. MERCURY ELIMINATION/REDUCTION FROM THE ASGM SUPPLY CHAIN (QUESTION 5) DESCRIPTIVE ANALYSIS – MERCURY REDUCTION IN THE ASGM SUPPLY CHAIN 201 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV According to OL’s information for 2017, the amount of mercury used to produce 1 ounce of gold amounted to 8.62 Hg/t. Taking this baseline and monitoring information from the PIRS and the M&E plan, we will prepare a descriptive analysis to detail mercury use between 2017 and 2020 among all MPUs supported by the program. This analysis will confirm significant differences between MPUs by municipalities and define the variables associated with higher or lower mercury consumption in the ASGM chain. MERCURY DETERMINANTS IN ASGM (OPTIONAL) To highlight probable links between mercury, use in ASGMs and the variables observed in MPUs, we propose to use an econometric model to find plausible determinants. This econometric model uses as observations units the MPUs targeted by OL in the formalization program and the dependent variable would be 𝑌𝑌𝑖𝑖 (mercury used by 1 MPU to produce 1 ounce of gold). This model would assess: • A vector of independent variables from OL’s M&E system, 𝑋𝑋1𝑖𝑖: availability of chemical supplies, productivity of the mining methods, type of mining (alluvial, underground, dredges), the length of the mining exploitation, number of miners, and MPU’s formalization indicator, among others. • Direct beneficiaries’ perception variables, 𝑋𝑋2𝑖𝑖: specific traits, perception of regulations, and variables associated with the mercury use behavior. • Municipal and departmental control variables, external to the program, that can help explain regional the ASGM use of mercury, 𝑋𝑋3𝑖𝑖: environmental variables, demographic variables, illicit economies indicators, and CAR management indicators, among others123. In line with the above, we propose the following econometric specification: 𝑌𝑌𝑖𝑖 = 𝛾𝛾0 + 𝛾𝛾1𝑋𝑋1𝑖𝑖 + 𝛾𝛾2𝑋𝑋2𝑖𝑖 + 𝑋𝑋3𝑚𝑚 + 𝑢𝑢𝑖𝑖 (6) Where: ● 𝑖𝑖: Mining Production Unit ● 𝑚𝑚: Municipalities ● 𝑢𝑢𝑖𝑖: Error The model described in equation (7) is optional because its results depend on the availability of primary information and obtaining MPU data to explain the use of mercury in gold production. In any case, this type of model depends on the collection of primary information available on February 12th (2021), which may be at risk due to the current COVID-19 pandemic. The information that is subsequently collected will be included as an annex of descriptive tables, however it will not be included in the final presentation of results. 123 Specific indicators for X1i, X2i and X3m are presented in ANNEX 2. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 202 PERCEIVED RELEVANCE AND IMPACT OF THE ACTIVITY (QUESTION 6) DESCRIPTIVE ANALYSIS AND FACTOR ANALYSIS Concerning the primary quantitative information that we will collect in this evaluation, we suggest a descriptive analysis to characterize the perception variables of OL’s impact on the mining. This analysis aims to compare groups of beneficiaries (mining formalization, reconversion, and rehabilitation of degraded areas) concerning the same set of perception variables124 and look for differences between their qualifications. Likewise, we will evaluate the perception of institutional actors associated with OL using the same set of variables. In a complementary way, we will use Factor Analysis to provide quantitative elements on the data’s underlying structure and provide the possibility of summarizing perception information about the importance of OL’s impacts among its beneficiaries. In any case, this type of analysis depends on the collection of primary information available on February 12th (2021), which may be at risk due to the current COVID-19 pandemic. The information that is subsequently collected will be included as an annex of descriptive tables, however, it will not be included in the final presentation of results. DATA COLLECTION CONSIDERATIONS The quantitative approach will answer 5 questions of the intervention (Q1, Q3, Q4, Q5, and Q6) using primary information. According to the Activity’s data, there are 1,245 direct beneficiaries in 20 municipalities of Antioquia and Chocó that are this evaluation’s universe. Direct beneficiaries, mine owners, or representatives of community councils, usually are the ideal source to supply information. The mining component has 167 beneficiaries, annatto has 345, apiculture 318, and rehabilitation of degraded areas 415 (see Table 26). TABLE 26: DIRECT BENEFICIARIES BY COMPONENT AND MUNICIPALITY DEPARTMENT MUNICIPALITY MINING ANNATTO BEEKEEPING REHAB TOTAL Antioquia Barbosa 3 3 Buriticá 10 10 Cáceres 23 24 47 Caucasia 103 30 133 Don Matías 5 5 El Bagre 72 315 387 Nechí 57 57 Remedios 11 11 124 The module (IV. Legal Gold Program) will be used, which contains perception variables associated with the program. Specific questions such as: 132. Before you joined the Oro Legal program, which of the following activities represented a source of income in your household? 133. Today, which of the following activities represent a source of income in your household? 134. On a scale of 1 to 5, where 1 is very dissatisfied and 5 is very satisfied. How satisfied are you with the ORO LEGAL program? 203 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 26: DIRECT BENEFICIARIES BY COMPONENT AND MUNICIPALITY DEPARTMENT MUNICIPALITY MINING ANNATTO BEEKEEPING REHAB TOTAL San Roque 11 11 Segovia 1 1 Tarazá 16 29 45 Zaragoza 1 57 43 101 Total Antioquia 81 0 318 412 811 Chocó Atrato (Yuto) 111 111 Cantón San Pablo 74 74 Certegui 11 2 13 Condoto 5 5 Quibdó 20 1 21 Río Quito 127 2 129 Tadó 41 41 Unión Panamericana 29 11 40 Total Chocó 86 345 0 3 434 Total 167 345 318 415 1,245 Given the characteristics of the universe of analysis, below are some guidelines to design quantitative samples by the information gathering firms125: 1. Mining: There is a database with the list of the 167 MPU of OL’s direct beneficiaries; for this universe, we expect a survey of census information. Of these beneficiaries, it is essential to mention that 121 have a contact telephone number, and the proposals must contemplate gathering the missing information to contact all the beneficiaries, either in person or virtually. 2. Annatto and Beekeeping: There is a database of the 345 families benefitting directly from annatto and 320 families benefitting directly from beekeeping in 11 municipalities in Antioquia and Chocó. We expect face-to-face or virtual surveys with this group included in the sample design. In general, the proposed sample must guarantee a margin of error of less than 5% and a confidence level equal to or greater than 95%. Indicatively, using these parameters, the minimum sample sizes for Achiote and Beekeeping beneficiaries are defined through the simple random sampling strategy, as illustrated in Table 27. If possible, we expect that the sample proposal contains data from all the targeted municipalities. The values included in 21 may vary with the proposed sampling strategy’s efficiency from 125 The firm must deliver a database with the information collected on February 12, 2020. This database will be the input for the quantitative analysis of the evaluation. Subsequently, the delivery of a final database with the results of the entire field work is expected according to the initial work plan. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 204 the gathering firms. Similarly, the proponents must present an oversampling percentage to account for possible replacements in non-responsive cases. Finally, the proponent should consider adjustments in the databases with expansion factors for non-responses. TABLE 27: MINIMUM SAMPLE ANNATTO BEEKEEPING Indicative Sample 261 245 One of the main challenges in collecting information is that, of the total of 345 beneficiaries of annatto, there are only 154 telephone numbers. Likewise, of the 320 beekeeping beneficiaries, there are only 234 telephone contacts. Therefore, the proposals must contain a face-to-face or virtual strategy to contact the direct beneficiaries selected in the samples that do not have telephone numbers, or their numbers are out of date. 3. Rehabilitation: According to the data, 2019 reports a portfolio of 16,813 rehabilitation hectares. Of this total, 1,781 are in Antioquia and 15,032 in Chocó. On the other hand, 412126 direct beneficiaries are in Antioquia and three direct beneficiaries in Chocó. We can explain this situation because the rehabilitation projects in the department of Chocó brought together several community councils, and their extension was considerably greater than the projects in Antioquia. Each rehabilitation project was developed with one council. They bring together the community but are implemented through their collective organization: the council. Considering these particularities, the proposed sample must guarantee a margin of error of less than 5% and a confidence level equal to or greater than 95%. Indicatively, with these parameters, using the simple random sampling strategy, the samples’ minimum sizes for rehabilitation beneficiaries in Antioquia are defined as illustrated in Table 28. TABLE 28: REHAB SAMPLE IN ANTIOQUIA REHABILITATION Indicative Sample 297 QUALITATIVE APPROACH METHODOLOGY We based the qualitative methodology on the phenomenology of perception and will use it to answer all the evaluation questions. In this approach, “the researcher identifies the essence of human experiences around a phenomenon according to how it is described by the study participants” (Creswell, 2003, p. 15). Therefore, the phenomenology of perception is useful for understanding and making sense of the everyday experiences and meanings that beneficiaries and stakeholders associated with Oro Legal have. We will complement the analysis with a review of the intervention’s Strengths, Weaknesses, 126 According to the program, nine reforestation beneficiaries are also beekeepers. The firm should decide if they would use the instruments designed for each component. 205 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Opportunities, and Threats (SWOT). This strategy identifies whether internal or external factors contribute to the perceived and effective success of the activity. We will use both primary and secondary information to meet the objectives of the qualitative approach. For that purpose, we will carry out semi-structured interviews. In this regard, we will collect all information, protecting the privacy of the interviewees. We will record each survey with the previous consent from the participants. Moreover, the evaluation will emphasize the importance of safeguarding interviewees’ rights and following the fieldwork protocol. All participants will receive an explanation of the principles that guide the evaluation, to which they will give consent, verbally, or in writing. This consent also states that participation is deliberate, participants can withdraw from the activity if they wish, and there is a plan to safeguard their privacy and confidentiality. It is also essential to clarify interviewees that the interviewers will handle data under statistical guidelines without disclosing the source. Additionally, we will use secondary information: mainly Oro Legal documentation and annual and quarterly reports. We will also employ OL bulletins, briefings, communication pieces, and diagnostic documents related to indexes and context information produced by national and local government agencies. These documents will help answer the evaluation’s questions and assess the activity’s performance and progress towards its main goals. The methodological strategy associated with the analysis will incorporate principles of the grounded theory. We will systematize, code, and analyze the information to identify findings and subsequent recommendations in a SWOT structure. We will use a codebook to sort the data (DeCuir-Gunby, Marshall, & McCulloch, 2011; Fereday & Muir-Cochrane, 2006; MacQueen, McLellan, Kay, & Milstein, 1998). We can modify this code later with the categories that arise from the analysis process (Auerbach & Silverstein, 2003; Hesse-Biber & Leavy, 2006). To accomplish this task, we will make use of the NVivo 12 Plus software. We select this software as it supports the organization, coding, visualization, and analysis of results. NVivo has several easy-to-use options (folders, sets, and attributes) that allow us to group data by subject, cross-section, geographic location, or document type. In terms of encoding processes, NVivo allows creating thematic nodes or analysis categories, case nodes, and sentiment nodes that help classify textual information and an easy way to merge different files containing encoded information from various actors under the categories defined in the evaluation codebook. Among the main visualization and analysis tools are word frequency queries and text search. Another tool that will be useful is the cluster analysis that will allow identifying whether there are associations between categories and thematic components through clusters that group categories either by the similarity of words or the coding of each node. In this same sense, the comparison diagrams will quickly identify between two case nodes: the categories they share, and which are not. Finally, NVivo can create three types of maps that will be useful throughout the coding and analysis process. The choice and degree of use of each of these tools will always support the organization, coding, visualization, and analysis of results, seeking to fully respond to the evaluation’s objectives and questions. We will use this coding process as the foundation to report the participants’ views and perceptions. In the qualitative analysis, it is worth clarifying that the frequency does not determine the value of a point USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 206 of view or an opinion (number of times it occurs) since this methodology tries to prioritize the interviews’ diversity and nuances. Finally, we will compare the qualitative strategy outcomes with those of the quantitative approach. The latter ensures that the results, findings, and recommendations of this evaluation will be sufficiently robust and confident. PRIMARY INFORMATION For the primary sources, we will conduct semi-structured interviews at national, regional, and local levels for all components of the assessment (O1 and O2). A data collection firm will be hired to interview participants and direct beneficiaries. The interviewers from this firm will receive a training session by the evaluation team, so that they understand OL context. All the actors interviewed will answer the context questions prepared by the evaluation team. Also, across all components, we will conduct semi-structured interviews with Oro Legal staff, including the Programmatic and Operational DCOPs and previous component managers, in charge of relevant tasks at the beginning of program operation. These actors will be contacted once again, if necessary, through or after the fieldwork, to discuss related issues further. Next, we present the qualitative sample for the evaluation, dividing between institutional actors and beneficiaries. We highlight the number of suitable informants to answer each guiding question in the matrix. Each actor will answer the modules that correspond to them according to their OL activities’ participation, with question 6 intended for all the actors. We expect a maximum of 102 applications. TABLE 29: SAMPLE FOR QUALITATIVE APPROACH127 INSTITUTIONS Q1 Q2 Q3 Q4 Q5 Q6 Academia 1 1 1 1 1 1 Mayor’s Offices 9 9 9 9 9 9 Commercial Allies 4 4 National Allies 4 4 4 7 Associations 1 1 1 1 CARs 4 4 4 4 4 Grantees 3 3 Governor’s Offices 2 2 2 2 2 2 National Government 3 4 1 3 4 National Operator 1 1 1 1 1 1 Regional Operator 2 2 2 2 2 2 127 Source: Own Production (2020). 207 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV SENA Regional 2 2 Formalization Subcontracts 2 2 2 Total 29 24 28 21 28 42 BENEFICIARIES Miners 4 4 4 4 4 Farmers 4 4 Rehab operators 4 4 4 4 4 Mining Beneficiaries* 16 16 16 16 16 Productive Projects Beneficiaries* 16 16 Rehab Beneficiaries* 16 16 16 16 16 Total 40 40 20 40 40 60 * The data collection firm will be in charge of carrying out these interviews TRIANGULATION When triangulating, we compare different types of data analysis with the same objective (Rodríguez, Pozo & Gutiérrez, 2006) as a way to validate, question, or expand the results found by contrasting different perspectives and sources of information (Turner & Turner, 2009). In this evaluation, triangulation becomes a means to analyze data to avoid bias and increase validity by comparing the results obtained by different research methods (Blaikie, 1991 and Gavira & Osuna, 2015). Therefore, we carried out the triangulation, as shown in Figure 1. In the first place, we will compare the quantitative methods establishing whether a result is met with the qualitative methods establishing the perceptions about the same phenomenon and why. This procedure will also allow strengthening the arguments in terms of findings and conclusions. Likewise, the evaluation team will triangulate the information from different types of actors involved in the project, such as national, regional, and local level actors or institutional actors, executors, and beneficiaries, so that the findings are enriched. Finally, the analyzes will integrate primary sources of information from interviews and surveys and secondary sources such as documents and information from the monitoring and evaluation system to contrast and strengthen the results. To carry out this work, the evaluation team used the question matrix in work sessions; the evidence will be classified, organized, and analyzed to compare the different findings according to the methods, agents, or sources of information and thus arrive at draw conclusions. Finally, it is important to mention that there will also be a triangulation exercise in the CLA meetings according to the review and analysis of each of the members of the evaluation team, which will allow to discuss the findings and conclusions and generate consensus. Triangulation compares systems eliminated biases associated with using a single method. This purpose also involves contrasting different types of stakeholders to help enrich findings. We will also collate primary and secondary information sources. We will carry out this work using the evaluation design USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 208 matrix to fill, classify, and organize the interviews’ answers to compare the different findings according to the methods, agents, or information sources. Exhibit 71: Triangulating the Information128 PILOT AND EARLY REPORT We will carry out a pilot for collection of qualitative information through virtual means between November 9th and 20th; this decision responds to the measures prescribed by the Government to reduce COVID-19 contagion. Additionally, on November 4th we will perform a CLA workshop with OL’s members to go in depth into the activity. We will use the pilot data to evaluate the degree of functionality, completeness, and coherence of the qualitative and quantitative collection instruments. We will also evaluate clarity, phrasing, the order of topics, the sequence of application and relevance of questions, performance, non-response rates, the suitability of informants, the duration of the application of the instruments, familiarize the team and data collectors with the instruments and identify advantages and difficulties that may arise during the operation regarding the handling of the samples129 and to produce an early report with some secondary information analysis; we will submit no later than Monday, December the 7th. Based on the guidelines outlined in this document, we will choose interviewees suitable for evaluating the two OL components stakeholders from national, regional, and local levels 128 Source: Own production (2020). 129 The evaluation team will carry out a meeting to assess all of these issues. 209 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV such as ANM, National Ministries, OL’s regional members, representatives from the Antioquia’s Governor’s office, CARs, and some municipalities. It is relevant to mention that a data collection firm hired for this purpose will pilot the quantitative survey instruments with mining production units (MPU) and the quantitative survey and the qualitative interviews instruments with direct beneficiaries in January 2021. MEL will hire this firm for such a purpose. Thus, we will implement an alternative process, described as follows, to produce the early report: a) Definition: MEL, USAID, and Implementing Partner agreed that the early report covers the analysis of secondary data and some primary qualitative information (we will gather this data virtually). b) Analysis of secondary information: At this point, we reviewed OL documentation (for example, the annual work plans and quarterly reports). We also revised qualitative data from the components (indicators) and administrative records (databases). c) Fieldwork: As part of the qualitative instruments piloting exercise, we will meet with OL representatives from all the components through the CLA workshop. Some of this work already began. Similarly, we propose to conduct the pilot with municipalities of Antioquia and Choco where the USAID team considers it likely to attend based on the local security conditions. d) Analysis and report: We will do a preliminary quantitative analysis, including detailed statistical and inferential data reports and a correspondence study. This work also involves studying geo￾referenced information and transcribing and analyzing qualitative data derived from the interviews. We will compare and triangulate all this information to further findings and inform discussions and conclusions. e) Limitations: After obtaining the preliminary results, we will address the restrictions found during the pilot. By doing so, the pilot exercise will help to improve information collecting tools for future fieldwork. LIMITATIONS OL’s evaluation must address some constraints that stem from its methodological design. One possible limitation in the assessment’s phenomenological approach occurs when it favors subjective experience over local social, economic, and political structures. However, we will address this issue using secondary information and triangulating research data to determine each region’s context. Another possible limitation is related to the exclusive use of a single methodology; the idea is to contrast diverse information to solve this issue. Furthermore, another concern has to do with the relevance of the chosen tools; the evaluation will use the pilot exercise to screen and improve quantitative and qualitative instruments. The evaluation might also face complications associated with beneficiaries: for example, problems may arise when trying to summon or make them participate in programmed activities. In this case, a methodological design with a sample that is more extensive than necessary may help resolve eventual difficulties. Likewise, other limitations have to do with the public order situation, exacerbated in some of the municipalities intervened by OL. For this purpose, MEL Activity will follow its security firm protocol, as well as USAID guidelines, additionally, the data collection firm will communicate with local authorities before the field-visit. Likewise, it is essential to mention that given the evaluation team’s experience in similar investigations, small miners tend to change contact numbers rapidly, which is the cause of a rapid outdating of this contact information. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 210 An additional limitation is related to the project’s end date, which implies a risk to gather information from people who end their contractual relationship in December 2020. Given this, the evaluation team will perform the CLA workshop in order to gather in one session, relevant OL members to establish open discussions for a better understanding of the activity. Only in exceptional cases will interviews be carried out in 2021. Another challenge is collecting primary environmental information, a task involving technology and specialists for each of the natural resources analyzed. The evaluation will use the primary information collected by the IP in the design and execution phases as a reference to overcome this issue. It is worth clarifying that if we have quality, reliable, standardized, and homologated environmental data, it may be a sufficient input to evaluate the activity’s environmental performance. However, suppose we find quality gaps or drawbacks in the information received from the IP. In that case, we will have to consider the need to subcontract a scouting team to assess the environmental effects, using primary information to determine geophysical, biotic, and abiotic consequences. In any case, we will use secondary information generated by public institutions (official country information) to mitigate this limitation. In conclusion, piloting the instruments and building the early report will help to highlight some limitations. Likewise, it is crucial to do a rigorous evaluation of some information sources to consider whether it is necessary to carry out primary information surveys. Some restrictions have to do with limited access to beneficiaries, lack of inclusion of some stakeholders, and organizing or systematizing some databases. Finally, one significant limitation affecting the evaluation is related to COVID-19. To mitigate participants’ risks of contamination, the evaluation team will work on early reporting and desk-based tasks from Bogotá. The evaluation team will also use video chatting apps, such as Meet, Teams, Skype, or Zoom, for meetings and interviews. For the time being, travel and tool testing will be suspended, especially with beneficiaries. Each week the situation will be assessed to make new decisions. CLA APPROACH AND COMMUNICATIONS STRATEGYCLA APPROACH Practices derived from the CLA methodology contribute to a more useful evaluation. This approach helps USAID and the team to learn during the process. In particular, the team will benefit from the CLA methodology in the following steps of the evaluation: 1. Planning First, the CLA component will create an atmosphere of trust among MEL, USAID, and the IP. This confidence-building will occur through meetings, where the evaluation team discusses the intervention, what the evaluation is for, and paths to its implementation. 2. Information analysis and report production Second, the CLA component will promote learning among MEL Activity actors, including the evaluation team members, and other related components. For this purpose, the learning approach will assure the alignment across findings, conclusions, and recommendations, through different pause and reflect moments that promote collaboration. 211 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 3. Presentation and final adjustments The CLA component will assist the evaluation team when preparing presentations, like the evaluation recommendations. It will also coordinate a learning workshop among the evaluation team, USAID, and the Implementing Partner to discuss recommendations and possible action plans arising from them. COMMUNICATIONS STRATEGY Assertive and effective communication that promotes the exchange of knowledge requires a strategic approach. The evaluation is useful to grasp the activity’s performance and results and share lessons and recommendations that contribute to give feedback to Oro Legal and provide data to inform future USAID/Colombia’s programming decisions. Therefore, it is essential to have a strategy that allows the correct dissemination of the evaluation findings, conclusions, and recommendations. The exercise of disseminating analytical products or the lessons learned from an evaluation implies systematizing previous steps, such as identifying primary and secondary audiences and constructing key messages based on their expectations. The dissemination and use plan will be prepared in February with the participation of Oro Legal´s COR, the Development Assistance Specialist, and the MEL Activity. INDICATORS FROM THE ECONOMETRIC MODEL OF MINING FORMALIZATION DETERMINANTS TABLE 30: INDICATORS FROM THE ECONOMETRIC MODEL OF MINING FORMALIZATION DETERMINANTS ECONOMETRIC MODEL COMPONENT VARIABLES AND SPECIFIC QUESTIONS FOR MEASUREMENT Response variables (𝑌𝑌𝑖𝑖) • Formalization and legalization levels indicator (equation 1) • Dichotomic variable that can be 1 if the MPU graduated with 75% of the required points or 0 in any other case (equation 2) • These indicators are calculated directly from information from the PIRS and M&E. MPU’s variables and attributes (𝑋𝑋1𝐼𝐼) • Type of mining: P203. ¿Por favor indique el tipo de operación minera en su Unidad de Producción Minera (UPM)? P204. ¿Qué tipo de explotación es desarrollada en su UPM? P205. ¿Cuál es la principal actividad que se desarrolla en su UPM? ● Length of the mining exploitation: Indicador tamaño de la UPM basado en información de M&E ● Number of miners: P209. ¿Incluido usted cuántas personas trabajaron en la UPM en el año 2020? ● Entry-level of the formalization indicator: Este indicador es calculado directamente a partir de la información de los PIRS y M&E. ● Use of installed capacity: USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 212 INDICATORS FROM THE ECONOMETRIC MODEL OF INCOME DETERMINANTS FOR BEEKEEPING AND ANNATTO PRODUCTION TABLE 31: INDICATORS FROM THE ECONOMETRIC MODEL OF INCOME DETERMINANTS FOR BEEKEEPING AND ANNATTO PRODUCTION ECONOMETRIC MODEL COMPONENT VARIABLES AND SPECIFIC QUESTIONS FOR MEASUREMENT Response variables (𝑌𝑌𝑖𝑖) ● Income received by the productive chains (equation 3): P320: ¿Cuál fue el valor de la producción de miel o ingresos operacionales de la Unidad de Producción Agropecuaria (UPA) para el año 2020? P421: ¿Cuál fue el valor de las ventas de Achiote o ingresos operacionales de la Unidad de Producción Agropecuaria (UPA) para el año 2020? P213. De 1 a 100 donde 100 significa que la UPM se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en su UPM? P212. ¿En promedio cuantos días del mes se trabaja en la UPM? ● Seniority or experience in production: P206. ¿Hace cuánto tiempo inició operaciones estas UPM? Variables from the formalization strategy (𝑋𝑋2𝐼𝐼) ● Technical assistance 240. ¿Para usted cuales son las principales ventajas de la formalización minera? Opciones: Acceso a capacitaciones ● Mining training P207. ¿Cuántos años de experiencia tiene USTED en la minería de oro? Opciones: Acceso a mejores prácticas. 240. ¿Para usted cuales son las principales ventajas de la formalización minera? ● Clean production training for processing mercury P223. ANTES de la participación en el programa Oro Legal ¿Por cada gramo de oro producido en su UPM cuantos gramos de mercurio se utilizaban? P224. ACTUALMENTE ¿Por cada gramo de oro producido en su UPM cuantos gramos de mercurio se utilizan? P226. En una escala de 1 a 5, donde 1 es muy en DESACUERDO y 5 es muy DE ACUERDO ¿Qué tan de acuerdo se encuentra con las siguientes afirmaciones? ● Other variables: P228. ¿El municipio de venta del oro producido es diferente al municipio donde se realizó la producción? P229. ¿La UPM cuenta con título minero? P230. ¿La UPM guarda registros de sus facturas de ventas o compras de insumos? P231. ¿La UPM cuenta con el Plan de Trabajo en Obras – PTO? P232. ¿En el PTO se orientó el beneficio de oro libre de mercurio? P233. ¿El Programa facilitó la implementación de las técnicas y tecnologías determinadas en el PTO? P234. ¿La UPM cuenta con Estudio de Impacto Ambiental? P235. ¿La UPM cuenta con Guía Ambiental? P243. ¿Continuaría con la actividad minera ante cambios en precios del oro que afectaran sus ingresos actuales (hasta que sus ingresos se reduzcan al X%)? P245. ¿La UPM cuenta con alguno de los siguientes servicios públicos? 213 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Microeconomic determinants (𝑋𝑋𝑖𝑖) ● Size of the UPA: P317_a: ¿Qué forma de tenencia tiene el lote que ocupa la actividad (Unidad de Producción Agropecuaria) de apicultura del programa Oro Legal? (Desagregue el área de esos lotes o predios según la forma de tenencia) P418_a: ¿Qué forma de tenencia tiene el lote que ocupa la actividad (Unidad de Producción Agropecuaria - UPA) de cultivo de achiote del programa Oro Legal? (Desagregue el área de esos lotes o predios según la forma de tenencia) ● Returns on the UPA (equation 4 & 5): Razón entre (P322/P321 & P423/P18_a) P321: En promedio ¿Cuántas colmenas tuvo la Unidad de Producción Agropecuaria (UPA) durante el año 2020? P322: ¿Cuál fue el total de kilogramos de miel producidos en la Unidad de Producción Agropecuaria (UPA) durante el año 2020? P423: ¿Del total de toneladas producidas en 2020, ¿cuál fue el total de toneladas de Achiote vendidas en la Unidad de Producción Agropecuaria (UPA) durante el año 2020? ● Investment level: P319: ¿Cuál es el valor de los activos totales con los que cuenta la Unidad de Producción Agropecuaria (UPA) dedicada a Apicultura para el año 2020? P420: ¿Cuál es el valor de los activos totales con los que cuenta la Unidad de Producción Agropecuaria (UPA) dedicada a Achiote para el año 2020? Valor en pesos: ● Use of installed capacity: P323: De 1 a 100 donde 100 significa que la UPA se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en la (UPA)? P424: De 1 a 100 donde 100 significa que la UPA se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en la (UPA)? ● Number of people dedicated to productive activity: P318: En promedio ¿Incluido usted cuántas personas trabajaron en la Unidad de Producción Agropecuaria (UPA) Apícola durante el año 2020? P419: En promedio ¿Incluido usted cuántas personas trabajaron en la Unidad de Producción Agropecuaria (UPA) dedicada al cultivo de Achiote durante el año 2020? ● Seniority or experience in production P303: ¿Hace cuánto tiempo es beneficiario o participante de la actividad de apicultura del programa Oro Legal? P404: ¿Hace cuánto tiempo es beneficiario o participante de la actividad de cultivo de achiote del programa Oro Legal? ● Technical assistance, associativity, among others: P324: ¿Qué aporta el programa Oro Legal a la actividad de apicultura en el convenio firmado con usted y su familia? y ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho cada uno de estos aportes? P425: ¿Qué aporta el programa Oro Legal a la actividad de cultivo de achiote en el convenio firmado con usted y su familia? y ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho cada uno de estos aportes? Territorial determinants (𝑍𝑍𝑖𝑖) ● The distance between the place of production and the marketplace, access to demand: P310: ¿Qué medios de transporte usa la mayoría de las veces para transportar su producto de la actividad de apicultura para comercializarlo? P412. ¿Qué medios de transporte usa la mayoría de las veces para transportar su producto de la actividad de cultivo de achiote para comercializarlo? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 214 P312: ¿Cuánto tiempo le toma, en promedio, transportar su producto de la actividad de apicultura para comercializarlo? P414: ¿Cuánto tiempo le toma, en promedio, transportar su producto de la actividad de cultivo de achiote para comercializarlo? ● Transportation costs: P311: ¿Cuánto le cuesta transportar habitualmente la producción de apicultura en toneladas para su comercialización? P412: ¿Cuánto le cuesta transportar habitualmente la producción de achiote en toneladas para su comercialización? ● Dispersion of UPAS: P326: ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la actividad apícola en su región? Opción: Aislamiento geográfico (las colmenas quedan en zonas muy alejadas) P427: ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la producción de Achiote en su región? Opción: Aislamiento geográfico (los cultivos de Achiote quedan en zonas muy alejadas) REFERENCES Auerbach, C. F., & Silverstein, L. B. (2003). Qualitative data: An introduction to coding and analysis. New York, NY, US: New York University Press. Bernal, R & Peña, X. (2011). Guía práctica para la evaluación de impacto. Universidad de los Andes. Facultad de Economía. Chapter 6. Blaikie, N. W. H. (1991): A critique of the use of triangulation in social research. Quality and Quantity. N. 25. Pp. 115-136. Creswell, J. (2003). Qualitative inquiry and research design: among five approaches. (2ª. ed) EUA: SAGE DeCuir-Gunby, J. T., Marshall, P. L., & McCulloch, A. W. (2011). Developing and Using a Codebook for the Analysis of Interview Data: An Example from a Professional Development Research Project. Field Methods, 23(2), 136–155. https://doi.org/10.1177/1525822X10388468 Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development. International Journal of Qualitative Methods, 5(1), 80–92. https://doi.org/10.1177/160940690600500107 Gavira, S. A., & Osuna, J. B. (2015). La triangulación de datos como estrategia en investigación educativa. Pixel-bit. Revista de medios y educación, (47), 73-88. Hesse-Biber, S. N., & Leavy, P. (2006). The Practice of Qualitative Research. SAGE. Lopez, L (2017). Bioestadística. Ed. Universidad Nacional de Colombia. Chapter 9. MacQueen, K. M., McLellan, E., Kay, K., & Milstein, B. (1998). Codebook Development for Team-Based Qualitative Analysis. CAM Journal, 10(2), 31–36. https://doi.org/10.1177/1525822X980100020301 Peña, D. (2002). Análisis de datos multivariantes. Editorial Mc Graw Hill. Chapter 7. P. 201-218. 215 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Rodríguez, C., Pozo, T. & Gutiérrez, J. (2006). La triangulación analítica como recurso para la validación de estudios de encuesta recurrentes e investigaciones de réplica en Educación Superior. RELIEVE. Revista electrónica de Investigación y Evaluación Educativa, 12 (2), p. 289-305. Turner, P., & Turner, S. (2009). Triangulation in practice. Virtual Reality, 13(3), 171-181. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 216 ANNEX V: DATA COLLECTION INSTRUMENTS QUANTITATIVE INSTRUMENT Oro Legal -OL Centro Nacional de Consultoría Calle 82 N° 6-51 Bogotá Teléfono: 339 4888 Instrumento Cuantitativo Centro de Costos: 6206_01 Fecha: enero 2021 Requirió Pr. Piloto Si [x] No [] SI N O O b Elaborado por: Panagora Revisado por: Juliana Márquez Revisado en Campo por: Antonio Ñungo PRESENTACIÓN AL ENCUESTADO Saludo a la persona seleccionada. Buenos días, tardes, mi nombre es (nombre y apellido), y pertenezco al Centro Nacional de Consultoría. Actualmente nos encontramos desarrollando un estudio sobre el proyecto Oro Legal y su información como beneficiario sería valiosa para los propósitos de la investigación. Su participación es voluntaria y su identidad, así como las respuestas suministradas serán de carácter confidencial y los datos se utilizarán únicamente para fines estadísticos. El beneficio de su participación es hacer valer en el estudio la opinión de personas que piensan como usted. El Centro Nacional de Consultoría pide su autorización para recolectar y compartir sus datos personales con fines de contacto para la aplicación de encuestas. Esta solicitud se hace de conformidad con la Ley 1581 de 2012. Con estas precisiones, le pido su autorización para tomar sus datos y hacer la encuesta, que nos tomará aproximadamente __30____ minutos. Autoriza: Sí______ No______ I. PREDILIGENCIAMIENTO 100. ¿A qué segmento pertenece la persona a encuestar, según la base de datos? Formalización Minera 01 Apicultura y Miel 02 Achiote 03 Rehabilitación con Acacia/Especies Nativas 04 217 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV II. CARACTERÍSTICAS DEMOGRÁFICAS 101. ¿Cuál es su Municipio de residencia?: Departamento Municipio Antioquia Barbosa 01 Buriticá 02 Cáceres 03 Caucasia 04 Don Matías 05 El Bagre 06 Nechí 07 Remedios 08 San Roque 09 Segovia 10 Tarazá 11 Zaragoza 12 Chocó Atrato (Yuto) 13 Cantón San Pablo 14 Certegui 15 Condoto 16 Quibdó 17 Rio Quito 18 Tadó 19 Unión Panamericana 20 Otro ¿cuál? Departamento_______________________ Municipio________________ 102. Nombre: 103. Sexo: Hombre 1 // Mujer 2 104. ¿Cuántos años cumplidos tiene usted? Años Cumplidos: |__|__| 105. SE ELIMINÓ ESTA PREGUNTA 106. ¿Usted presenta alguna discapacidad? Sí 01 No 02 107. Vive en la cabecera municipal: Sí 01 No 02 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 218 108. ¿Cuál es su estado civil? (R.U.) Soltero(a) 01 Casado(a) 02 En unión libre 03 Divorciado(a) 04 Viudo(a) 05 109. ¿Cuántas personas componen su hogar?? |__|__| 110. ¿Es usted el/la jefe del hogar? Sí 01 No 02 111. ¿Alguien en su hogar ha sido beneficiario de alguno(s) de los siguientes programas o ayudas? (R.M.) LEER respuestas) Familias en Acción 01 Programas para el adulto mayor 02 Programas de formación para el trabajo 03 Jóvenes en Acción 04 Red Juntos/Unidos 05 Programas del ICBF 06 Subsidio al desempleo 07 Ayudas para desastres naturales 08 Ayudas para desplazados 09 Programa Nacional Integral de Sustitución de cultivos (PNIS) 10 Programa de restitución de tierras 11 Antioquia Libre de Minas 12 Ninguno 13 Otro, ¿Cuál? 88 . 112. De acuerdo con su cultura, pueblo o rasgos físicos . . . es o se reconoce como: (R.U.) Indígena 1 Gitano(a) o Rrom 2 Raizal 3 Palenquero(a) 4 Negro(a), mulato(a), afrodescendiente, afrocolombiano(a) 5 Ningún grupo étnico 6 113. ¿Sabe leer y escribir? Sí 01 No 02 114. ¿Cuál es el nivel educativo más alto aprobado por usted? (R.U) (E: El último año o grado aprobado.) Ninguno o preescolar 01 Básica primaria o básica secundaria 02 Media 03 Superior o universitario (1 a 4 años) 04 Superior o universitario (5 años o más) 05 115. ¿Está afiliado a salud y pensiones? Sí 01 No 02 116. ¿A cuál de los siguientes regímenes de seguridad social está afiliado? Contributivo (EPS) 1 Especial 2 Subsidiado 3 No sabe, no informa 4 . 219 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV III. INGRESOS 117. ¿A cuál de las siguientes actividades se dedica usted principalmente? (RM) Formalización Minera 01 Apicultura y Miel 02 Achiote 03 Rehabilitación con Acacia/Especies Nativas 04 118. Durante los últimos 12 meses, ¿cuánto es el ingreso promedio mensual de las actividades productivas o de generación de ingresos a la que usted se dedica principalmente? MINERÍA: _________________________ APICULTURA: _____________________ ACHIOTE:_________________________ REHABILITACIÓN:___________ (PROG: CAMPO NUMERICO RANGO DE 10.000 EN ADELANTE) (ENC: RECUERDELE A LA PERSONA QUE ESTA HABLANDO DEL PROMEDIO DE LA ACTIVIDAD PRODUCTIVA PRINCIPAL REALIZADA DURANTE LOS ULTIMOS 12 MESES, PEDIR EL MONTO MENSUAL EN PESOS) 119. ¿Algún otro miembro de su hogar aporta ingresos para el hogar? Sí 01 No 02 120. ¿De cuánto es el aporte en dinero que los otros miembros hacen al hogar? _____________(PROG: CAMPO NUMERICO RANGO DE 10.000 EN ADELANTE) (Aplica si (1) en la pregunta 119) 121. ¿Hay alguna otra fuente de ingresos para el hogar? RU (ENC: INDIQUE AL ENCUESTADO QUE SE HABLA DE INGRESOS EN DINERO DIFERENTES A LOS QUE APORTAN LOS MIEMBROS DEL HOGAR) Sí 01 No 02 122. ¿Cuál es el valor en dinero de estos ingresos adicionales? _____________(PROG: CAMPO NUMERICO RANGO DE 10.000 EN ADELANTE) (ENC: PEDIR RESPUESTA EN PESOS) (Aplica si (1) en la pregunta 121) 123. ¿Cuál es el ingreso promedio mensual de su hogar en pesos durante los últimos 12 meses? _____________(PROG: CAMPO NUMERICO RANGO DE 10.000 EN ADELANTE, EL MONTO REGISTRADO NO DEBE SER MENOR A LA SUMA DE LAS RESPUESTAS DADAS EN Q20; Q22 Y Q25) (ENC: RECUERDELE AL ENCUESTADO QUE ESTA HABLANDO DE TODOS LOS INGRESOS EN DINERO QUE HA TENIDO EL HOGAR EN EL ULTIMO AÑO, EJEMPLO SUELDOS O APORTES DE LOS MIEMBROS DEL HOGAR, SUBSIDIOS, VENTAS, ETC.) 124. ¿Cuál es su ingreso promedio mensual individual en pesos durante los últimos 12 meses? _____________ (PROG: CAMPO NUMERICO RANGO DE 10.000 EN ADELANTE) (ENC: RECUERDELE AL ENCUESTADO QUE ESTA HABLANDO DEL INGRESO QUE HA RECIBIDO DE TODAS LAS ACTIVIDADES QUE REALIZA) 125. ¿Qué proporción de los ingresos en su hogar se dedica al pago de deudas? Menos de la mitad 01 La mitad 02 126. ¿Indique el tipo de deudas que tiene su hogar en este momento? Créditos de consumo 01 Créditos para compra de insumos para actividad minera 02 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 220 Más de la mitad 03 Créditos para la actividad agrícola 03 Créditos de libre inversión 04 Créditos de vivienda / automóviles 05 Créditos educativos 06 IV. PROGRAMA ORO LEGAL E leer: A continuación, voy a hacerle unas preguntas sobre el programa Oro Legal del cual usted o su familia es beneficiario. El Programa Oro Legal busca formalizar la actividad minera y ayudar al cuidado del medio ambiente. 129. ¿Cómo se enteró por primera vez del programa de Oro Legal? A través de una Unidad de producción Minera 1 Por una asociación a la que pertenece 2 Por líderes de su comunidad 3 A través de la alcaldía 4 A través de los funcionarios del programa Oro Legal 5 Por un miembro de familia o amigo 6 Otro ¿Cuál? 7 Ns/Nr 99 130. ¿Cuántos miembros del hogar son beneficiarios del programa de Oro Legal? (E: esto significa que trabajan en actividades apoyadas por el programa Oro Legal) |__|__| (E: verificar que este número sea menor o igual a la respuesta 109) 131. ¿Es usted miembro de alguna asociación? (R.M) 1 Sí� 1 Minería 2 Productores 3 Consejo Comunitario 4 Cabildo indígena 5 Asociación de mujeres 6 Deportiva 7 Cultural 8 Víctimas 88 Otra. ¿Cuál? 2 No 99 Ns/Nr 132. Antes de que perteneciera al programa Oro Legal, ¿Cuáles de las siguientes actividades representaban una fuente de ingresos en su hogar? (R.M) (E: Leer respuestas) Minería 01 Agropecuario (diferente a apicultura o achiote) 02 133. Hoy en día, ¿Cuáles de las siguientes actividades representan una fuente de ingresos en su hogar? (R.M) (E: Leer respuestas) Minería 01 221 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Apicultura y miel 03 Siembra y comercialización de Achiote 04 Siembra de Acacia 05 Artesanías 06 Comercio 07 Transporte 08 Servicios sociales, comunales y personales 09 Otro. ¿Cuál? 88 Ns/Nr 99 Agropecuario (diferente a apicultura y achiote) 02 Apicultura y miel 03 Siembra y comercialización de Achiote 04 Siembra de Acacia 05 Artesanías 06 Comercio 07 Transporte 08 Servicios sociales, comunales y personales 09 Otro. ¿Cuál? 88 Ns/Nr 99 134. En una escala de 1 a 5, donde 1 es muy insatisfecho y 5 es muy satisfecho ¿Qué tan satisfecho se encuentra con el programa ORO LEGAL? RU (ENC. NO LEER OPCIONES) 1.MUY INSATISFECHO 2. 3. 4. 5. MUY SATISFECHO V. FORMALIZACIÓN MINERA Solo para los que respondieron 01 en la pregunta 117 201. ¿Usted y su familia participan en la actividad minera del programa Oro Legal? 1 Sí Continúe 2 No Finalice la encuesta 202. ¿La principal actividad económica suya o de su familia estaba relacionada con minería antes de la llegada del programa Oro Legal? Sí 01 No 02 203. ¿Por favor indique el tipo de operación minera en su Unidad de Producción Minera (UPM)? RU: (Leer respuestas) Aluvial 01 Socavón 02 Minidragas 03 204. ¿Qué tipo de explotación es desarrollada en su UPM? RU: (Leer respuestas) Mecanizado 01 Semi-Mecanizado 02 Manual (artesanal) 03 . 205. ¿Cuál es la principal actividad que se desarrolla en su UPM? RU (LEER OPCIONES) Exploración 01 Construcción 02 Extracción 03 Beneficio 04 206. ¿Hace cuánto tiempo inició operaciones estas UPM? Años |__|__| Meses |__|__| 207. ¿Cuántos años de experiencia tiene USTED en la minería de oro? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 222 Años |__|__| Meses |__|__| 208. ¿Por favor indique cuantos trabajadores tiene la UPM en este momento? |__|__| 209. ¿Incluido usted cuántas personas trabajaron en la UPM en el año 2020? |__|__| 210. ¿De las personas que trabajan en la UPM cuantas hacen parte de su núcleo familiar? |__|__| 211. ¿Qué tipo de familiares suyos trabajan o han trabajado en la UPM? (R.M: Leer opciones) Padres 01 Hermanos 02 Hijos 03 Cónyuge 04 Otros:_______________________________ 05 212. ¿En promedio cuantos días del mes se trabaja en la UPM? |__|__| (Nota: No debe superar 30 días) 213. De 1 a 100 donde 100 significa que la UPM se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en su UPM? |__|__|__| (Nota: No debe superar 100) 214. ¿Los trabajadores de esta mina están afiliados a seguridad social (salud y pensiones)? Sí 01 No 02 215. ¿Con que implementos de seguridad industrial cuenta la UPM para sus trabajadores? Cascos 01 Lámparas 02 Botas 03 Audífonos 04 Gafas 05 Guantes 06 Uniformes 07 Todos los anteriores 08 216. ¿Cuál es la principal forma de pago de los salarios de los trabajadores? Efectivo 01 A través del sistema financiero 02 En especie (oro) 03 NS/NR 99 217. ¿Cuál es la principal forma de pago para la compra de insumos relacionados con la producción de oro? Efectivo 01 A través del sistema financiero 02 En especie (oro) 03 NS/NR 99 218. ¿La UPM en la actualidad cuenta con algún tipo de financiación? Sí 01 No 02 219. ¿Por favor indique la fuente de esta financiación? (E. En caso de que responda SI en 218) Sistema financiero 01 Préstamo de un familiar 02 Prestamista 03 Paga diario 04 Otro:___________________________ 88 220. ¿Cuál es el principal proveedor de insumos de la UPM? Comerciante local 01 Comerciante regional 02 Empresa nacional 03 Socio 04 Asociación/Cooperativa 05 Otro:__________________ 88 221. ¿Cuál es el principal comprador del oro producido en esta UPM? Comercializadora internacional 01 Compra-Venta 02 Fundidora 03 Empresa minera gran escala 04 Proveedor como parte de pago 05 Otro:___________________________ 88 223 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 222. ¿Qué tipo de químicos y/o minerales utiliza para la producción de oro? Explosivos 01 Mercurio 02 Cianuro 03 223. ANTES de la participación en el programa Oro Legal ¿Por cada gramo de oro producido en su UPM cuantos gramos de mercurio se utilizaban? (Se responde si en 222 es opción 02) |__|,|__| gramos 224. ACTUALMENTE ¿Por cada gramo de oro producido en su UPM cuantos gramos de mercurio se utilizan? (Se responde si en 222 es opción 02) |__|,|__| gramos 225. ¿Considera que los costos del mercurio limitan actualmente su uso? Sí 01 No 02 226. En una escala de 1 a 5, donde 1 es muy en DESACUERDO y 5 es muy DE ACUERDO ¿Qué tan de acuerdo se encuentra con las siguientes afirmaciones? Afirmación: 1. En desacuerdo 2. Algo en desacuerdo 3. Ni de acuerdo ni en desacuerdo 4. Algo de acuerdo 5. Muy de acuerdo a. Las capacitaciones del Programa Oro Legal ayudaron en la reducción del uso de Mercurio b. El cambio tecnológico ayuda en el no uso de mercurio para el beneficio de oro c. El beneficio que ud obtiene de oro ahora es más alto, gracias a las nuevas prácticas libres de mercurio. d. Las nuevas prácticas de beneficio han mejorado el medio ambiente . 227. ¿Cuáles son sus principales motivaciones para no usar mercurio? (R.M: Espontanea) Mejorar condiciones de salud 01 Proteger las fuentes hídricas 02 Proteger el aire 03 Mantenerse en la legalidad 04 Evitar el cierre de la mina 05 Evitar la destrucción de la maquinaria 06 Evitar ser judicializado 07 Otro:________________________________ 88 . 228. ¿El municipio de venta del oro producido es diferente al municipio donde se realizó la producción? Sí 01 No 02 229. 229. ¿La UPM cuenta con título minero? Sí 01 No 02 230. ¿La UPM guarda registros de sus facturas de ventas o compras de insumos? 231. ¿La UPM cuenta con el Plan de Trabajo en Obras – PTO? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 224 Sí 01 No 02 Sí 01 No 02 232. ¿En el PTO se orientó el beneficio de oro libre de mercurio? Sí 01 No 02 233. ¿El Programa facilitó la implementación de las técnicas y tecnologías determinadas en el PTO? Sí 01 No 02 234. ¿La UPM cuenta con Estudio de Impacto Ambiental? Sí 01 No 02 235. ¿La UPM cuenta con Guía Ambiental? Sí 01 No 02 236. ¿Con cuál de las siguientes autorizaciones ambientales cuenta la UPM? Licencia ambiental 01 PMA 02 PMRRA 03 Otra:___________________________________ 88 237. ¿Con cual de los siguientes permisos ambientales cuenta la UPM? Captación de agua 01 Vertimientos 02 Aprovechamiento forestal 03 Otra:___________________________________ 88 238. ¿La UPM tiene un lugar de almacenamiento del material de arrastre (estéril)? Sí 01 No 02 239. ¿Cómo dispone de las colas/relave luego del proceso de beneficio? Lo arroja al rio 01 Lo entierra 02 Lo arroja al campo abierto 03 Lo procesa 04 Lo almacena 05 No produce relave 06 Otro:________________________________ 88 240. ¿Para usted cuales son las principales ventajas de la formalización minera? (R.M: Espontanea) Acceso a capacitaciones 01 Acceso a mercados con precios más justos 02 Acceso a mejores practicas 03 Simplificación de la titulación 04 Precios de insumos más justos 05 Protección ante riesgos profesionales 06 Mejora en el acceso al crédito 07 Disminución en los costos de titulación 08 Seguridad y tranquilidad personal 09 Ninguna 10 Otra:________________________________ 88 241. ¿Para usted cuales son las principales barreras de la formalización? (R.M: Espontanea) Desconoce el procedimiento 01 Es muy costoso 02 El proceso es complicado 03 El pago de impuestos 04 Rentabilidad insuficiente 05 Otra:_______________________________ 88 225 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 242. ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la actividad minera en su región? (R.M: Leer opciones) Falta de financiamiento 01 Baja productividad 02 Falta de capacitación técnica 03 Falta de capacitación normativa 04 Falta de permisos legales 05 Falta de maquinaria 06 Falta de mano de obra 07 Problemas de orden público 08 Aislamiento geográfico (minas quedan en zonas muy alejadas) 09 Otra:________________________________ 88 243. ¿Continuaría con la actividad minera ante cambios en precios del oro que afectaran sus ingresos actuales (hasta que sus ingresos se reduzcan al X%)? Reducción de hasta el 25% de sus ingresos 01 - Si 02 - No Reducción de hasta el 50% de sus ingresos 01 - Si 02 - No Reducción de hasta el 75% de sus ingresos 01 - Si 02 - No 244. ¿Qué medios de transporte usa la mayoría de las veces para transportar su producto de la actividad minera para comercializarlo? Enc: Si requiere más de un medio de transporte indicar los que apliquen Caballo o mula 1 Lancha o canoa 2 Carro 3 Moto 5 Camión 6 Caminando 7 Otro cual___________ 88 245. ¿La UPM cuenta con alguno de los siguientes servicios públicos? (R.M: Leer opciones) Acueducto 01 Alcantarillado 02 Energía Eléctrica 03 Telefonía 04 Internet 05 Ninguno 06 VI. CADENA PRODUCTIVA APICULTURA Y MIEL Solo para los que respondieron 02 en la pregunta 117 301. ¿Usted y su familia participan en la actividad de apicultura del programa Oro Legal? E: leer El Proyecto productivo Apícola busca implementar una alternativa de desarrollo productivo que permita generar ingresos y ayude al cuidado del medio ambiente. 1 Sí Continúe 2 No Pase a 501 . 302. ¿La principal actividad económica suya o de su familia estaba relacionada con minería antes de la llegada del programa Oro Legal? 1 Sí 2 No . USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 226 303. ¿Hace cuánto tiempo es beneficiario o participante de la actividad de apicultura del programa Oro Legal? (E: Meses cumplidos) Número de meses ______________ 304. ¿Considera usted que la actividad de apicultura y miel es más viable que la actividad minera? 1 Sí 2 No 305. ¿Cuál fue su principal motivación para participar en la actividad de apicultura del programa Oro Legal? (R.U.) (E:Leer) 1º Mención Poder comercializar más fácilmente el producido 1 Mejorar ingresos 2 Proteger el medio ambiente 3 Mejorar condiciones de salud suyas y de su familia 4 Evitar problemas con las autoridades 5 Participar en una actividad menos riesgosa 6 Otro, ¿Cuál? 7 Ns/Nr 99 306. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones: (E: Mostrar tarjeta) La actividad de apicultura y miel le ha ayudado a: Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo a. Mejorar sus condiciones de trabajo 1 2 3 4 5 b. Disminuir los riesgos para su salud 1 2 3 4 5 c. Tener más tiempo para atender el hogar, la familia y asuntos personales 1 2 3 4 5 d. Reducir la incertidumbre sobre el futuro 1 2 3 4 5 e. Tener una menor exposición a riesgos para usted y su familia 1 2 3 4 5 f. Mejorar las condiciones ambientales en su entorno 1 2 3 4 5 307. Desde su participación en el programa ¿Usted o su familia ha recibido algún ingreso de la actividad de apicultura? Sí 1 Continúe No 2 Pase a 314 Ns/N r 9 Pase a 314 308. ¿Con qué frecuencia se reciben ingresos por el desarrollo de la actividad de apicultura dentro del programa de Oro Legal? Diario 1 Semanal 2 Quincenal 3 Mensual 4 Bimensual 5 Trimestral 6 Semestral 7 Annual 8 Ns/Nr 99 309. ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho la frecuencia con la que se reciben ingresos por la actividad de apicultura dentro del programa de Oro Legal? 1.MUY INSATISFECHO 2. 3. 4. 5. MUY SATISFECHO 227 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 310. ¿Qué medios de transporte usa la mayoría de las veces para transportar su producto de la actividad de apicultura para comercializarlo? Enc: Si requiere más de un medio de transporte indicar los que apliquen Caballo o mula 1 Lancha o canoa 2 Carro 3 Moto 5 Camión 6 Caminando 7 Otro ¿cuál?__________ 88 311. ¿cuánto le cuesta transportar habitualmente la producción de apicultura en toneladas para su comercialización? Valor en pesos _________ Enc: Si son varios transportes los que toma, indicar la suma total de los valores. 311.A. ¿El valor que me indicó es para transportar en promedio cuántas toneladas? Valor en número _________ 312. ¿Cuánto tiempo le toma, en promedio, transportar su producto de la actividad de apicultura para comercializarlo? Menos de 15 minutos 1 Entre 16 y 30 minutos 2 Entre 31 y 45 minutos 3 Entre 46 y 60 minutos 4 Entre 1 y 2 horas 5 Entre 2 y 3 horas 6 Entre 3 y 4 horas 7 Más de 4 horas 8 313. De su producción de miel, ¿Cuánto dedica al acopio y venta asociativa y cuánto destina a comercialización individual? Nada Menos de la mitad La mitad Más de la mitad Todo a. Acopio y venta asociativa 1 2 3 4 5 b. Comercialización individual 1 2 3 4 5 314. ¿La principal actividad económica suya o de su familia estaba relacionada con apicultura o producción y comercialización de miel antes de la llegada del programa Oro Legal? Sí 1 No 2 Ns/Nr 99 315. ¿Tuvo que firmar un contrato para participar en la actividad de apicultura del programa Oro Legal? (R.U.) Sí 1 No 2 Ns/Nr 99 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 228 316. ¿Qué compromisos adquirió usted y su familia con el programa de Oro Legal al incorporarse a la actividad de apicultura? (R.M.) Aportar mano de obra 01 Aportar dinero 02 Aportar tierra 03 Aportar materiales 04 Capacitarse 05 Seguir recomendaciones de asesores 06 Dar al programa información verdadera 07 Evitar la deforestación 08 Nada 09 Otro, ¿Cuál? 88 Ns/Nr 99 317. ¿Cuántos lotes ocupa la actividad (Unidad de Producción Agropecuaria) de apicultura del programa Oro Legal? Valor en Número______ 317.A. ¿Qué forma de tenencia tiene el lote que ocupa la actividad (Unidad de Producción Agropecuaria) de apicultura del programa Oro Legal? (Desagregue el área de esos lotes o predios según la forma de tenencia) (R.U) (E: lea cada ítem y espere respuesta para cada uno, llenando inmediatamente el área y la unidad de medida) Forma de tenencia Área Unidad de Medida: 1. Hectárea 2. M2 3. Fanegada 4. Otro 1. Propia sin título _ _ _ _._ _ 1 2 3 4 2. Propia con título _ _ _ _._ _ 1 2 3 4 3. en Arriendo o subarriendo _ _ _ _._ _ 1 2 3 4 4. Aparcería _ _ _ _._ _ 1 2 3 4 5. En usufructo _ _ _ _._ _ 1 2 3 4 6. En concesión _ _ _ _._ _ 1 2 3 4 7. En sucesión con título _ _ _ _._ _ 1 2 3 4 8. En sucesión sin título _ _ _ _._ _ 1 2 3 4 229 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 9. Propiedad colectiva _ _ _ _._ _ 1 2 3 4 . 318. En promedio ¿Incluido usted cuántas personas trabajaron en la Unidad de Producción Agropecuaria (UPA) Apícola durante el año 2020? Valor en númerico |__|__|__| 319. ¿Cuál es el valor de los activos totales con los que cuenta la Unidad de Producción Agropecuaria (UPA) dedicada a Apicultura para el año 2020? Valor en pesos: 320. ¿Cuál fue el valor de la producción de miel o ingresos operacionales de la Unidad de Producción Agropecuaria (UPA) para el año 2020? Valor en pesos: 321. En promedio ¿Cuántas colmenas tuvo la Unidad de Producción Agropecuaria (UPA) durante el año 2020? Valor numérico |__|__|__| 322. ¿Cuál fue el total de kilogramos de miel producidos en la Unidad de Producción Agropecuaria (UPA) durante el año 2020? Valor numérico |__|__|__| 323. 311. De 1 a 100 donde 100 significa que la UPA se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en la (UPA)? |__|__|__| (Nota: No debe superar 100) 324. ¿Qué aporta el programa Oro Legal a la actividad de apicultura en el convenio firmado con usted y su familia? y ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho cada uno de estos aportes? (R.M.) (E: Mostrar Tarjeta 4) No ↓ Sí� Muy insatisfecho Insatisfecho Ni satisfecho ni Insatisfecho Satisfecho Muy satisfecho a. Asesoría legal 2↓ 1� 1 2 3 4 5 b. Asistencia técnica ambiental 2↓ 1� 1 2 3 4 5 c. Asesoría técnica en temas de abejas 2↓ 1� 1 2 3 4 5 d. Asesoría técnica en comercialización de miel y subproductos 2↓ 1� 1 2 3 4 5 e. Asistencia técnica en manipulación de equipos y herramientas 2↓ 1� 1 2 3 4 5 f. Capacitaciones técnicas apícolas 2↓ 1� 1 2 3 4 5 g. Capacitación seguridad en el trabajo 2↓ 1� 1 2 3 4 5 h. Capacitación empresarial 2↓ 1� 1 2 3 4 5 i. Capacitación manejo ambiental 2↓ 1� 1 2 3 4 5 j. Colmenas y abejas 2↓ 1� 1 2 3 4 5 k. Equipos de protección 2↓ 1� 1 2 3 4 5 l. Proceso de convenio para comercialización 2↓ 1� 1 2 3 4 5 m. Otro, ¿Cuál? 2↓ 1� 1 2 3 4 5 . 325. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones. (E: Mostrar tarjeta) Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo g. El apoyo de Oro Legal para mi actividad de apicultura ha contribuido a mejorar la calidad de vida de su familia 1 2 3 4 5 h. El apoyo de Oro Legal para su actividad de apicultura ha hecho que mejoren los ingresos de su hogar 1 2 3 4 5 i. El apoyo de Oro Legal para su actividad de apicultura ha contribuido a que los ingresos en su hogar sean más estables 1 2 3 4 5 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 230 j. Tal y como se estableció el negocio apícola con Oro Legal de aquí a 5 años seguirá funcionando y creciendo 1 2 3 4 5 k. Si no hubiera más apoyo del programa de Oro Legal, seguiría trabajando en la apicultura 1 2 3 4 5 l. La actividad apícola impulsada por Oro Legal ha impedido que se siga afectando el medio ambiente en el territorio. 1 2 3 4 5 m. La actividad apícola impulsada por Oro Legal ha contribuido a mejorar las condiciones de seguridad del territorio. 1 2 3 4 5 326. ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la actividad apícola en su región? (R.M: Leer opciones) Falta de convenios comerciales suficientes y estables 01 Falta de capacidad de compra del acopiador regional (Campo Dulce) 02 Calidad insuficiente del producto de la región 03 Falta de financiamiento 04 Baja productividad 05 Falta de capacitación técnica 06 Falta de equipos apropiados para la extracción y procesamiento de la miel 07 Falta de mano de obra 08 Problemas de orden público 09 Aislamiento geográfico (las colmenas quedan en zonas muy alejadas) 10 Otra:________________________________ 77 327. ¿Continuaría con la actividad apícola si se llegaran a afectar sus ingresos actuales (hasta que sus ingresos se reduzcan al X%)? Reducción de hasta el 25% de sus ingresos 01 - Si 02 - No Reducción de hasta el 50% de sus ingresos 01 - Si 02 - No Reducción de hasta el 75% de sus ingresos 01 - Si 02 - No VII. CADENA PRODUCTIVA CULTIVO ACHIOTE Solo para los que responden 03 en la pregunta 117 401. ¿Usted y su familia participan en la actividad de cultivo de achiote del programa Oro Legal? 1 Sí� Continúe 2 No� pase a 501 402. ¿La principal actividad económica suya o de su familia estaba relacionada con minería antes de la llegada del programa Oro Legal? 1 Sí 2 No 403. ¿Cómo participa usted en la actividad de cultivo de Achiote? (R.M) 1 Mano de obra 2 Prestando la tierra de su propiedad 404. ¿Hace cuánto tiempo es beneficiario o participante de la actividad de cultivo de achiote del programa Oro Legal? (E: Meses completados) 406. ¿Cuál fue su principal motivación para participar en la actividad de cultivo de achiote del programa Oro Legal? (R.U.) (E:Leer) 231 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Número de meses: ______________ 405. ¿Considera usted que el cultivo de Achiote es más viable que la actividad minera? 1 Sí 2 No 1º Mención Poder comercializar más fácilmente el producido 1 Mejorar ingresos 2 Proteger el medio ambiente 3 Mejorar condiciones de salud suyas y de su familia 4 Evitar problemas con las autoridades 5 Participar en una actividad menos riesgosa 6 Otro, ¿Cuál? 7 Ns/Nr 99 . 407. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones: (E: Mostrar tarjeta) La actividad de Cultivo de Achiote me ha ayudado a: Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo n. Mejorar sus condiciones de trabajo 1 2 3 4 5 o. Disminuir los riesgos para su salud 1 2 3 4 5 p. Tener más tiempo para atender el hogar, la familia y asuntos personales 1 2 3 4 5 q. Reducir la incertidumbre sobre el futuro 1 2 3 4 5 r. Tener una menor exposición a riesgos para usted y su familia 1 2 3 4 5 s. Mejorar las condiciones ambientales en su entorno 1 2 3 4 5 408. Desde su participación en el programa ¿Usted o su familia ha recibido algún ingreso proveniente de la venta de achiote? Sí 1 No 2 Ns/N r 99 409. Desde su participación en el programa ¿Usted o su familia ha recibido algún jornal proveniente del programa Oro legal? Sí 1 No Pase a 415 Ns/Nr Pase a 415 410. 410.A.¿Con qué frecuencia se reciben jornales por el desarrollo de la actividad de cultivo de achiote dentro del programa de Oro Legal? Programa OL Diario 1 Semanal 2 Quincenal 3 Mensual 4 Bimensual 5 Trimestral 6 Semestral 7 Anual 8 Ns/Nr 99 410.B.¿Con qué frecuencia se reciben ingresos por ventas en el desarrollo de la actividad de cultivo de achiote dentro del programa de Oro Legal? Ventas Diario 1 Semanal 2 Quincenal 3 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 232 Mensual 4 Bimensual 5 Trimestral 6 Semestral 7 Anual 8 Ns/Nr 99 411. ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho la frecuencia con la que se reciben ingresos por la actividad de cultivo de achiote dentro del programa de Oro Legal? 1.MUY INSATISFECHO 2. 3. 4. 5. MUY SATISFECHO 412. ¿Qué medios de transporte usa la mayoría de las veces para transportar su producto de la actividad de cultivo de achiote para comercializarlo? Enc: Si requiere más de un medio de transporte indicar los que apliquen Caballo o mula 1 Lancha o canoa 2 Carro 3 Moto 5 Camión 6 Caminando 7 Otro cual___________ 88 413. ¿cuánto le cuesta transportar habitualmente la producción de achiote en toneladas para su comercialización? Valor en pesos _________ Enc: Si son varios transportes los que toma, indicar la suma total de los valores. 413.A. ¿El valor que me indicó es para transportar en promedio cuántas toneladas? Valor en número _________ 414. ¿Cuánto tiempo le toma, en promedio, transportar su producto de la actividad de cultivo de achiote para comercializarlo? Menos de 15 minutos 1 Entre 16 y 30 minutos 2 Entre 31 y 45 minutos 3 Entre 46 y 60 minutos 4 Entre 1 y 2 horas 5 Entre 2 y 3 horas 6 233 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Entre 3 y 4 horas 7 Más de 4 horas 8 415. ¿La principal actividad económica de usted o de su familia estaba relacionada con cultivo o comercialización de achiote antes de la llegada del programa Oro Legal? Sí 1 No 2 Ns/Nr 99 416. ¿Tuvo que firmar un contrato para participar en la actividad de cultivo de achiote del programa Oro Legal? (R.U.) Sí 1 No 2 Ns/Nr 99 417. ¿Qué compromisos adquirió usted y su familia con el programa de Oro Legal al incorporarse a la actividad de cultivo de achiote? (RM) Aportar mano de obra 01 Aportar dinero 02 Aportar tierra 03 Aportar materiales 04 Capacitarse 05 Seguir recomendaciones de asesores 06 Dar al programa información verdadera 07 Evitar la deforestación 08 Nada 09 Otro, ¿Cuál? 88 Ns/Nr 99 418. ¿Cuántos lotes ocupa la actividad (Unidad de Producción Agropecuaria - UPA) de cultivo de achiote del programa Oro Legal? Valor en número___________ 418.A.¿Qué forma de tenencia tiene el lote que ocupa la actividad (Unidad de Producción Agropecuaria - UPA) de cultivo de achiote del programa Oro Legal? (Desagregue el área de esos lotes o predios según la forma de tenencia) (R.U) (E: lea cada ítem y espere respuesta para cada uno, llenando inmediatamente el área y la unidad de medida) Forma de tenencia Área Unidad de Medida: 1. Hectárea 2. M2 3. Fanegada 4. Otro 1. Propia sin título _ _ _ _._ _ 1 2 3 4 2. Propia con título _ _ _ _._ _ 1 2 3 4 3. en Arriendo o subarriendo _ _ _ _._ _ 1 2 3 4 4. Aparcería _ _ _ _._ _ 1 2 3 4 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 234 5. En usufructo _ _ _ _._ _ 1 2 3 4 6. En concesión _ _ _ _._ _ 1 2 3 4 7. En sucesión con título _ _ _ _._ _ 1 2 3 4 8. En sucesión con título _ _ _ _._ _ 1 2 3 4 9. Propiedad colectiva _ _ _ _._ _ 1 2 3 4 . 419. En promedio ¿Incluido usted cuántas personas trabajaron en la Unidad de Producción Agropecuaria (UPA) dedicada al cultivo de Achiote durante el año 2020? Valor en numérico |__|__|__| 420. ¿Cuál es el valor de los activos totales con los que cuenta la Unidad de Producción Agropecuaria (UPA) dedicada a Achiote para el año 2020? Valor en pesos: 421. ¿Cuál fue el valor de las ventas de Achiote o ingresos operacionales de la Unidad de Producción Agropecuaria (UPA) para el año 2020? Valor en pesos: 422. ¿Cuál fue el total de toneladas de Achiote producidas en la Unidad de Producción Agropecuaria (UPA) durante el año 2020? Valor numérico y texto |__|__|__| 423. ¿Cuál fue el total de toneladas de Achiote vendidas en la Unidad de Producción Agropecuaria (UPA) durante el año 2020? Valor numérico y texto |__|__|__| 424. De 1 a 100 donde 100 significa que la UPA se encuentra en su máxima capacidad de producción (capacidad instalada). ¿Cómo calificaría el nivel de capacidad instalada en la (UPA)? |__|__|__| (Nota: No debe superar 100) 425. ¿Qué aporta el programa Oro Legal a la actividad de cultivo de achiote en el convenio firmado con usted y su familia? y ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho cada uno de estos aportes? (R.M.) (E: Mostrar tarjeta 4) No ↓ Sí� Muy insatisfecho Insatisfecho Ni satisfecho ni Insatisfecho Satisfecho Muy satisfecho a. Asesoría legal 2↓ 1� 1 2 3 4 5 b. Asistencia técnica ambiental 2↓ 1� 1 2 3 4 5 c. Asistencia técnica en el manejo del cultivo, cosecha y poscosecha. 2↓ 1� 1 2 3 4 5 d. Asesoría técnica en los procesos de transformación 2↓ 1� 1 2 3 4 5 e. Capacitaciones técnicas 2↓ 1� 1 2 3 4 5 f. Capacitación seguridad en el trabajo 2↓ 1� 1 2 3 4 5 g. Capacitación empresarial 2↓ 1� 1 2 3 4 5 h. Capacitación manejo ambiental 2↓ 1� 1 2 3 4 5 i. Insumos 2↓ 1� 1 2 3 4 5 j. Equipos de protección 2↓ 1� 1 2 3 4 5 k. Proceso de convenio para comercialización 2↓ 1� 1 2 3 4 5 l. Otro, ¿Cuál? 2↓ 1� 1 2 3 4 5 426. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones. (E: Mostrar tarjeta 3) 235 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo t. El apoyo de Oro Legal para su actividad de producción de achiote ha contribuido a mejorar la calidad de vida de su familia 1 2 3 4 5 u. El apoyo de Oro Legal para su actividad de producción de achiote ha hecho que mejoren los ingresos de su hogar 1 2 3 4 5 v. El apoyo de Oro Legal para su actividad de producción de achiote ha contribuido a que los ingresos en su hogar sean más estables 1 2 3 4 5 w. Tal y como se estableció el negocio apícola con Oro Legal de aquí a 5 años seguirá funcionando y creciendo 1 2 3 4 5 x. Si no hubiera más apoyo del programa de Oro Legal, seguiría trabajando en el cultivo de achiote. 1 2 3 4 5 y. El cultivo de achiote impulsado por Oro Legal ha impedido que se siga afectando el medio ambiente en el territorio. 1 2 3 4 5 z. El cultivo de achiote impulsado por Oro Legal ha contribuido a mejorar las condiciones de seguridad del territorio. 1 2 3 4 5 427. ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la producción de Achiote en su región? (R.M: Leer opciones) Falta de convenios comerciales suficientes y estables 01 Falta de capacidad de acopio y trillado 02 Falta de capacidad de procesamiento del achiote para la venta 03 Calidad insuficiente del producto de la región 04 Falta de financiamiento 05 Baja productividad 06 Falta de capacitación técnica 07 Falta de maquinaria apropiada de transformación 08 Falta de mano de obra 09 Problemas de orden público 10 Aislamiento geográfico (los cultivos de Achiote quedan en zonas muy alejadas) 11 Otra:________________________________ 88 428. ¿Continuaría con la actividad de Achiote ante cambios en que afecten sus ingresos actuales (hasta que sus ingresos se reduzcan al X%)? Reducción de hasta el 25% de sus ingresos 01 - Si 02 - No Reducción de hasta el 50% de sus ingresos 01 - Si 02 - No Reducción de hasta el 75% de sus ingresos 01 - Si 02 - No VIII. REHABILITACIÓN DE ZONAS DEGRADADAS 501. ¿Usted y su familia participan en la actividad de rehabilitación de zonas degradadas del programa Oro Legal? 1 Sí� Continúe 2 No� Pase a 601 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 236 502. ¿La principal actividad económica suya o de su familia estaba relacionada con minería antes de la llegada del programa Oro Legal? 1 Sí 2 No . 503. ¿Cómo es su participación en la actividad de rehabilitación? 1 Búsqueda de semillas 2 Cultivo de plántulas en vivero 3 Siembra de árboles 504. ¿Hace cuánto tiempo es beneficiario de la actividad de rehabilitación del programa Oro Legal? (E: Meses cumplidos) Número de meses ______________ 505. ¿Estuvo de acuerdo con las especies empleadas para la rehabilitación de las áreas degradadas por minería? 1 Sí 2 No 506. ¿Cuál fue su principal motivación para participar en la actividad de Rehabilitación del programa Oro Legal? (E: Leer las respuestas) Puede marcar más de una opción 1º Mención Rehabilitar 1 Recuperar áreas previamente degrades por minería 2 Proteger el medio ambiente 3 Recuperar especies nativas 4 Recuperar especies importantes para el uso de la comunidad 5 Mejorar condiciones de salud suyas y de su familia 6 Buscar en un futuro seguridad alimentaria 7 Participar en una actividad menos riesgosa 8 Otro, ¿Cuál? 88 Ns/Nr 99 507. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones. (E: Mostrar Tarjeta) Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo a. Las expectativas que tenía de recuperar las áreas degradadas se cumplieron 1 2 3 4 5 b. Fue clara la propuesta de rehabilitación del Programa Oro Legal 1 2 3 4 5 c. Considera que se recuperó el suelo 1 2 3 4 5 d. Considera que se tuvo efectos positivos en el agua 1 2 3 4 5 e. Se ha recuperado el paisaje que se tenía antes de la degradación por minería 1 2 3 4 5 f. Se modificó el paisaje, en comparación con el que se tenía antes de la degradación minera 1 2 3 4 5 g. Los efectos de la recuperación son positivos para la comunidad 1 2 3 4 5 237 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 508. ¿Cómo califica en una escala de 1 a 5, el control que se ha desarrollado para el no retorno de actividades extractivas en la zona intervenida? 1.MUY INSATISFECHO 2. 3. 4. 5. MUY SATISFECHO 509. Si conoce los costos de la implementación, ¿Le parecieron? Altos 1 Adecuados 2 Bajos 3 Ns/ Nr 99 Otro cual___________ 88 510. ¿Espera que la rehabilitación adelantada tenga una retribución económica para usted y su comunidad, mediante algún proyecto de carbono o pago por servicios ambientales? 1 Sí 2 No 511. ¿El trabajo realizado tuvo acompañamiento de alguna de las siguientes entidades? 1.Si 2. No a) Alcaldía b) Corporación autónoma regional o de Desarrollo Sostenible c) Instituto de Investigaciones Ambientales del Pacífico Otro, ¿cuál? 512. ¿Le han indicado cómo se realizará el aprovechamiento de las plantaciones de Acacia? Solo aplica para Antioquia Sí 1 No 2 Ns/Nr 99 513. ¿Tuvo que firmar un contrato para participar en la actividad de rehabilitación con acacia del programa Oro Legal? (R.U.) Sí 1 No 2 Ns/Nr 99 514. ¿Qué compromisos adquirió usted y su familia con el programa de Oro Legal al incorporarse a la actividad de rehabilitación con Acacia (R.M.) Aportar mano de obra 01 Aportar dinero 02 Aportar tierra 03 Aportar materiales 04 Capacitarse 05 USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 238 Seguir recomendaciones de asesores 06 Dar al programa información verdadera 07 Evitar la deforestación 08 Nada 09 Otro, ¿Cuál? 88 Ns/Nr 99 515. ¿Cuántos lotes/parcelas ocupa la actividad de rehabilitación con acacia del programa Oro Legal? Valor en número ____________ 515.A. ¿Qué forma de tenencia tiene el lote que ocupa la actividad de rehabilitaciónn con acacia del programa Oro Legal? (Desagregue el área de esos lotes o predios según la forma de tenencia) (R.U) (E: lea cada ítem y espere respuesta para cada uno, llenando inmediatamente el área y la unidad de medida) Forma de tenencia Área Unidad de Medida: 1. Hectárea 2. M2 3. Fanegada 4. Otro 1. Propia sin título _ _ _ _._ _ 1 2 3 4 2. Propia con título _ _ _ _._ _ 1 2 3 4 3. en Arriendo o subarriendo _ _ _ _._ _ 1 2 3 4 4. Aparcería _ _ _ _._ _ 1 2 3 4 5. En usufructo _ _ _ _._ _ 1 2 3 4 6. En concesión _ _ _ _._ _ 1 2 3 4 7. En sucesión con título _ _ _ _._ _ 1 2 3 4 8. En sucesión sin título _ _ _ _._ _ 1 2 3 4 9. Propiedad colectiva _ _ _ _._ _ 1 2 3 4 . 516. ¿Qué aporta el programa de oro legal para la actividad de rehabilitación con acacia en el convenio firmado con usted y su familia? Y ¿Cómo califica en una escala de 1 a 5 siendo 1 Muy insatisfecho y 5 Muy satisfecho cada uno de estos aportes? (R.M.) (E: Mostrar Tarjeta 4) 239 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV No ↓ Sí� Muy insatisfecho Insatisfecho Ni satisfecho ni Insatisfecho Satisfecho Muy satisfecho a. Asesoría legal 2↓ 1� 1 2 3 4 b. Asistencia técnica ambiental 2↓ 1� 1 2 3 4 5 c. Asesoría técnica en siembra, poda y mantenimiento de la acacia 2↓ 1� 1 2 3 4 5 d. Asesoría técnica en corte, acopio, transporte y comercialización 2↓ 1� 1 2 3 4 5 e. Asistencia técnica en manipulación de equipos y herramientas 2↓ 1� 1 2 3 4 5 f. Capacitaciones técnicas 2↓ 1� 1 2 3 4 5 g. Capacitación seguridad en el trabajo 2↓ 1� 1 2 3 4 5 h. Capacitación empresarial 2↓ 1� 1 2 3 4 5 i. Capacitación manejo ambiental 2↓ 1� 1 2 3 4 5 j. Insumos 2↓ 1� 1 2 3 4 5 k. Equipos de pr tección 2↓ 1� 1 2 3 4 5 l. Gestión para vender producto a mejores precios 2↓ 1� 1 2 3 4 5 m. Otro, ¿Cuál? 2↓ 1� 1 2 3 4 5 517. En una escala de 1 a 5 dónde 1 es completamente en desacuerdo y 5 completamente de acuerdo, qué tan de acuerdo está con cada una de las siguientes afirmaciones. (E: Mostrar Tarjeta) Completamente en desacuerdo Desacuerdo Ni de acuerdo ni en desacuerdo De acuerdo Completamente de acuerdo h. El apoyo de Oro Legal para su actividad de rehabilitación con acacia ha contribuido a mejorar la calidad de vida de su familia 1 2 3 4 5 i. El apoyo de Oro Legal para su actividad de rehabilitación con acacia ha mejorado la salud de su familia 1 2 3 4 5 j. El apoyo de Oro Legal para su actividad de rehabilitación con acacia ha contribuido a que los ingresos en su hogar sean más estables 1 2 3 4 5 k. Quisiera continuar en 10 años con la actividad rehabilitación con acacia del programa Oro Legal 1 2 3 4 5 l. Si no hubiera más apoyo del programa de Oro Legal, seguiría trabajando en la rehabilitación con acacia 1 2 3 4 5 . 518. ¿Para usted cuales de los siguientes factores son obstáculos para el desarrollo de la actividad de rehabilitación en su región? (R.M: Leer opciones) Falta de financiamiento 01 Baja productividad 02 Falta de capacitación técnica 03 Falta de capacitación normativa 04 Falta de permisos legales 05 Falta de maquinaria 06 519. ¿Continuaría con la actividad de rehabilitación ante cambios que afecten sus ingresos actuales (hasta que sus ingresos se reduzcan al X%)? Reducción de hasta el 25% de sus ingresos 01 – Si 02 – No Reducción de hasta el 50% de sus ingresos 01 – Si 02 – No Reducción de hasta el 75% de sus ingresos 01 – Si 02 – No USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 240 Falta de mano de obra 07 Problemas de orden público 08 Aislamiento geográfico (las zonas rehabilitadas quedan en zonas muy alejadas) 09 Otra:________________________________ 88 IX. Actualización de Datos 601. ¿Usted conoce a _________? Sí 1 No Pase a 603 602. ¿Me podría dar el número de contacto de esa persona por favor? ________________________ 603. ¿Usted conoce a _________? Sí 1 No Pase a 604 604. ¿Me podría dar el número de contacto de esa persona por favor? ________________________ 605. ¿Usted conoce a _________? Sí 1 No Pase a 607 606. ¿Me podría dar el número de contacto de esa persona por favor? ________________________ 607. ¿Conoce a alguna otra persona beneficiaria del Programa Oro Legal que no viva con usted? Sí 1 No Finalizar encuesta 608. ¿Me podría dar el número de contacto de esa persona por favor? ________________________ Agradezco en nombre del Centro Nacional de Consultoría su tiempo y colaboración. Le recuerdo que Nuestra Política de Tratamiento de Datos Personales puede encontrarla en la página www.centronacionaldeconsultoria.com y cualquier consulta puede dirigirla al correo cnc@cnccol.com o al Tel: 3394888 de Bogotá. Le deseo un feliz día. QUALITATIVE INSTRUMENTS GUÍA DE ENTREVISTA – ACADEMIA Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. 241 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. Cuénteme un poco en qué consiste el programa Oro Legal - OL (Indagar por los objetivos) 2.2. En líneas generales, ¿tiene conocimiento de qué acciones se han implementado OL en la región? Indague. 2.3. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? 2.4. ¿Cómo se articuló la Universidad y en particular la academia con el programa OL? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la Universidad con el programa OL? 3. Gobernanza y políticas mineras 3.1. ¿Qué percepción tiene del proceso de articulación con las autoridades del gobierno nacional, regionales y municipales? (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas Mineras) 3.2. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades del territorio? 3.3. ¿Qué percepción tiene del hecho de que USAID fuese el financiador del programa OL? 3.4. ¿Cuáles son los factores externos que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) ¿Qué rol juega la academia en esto? 3.5. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? ¿Qué rol cree que puede jugar la academia en esto? 4. Desarrollo de cadenas de valor y medios de vida alternativos 4.1. ¿Qué tan consolidado considera usted que se encuentra el encadenamiento de achiote que apoyó OL como para poder continuar su operación después de la terminación del programa? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 242 4.2. ¿Cuáles son los principales riesgos que usted percibe que puedan afectar la sostenibilidad del agronegocio de los productores en esta cadena? ¿cómo están preparados los productores y sus organizaciones para enfrentarlos? 4.3. ¿Cómo valora usted los aliados comerciales y los mercados con los que se ha encadenado esta iniciativa en términos de estabilidad, requerimientos de calidad y potencial de crecimiento? 4.4. ¿En qué etapa de desarrollo se encuentran el producto resultante del procesamiento de achiote en la transformación que se realiza en la cadena apoyada por OL a través de Achiote y Agros del Chocó (A&ACH)? 4.5. ¿Cuáles son las oportunidades que cree que tiene este producto en la dinámica del mercado actual de la bixina? 4.6. ¿Cómo percibe las capacidades productivas y de procesamiento de los productores y sus organizaciones en cuanto a calidades y volúmenes requeridos en el mercado o mercados que participan? 4.7. ¿Cuáles cree que son los principales retos que deben asumir de aquí en adelante los productores y sus organizaciones para asegurar su sostenibilidad en la cadena? ¿Qué herramientas considera que tienen para afrontarlos? 4.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad y replicabilidad? 5. Percepción de pertinencia y efectividad de la actividad 5.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico a nivel regional, ¿Qué tan pertinente considera que han sido las intervenciones de OL? 5.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 5.3. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio) 5.4. ¿Cuáles cree que han sido las contribuciones de OL en temas de desarrollo económico? 5.5. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por cadenas productivas) 5.6. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) 5.7. ¿Cómo cree que la academia puede llegar a jugar un rol de mayor impacto en programas o intervenciones del tipo OL? ¿Cuál sería ese rol? ¿Cómo lo haría? GUÍA DE ENTREVISTA – ALCALDÍAS Introducción Buenos días/tardes, 243 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal en el municipio? Indagar por actividades con los diferentes actores del territorio 2.3. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? ¿Cómo se alinean esas expectativas con sus intereses profesionales? 2.4. ¿Cómo se articula la Alcaldía con el programa OL? ¿A qué niveles se dio esa articulación (a nivel directivo o equipos técnicos en territorio)? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la Alcaldía con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 2.6. ¿La Alcaldía cofinanció alguna actividad de OL? ¿Cuáles? 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras? ¿La Alcaldía recibió recomendaciones de OL para fortalecer la institucionalidad o las normas relacionadas con la formalización? (Tener en cuenta PTO, Política de formalización minera y permiso de aprovechamiento minero, estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 244 3.2. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mantenerse formales/legales y mejorar el desempeño ambiental de sus operaciones mineras? 3.3. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.4. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y MME para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.5. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? ¿Cuál podría ser el rol de la Gobernación para apoyar la sostenibilidad de los procesos de formalización minera? 3.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿Cómo ha sido el proceso de articulación del programa OL con las autoridades del gobierno nacional, regionales y municipales? ¿OL contribuyó a mejorar estos procesos de coordinación? (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas mineras) 4.2. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.3. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.4. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades del territorio? 5. Desarrollo de cadenas de valor y medios de vida alternativos (No aplica para Condoto) 5.1. ¿Cuál es su percepción frente a incluir alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 5.2. ¿Qué importancia tiene el renglón productivo de miel (Antioquia)/achiote (Chocó) en su municipio y cómo lo está apoyando o tiene planeado apoyarlo la alcaldía durante este gobierno? Solo para Alcaldías de Antioquia: ¿Cómo fue la participación del municipio en los proyectos apícolas de Oro Legal? 5.3. De acuerdo con las dinámicas del territorio ¿considera que los ingresos por actividades productivas alternativas como miel (Antioquia)/achiote (Chocó) son suficientes para disuadir la incursión de los beneficiarios en actividades de minería artesanal? 245 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 5.4. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL para el desarrollo de la cadena productiva de miel (Antioquia)/Achiote (Chocó)? 5.5. ¿Qué cambios en la situación de orden público en las zonas de producción de achiote (Chocó)/miel (Antioquia) se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 5.6. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socioempresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.7. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad y replicabilidad? 6. Rehabilitación de áreas degradadas por minería ilegal 6.1. ¿En qué medida las actividades de rehabilitación de OL se alinean con las directrices del Plan de Desarrollo Municipal? 6.2. ¿En qué medida cree que los modelos de rehabilitación de OL han sido efectivos y pertinentes para lograr una recuperación ecológica en las áreas intervenidas (Acacia mangium) y en Chocó (especies endémicas)? ¿Considera que las zonas intervenidas corresponden a las más afectadas por minería? ¿Por qué? 6.3. ¿La Alcaldía apoyó y podría seguir soportando el proceso de rehabilitación de las áreas intervenidas en el Programa OL? 6.4. ¿Qué incentivos existen para promover la sostenibilidad de los modelos de rehabilitación? ¿Qué otros incentivos podrían desarrollarse para lograr la sostenibilidad? 6.5. ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿Qué instrumentos y enfoques tiene la Alcaldía para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 7.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 7.3. ¿Cuáles cree que son los incentivos o motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 7.4. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 7.5. ¿Cuáles son los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias, precio del mercurio) 7.6. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 246 8.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 8.3. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.4. ¿Cuáles cree que han sido las contribuciones de OL en temas de desarrollo económico? 8.5. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 8.6. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – ALIADOS COMERCIALIZACIÓN DE ORO Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y iv) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 247 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 2.2. ¿Conoce el número de unidades productivas mineras (UPM) de pequeña minería beneficiarias de OL que comercializan el oro a través de su compañía? 2.3. ¿Qué expectativas tenían del programa OL y sus iniciativas productivas y comercialización cuando iniciaron la alianza y en qué medida estas se cumplieron? 2.4. ¿Cuáles diferencias ha identificado su compañía entre las UPM beneficiarias del programa OL y las no beneficiarias? 2.5. ¿Cuántos intermediarios estima que hay entre los beneficiarios de OL y su compañía? 2.6. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de su empresa con la iniciativa de comercialización del programa OL? 3. Formalización minera 3.1. ¿Cuáles fueron las contribuciones técnicas y jurídicas de OL para mejorar los procesos de comercialización del oro de las UPM beneficiarias? 3.2. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de comercialización del oro dentro de los procesos de formalización minera de OL? 3.3. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en la comercialización? 3.4. ¿Cuáles fueron las herramientas o los incentivos que brindó OL a los mineros de MAPE para mejorar la comercialización del oro? (Indagar por la participación de intermediarios, ventas no registradas, uso de efectivo y trueque por encima de la bancarización) 3.5. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? 3.6. ¿Cuál cree que será el futuro de las UPM beneficiarias de OL al concluir el programa? 3.7. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la alianza en la comercialización del oro entre su compañía, OL y los beneficiarios? 3.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de comercialización para incrementar su efectividad y replicabilidad? 3.9. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales sobre los mecanismos de comercialización del oro? 3.10. ¿En qué medida usted cree que las UPM están satisfechas con los resultados de la alianza de comercialización del oro entre su compañía y OL? 4. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 4.1. ¿Cómo cree que la alianza entre su compañía y OL ha contribuido a la eliminación del mercurio en las UPM beneficiadas? 4.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la erradicación del mercurio? (Indagar por costos, grado de dificultad técnica) 4.3. ¿Qué tan claro considera que es el marco normativo sobre eliminación de mercurio? 4.4. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales frente a los lineamientos sobre eliminación de mercurio? 4.5. ¿Cómo cree que puede aportar la cadena de comercialización del oro en la eliminación del uso de mercurio? 4.6. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 248 4.7. ¿Cuáles son los factores facilitadores y los cuellos de botella para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpia, factores externos a OL) 4.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Qué tan pertinente considera las intervenciones de OL en cuanto a la comercialización del oro proveniente de las UPM beneficiarias? 5.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 5.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 5.4. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 5.5. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el órden público? GUÍA DE ENTREVISTA – ALIADOS COMERCIALES PROYECTOS PRODUCTIVOS Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; y ii) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 249 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 2.2. En líneas generales, ¿qué acciones conoce usted que haya implementado Oro Legal y en dónde? Indagar especialmente por actividades de los proyectos productivos 2.3. ¿En qué consistió su participación y la de su empresa, con las iniciativas productivas del programa OL? 2.4. ¿Qué expectativas tenían del programa OL y sus iniciativas productivas cuando iniciaron su participación y en qué medida estas se cumplieron? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de su empresa con la iniciativa productiva de miel/achiote del programa OL? 3. Desarrollo de cadenas de valor y medios de vida alternativos 3.1. ¿Cómo entiende usted que se incluyan alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 3.2. ¿Por favor describa en qué estado se encuentra la relación comercial de su empresa con la organización o empresa que representa la producción de los agricultores/apicultores del Chocó/Bajo Cauca antioqueño? El estado de la relación puede ser en fase inicial, en negociaciones, consolidada, inexistente, distante, etc. 3.3. ¿Su empresa tiene definida una proyección de compras de achiote/miel a los productores que apoyó OL? ¿Esta proyección ha sido acordada con la organización o empresa que los representa? ¿Cómo se acuerdan estas proyecciones de compra? Indagar si nos pueden compartir las proyecciones en caso de tenerlas explicando la confidencialidad de la información. 3.4. ¿Cuáles son y cómo se definieron las condiciones comerciales pactadas con la organización o empresa que representa a los productores de achiote/miel? 3.5. ¿Además de la compra de los productos, en esta relación comercial se acordó también apoyo técnico, logístico o de algún otro tipo? 3.6. ¿Cómo perciben las capacidades productivas y de procesamiento de estos productores y sus organizaciones para poder cumplir a los requerimientos de calidad, volumen y oportunidad de su empresa? 3.7. ¿Cuáles son las razones principales de su empresa para establecer una relación comercial con los productores apoyados por OL y con sus organizaciones? Indagar por especificaciones de calidad propias de sus productos, origen de los mismos, responsabiliad social, precios competitivos, volúmenes atractivos. 3.8. ¿Cómo perciben las capacidades administrativas y financieras de los productores y sus organizaciones para manejar la relación comercial con su empresa? 3.9. ¿Cuáles son los principales riesgos que usted percibe que puedan afectar la relación comercial con los productores en esta cadena? ¿cómo considera que están preparados los productores y sus organizaciones para enfrentarlos? 3.10. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos del programa OL para incrementar su efectividad, replicabilidad y sostenibilidad después de la intervención? Indagar por el modelo de negocio. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 250 4. Percepción de pertinencia y efectividad de la actividad 4.1. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 4.2. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? 4.3. ¿Qué sabe usted de la participación de las mujeres y jóvenes en el desarrollo de las cadenas productivas del programa OL y cómo ha incidido en la calidad de vida de las familias? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos) 4.4. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el órden público? GUÍA DE ENTREVISTA – ALIADOS PROYECTOS PRODUCTIVOS Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; y ii) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 2.2. En líneas generales, ¿qué acciones conoce usted que haya implementado Oro Legal y en dónde? Indagar especialmente por actividades de los proyectos productivos 2.3. ¿En qué consistió su participación y la de su entidad, en el programa OL? 251 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 2.4. ¿Qué expectativas tenía del programa OL cuando inició su participación y en qué medida estas se cumplieron? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para su articulación y la de su entidad con el programa OL? 3. Desarrollo de cadenas de valor y medios de vida alternativos 3.1. ¿Cómo entiende usted que se incluyan alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 3.2. ¿Qué tan consolidado considera usted que se encuentra el encadenamiento de achiote/miel que apoyó OL como para poder continuar su operación después de la terminación del programa? 3.3. ¿Cuáles son los principales riesgos que usted percibe que puedan afectar la sostenibilidad del agronegocio de los productores en esta cadena? ¿cómo están preparados los productores y sus organizaciones para enfrentarlos? 3.4. ¿Cómo perciben las capacidades productivas y de procesamiento de los productores y sus organizaciones en cuanto a calidades y volúmenes requeridos en el mercado o mercados que participan? 3.5. ¿Qué concepto tienen de las capacidades desarrolladas y recursos destinados en el territorio para la prestación de asistencia técnica a los productores de ahora en adelante para garantizar su continuidad y mejoramiento productivo? 3.6. ¿Cómo perciben las capacidades socioempresariales, administrativas y financieras de los productores y sus organizaciones para manejar los diferentes eslabones que les corresponden del agronegocio? miel: asociaciones apícolas y Campo Dulce, achiote: centros de acopio de los consejos comunitarios y A&ACH 3.7. ¿Cómo valora usted los aliados comerciales y los mercados con los que se ha encadenado esta iniciativa en términos de estabilidad, condiciones comerciales, capacidad de compra y potencial de crecimiento? 3.8. ¿Cuáles cree que son los principales retos que deben asumir de aquí en adelante los productores y sus organizaciones para asegurar su sostenibilidad en la cadena? ¿Qué herramientas considera que tienen para afrontarlos? 3.9. Con respecto a los ingresos para las familias de los productores generados por esta actividad, ¿qué tan significativos cree que son, o pueden llegar a ser, en relación con los ingresos totales de estas familias? ¿Cree que podrán completar un ingreso suficiente para mejorar su calidad de vida en el territorio? 3.10. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos del programa OL para incrementar su efectividad, replicabilidad y sostenibilidad después de la intervención? Indagar por el modelo de negocio. 4. Percepción de pertinencia y efectividad de la actividad 4.1. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a rehabilitación y manejo sostenible de recursos) 4.2. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 252 4.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? 4.4. ¿Qué sabe usted de la participación de las mujeres y jóvenes en el desarrollo de las cadenas productivas del programa OL y cómo ha incidido en la calidad de vida de las familias? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos) 4.5. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el órden público? GUÍA DE ENTREVISTA – AGENCIA NACIONAL DE MINERÍA Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos y acciones implementadas) 2.2. ¿Qué expectativas tenía de OL cuando lo conoció y en qué medida estas se cumplieron? 2.3. ¿Cómo se articula la Agencia Nacional de Minería con el programa OL? ¿A qué niveles se dio esa articulación (a nivel directivo o equipos técnicos en territorio)? 253 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 2.4. ¿Cómo fueron las interacciones entre la ANM y OL, para los procesos de contratación y fiscalización de mineros en formalización dentro de los procesos de OL? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la ANM con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras? ¿Recibió la ANM recomendaciones de OL para fortalecer la institucionalidad o las normas relacionadas con la formalización? (Tener en cuenta PTO, Política de formalización minera y permiso de aprovechamiento minero, estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) 3.2. ¿En qué medida las actividades de formalización minera de OL estuvieron alineadas con el Programa de Formalización Minera? 3.3. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.4. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y ANM para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.5. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. ¿Desde la fiscalización minera, se tiene algún seguimiento al desempeño minero y ambiental de los beneficiarios de los procesos de formalización? 3.8. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? ¿cuál podría ser el rol de la ANM para apoyar la sostenibilidad de los procesos de formalización minera? 3.9. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿En qué medida considera que el Programa brindó apoyo técnico o jurídico a la Agencia Nacional de Minería y a las autoridades mineras en relación con el cumplimiento de las normas para el desarrollo de actividades de OL? 4.2. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.3. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 254 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿Cuál es su percepción frente a incluir alternativas productivas en un programa de formalización minera? (Indagar por pertinencia y sostenibilidad para que los mineros no regresen a la extracción informal de oro) 6. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 6.1. ¿Qué instrumentos y enfoques tiene la ANM para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 6.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 6.3. ¿Qué tan claro considera que es el marco normativo del Ministerio de Ambiente sobre eliminación de mercurio? ¿Qué nivel de conocimiento considera que tiene el Ministerio de Ambiente frente a los lineamientos sobre eliminación de mercurio? 6.4. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 6.5. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 6.6. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 6.7. ¿Cuáles son los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias, factores externos a OL) 6.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 7. Percepción de pertinencia y efectividad de la actividad 7.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 7.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 7.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – ASOCIACIÓN DE MINEROS DEL BAJO CAUCA Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este 255 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y iv) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 2.2. ¿Cuántas unidades productivas mineras de pequeña minería pertenecen a la asociación y cuántas fueron beneficiarias del programa Oro Legal? 2.3. ¿Qué expectativas tenían del programa OL y sus iniciativas productivas cuando iniciaron su participación y en qué medida estas se cumplieron? 2.4. ¿Cómo fue la articulación entre OL, la asociación de mineros del Bajo Cauca, las UPM, las autoridades mineras y ambientales, durante el proceso de legalización de los contratos? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la asociación de mineros del Bajo Cauca con la iniciativa productiva del programa OL? 3. Formalización minera 3.1. ¿Cuáles fueron las contribuciones técnicas y jurídicas de OL para el proceso de formalización minera en las UPM pertenecientes a la asociación? 3.2. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.3. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y ANM para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.4. ¿En qué medida OL sirvió de intermediario entre las minas y las autoridades mineras y ambientales? (indagar sobre acompañamiento efectivo frente a trámites complejos). 3.5. ¿Cuáles fueron las herramientas o los incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 256 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? 3.8. ¿Cuál cree que será el futuro de las UPM beneficiarias de OL al concluir el programa? 3.9. ¿Cuántas entidades, públicas y privadas, han realizado actividades relacionadas con la formalización mineras para las UPM pertenecientes a la asociación? 3.10. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 3.12. ¿Qué conoce sobre la licencia ambiental temporal y su impacto sobre el proceso de formalización minera? 4. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 4.1. ¿Cuáles acciones fueron apoyadas por OL para contribuir a la eliminación del mercurio en las UPM pertenecientes a la asociación? 4.2. ¿De uno a diez, siendo diez la erradicación total de mercurio, con cuánto calificaría los logros de los beneficiarios del programa OL? 4.3. ¿Qué instrumentos y enfoques tiene la asociación para contribuir en la eliminación de mercurio en las UPM? (Indagar por avance en nuevas tecnologías, asistencia técnica) 4.4. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 4.5. ¿Qué tan claro considera que es el marco normativo sobre eliminación de mercurio? 4.6. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales frente a los lineamientos sobre eliminación de mercurio? 4.7. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 4.8. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 4.9. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpia promovidas por OL? 4.10. ¿Cuáles son los factores facilitadores y los cuellos de botella para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpia, factores externos a OL) 4.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Qué tan pertinente considera las intervenciones de OL en cuanto a las necesidades de formalización minera, el desempeño ambiental y el desarrollo económico? 5.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 257 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 5.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 5.4. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 5.5. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el órden público? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 258 GUÍA ENTREVISTA – POBLACIÓN BENEFICIARIA Centro Nacional de Consultoría. Calle 82 N° 6-51 Bogotá Teléfono: 339 4888 Proveer a la Actividad MEL de USAID Colombia con servicios técnicos relacionados con recolección de datos. Guía entrevista – Población beneficiaria Centro de Costos: XXX Fecha: 17/01/2021 Diligenciado en Campo por: Elaborado por: USAID Revisado por: Juliana Márquez Revisado en Campo por: GUÍA DE ENTREVISTA – BENEFICIARIOS DIRECTOS Para el Moderador: Antes de iniciar la sesión, los datos que se presentan a continuación deben quedar registrados de viva voz al principio de la grabación. En caso de no tener la autorización de grabar, regístrelos en la presente guía. Nombre Moderador: Fecha y Hora: Grupo / Segmento: Departamento / Municipio / Vereda Día /___/___/ Mes /___/___/ Año /______/ Hora de inicio /___/:/____/ Hora fin /___/:/____/ Se recomienda tener en cuenta las siguientes instrucciones: ● Cuando la entrevista es virtual, el participante será citado 10 minutos antes de la hora prevista para el inicio de la sesión con el objetivo de acompañar el proceso de conexión de cámara y audio según el protocolo de conectividad anexo. ● Al iniciar la sesión realice una breve presentación de usted y del Centro Nacional de Consultoría de acuerdo con la introducción de este documento. ● Lea el consentimiento informado a los participantes y solicite la aprobación para que quede en la grabación. 259 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ● Vigile el tiempo de la sesión procurando que no sea mayor de 60 minutos. ● La Guía es el instrumento que se utiliza como mecanismo para garantizar que se cubre la temática requerida en la investigación. Igualmente se recomienda al moderador no limitarse estrictamente a las preguntas guías y formular nuevas preguntas que permitan profundizar la información. Es de vital importancia facilitar la interacción entre los participantes. Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. Grabación y presentación. Dada la importancia del trabajo que vamos a realizar, vamos a grabar la sesión. Lo vamos a hacer para no perder los valiosos aportes brindados por usted. Nuestro compromiso es guardar la confidencialidad de sus opiniones, así que la información que de aquí surja será de carácter confidencial y no utilizaremos los nombres o los datos de las personas del proyecto para dar información sobre lo que aquí conversemos. ¿Tienen alguna pregunta o inquietud antes de comenzar? 1. PRESENTACIÓN DEL ENTREVISTADO (TODOS) 1.1. ¿Cuál es su nombre? 1.2. ¿En qué municipio vive? 1.3. ¿Usted ha sido beneficiario directo o indirecto del programa OL? ¿En qué componentes? En caso de que la respuesta sea NO, por favor verifique, de lo contrario termine la entrevista. 2. CONOCIMIENTOS Y RELACIÓN CON LA ACTIVIDAD (TODOS) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 260 2.1. ¿Qué conoce del programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar por los objetivos y acciones implementadas en los tres componentes (Formalización minera, cadenas productivas y rehabilitación). 2.2. ¿Qué expectativas tenía del programa OL y sus iniciativas cuando se vinculó y en qué medida estas se cumplieron? 3. MÓDULO FORMALIZACIÓN MINERA 3.1. ¿Usted ha participado en actividades de formalización minera con el programa Oro Legal? Cuando la respuesta sea SÍ, continuar. Cuando la respuesta sea NO, pasar al siguiente módulo (5. MÓDULO CADENA PRODUCTIVA – APICULTURA) 3.2. ¿Usted pertenece a alguna comunidad o consejo comunitario? ¿Cuál? 3.3. ¿Cuál es su cargo en la mina? ¿Hace cuánto tiempo está en este cargo? 3.4. ¿A cuál tipo de titulación minera accedió la UPM? a) Beneficiario de área de reserva especial - ARE, b) Subcontrato de Formalización Minera, c) Extensión del programa de legalización de minería de hecho, d) solicitud de título minero. 3.5. ¿Cómo se enteró del programa OL y cómo se dio el acercamiento del programa a su mina? (Indagar por el proceso de inclusión como beneficiari@ del programa) 3.6. ¿En qué consistió el acompañamiento de OL en las actividades de formalización minera? ¿Durante cuánto tiempo recibió ese acompañamiento? 3.7. ¿Qué lo/la motivó a buscar la formalización minera de su mina? 3.8. ¿Qué herramientas o incentivos brindó OL a la mina para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.9. ¿Cómo fue la articulación entre OL, la mina, las autoridades mineras y ambientales, durante el proceso de formalización? 3.10. ¿En qué medida OL sirvió de intermediario entre la mina y las autoridades mineras y ambientales? (indagar sobre acompañamiento efectivo frente a trámites complejos). 3.11. ¿Cuáles fueron las contribuciones técnicas y jurídicas de OL para el proceso de formalización minera en la mina? 3.12. ¿Cuáles fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.13. ¿Cuáles fueron los principales logros alcanzados por la mina con el acompañamiento de OL? (obtención de título minero, licencia ambiental, aplicación de normas de seguridad laboral, etc.) 3.14. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.15. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuál cree que será el futuro de la mina al concluir el programa de OL? 3.16. ¿Además de OL, cuáles entidades, públicas y privadas, han realizado actividades relacionadas con la formalización mineras en su mina? (Indagar por posibles excesos de asistencia no coordinada entre múltiples actores institucionales, académicos, internacionales, etc.) 3.17. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera apoyada por OL para incrementar su efectividad y replicabilidad? 3.18. Para beneficiarios de subcontrato de formalización: ¿Qué tan satisfecho se encuentra con las condiciones establecidas en el subcontrato de formalización? 261 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 3.19. ¿Qué conoce sobre la licencia ambiental temporal y su impacto sobre el proceso de formalización minera? 4. ELIMINACIÓN/REDUCCIÓN DEL MERCURIO EN LA CADENA DE SUMINISTRO DE LA MAPE (SOLO PARA QUIENES HAN PARTICIPADO EN ACTIVIDADES DE FORMALIZACIÓN MINERA) 4.1. ¿Cuáles acciones fueron apoyadas por OL para contribuir a la eliminación del mercurio en la mina? 4.2. ¿Cuáles han sido los avances en nuevas tecnologías en su mina para reducir o eliminar el mercurio? 4.3. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esas nuevas tecnologías? (Indagar por costos, grado de dificultad técnica) 4.4. ¿Qué opinión tiene sobre los avances en la reducción de mercurio dentro de la mina con el acompañamiento de OL? 4.5. ¿Qué tan claro considera que es el marco normativo sobre eliminación de mercurio? 4.6. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales frente a los lineamientos sobre eliminación de mercurio? 4.7. ¿Cuáles son las principales motivaciones que tiene para eliminar el mercurio en la operación minera? 4.8. ¿En qué medida se siente satisfecho con la implementación de las técnicas de producción más limpia promovidas por OL? ¿Por qué? 4.9. ¿Cuáles son los factores facilitadores y los cuellos de botella para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias, factores externos a OL) 4.10. En su municipio o región ¿cuáles son las estrategias que se han utilizado para sensibilizar sobre la eliminación del uso de mercurio en la actividad minera de oro? 4.11. ¿Qué ajustes cree que podrían hacerse a las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 5. MÓDULO CADENA PRODUCTIVA – APICULTURA (INDAGAR SOLO A BENEFICIARIOS DE ANTIOQUIA) 5.1. ¿Usted ha participado en actividades de apicultura con el programa Oro Legal? Cuando la respuesta sea SÍ, continuar. Cuando la respuesta sea NO, pasar al siguiente módulo (7. MÓDULO REHABILITACIÓN) 5.2. ¿A qué organización de apicultores pertenece? ¿Hace cuánto? 5.3. ¿Ocupa algún cargo en esta organización? ¿Cuál? 5.4. ¿Usted o su familia realizaba actividades mineras o apícolas antes del programa OL? 5.5. ¿Cómo se enteró del programa OL y cómo se dio el acercamiento del programa a su comunidad? (Indagar por el proceso de inclusión como beneficiari@ del programa). 5.6. ¿En qué consistió el acompañamiento de OL en las actividades apícolas? ¿Durante cuánto tiempo recibió ese acompañamiento? 5.7. ¿Cuáles fueron las propuestas iniciales del programa para fortalecer la producción y comercialización apícola en el territorio y en qué medida se cumplieron? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 262 5.8. ¿Cuáles considera que son los aprendizajes y capacidades que usted adquirió como productor de miel gracias al programa de OL? 5.9. Después de finalizado el programa OL, ¿cómo quedaron establecidas las condiciones de acceso a insumos, material genético y asistencia técnica para poder desarrollar su actividad apícola de manera rentable? 5.10. ¿Considera que la actividad apícola que usted desarrolla es una actividad individual o es asociativa? ¿Por qué? ¿Cuál es el papel de la asociación de apicultores en sus procesos productivos y de comercialización? 5.11. ¿Qué concepto tiene usted de los aliados comerciales y de los mercados en los cuales se está vendiendo la miel que usted produce? 5.12. ¿Cuáles han sido las principales dificultades y aciertos que han tenido en la comercialización de la miel que producen? 5.13. Desde su conocimiento, por favor describa ¿cómo quedó funcionando el proceso de producción y comercialización de miel en el territorio después de la intervención de OL? 5.14. ¿Cuáles son los planes que tienen como productores y organización de apicultores para mantener y mejorar la producción y comercialización de miel en el futuro? ¿Qué herramientas, recursos y apoyos considera que tienen a su disposición para hacerlo? 5.15. Con la dotación y capacidades adquiridas por usted con el programa, ¿cómo percibe que cambiarán sus ingresos familiares debido a las actividades apícolas? 5.16. ¿Considera que los ingresos originados por la actividad apícola son suficientes para que usted y su familia mejoren su calidad de vida en el territorio? 5.17. Además de los ingresos monetarios, ¿qué otros beneficios cree que le ha dejado la actividad apícola a su familia? Indagar por tranquilidad, salud, seguridad, tiempo en familia, redes de apoyo comunitarias. 5.18. ¿Cuál es su opinión del negocio apícola impulsado por OL como una alternativa a las actividades mineras en el territorio? ¿En su familia, o en otras familias que usted conozca, con el programa OL se han dado cambios de la actividad minera para dedicarse a las labores apícolas? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 5.19. ¿Cómo fue la participación de las mujeres y los jóvenes en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? 5.20. ¿Qué ajustes o recomendaciones cree que podrían realizarse en el apoyo a este tipo de proyectos para mejorar sus resultados y sostenibilidad a futuro? 6. MÓDULO CADENA PRODUCTIVA – ACHIOTE (INDAGAR SOLO A BENEFICIARIOS DE CHOCÓ) 6.1. ¿Usted ha participado en actividades productivas de achiote con el programa Oro Legal? Cuando la respuesta sea SÍ, continuar. Cuando la respuesta sea NO, pasar al siguiente módulo (7. MÓDULO REHABILITACIÓN DE ÁREAS DEGRADADAS POR MINERÍA) 6.2. ¿A qué Consejo comunitario pertenece? ¿Hace cuánto? 6.3. ¿Ocupa algún cargo en el consejo comunitario? ¿Cuál? 6.4. ¿Usted o su familia realizaba actividades mineras o de producción de achiote antes del programa OL? 263 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 6.5. ¿Cómo se enteró del programa OL y cómo se dio el acercamiento del programa a su comunidad? (Indagar por el proceso de inclusión como beneficiari@ del programa). 6.6. ¿En qué consistió el acompañamiento de OL en las actividades de producción de achiote? ¿Durante cuánto tiempo recibió ese acompañamiento? 6.7. ¿Cuáles fueron las propuestas iniciales del programa para fortalecer la producción, transformación y comercialización de achiote en el territorio y en qué medida se cumplieron? 6.8. ¿Cuáles considera que son los aprendizajes y capacidades que usted adquirió como productor de achiote gracias al programa de OL? 6.9. Después de finalizado el programa OL, ¿cómo quedaron establecidas las condiciones de acceso a insumos, asistencia técnica y acopio del achiote, para poder desarrollar su actividad productiva de manera rentable? 6.10. ¿Cuál es el papel del Consejo Comunitario, como organización de productores, en sus procesos productivos y de comercialización? 6.11. ¿Qué opinión tiene sobre la constitución de la empresa Achiote y Agros del Choco, con participación de los consejos comunitarios, como un eslabón de transformación del achiote para su comercialización? 6.12. ¿Qué concepto tiene usted de los aliados comerciales y de los mercados en los cuales se está vendiendo el achiote que usted produce? 6.13. ¿Cuáles han sido las principales dificultades y aciertos que han tenido en la transformación y comercialización del achiote que producen? 6.14. Desde su conocimiento, por favor describa ¿cómo quedó funcionando el proceso de producción, transformación y comercialización de achiote en el territorio después de la intervención de OL? 6.15. ¿Cuáles son los planes que tienen como productores y como consejo comunitario para mantener y mejorar la producción, transformación y comercialización de achiote en el futuro? ¿Qué herramientas, recursos y apoyos considera que tienen a su disposición para hacerlo? 6.16. Con la dotación y capacidades adquiridas por usted con el programa, ¿cómo percibe que cambiarán sus ingresos familiares debido a las actividades de producción de achiote? 6.17. ¿Considera que los ingresos originados por la actividad productiva del achiote son suficientes para que usted y su familia mejoren su calidad de vida en el territorio? 6.18. Además de los ingresos monetarios, ¿Qué otros beneficios considera que le ha dejado la actividad productiva de achiote a su familia? Indagar por tranquilidad, salud, seguridad, tiempo en familia, redes de apoyo comunitarias. 6.19. ¿Cuál es su opinión del agronegocio de achiote impulsado por OL como una alternativa a las actividades mineras en el territorio? ¿En su familia, o en otras familias que usted conozca, con el programa OL se han dado cambios de la actividad minera para dedicarse a las labores productivas de achiote? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 6.20. ¿Cómo fue la participación de las mujeres y los jóvenes en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? 6.21. Antes del programa OL ¿qué uso le daba al área en donde estableció su cultivo de achiote? ¿tuvo que cumplir alguna norma o protocolo que impidiera deforestar áreas boscosas para establecer su cultivo? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 264 6.22. ¿Qué ajustes o recomendaciones cree que podrían realizarse en el apoyo a este tipo de proyectos para mejorar sus resultados y sostenibilidad a futuro? 7. MÓDULO REHABILITACIÓN DE ÁREAS DEGRADADAS POR MINERÍA 7.1. ¿Usted ha participado en actividades de rehabilitación con el programa Oro Legal? Cuando la respuesta sea SÍ, continuar. Cuando la respuesta sea NO, pasar al siguiente módulo (8. MÓDULO PERCEPCIÓN DE PERTINENCIA Y EFECTIVIDAD DE LA ACTIVIDAD) Hacer las preguntas 7.2 a 7.5 si no se han contestado previamente: 7.2. ¿Pertenece a alguna organización o consejo comunitario? ¿Cuál? 7.3. ¿Ocupa algún cargo en la organización/consejo comunitario? ¿Cuál? 7.4. ¿Usted o su familia realizaba actividades de rehabilitación de zonas degradadas antes del programa OL? 7.5. ¿Cómo se enteró del programa OL y cómo se dio el acercamiento del programa a su comunidad? (Indagar por el proceso de inclusión como beneficiari@ del programa) 7.6. ¿Hace cuánto tiempo Ud. identificaba esa zona como un área degradada por minería?, ¿cómo describiría la condición ambiental de dicha zona? 7.7. ¿En qué consistió la actividad de rehabilitación de las zonas degradadas por minería? 7.8. ¿Sabe si se trabajó con alguna entidad del Estado en esta actividad? (indagar por la planeación del programa y por la sostenibilidad de las áreas rehabilitadas) 7.9. ¿Qué lo motivó a desarrollar la rehabilitación de las zonas degradadas por minería? (Indagar por Reforestar, Volver a la condición inicial del paisaje, Recuperar especies que se habían perdido, Generar ingresos y Dejar unas mejores condiciones para sus hijos) 7.10. ¿Cómo se desarrolló el crecimiento de las semillas y plántulas en los viveros? ¿Qué personas realizaron esta actividad? 7.11. ¿Cuál fue la estrategia de siembra? (Indagar por el distanciamiento entre árboles, complemento con otras especies de plantas) 7.12. ¿En qué consistió el acompañamiento de OL en las actividades de rehabilitación? ¿Durante cuánto tiempo recibió ese acompañamiento? 7.13. ¿Cuáles considera que son los aprendizajes y capacidades que usted adquirió participando en la rehabilitación de la zona degradada? (Indagar por capacitaciones recibidas y percepción sobre las mismas) 7.14. ¿Usted recibió algún tipo de ingreso por participar en el proceso de rehabilitación? (indagar si fueron los mismos actores en cada etapa de la rehabilitación) 7.15. ¿Qué otros pasos, adicionales a la siembra de los árboles, se ejecutaron para rehabilitar la zona? 7.16. ¿Qué efectos ha tenido hasta ahora la siembra de los árboles en la zona? (Indagar por biodiversidad, suelos, agua, entre otros) 7.17. Solo para beneficiarios de Antioquia: ¿Cómo esperan hacer uso de la madera de los árboles de Acacia? 7.18. ¿En qué medida usted cree que se ha generado compromiso y apropiación por parte de la comunidad para mantener los modelos de rehabilitación? 7.19. ¿Considera apropiados los modelos de rehabilitación utilizados por OL? 265 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 7.20. De las acciones realizadas en las actividades de rehabilitación, ¿Qué hubiera hecho diferente o qué podría hacerse mejor si se quisiera continuar con el apoyo? 8. MÓDULO PERCEPCIÓN DE PERTINENCIA Y EFECTIVIDAD DE LA ACTIVIDAD (TODOS) 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico en su región ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿Cuáles cree que han sido las contribuciones de OL para usted y para su región? (Indagar por temas ambientales y minería artesanal y de pequeña escala) 8.3. ¿Considera que las acciones apoyadas por OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 8.4. ¿En qué medida las actividades apoyadas por OL en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? 8.5. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 8.6. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) CIERRE Para terminar, por favor dígame en una frase ¿Cómo resume usted el Programa OL? En nombre del Centro Nacional de Consultoría, agradecemos por su participación y su tiempo para la realización de este estudio. Tenga usted un feliz día. GUÍA DE ENTREVISTA – CORPORACIONES AUTÓNOMAS REGIONALES Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 266 para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos y acciones implementadas) 2.2. ¿Qué expectativas tenía de OL cuando lo conoció y en qué medida estas se cumplieron? 2.3. ¿Cómo se articula la Corporación con el programa OL? 2.4. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la Corporación con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras en los temas ambientales? (Tener en cuenta estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) 3.2. En caso de realizar programas similares en un futuro, ¿cómo podría USAID desarrollar una gestión que fortalezca o apoye procesos de la Corporación en materia de formalización minera? 3.3. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar el desempeño ambiental de sus operaciones mineras? 3.4. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL para minimizar los impactos ambientales generados por la actividad minera? 3.5. ¿En qué medida considera que el desempeño ambiental de las operaciones mineras formalizadas ha mejorado por la intervención de OL? 3.6. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y MME para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.7. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿Cómo ha sido el proceso de articulación del programa OL con las autoridades del gobierno nacional, regionales y municipales? ¿OL contribuyó a mejorar estos procesos de coordinación? 267 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas mineras) 4.2. ¿En qué medida considera que el Programa brindó apoyo técnico o jurídico a la Corporación u otros actores ambientales en relación con el cumplimiento de las normas ambientales para el desarrollo de actividades de OL? 4.3. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.4. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.5. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades del territorio? 4.6. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿En qué medida considera que OL contribuye al mejoramiento ambiental de las regiones al promover actividades productivas agropecuarias (miel y achiote) como alternativas a la minería artesanal? 5.2. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socioempresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.3. Teniendo en cuenta el desarrollo de las cadenas productivas, ¿podrían considerarse estas cadenas como parte de los programas de pago por servicios ambientales o acciones de mitigación de cambio climático o negocios verdes? 6. Rehabilitación de áreas degradadas por minería ilegal 6.1. ¿En qué medida son apropiados los modelos de rehabilitación utilizados por OL en Antioquia (Acacia mangium) y en Chocó (especies endémicas)? ¿Esto se enmarca con los lineamientos de la política ambiental? (Tener en cuenta Plan Nacional de Restauración, lineamientos del IIAP) 6.2. ¿Considera que las zonas intervenidas corresponden a las más afectadas por minería? ¿Por qué? 6.3. ¿En qué medida cree que los modelos de rehabilitación han sido efectivos para lograr una recuperación ecológica en las áreas intervenidas? 6.4. ¿La Corporación apoya y podría seguir soportando el proceso de rehabilitación de las dos áreas intervenidas en el Programa OL? 6.5. ¿Qué incentivos existen para promover la sostenibilidad de los modelos de rehabilitación? ¿Qué otros incentivos podrían desarrollarse para lograr la sostenibilidad? 6.6. ¿Existen diagnósticos sobre los pasivos huérfanos en Antioquia y Chocó, así como modelos de rehabilitación y los costos de su implementación? ¿Esta información es pública? ¿Los implementadores en territorio tienen acceso a esta información? 6.7. ¿Cree que ha existido compromiso y apropiación de los beneficiarios para mantener los modelos de rehabilitación? ¿Por qué? 6.8. ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 268 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿Qué instrumentos y enfoques tiene la Corporación para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 7.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 7.3. ¿Qué tan claro considera que es el marco normativo del Ministerio de Minas sobre eliminación de mercurio? ¿Qué nivel de conocimiento considera que tiene el Ministerio de Minas frente a los lineamientos ambientales sobre eliminación de mercurio? 7.4. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – COORDINADORES REGIONALES ORO LEGAL Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. 269 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. Cuénteme un poco en qué consiste el programa Oro Legal (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal en la región? Indagar por actividades con los diferentes actores del territorio 2.3. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? ¿Cómo se alinean esas expectativas con sus intereses profesionales? 2.4. ¿Cómo está estructurado el equipo de trabajo de la regional? ¿Este equipo es suficiente para el desarrollo de las actividades del programa? ¿Cómo es la articulación con la dirección del programa y los territorios? 3. Formalización minera 3.1. ¿Cómo fue el proceso de focalización y selección de los beneficiarios de formalización minera? 3.2. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mantenerse formales/legales y mejorar el desempeño ambiental de sus operaciones mineras? 3.3. ¿Cuáles fueron los cuellos de botella y factores facilitadores para las actividades de formalización minera? (indagar por temas mineros y ambientales) 3.4. ¿Cuáles aspectos se identificaron como útiles o necesarios para reducir la complejidad en el proceso de formalización? ¿Cuál fue la contribución de OL para reducir la complejidad? (indagar por temas mineros y ambientales) 3.5. ¿Cómo percibió OL la valoración que dan los mineros al apoyo ofrecido para su formalización? ¿Habría más compromiso si los mineros tuvieran que pagar por algunos trámites? 3.6. ¿Cómo planeó OL que al finalizar el proyecto los mineros sigan cumpliendo con un aprovechamiento minero legal? 3.7. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿Cómo ha sido el proceso de articulación con las autoridades del gobierno nacional, regionales y municipales? (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas Mineras) 4.2. ¿Cuáles fueron los mecanismos de participación de los beneficiarios de OL para la toma de decisiones sobre el programa? (Indagar por formalización minera, cadenas productivas y rehabilitación) 4.3. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades del territorio? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 270 4.4. ¿Cuáles fueron los beneficios o inconvenientes que generó el hecho de que USAID fuera el financiador del programa OL? 4.5. ¿Cuáles son los factores externos que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.6. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿Cómo fue el proceso de focalización y selección de los beneficiarios de las cadenas de valor desarrolladas por OL? (Indagar por apicultura y achiote) 5.2. ¿Cuáles son las principales capacidades con las que el programa fortaleció a los pequeños productores de achiote/miel para que aseguren un ingreso suficiente? (Indagar por los 3 eslabones de la cadena productiva) 5.3. ¿En qué medida considera que el fortalecimiento de actividades productivas alternativas a la minería reduce el impacto ambiental en la zona? ¿El programa hizo un análisis previo sobre los posibles impactos ambientales del desarrollo de las cadenas productivas? ¿Qué estrategias se adelantaron para mitigar estos efectos? 5.4. ¿Qué cambios en la situación de orden público en las zonas de producción de achiote (Chocó)/miel (Antioquia) se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 5.5. ¿Cómo fue la participación de las mujeres y los jóvenes de los núcleos familiares en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? 5.6. (Antioquia) ¿Qué rol desempeñan las asociaciones de productores de miel y cómo las preparó el programa para prestar servicios a sus asociados en el desarrollo del negocio apícola? 5.7. (Antioquia) ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de miel? (Tener en cuenta los eslabones: productores, asociaciones apícolas, Campo Dulce y aliados comerciales) 5.8. (Chocó) ¿Qué rol desempeñan los consejos comunitarios y cómo los preparó el programa para el desarrollo de la cadena de achiote? 5.9. (Chocó) ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de achiote? (Tener en cuenta los eslabones: productores, centros de acopio y trillado de los consejos comunitarios, A&ACH y Naranja Madura) 5.10. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socio-empresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad y replicabilidad? 6. Rehabilitación de áreas degradadas por minería ilegal 6.1 ¿Cómo se seleccionaron las zonas a rehabilitar y los beneficiarios? (Indagar por diagnósticos de pasivos huérfanos) 271 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 6.2 ¿La estrategia de rehabilitación se enmarca en los lineamientos ambientales de la política ambiental? (Tener en cuenta Plan Nacional de Restauración, lineamientos de IIAP (Chocó)) 6.3 ¿En qué medida cree que los modelos de rehabilitación han sido efectivos para lograr una recuperación ecológica en las áreas intervenidas? 6.4 ¿Cree que ha existido compromiso y apropiación de los beneficiarios para mantener los modelos de rehabilitación? ¿Por qué? 6.5 ¿Qué incentivos cree que tienen las familias para mantener en el tiempo los modelos de rehabilitación? 6.6 ¿Cuáles fueron los factores facilitadores y cuellos de botella para las actividades de rehabilitación? 6.7 ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿En qué consistió la estrategia técnica brindada a los mineros para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por asistencia técnica, cumplimiento de normativa, tecnificación, sensibilización riesgos a la salud y ambientales) 7.2. ¿Cuáles cree que fueron las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 7.3. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 7.4. ¿Cuáles fueron los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias) 7.5. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿En qué medida las actividades de OL estuvieron alineadas con las directrices de los instrumentos de planeación y ordenamiento territorial? 8.3. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 8.4. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.5. ¿Cuáles cree que han sido las contribuciones de OL en temas de desarrollo económico? 8.6. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 8.7. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 272 GUÍA DE ENTREVISTA – DONATARIOS ACHIOTE Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; y ii) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿A que consejo comunitario pertenece? 1.3. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. Según su conocimiento, cuénteme en qué consiste el programa Oro Legal. (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal en la región? Indagar por actividades con los diferentes actores del territorio 2.3. ¿Cómo fue el proceso por el cual se decidió y se aceptó la participación de su organización en el programa OL? 2.4. ¿En qué consistió la participación y cuál fue el rol de su organización en el programa OL? 2.5. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? 2.6. ¿Cuáles fueron los cuellos de botella y los factores facilitadores para su articulación y la de su organización con el programa OL? 3. Desarrollo de cadenas de valor y medios de vida alternativos 273 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 3.1. ¿Cómo entiende usted que se incluyan alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 3.2. Desde su perspectiva, ¿cuál era la estrategia que tenía el programa en su inicio para el fortalecimiento de la cadena de achiote y en qué medida se cumplió? 3.3. A la finalización del programa y según su comprensión, ¿considera que durante la ejecución se estructuró y se implementó un modelo de negocio factible para los productores de achiote y para sus organizaciones en el largo plazo? 3.4. ¿Cuáles son los eslabones de la cadena de achiote que quedaron conformados o fortalecidos con el proyecto y qué funciones cumple cada uno? (productores, consejos comunitarios, centros de acopio y trillado, empresa de transformación comunitaria A&ACH) 3.5. ¿Cómo fue el proceso de focalización y selección de los productores beneficiarios que participaron en el proyecto de achiote? 3.6. ¿Cuáles son las principales capacidades con las que el programa fortaleció a los pequeños productores de achiote para que puedan mejorar sus ingresos a través de esta actividad? 3.7. Hablando de los consejos comunitarios como organizaciones de productores de achiote, ¿cuáles fueron las capacidades que se fortalecieron con el programa en aspectos técnico productivos, socioempresariales y comerciales? ¿Cómo cree que esto impacta en la sostenibilidad de este agronegocio? 3.8. ¿Cuáles alternativas dejó establecidas el programa para la prestación de servicios de acompañamiento y asistencia técnica a los productores luego de finalizar la intervención y cómo se van a financiar? 3.9. Teniendo en cuenta que Achiote y Agros del Choco (A&ACH sas) es una empresa recientemente constituida, ¿en qué condiciones se encuentra para continuar con el negocio? ¿Requiere de recursos, herramientas o apoyos adicionales para poder sostenerse? 3.10. ¿Cómo valora usted los aliados comerciales y los mercados con los que se ha encadenado esta iniciativa en términos de estabilidad, condiciones comerciales, capacidad de compra y potencial de crecimiento? 3.11. ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de achiote dentro y fuera del territorio? (Tener en cuenta los eslabones: productores, consejos comunitarios, centros de acopio y trillado, empresa de transformación comunitaria A&ACH, Naranja Madura y aliados comerciales) 3.12. ¿Cuáles cree que son los principales retos que deben asumir de aquí en adelante los productores y sus organizaciones para asegurar su sostenibilidad en la cadena? ¿Con qué herramientas, recursos y apoyos considera que cuentan para afrontarlos? 3.13. Con respecto a los ingresos para las familias de los productores generados por esta actividad, ¿qué tan significativos cree que son, o pueden llegar a ser, en relación con los ingresos totales de estas familias? ¿Cree que podrán completar un ingreso suficiente para mejorar su calidad de vida en el territorio? 3.14. Desde su perspectiva, para las familias que antes del proyecto trabajaban en minería ¿se percibieron cambios significativos de actividad en favor de las labores relacionadas con la producción, transformación y comercialización del achiote? 3.15. ¿Cómo fue la participación de las mujeres y los jóvenes de los núcleos familiares en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 274 3.16. ¿En qué medida considera que el fortalecimiento de actividades productivas del achiote como alternativa a la minería podría contribuir a reducir el impacto ambiental en la zona? 3.17. ¿Qué cambios en la situación de orden público en las zonas de producción y transformación de achiote se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 3.18. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad, replicabilidad y sostenibilidad? 4. Rehabilitación de áreas degradadas por minería ilegal 6.8 En general, antes del programa OL ¿qué uso tenían las áreas en donde se estableció el achiote? ¿se tenía alguna norma o protocolo que impidiera deforestar áreas boscosas para establecer los cultivos? 6.9 ¿Las zonas de la rehabilitación forestal del programa OL están relacionadas con las zonas y beneficiarios de la cadena de achiote? ¿De qué manera? 6.10 ¿En qué medida cree que el modelo de rehabilitación de OL ha sido efectivo para lograr una recuperación ecológica en las áreas intervenidas? 6.11 ¿Qué incentivos cree que tienen los Consejos Comunitarios y las familias para mantener en el tiempo los modelos de rehabilitación? 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a rehabilitación y manejo sostenible de recursos) 5.2. ¿Considera que las acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? Indagar por todos los componentes, no sólo cadenas productivas. 5.3. ¿En qué medida las actividades apoyadas por OL en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? 5.4. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? Indagar por todos los componentes, no sólo cadenas productivas. 5.5. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – DONATARIOS APICULTURA Introducción Buenos días/tardes, 275 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; y ii) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. Según su conocimiento, cuénteme en qué consiste el programa Oro Legal. (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal en la región? Indagar por actividades con los diferentes actores del territorio 2.3. ¿Cómo fue el proceso por el cual se decidió y se aceptó la participación de su entidad en el programa OL? 2.4. ¿En qué consistió la participación y cuál fue el rol de su entidad en el programa OL? 2.5. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? 2.6. ¿Cuáles fueron los cuellos de botella y los factores facilitadores para su articulación y la de su entidad con el programa OL? 3. Desarrollo de cadenas de valor y medios de vida alternativos 3.1. ¿Cómo entiende usted que se incluyan alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 3.2. Desde su perspectiva, ¿cuál era la estrategia que tenía el programa en su inicio para el fortalecimiento de la cadena apícola y en qué medida se cumplió? 3.3. A la finalización del programa y según su comprensión, ¿considera que durante la ejecución se estructuró y se implementó un modelo de negocio factible para los productores apícolas y para sus organizaciones en el largo plazo? 3.4. ¿Cuáles son los eslabones de la cadena apícola que quedaron conformados o fortalecidos con el proyecto y qué funciones cumple cada uno? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 276 3.5. ¿Cómo fue el proceso de focalización y selección de los productores beneficiarios que participaron en el proyecto apícola? 3.6. ¿Cuáles son las principales capacidades con las que el programa fortaleció a los pequeños productores de miel para que puedan mejorar sus ingresos a través de esta actividad? 3.7. Hablando de las organizaciones de productores apícolas ¿cuáles fueron las capacidades que se fortalecieron con el programa en aspectos técnico productivos, socioempresariales y comerciales? ¿Cómo cree que esto impacta en la sostenibilidad de este agronegocio? 3.8. ¿Cuáles alternativas dejó establecidas el programa para la prestación de servicios de acompañamiento y asistencia técnica a los productores luego de finalizar la intervención y cómo se van a financiar? 3.9. ¿Cómo valora usted los aliados comerciales y los mercados con los que se ha encadenado esta iniciativa en términos de estabilidad, condiciones comerciales, capacidad de compra y potencial de crecimiento? 3.10. ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de miel dentro y fuera del territorio? (Tener en cuenta los eslabones: productores, asociaciones apícolas, Campo Dulce y aliados comerciales) 3.11. ¿Cuáles cree que son los principales retos que deben asumir de aquí en adelante los productores y sus organizaciones para asegurar su sostenibilidad en la cadena? ¿Qué herramientas, recursos y apoyos considera que tienen para afrontarlos? 3.12. Con respecto a los ingresos para las familias de los productores generados por esta actividad, ¿qué tan significativos cree que son, o pueden llegar a ser, en relación con los ingresos totales de estas familias? ¿Cree que podrán completar un ingreso suficiente para mejorar su calidad de vida en el territorio? 3.13. Desde su perspectiva, para las familias que antes del proyecto trabajaban en minería ¿se percibieron cambios significativos de actividad en favor de las labores apícolas? 3.14. ¿Cómo fue la participación de las mujeres y los jóvenes de los núcleos familiares en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? 3.15. ¿En qué medida considera que el fortalecimiento de actividades apícolas como alternativa a la minería podría contribuir a reducir el impacto ambiental en la zona? 3.16. ¿Qué cambios en la situación de orden público en las zonas de producción de miel se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 3.17. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad, replicabilidad y sostenibilidad? 4. Rehabilitación de áreas degradadas por minería ilegal 6.12 ¿Cómo se relacionaron las zonas y los beneficiarios de la rehabilitación con Acacia magnium con los de la cadena apícola? 6.13 ¿En qué medida cree que este modelo de rehabilitación ha sido efectivo para lograr una recuperación ecológica en las áreas intervenidas? 6.14 ¿Qué incentivos cree que tienen las familias para mantener en el tiempo los modelos de rehabilitación? 277 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a rehabilitación y manejo sostenible de recursos) 5.2. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? Indagar por todos los componentes, no solo cadenas productivas. 5.3. ¿En qué medida las actividades apoyadas por OL en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? 5.4. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? Indagar por todos los componentes, no solo cadenas productivas. 5.5. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – DONATARIO-PROYECTO MINERO Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y iv) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 278 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 2.2. ¿Cuántas unidades productivas mineras de pequeña minería hay dentro del consejo comunitario y cuántas fueron beneficiarias del programa Oro Legal? 2.3. ¿Qué expectativas tenían del programa OL y sus iniciativas productivas cuando iniciaron su participación y en qué medida estas se cumplieron? 2.4. ¿Cómo fue la articulación entre OL, el consejo comunitario, las UPM, las autoridades mineras y ambientales, durante el proceso de legalización de los contratos? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación del consejo comunitario con la iniciativa productiva del programa OL? 3. Formalización minera 3.1. ¿Cuáles fueron las contribuciones técnicas y jurídicas de OL para el proceso de formalización minera en las áreas del consejo comunitario? 3.2. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.3. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y ANM para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.4. ¿En qué medida OL sirvió de intermediario entre las minas y las autoridades mineras y ambientales? (indagar sobre acompañamiento efectivo frente a trámites complejos). 3.5. ¿Cuáles fueron las herramientas o los incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? 3.8. ¿Cuál cree que será el futuro de las UPM beneficiarias de OL al concluir el programa? 3.9. ¿Cuántas entidades, públicas y privadas, han realizado actividades relacionadas con la formalización mineras para las UPM ubicadas dentro del consejo comunitario? 3.10. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 3.12. ¿Qué conoce sobre la licencia ambiental temporal y su impacto sobre el proceso de formalización minera? 4. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 4.1. ¿Cuáles acciones fueron apoyadas por OL para contribuir a la eliminación del mercurio en las UPM ubicadas dentro del consejo comunitario? 279 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 4.2. ¿De uno a diez, siendo diez la erradicación total de mercurio, con cuánto calificaría los logros de los beneficiarios del programa OL? 4.3. ¿Qué instrumentos y enfoques tiene el consejo comunitario para eliminar el uso de mercurio en la actividad minera de oro de las UPM? (Indagar por avance en nuevas tecnologías, asistencia técnica) 4.4. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 4.5. ¿Qué tan claro considera que es el marco normativo sobre eliminación de mercurio? 4.6. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales frente a los lineamientos sobre eliminación de mercurio? 4.7. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 4.8. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 4.9. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpia promovidas por OL? 4.10. ¿Cuáles son los factores facilitadores y los cuellos de botella para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpia, factores externos a OL) 4.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Qué tan pertinente considera las intervenciones de OL en cuanto a las necesidades de formalización minera, el desempeño ambiental y el desarrollo económico? 5.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 5.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 5.4. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 5.5. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? GUÍA DE ENTREVISTA – DONATARIO REHABILITACIÓN Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 280 programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿A qué organización/asociación (Antioquia) o consejo comunitario (Chocó) pertenece? 1.3. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 1.4. ¿En qué consistió la actividad de rehabilitación de las zonas degradadas por minería? 1.5. ¿Cuál fue el rol de la organización/asociación XXX (Antioquia) o del consejo comunitario XXX (Chocó) en el programa Oro Legal? 1.6. ¿Cuál fue su rol en el desarrollo del Programa Oro Legal? 2. Planteamiento de la intervención 2.1. ¿Hace cuánto tiempo Ud. identificaba esa zona como un área degradada por minería?, ¿cómo describiría la condición ambiental de dicha zona? 2.2. ¿Cuál era el uso que se estaba dando a las zonas trabajadas? 2.3. ¿Conoció estudios que se hubieran realizado en la zona trabajada, relacionados con el estado de los suelos o de la biodiversidad, contaminantes del suelo, geografía, hidrología, flora, fauna, características atmosféricas, regiones de frontera, aspectos sociales? 2.4. ¿Cómo se priorizaron las zonas a rehabilitar? (Indagar por Facilidad con los dueños de la tierra, Grado de degradación, Condiciones sociales asociadas a la zona degradada, Relacionamiento con zonas protegidas e Importancia por relacionamiento con cuencas hídricas) 2.5. ¿Encuentra un relacionamiento de las zonas intervenidas con la Identificación reservas de Ley 2da y/o identificación de áreas protegidas? 2.6. ¿Sabe si se trabajó con alguna entidad del Estado para determinar el alcance de la intervención? ¿Cuáles? 2.7. Desde su punto de vista ¿Qué motivó a los dueños de los predios para desarrollar la rehabilitación de las zonas degradadas por minería? (Indagar por Reforestar, Volver a la 281 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV condición inicial del paisaje, Recuperar especies que se habían perdido, Generar ingresos y Dejar unas mejores condiciones para sus hijos) 3. Desarrollo de la intervención 3.1. ¿Considera apropiados los modelos de rehabilitación utilizados por OL? 3.2. ¿Cómo se determinaron las especies que se sembraron y sus porcentajes respecto a las especies nativas (para el caso de Acacia)? 3.3. ¿Cuál fue la estrategia de siembra? (Indagar por el distanciamiento entre árboles, complemento con otras especies) 3.4. ¿Cuál era el compromiso que asumía la comunidad para que el Programa prestara el apoyo? 3.5. ¿Cómo se desarrolló el crecimiento de las plántulas en los viveros? ¿Qué personas se vincularon en este ejercicio? 3.6. ¿Cuáles considera que son las fortalezas y debilidades del proceso de rehabilitación llevado a cabo con apoyo de OL en las zonas degradadas? 3.7. Desde su punto de vista ¿Qué ganaron o esperaban ganar los titulares de la tierra? ¿Conoce si esto se acordó en algún tipo de contrato? 3.8. ¿Qué otros pasos, adicionales a la siembra de los árboles, se ejecutaron para rehabilitar la zona? 3.9. ¿Cuál es el resultado final al que se quiere llegar con posterioridad a la siembra de los árboles? 3.10. ¿Conoce cuáles fueron los costos de adelantar la rehabilitación? 3.11. ¿La organización/asociación (Antioquia) o consejo comunitario (Chocó) participó en capacitaciones o procesos de formación apoyados por OL? ¿Cómo valora las capacitaciones que les ofrecieron? 4. Resultados de la intervención 4.1. ¿Qué efectos ha tenido hasta ahora la siembra de los árboles en la zona? Indagar por biodiversidad, suelos, agua, entre otros) 4.2. ¿Cómo se propuso o espera hacer uso de la madera de los árboles de Acacia? 4.3. Desde su conocimiento y experiencia ¿cree que el suelo rehabilitado se ha ido recuperando? ¿Cómo se evidencian esos resultados? 4.4. ¿En qué medida usted cree que se ha generado compromiso y apropiación por parte de la comunidad para mantener los modelos de rehabilitación? 5. Sostenibilidad 5.1. ¿Qué incentivos tienen las familias para mantener en el tiempo los modelos de rehabilitación? 5.2. ¿Se cuenta con recursos adicionales a los ya provistos por el Programa OL para mantener los procesos adelantados? 5.3. ¿Cuál fue la participación de instituciones como la alcaldía, secretaría de agricultura, corporación autónoma regional o alguna otra institución pública en las actividades de rehabilitación? ¿Considera que estas entidades están comprometidas para mantener las zonas rehabilitadas? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 282 6. Replicabilidad 6.1. ¿Usted cree que el proyecto tal como se desarrolló, podría repetirse para abarcar nuevas áreas que se encuentran degradadas por minería? 7. Recomendaciones 7.1. De las acciones realizadas en las actividades de rehabilitación, ¿qué hubiera hecho diferente o qué podría hacerse mejor si se quisiera continuar con el apoyo? 8. Percepción de pertinencia y efectividad de la actividad 8.1. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a rehabilitación y manejo sostenible de recursos) 8.2. ¿Considera que las acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 8.3. ¿En qué medida las actividades apoyadas por OL en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? 8.4. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? 8.5. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad) GUÍA DE ENTREVISTA – GOBERNACIONES Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. 283 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal en la región? Indagar por actividades con los diferentes actores del territorio 2.3. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? ¿Cómo se alinean esas expectativas con sus intereses profesionales? 2.4. ¿Cómo se articula la Gobernación con el programa OL? ¿A qué niveles se dio esa articulación (a nivel directivo o equipos técnicos en territorio)? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de la Gobernación con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 2.6. ¿La Gobernación cofinanció alguna actividad de OL? ¿Cuáles? 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras? ¿La gobernación recibió recomendaciones de OL para fortalecer la institucionalidad o las normas relacionadas con la formalización? (Tener en cuenta PTO, Política de formalización minera y permiso de aprovechamiento minero, estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) 3.2. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mantenerse formales/legales y mejorar el desempeño ambiental de sus operaciones mineras? 3.3. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.4. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y MME para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.5. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 284 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? ¿Cuál podría ser el rol de la Gobernación para apoyar la sostenibilidad de los procesos de formalización minera? 3.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿Cómo ha sido el proceso de articulación del programa OL con las autoridades del gobierno nacional, regionales y municipales? ¿OL contribuyó a mejorar estos procesos de coordinación? (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas mineras) 4.2. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.3. (Antioquia) ¿En qué medida considera que el Programa brindó apoyo técnico o jurídico a la Gobernación para el cumplimiento de la normativa minera? 4.4. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.5. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades del territorio? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿Cuál es su percepción frente a incluir alternativas productivas en un programa de formalización minera? (Indagar por pertinencia, sostenibilidad para que los mineros no regresen a la extracción informal de oro, mejoramiento ambiental) 5.2. ¿Cómo está posicionada la línea productiva apícola (Antioquia)/Achiote (Chocó) en la región dentro del plan de desarrollo departamental y qué impacto tienen los proyectos productivos de Oro Legal en los objetivos y metas de este? 5.3. De acuerdo con las dinámicas del territorio ¿considera que los ingresos por actividades productivas alternativas como miel (Antioquia)/achiote (Chocó) son suficientes para disuadir la incursión de los beneficiarios en actividades de minería artesanal? 5.4. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL para el desarrollo de la cadena productiva de miel (Antioquia)/Achiote (Chocó)? 5.5. ¿Qué cambios en la situación de orden público en las zonas de producción de achiote (Chocó)/miel (Antioquia) se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 5.6. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socioempresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.7. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad y replicabilidad? 6. Rehabilitación de áreas degradadas por minería ilegal 285 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 6.1. ¿En qué medida las actividades de rehabilitación de OL se alinean con las directrices del Plan de Desarrollo Departamental? 6.2. ¿En qué medida cree que los modelos de rehabilitación de OL han sido efectivos y pertinentes para lograr una recuperación ecológica en las áreas intervenidas (Acacia mangium) y en Chocó (especies endémicas)? ¿Usted cree que las zonas intervenidas corresponden a las más afectadas por minería? ¿Por qué? 6.3. ¿La Gobernación apoyó y podría seguir soportando el proceso de rehabilitación de las áreas intervenidas en el Programa OL? 6.4. ¿Qué incentivos existen para promover la sostenibilidad de los modelos de rehabilitación? ¿Qué otros incentivos podrían desarrollarse para lograr la sostenibilidad? 6.5. ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿Qué instrumentos y enfoques tiene la Gobernación para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 7.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 7.3. ¿Cuáles cree que son los incentivos o motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 7.4. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 7.5. ¿Cuáles son los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias, precio del mercurio) 7.6. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 8.3. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.4. ¿Cuáles cree que han sido las contribuciones de OL en temas de desarrollo económico? 8.5. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 8.6. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 286 GUÍA DE ENTREVISTA – MINISTERIO DE AMBIENTE Y DESARROLLO SOSTENIBLE Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos y acciones implementadas) 2.2. ¿Qué expectativas tenía de OL cuando lo conoció y en qué medida estas se cumplieron? 2.3. ¿Cómo se articula el Ministerio de Ambiente con el programa OL? 2.4. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación del ministerio con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras en los temas ambientales? (Tener en cuenta estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) 287 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 3.2. En caso de realizar programas similares en un futuro, ¿cómo podría USAID desarrollar una gestión que fortalezca o apoye procesos del ministerio en materia de formalización minera? 3.3. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar el desempeño ambiental de sus operaciones mineras? 3.4. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL para minimizar los impactos ambientales generados por la actividad minera? 4. Gobernanza y políticas mineras 4.1. ¿En qué medida considera que el Programa brindó apoyo técnico o jurídico al Ministerio, o a las Corporaciones u otros actores ambientales en relación con el cumplimiento de las normas ambientales para el desarrollo de actividades de OL? 4.2. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.3. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.4. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿En qué medida considera que OL contribuye al mejoramiento ambiental de las regiones al promover actividades productivas agropecuarias (miel y achiote) como alternativas a la minería artesanal? 5.2. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socioempresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.3. Teniendo en cuenta el desarrollo de las cadenas productivas, ¿podrían considerarse estas cadenas como parte de los programas de pago por servicios ambientales o acciones de mitigación de cambio climático? 6. Rehabilitación de áreas degradadas por minería ilegal 6.1. ¿En qué medida son apropiados los modelos de rehabilitación utilizados por OL en Antioquia (Acacia mangium) y en Chocó (especies endémicas)? ¿Esto se enmarca con los lineamientos de la política ambiental? (Tener en cuenta Plan Nacional de Restauración, lineamientos del IIAP) 6.2. ¿En qué medida cree que los modelos de rehabilitación han sido efectivos para lograr una recuperación ecológica en las áreas intervenidas? 6.3. ¿El ministerio conoce, apoya y podría seguir soportando el proceso de rehabilitación de las dos áreas intervenidas en el Programa OL? 6.4. ¿Qué incentivos existen para promover la sostenibilidad de los modelos de rehabilitación? ¿Qué otros incentivos podrían desarrollarse para lograr la sostenibilidad? 6.5. ¿Existen diagnósticos sobre los pasivos huérfanos en Antioquia y Chocó, así como modelos de rehabilitación y los costos de su implementación? ¿Esta información es pública? ¿Los implementadores en territorio tienen acceso a esta información? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 288 6.6. ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿Qué instrumentos y enfoques tiene el ministerio para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 7.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 7.3. ¿Qué tan claro considera que es el marco normativo del Ministerio de Minas sobre eliminación de mercurio? ¿Qué nivel de conocimiento considera que tiene el Ministerio de Minas frente a los lineamientos ambientales sobre eliminación de mercurio? 7.4. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – MINISTERIO DE MINAS Y ENERGÍA Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. 289 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Cuénteme un poco en qué consiste el programa (Indagar por los objetivos y acciones implementadas) 2.2. ¿Qué expectativas tenía de OL cuando lo conoció y en qué medida estas se cumplieron? 2.3. ¿Cómo se articula el Ministerio de Minas con el programa OL? ¿A qué niveles se dio esa articulación (a nivel directivo o equipos técnicos en territorio)? 2.4. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación del ministerio con el programa OL? (Indagar por los tres componentes: formalización minera, cadenas de valor y rehabilitación) 3. Formalización minera 3.1. ¿En qué medida OL contribuyó en aspectos técnicos o jurídicos para la formalización de las actividades mineras? ¿Recibió el MME recomendaciones de OL para fortalecer la institucionalidad o las normas relacionadas con la formalización? (Tener en cuenta PTO, Política de formalización minera y permiso de aprovechamiento minero, estudios de impacto ambiental, guías minero-ambientales y planes de manejo ambiental) 3.2. ¿En qué medida las actividades de formalización minera de OL estuvieron alineadas con el Programa de Formalización Minera del MME? 3.3. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.4. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y MME para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.5. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? ¿cuál podría ser el rol del Ministerio de Minas para apoyar la sostenibilidad de los procesos de formalización minera? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 290 3.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿En qué medida considera que el Programa brindó apoyo técnico o jurídico al Ministerio, a la Agencia Nacional de Minería y a las autoridades mineras en relación con el cumplimiento de las normas para el desarrollo de actividades de OL? 4.2. ¿Cuáles son los beneficios o inconvenientes que genera el hecho de que USAID sea el financiador de un programa como OL? 4.3. ¿Cuáles son los factores que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.4. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿Cuál es su percepción frente a incluir alternativas productivas en un programa de formalización minera? (Indagar por pertinencia y sostenibilidad para que los mineros no regresen a la extracción informal de oro) 6. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 6.1. ¿Qué instrumentos y enfoques tiene el ministerio para eliminar el uso de mercurio en la actividad minera de oro? (Indagar por avance en nuevas tecnologías, asistencia técnica) 6.2. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 6.3. ¿Qué tan claro considera que es el marco normativo del Ministerio de Ambiente sobre eliminación de mercurio? ¿Qué nivel de conocimiento considera que tiene el Ministerio de Ambiente frente a los lineamientos sobre eliminación de mercurio? 6.4. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 6.5. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 6.6. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 6.7. ¿Cuáles son los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias, factores externos a OL) 6.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 7. Percepción de pertinencia y efectividad de la actividad 7.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 291 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 7.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 7.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) GUÍA DE ENTREVISTA – SUBCONTRATO DE FORMALIZACIÓN Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y iv) las percepciones generales sobre la relevancia y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. ¿Conoce usted el programa Oro Legal? Según su conocimiento, cuénteme en qué consiste el programa. Indagar sobre los tres componentes. 2.2. ¿Cuántas unidades productivas mineras de pequeña minería hay en los títulos de la compañía y cuántas fueron beneficiarias del programa Oro Legal? 2.3. ¿Qué expectativas tenían del programa OL y sus iniciativas productivas cuando iniciaron su participación y en qué medida estas se cumplieron? USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 292 2.4. ¿Cómo fue la articulación entre OL, la compañía, las UPM, las autoridades mineras y ambientales, durante el proceso de legalización de los subcontratos? 2.5. ¿Cuáles fueron los cuellos de botella y factores facilitadores para la articulación de su empresa con la iniciativa productiva del programa OL? 3. Formalización minera 3.1. ¿Cuáles fueron las contribuciones técnicas y jurídicas de OL para el proceso de formalización minera en las áreas de la compañía? 3.2. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.3. ¿En qué medida usted considera que el programa OL contribuye a reducir la complejidad en el proceso de formalización? (Indagar sobre coordinación entre OL y ANM para solucionar las dificultades halladas en el desarrollo de la formalización minera) 3.4. ¿En qué medida OL sirvió de intermediario entre las minas y las autoridades mineras y ambientales? (indagar sobre acompañamiento efectivo frente a trámites complejos). 3.5. ¿Cuáles fueron las herramientas o los incentivos que brindó OL a los mineros de MAPE para mejorar las operaciones mineras? (Indagar por disminución de riesgos laborales y de salud, productividad, uso de nuevas tecnologías) 3.6. ¿Qué percepción tiene sobre la calidad de la asistencia técnica brindada por OL? (Indagar por nuevas tecnologías, reducción de mercurio, conocimiento del marco normativo, mitigación de riesgos laborales y de salud) 3.7. Pensando en la sostenibilidad de este tipo de iniciativas, ¿Cuáles cree que son los principales incentivos que OL brinda para promover la continuidad de la formalización en los beneficiarios? 3.8. ¿Cuál cree que será el futuro de las UPM beneficiarias de OL al concluir el programa? 3.9. ¿Cuántas entidades, públicas y privadas, han realizado actividades relacionadas con la formalización mineras para las UPM ubicadas dentro de los títulos de la compañía? 3.10. ¿Cuáles considera que fueron las principales fortalezas y debilidades de la estrategia de formalización minera de OL? 3.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 3.12. ¿Qué conoce sobre la licencia ambiental temporal y su impacto sobre el proceso de formalización minera? 4. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 4.1. ¿Cuáles acciones fueron apoyadas por OL para contribuir a la eliminación del mercurio en las UPM ubicadas dentro de los títulos de la compañía? 4.2. ¿De uno a diez, siendo diez la erradicación total de mercurio, con cuánto calificaría los logros de los beneficiarios del programa OL? 4.3. ¿Qué instrumentos y enfoques tiene la compañía para eliminar el uso de mercurio en la actividad minera de oro de las UPM? (Indagar por avance en nuevas tecnologías, asistencia técnica) 4.4. ¿Cuáles han sido los factores facilitadores y cuellos de botella para la implementación de esos instrumentos o enfoques? (Indagar por costos, grado de dificultad técnica) 4.5. ¿Qué tan claro considera que es el marco normativo sobre eliminación de mercurio? 293 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 4.6. ¿Qué nivel de conocimiento considera que tienen las autoridades mineras y ambientales frente a los lineamientos sobre eliminación de mercurio? 4.7. ¿Cuáles son las estrategias de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro y cómo estás podrían mejorarse? 4.8. ¿Cuáles cree que son las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? 4.9. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpia promovidas por OL? 4.10. ¿Cuáles son los factores facilitadores y los cuellos de botella para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpia, factores externos a OL) 4.11. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 5. Percepción de pertinencia y efectividad de la actividad 5.1. ¿Qué tan pertinente considera las intervenciones de OL en cuanto a las necesidades de formalización minera, el desempeño ambiental y el desarrollo económico? 5.2. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 5.3. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 5.4. ¿Considera que estas acciones de OL han contribuido al desarrollo económico en los territorios en que se han implementado? ¿De qué manera? 5.5. ¿Estima que estas actividades en el territorio tienen alguna incidencia en el mejoramiento de la seguridad y el orden público? GUÍA DE ENTREVISTA – USAID Introducción Buenos días/tardes, Soy ______, hago parte del equipo de evaluación del programa Oro Legal el cual ha sido contratado por USAID. El objetivo de esta entrevista es identificar cómo ha venido funcionando en la práctica este programa e indagar sobre la efectividad, impacto y pertinencia que usted ha percibido en el trabajo que se realiza con los diferentes actores institucionales y con los beneficiarios. Agradezco su participación en esta entrevista, que servirá como insumo para determinar: i) en qué medida Oro Legal brinda incentivos para la formalización minera; ii) los factores que inciden en el progreso del programa para lograr gobernanza minera y ambiental; iii) la percepción sobre el impacto de las cadenas de valor desarrolladas como medios de vida alternativos; iv) la pertinencia y efectividad de los modelos de rehabilitación desarrollados por el programa; v) la eficiencia de los instrumentos y enfoque utilizados para reducir el uso del mercurio en la minería de oro; y vi) las percepciones generales sobre la relevancia USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 294 y efectividad del programa. Igualmente nos permitirá identificar diferencias y similitudes entre la forma en que se concibió el programa y la manera en que se ha implementado. Es importante señalar que la evaluación no se constituye en un proceso de auditoría y que, en su desarrollo, la información levantada en campo es confidencial y anónima, y no tendrá repercusiones sobre los entrevistados. Así mismo, su participación es voluntaria. * Solicitud de autorización para grabar audio de la actividad. 1. Presentación del entrevistado 1.1. ¿Cuál es su nombre? 1.2. ¿Cuál es su cargo? ¿Hace cuánto tiempo está en este cargo? 2. Conocimientos y relación con la actividad 2.1. Cuénteme un poco en qué consiste el programa Oro Legal (Indagar por los objetivos) 2.2. En líneas generales, ¿qué acciones ha implementado Oro Legal? Indagar por actividades con los diferentes actores 2.3. ¿Qué expectativas tenía del OL cuando lo conoció y en qué medida estas se cumplieron? 2.4. ¿Cómo está estructurado el equipo de trabajo en USAID? ¿Este equipo es suficiente para el desarrollo de las actividades del programa? ¿Cómo es la articulación con las coordinaciones regionales y los territorios? 3. Formalización minera 3.1. ¿Cómo fue el proceso de focalización y selección de los beneficiarios de formalización minera? ¿Hubo dentro de los criterios alguna priorización a poblaciones vulnerables (mujeres, jóvenes, personas con pertenencia étnica)? 3.2. ¿Cuáles fueron las herramientas/incentivos que brindó OL a los mineros de MAPE para mantenerse formales/legales y mejorar el desempeño ambiental de sus operaciones mineras? 3.3. ¿Cuáles fueron los cuellos de botella y factores facilitadores para las actividades de formalización minera? (indagar por temas mineros y ambientales) 3.4. ¿Cuáles aspectos se identificaron como útiles o necesarios para reducir la complejidad en el proceso de formalización? ¿Cuál fue la contribución de OL para reducir la complejidad? (indagar por temas mineros y ambientales) 3.5. ¿Cómo percibió OL la valoración que dan los mineros al apoyo ofrecido para su formalización? ¿Cree que habría más compromiso si los mineros tuvieran que pagar por algunos trámites? 3.6. ¿Cómo planeó OL que al finalizar el proyecto los mineros sigan cumpliendo con un aprovechamiento minero legal? 3.7. ¿Qué ajustes cree que podrían hacerse a la estrategia de formalización minera para incrementar su efectividad y replicabilidad? 4. Gobernanza y políticas mineras 4.1. ¿Cuál fue el rol de USAID en el proceso de articulación con las autoridades del gobierno nacional, regionales y municipales? 295 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 4.2. ¿Cómo percibe el proceso de articulación con las autoridades del gobierno nacional, regionales y municipales del programa OL? (Indagar por Ministerio de Minas, ANM, Ministerio de Ambiente, CARs, Gobernaciones y Alcaldías, Mesas Mineras) 4.3. ¿Cuáles fueron los mecanismos de participación de los beneficiarios de OL para la toma de decisiones sobre el programa? (Indagar por formalización minera, cadenas productivas y rehabilitación) ¿Hubo diferencias en la participación por parte de mujeres? 4.4. ¿Qué tan rígido o adaptable fue el programa a las condiciones y necesidades de los territorios? (Indagar si se tuvieron en cuenta características y condiciones particulares de grupos vulnerables étnicos/mujeres/jóvenes) 4.5. ¿Cuáles son los factores externos que facilitan o dificultan la gobernanza sobre las políticas mineras y ambientales? (indagar por conocimiento de la normatividad sobre formalización y no uso de mercurio, complejidad del sector, seguridad laboral) 4.6. ¿Qué ajustes cree que podrían hacerse a la estrategia de acercamiento y articulación con los aliados institucionales y beneficiarios para mejorar la gobernanza y políticas mineras? 5. Desarrollo de cadenas de valor y medios de vida alternativos 5.1. ¿Cómo fue el proceso de focalización y selección de los beneficiarios de las cadenas de valor desarrolladas por OL? (Indagar por apicultura y achiote) ¿Hubo dentro de los criterios alguna priorización a poblaciones vulnerables (mujeres, jóvenes, personas con pertenencia étnica)? 5.2. ¿Cuáles son las principales capacidades con las que el programa fortaleció a los pequeños productores de achiote/miel para que aseguren un ingreso suficiente? (Indagar por los 3 eslabones de la cadena productiva) 5.3. ¿En qué medida considera que el fortalecimiento de actividades productivas alternativas a la minería reduce el impacto ambiental en la zona? ¿El programa hizo un análisis previo sobre los posibles impactos ambientales del desarrollo de las cadenas productivas? ¿Qué estrategias se adelantaron para mitigar estos efectos? 5.4. ¿Qué cambios en la situación de orden público en las zonas de producción de achiote (Chocó)/miel (Antioquia) se percibieron durante la ejecución del programa y cómo cree que se relacionan con los proyectos productivos? (Indagar por la presión de grupos armados ilegales sobre los beneficiarios) 5.5. Dadas las condiciones de seguridad y orden público del territorio, ¿Qué prácticas o medidas tomaba al realizar su trabajo para sentirse seguro? ¿Algunas de estas prácticas cambiaron en el tiempo de desarrollo del programa? ¿Cómo? 5.6. ¿Cómo fue la participación de las mujeres y los jóvenes en las actividades del proyecto y cómo se refleja ahora en las actividades productivas? ¿Cómo cree que afectó esta participación las cargas de cuidado al interior de las familias? 5.7. En el caso de Antioquia ¿Qué rol desempeñan las asociaciones de productores de miel y cómo las preparó el programa para prestar servicios a sus asociados en el desarrollo del negocio apícola? 5.8. En el caso de Antioquia ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de miel? (Tener en cuenta los eslabones: productores, asociaciones apícolas, Campo Dulce y aliados comerciales) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 296 5.9. En el caso de Chocó ¿Qué rol desempeñan los consejos comunitarios y cómo los preparó el programa para el desarrollo de la cadena de achiote? (Indagar si hay alguna percepción del rol de las mujeres dentro de estos consejos en relación con el programa y la cadena productiva) 5.10. En el caso de Chocó ¿Cuáles han sido las principales dificultades y factores facilitadores para desarrollar los acuerdos y relaciones entre los diferentes eslabones del encadenamiento de achiote? (Tener en cuenta los eslabones: productores, centros de acopio y trillado de los consejos comunitarios, Achiote y Agros del Chocó y Naranja Madura) 5.11. ¿Qué elementos van a permitir la sostenibilidad de las cadenas productivas? (Indagar por encadenamientos con mercados formales, capacidades socio-empresariales de los beneficiarios y organizaciones y acceso a asistencia técnica) 5.12. ¿Qué ajustes cree que podrían hacerse a la estrategia de desarrollo de cadenas de valor y medios de vida alternativos para incrementar su efectividad y replicabilidad? 6. Rehabilitación de áreas degradadas por minería ilegal 6.15 ¿Cómo se seleccionaron las zonas a rehabilitar y los beneficiarios? (Indagar por diagnósticos de pasivos huérfanos) 6.16 ¿En qué medida la estrategia de rehabilitación se enmarca en los lineamientos ambientales de la política ambiental? ¿Algunos pasos de la intervención se trabajaron con las entidades ambientales públicas? (Tener en cuenta Plan Nacional de Restauración, lineamientos de IIAP (Chocó)) 6.17 En el momento en que se diseñó la estrategia de rehabilitación ¿cuál era el resultado esperado para esta actividad? 6.18 ¿En qué medida cree que los modelos de rehabilitación han sido efectivos para lograr una recuperación ecológica en las áreas intervenidas? 6.19 ¿Cree que ha existido compromiso y apropiación de los beneficiarios para mantener los modelos de rehabilitación? ¿Por qué? 6.20 ¿Qué incentivos cree que tienen las familias para mantener en el tiempo los modelos de rehabilitación? 6.21 ¿Cuáles fueron los factores facilitadores y cuellos de botella para las actividades de rehabilitación? 6.22 ¿Qué ajustes cree que podrían hacerse en los modelos de rehabilitación para incrementar su efectividad y replicabilidad? 6.23 ¿Cómo se realizó el costeo de los dos modelos de intervención? 7. Eliminación/reducción del mercurio en la cadena de suministro de la MAPE 7.1. ¿En qué consistió la estrategia técnica brindada a los mineros para disminuir el uso de mercurio en el aprovechamiento de oro? (Indagar por asistencia técnica, cumplimiento de normativa, tecnificación, sensibilización riesgos a la salud y ambientales) 7.2. ¿Cuáles cree que fueron las motivaciones de las UPM para reducir el uso de mercurio en el aprovechamiento de oro? (Indagar si hay diferencias en estas motivaciones entre mineros hombres y mujeres) 7.3. ¿En qué medida usted cree que las UPM están satisfechas con la implementación de técnicas de producción más limpias promovidas por OL? 7.4. ¿Apoyaron el desarrollo de los estudios geológicos para los PTO, y estos a su vez determinaron cómo serían las técnicas de beneficio de oro? 297 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 7.5. ¿Cuáles fueron los factores facilitadores y cuellos de botella para la disminución del uso de mercurio en el aprovechamiento de oro? (Indagar por grado de dificultad técnica o económica en las técnicas de producción más limpias) 7.6. ¿Cuál fue el objetivo de realizar las mediciones de mercurio en aire? ¿Por qué se desarrollaron esas mediciones? ¿En algún momento contemplaron hacer más mediciones? 7.7. ¿Monitorearon de alguna manera el comportamiento del comercio de mercurio en las regiones intervenidas? 7.8. ¿Qué ajustes cree que podrían hacerse a la estrategia de sensibilización para la eliminación del uso de mercurio en la actividad minera de oro? 8. Percepción de pertinencia y efectividad de la actividad 8.1. Dado el contexto y las necesidades de formalización minera, medio ambiente y desarrollo económico (acotar al nivel nacional, regional o municipal dependiendo del entrevistado) ¿Qué tan pertinente considera que han sido las intervenciones de OL? 8.2. ¿En qué medida las actividades de OL estuvieron alineadas con las directrices de los instrumentos de planeación y ordenamiento territorial? 8.3. ¿Cuáles cree que han sido las contribuciones de OL en cuanto a MAPE? 8.4. ¿Cuáles cree que han sido las contribuciones de OL en temas de medio ambiente? (Indagar por contribuciones asociadas a reducción de mercurio y rehabilitación) 8.5. ¿Cuáles cree que han sido las contribuciones de OL en temas de desarrollo económico? 8.6. ¿Qué efectos cree que ha traído OL en términos de calidad de vida para las familias beneficiaras del programa? (Indagar por los tres componentes: Formalización minera, cadenas productivas y rehabilitación) 8.7. ¿Cómo cree que la participación de las mujeres y jóvenes en los diferentes componentes del programa ha incidido en su calidad de vida y la de sus familias? (Indagar por ingresos, tranquilidad, más tiempo en familia, acceso a servicios básicos, seguridad en los 3 componentes: formalización minera, cadenas productivas y rehabilitación) USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 298 ANNEXI VI: SOURCES OF INFORMATION In general, this evaluation used primary information gathered through telephone and in-person surveys, to understand the components of mining formalization, rehabilitation of degraded areas, and value chains for annatto and beekeeping. TABLE 32. SURVEYS FOR THE OL EVALUATION COMPONENT ANTIOQU IA CHOCÓ TOTAL Annatto 0 212 212 Beekeeping 159 0 159 Mining formalization 24 73 97 Rehabilitation 199 2 201 Total 382 287 669 In addition, we carried out 120 interviews with relevant actors and beneficiaries from all components of the Activity. The distribution of the interviews is shown in Table 33. TABLE 33. INTERVIEWS CARRIED OUT FOR OL EVALUATION TYPE OF ACTOR LEVEL OF THE ACTOR NATIONAL DEPT. MUNICIPAL TOTAL Institutional actors 5 10 7 22 Implementer and partners 10 7 17 Grantees – executors and direct benefciaries 6 75 81 Total 15 23 82 120 We also analyzed secondary information supplied by the implementing partner as multiple files. We reviewed the following documents: • Oro Legal - Scope of work • Monitoring, Evaluation, and Learning Plan for AMELP Activities • Annual Work Plans − 2015 − 2016 − 2017 − 2018 − 2019 • Annual reports − 2016 299 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV − 2017 − 2018 − 2019 • Quarterly reports − OL Quarterly Report 2016 Q1 (October - December) − OL Quarterly Report 2016 Q2 (January - March) − OL Quarterly Report 2016 Q3 (April - June) − OL Quarterly Report 2017 Q1 (October - December) − OL Quarterly Report 2017 Q2 (January - March) − OL Quarterly Report 2017 Q3 (April - June) − OL Quarterly Report 2018 Q1 (October - December) − OL Quarterly Report 2018 Q2 (January - March) − OL Quarterly Report 2018 Q3 (April - June) − OL Quarterly Report 2019 Q1 (October - December) − OL Quarterly Report 2019 Q2 (January - March) − OL Quarterly Report 2019 Q3 (April - June) − OL Quarterly Report 2020 Q1 (October - December) − OL Quarterly Report 2020 Q2 (January - March) • Brief updates − 2016 − 2017 − 2018 − 2019 − 2020 • Oro Legal Indicators - baseline • Communication pieces − Digital bulletins − Communication pieces − Communication products − TV Communication pieces − Videos • Oro Legal Indicators – MEL Activity • Oro Legal Environmental Assesments • Oro Legal Mid-term Evaluation • Gender and vulnerable population • Beneficiary database • Key actors database • Presentation on Oro Legal Oro Legal activities We complemented these documents with additional inputs provided to the evaluation team. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 300 MINING FORMALIZATION • Foramlization Recommendations for the Government • UNODC – Alluvial gold mining • SST annexes provided to the MPUs • Environmental annexes provided to the MPUs • Business annexes provided to the MPUs • Final report Memorandum of understanding Oro Legal – Gran Colombia Gold • Histórico Asistencia Técnica Nordeste • UPM Contactadas razones de no intervención MERCURY • Airborne Mercury Concentration Sampling • Oro_Legal_Mercury_Monitoring_Plan (December 2015) VALUE CHAINS • Annatto and beekeeping GIS • Annatto production in the department of Chocó • Production of honey in the case of Bajo Cauca Antioqueño • EMA CC San Isidro • EMA Coltapicola REHABILITATION • Evaluation of species diversity in rehabilitated areas of Bajo Cauca • Rehabilitation GIS Antioquia • Rehabilitation GIS Chocó • Long-term agreement – Consejo Mayor de San Isidro • Rehabilitation of soils degraded by informal gold mining. Cases of Antioquia and Chocó • Academic documents on rehabilitation models 301 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ANNEX VII: SCOPE OF WORK OF THIS PERFORMANCE EVALUATION PURPOSE OF THE EVALUATION The purpose of this final performance evaluation is to conduct a systematic analysis around the performance of the Oro Legal Activity to assess to what extent the achievements at the different sub￾objectives of the intervention are perceived effective and impactful, and which factors may be playing a role to modulate the performance toward those sub-objectives, the Activity´s main goal, and USAID´s strategic development goals. In accomplishing this evaluation, the Contractor will collect all necessary data, perform formal analyses by using mixed methods, and produce empirical quantitative and qualitative information of Oro Legal´s performance, stakeholders perceptions, and the context to: (a) respond technically to the purpose of this evaluation, (b) answer the evaluation questions of herein based on the data collected and analysis performed, and (c) provide evaluation findings, actionable and context-specific recommendations, conclusions, and lessons learned -if any- constructed through evidence-based knowledge. The evaluation will support USAID/Colombia and Oro Legal Implementing Partner (IP) accountability among stakeholders. The evaluation must also provide lessons learned and insightful recommendations as input to help USAID/Colombia making programming decisions as well as to share knowledge gained with relevant stakeholders in the Government of Colombia (GOC), Mining Sector, International Donors, and Communities. This evaluation is scheduled to start and develop along the Oro Legal Activity final phase of implementation. Its design and methodology must be replicable in order to have technical validity and to be heuristic for other performance evaluation in the portfolio of USAID. SUMMARY INFORMATION OF THE ACTIVITY TABLE 34: ACTIVITY SNAPSHOT Activity Name: Artisanal Gold Mining Environmental Impact Reduction - Oro Legal Period of Performance: 9/23/2015 - 3/31/2021 Contract No: AID-514-C-15-00003 Contracting Mechanism: Cost plus fixed fee contract. Total Estimated Cost (TEC): $22,122,365 Contractor: Chemonics Inc. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 302 Alignment to the USAID/Colombia Country Development Cooperation Strategy (CDCS) 2014- 2020: DO 4: “Environmental resilience and low carbon development strengthened”. Intermediate Results: IR 4.1 “Natural resource management strengthened”. Sub-IR 4.1.1 “Legality, rehabilitation, and reduction of mercury use in small, artisanal mining operations improved” Purpose: 1) Building effective governance capacity for gold mining activities via: (i) strengthening Colombian government capacity to enforce gold mining legislation; (ii) enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs, and (iii) providing training and technical assistance to artisanal miners; 2) Increasing the capacity of Colombian government entities to address the environmental impact caused by illegal mining through: (i) reforestation of degraded areas; (ii) generating alternative livelihoods for communities that cannot or should not be involved in gold mining, and (iii) improve drinking water quality in mining areas. Contracting Officer Representative (COR): Gustavo Vargas - Project Management Specialist (Environment) Alternate Contracting Officer Representative: Daniel García - Financial Specialist Monitoring and Evaluation Specialist: Catalina Correa Evaluation Activity Manager (EAM): Omar López, Monitoring and Evaluation Specialist (Program Office) - COR to the MEL Activity BACKGROUND Colombia’s gold mining sector suffers from pervasive illegality. The vast majority of the yearly average 50 metric tons of gold produced in Colombia are done so by illegal and informal actors, driving widespread deforestation, riverbanks sedimentation and mercury contamination, posing a threat to public health, and costing the state tens of millions of dollars in lost tax revenues and environmental liabilities left behind. As a result, addressing illegal mining has become a major concern in Colombia’s post conflict scenario. Gold mining in Colombia has increased by over 300% since 2006 and illegal and informal gold mining is a growing challenge. The scale of mining has grown faster than the institutional capacity to adequately regulate and control mining activities. Illegal mining is part of criminal economies that also include illicit coca cultivation, land appropriation, logging, trafficking in wildlife, trafficking in persons, and extortion and money laundering by the criminal organizations involved in these activities. According to the GOC, illegal gold mining has replaced drug trafficking as the primary source of income for organized crime, armed groups, and illicit actors. This is fueling violence as there is a strong link between the criminal organizations responsible for the illicit coca and those involved in trading the gold from illegal mining. At national level, 43 percent of Colombia’s territory that is affected by illegal mining is also affected by illicit coca cultivation, and Antioquia and Chocó bear the heaviest lift of the phenomenon. Uncontrolled gold mining is not only a major stimulus for Colombia’s illicit economy, but is also a major driver of environmental degradation, causing large-scale deforestation, riverbanks destruction, and mercury contamination; scarring the landscape with a legacy of highly degraded and contaminated 303 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV abandoned mining sites. In 2019, nationally an estimated 98,000 hectares of land were degraded by alluvial gold exploitation and 78% of this detected area was concentrated in the departments of Antioquia and Chocó. The resulting biodiversity loss in Choco is of particular concern as this is one of the most biodiverse regions in the world. In addition, these highly degraded sites are so devastated that they cease to be part of the productive economy, no longer providing any viable livelihood options, unless and until major investments in remediation and restoration are made. Due to the use of mercury to amalgamate gold in artisanal gold mining, Colombia is the world's largest mercury polluter per capita. As a result, some towns in the department of Antioquia have the highest urban per capita air mercury pollution in the world (as of 2011) (Cordy et al, 2011).130 The country has passed legislation banning the use of mercury in mining that went into effect in July 2018, but a full implementation of this ambitious goal presents yet major challenges along the chain of control. The USAID´s strategic development goals can be divided in two levels: 1) An Agency´s overarching objective known as “Journey to Self-Reliance” -J2SR- which looks for (a) helping countries to build capacity and (b) to impact behaviors triggering positive changes; so the ultimate goal of foreign assistance is ceasing to exist in the future. 2) A USAID/Colombian Mission´s Environmental Development Objective of improving the environmental resiliency by strengthening the management of natural resources [Intermediate Result 4.1 / CDCS 2014-2020]. USAID supports the GOC’s efforts to create a durable and inclusive peace in the wake of the country's 50-year armed conflict. Tackling illegal activities, including illegal mining, and providing alternative incomes through the licit economy, is paramount for a sustainable and participatory peace. Given the important challenges posed by uncontrolled gold mining, USAID has responded with the Agency’s largest and most ambitious artisanal gold mining activity to build effective governance capacity for gold mining activities, aimed to strengthen Colombian Government capacity to enforce gold mining legislation and address the environmental impact caused by illegal mining. The Artisanal Mining Program (Oro Legal) promotes responsible legal mining, alternative livelihoods, and environmental rehabilitation. The main goal of the Artisanal Gold Mining - Oro Legal Activity (the Activity, from now on) is to help the Government of Colombia (GOC), the mining sector, and the communities to address the informality of the artisanal and small-scale gold mining (ASGM) to reduce its environmental liabilities through: 1) Building effective governance capacity for gold mining activities via: (a) Strengthening Colombian government capacity to enforce gold mining legislation; (b) Enhancing the participation of artisanal gold mining associations and Afro Colombian and indigenous communities in mining formalization programs; and (c) Providing training and technical assistance to artisanal miners; and 2 ) Increasing the capacity of Colombian Government (GOC entities to address the environmental impact caused by illegal mining through: (a) Restoration of degraded areas; (b) Generating alternative livelihoods for communities that cannot or should not be involved in gold mining, and (c) Improve drinking water quality in mining areas. 130 Cordy P, Veiga MM, Salih I, Al-Saadi S, Console S, Garcia O, Mesa LA, Velásquez-López PC, Roeser M. Mercury contamination from artisanal gold mining in Antioquia, Colombia: The world's highest per capita mercury pollution. Sci Total Environ. 2011 Dec 1;410-411:154-60. doi: 10.1016/j.scitotenv.2011.09.006. Epub 2011 Oct 15. PMID: 22000915. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 304 This Artisanal Mining Program contract totaling almost $22.1 million follows on to the mining component of the previous USAID/Colombia BioREDD+ activity implemented by Chemonics from July 2013 to May 2015. DESCRIPTION OF THEORY OF CHANGE AND RESULTS FRAMEWORK The Theory of Change (TOC) underlying the Oro Legal Activity is that if the program strengthens government capacity to effectively manage gold mining and address environmental damages, then the social, environmental, and economic performance of gold mining operations will be improved, thereby supporting the GOC to create a sustainable and inclusive peace. Each of the two objectives of the Oro Legal has its own TOC. The Oro Legal Activity has two Objectives: the first objective focuses on supporting mining sector institutions and organizations to improve and/or better enforce regulations covering gold mining in target regions, thus reducing the negative environmental and social impacts often implied by these activities. The second objective seeks to increase the capacities of the GOC, communities and the private sector to address the environmental degradation caused by unauthorized gold mining. The results framework (see chart below) guides Oro Legal’s approach to implementing USAID’s vision for the Activity is linked directly to USAID/Colombia Strategy at CDCS 2014-2020. It incorporates crosscutting responses to tasks, including a focus on our differentiated approach on gender inequity and vulnerable population. 305 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV Exhibit 72: The Oro Legal Results Framework *ASM: Artisanal and Small-Scale Miners. NAMA: Nationally Appropriate Mitigation Actions. The following are the underlying critical assumptions accompanying the results framework and acting as external potential factors influencing objectives: • The security situation in target municipalities is conducive to implementation. • The GOC continues supporting Oro Legal objectives through the implementation of policies and legal reforms conducive to responsible artisanal gold mining. • There are a sufficient number of MPU131s in the defined geographies, able and willing to become legal and formal. • Cooperation and goodwill between key partners are maintained: GOC, municipal administrations, mining associations and private mining companies, etc. The set of Oro Legal goals depends on the following factors that fall outside Activity control: 131 MPU: Mining Production Unit USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 306 • Fluctuation in gold prices, which affects miner earnings and provides incentives/disincentives for operators to enter or exit this activity. • The opportunity cost to mining of prices of alternate goods and services that produce returns on investment for private investors, and returns to labor and investment for small-scale miners, affecting the decision to mine. Geography of Intervention of Oro Legal: See Annex A. LINES OF INTERVENTION Based on the objectives and sub-objectives previously set forth in the Oro Legal Results Framework, the Activity develops four main lines of work: A: TO PROMOTE SUSTAINABLE RURAL DEVELOPMENT Mining Legalization and Formalization: Legalization (titling) and formalization (compliance with all relevant regulatory requirements) is one of the best means to reduce illegal activity and environmental devastation caused by illegal mining, which includes mercury contamination, deforestation, and a costly legacy of abandoned mining sites. Oro Legal’s goal is to support small and medium miners to legalize $155 million dollars worth of gold, effectively removing it from the illegal economy, while averting the discharge of 55 tons of mercury to the environment. Alternative Livelihoods: A key component to combating illegal economies is strengthening opportunities for licit economic activities. Where responsible mining is not economically, technically, or environmentally feasible, which is the case for the vast majority of artisanal miners, Oro Legal provides assistance to generate licit economic alternatives to illegal mining. Oro Legal is supporting commercial honey production in Antioquia and annatto production in Choco. Together, these two value chains are expected to benefit 1,000 families and produce an income per participant of approximately twice the Colombian minimum wage. B: TO PROMOTE ENVIRONMENTAL REHABILITATION: Rehabilitation of Degraded Lands: Oro Legal aims to rehabilitate 11,500 hectares of highly degraded land using reforestation with Acacia mangium in Antioquia and assisted natural regeneration in Choco. In Chocó, agreements have been signed with Afro Colombian Community Councils (CCs) to rehabilitate 8,220 hectares (18%) of an estimated total of 45,000 hectares of degraded land. In Antioquia, 8 projects with private investors, local governments and communities will rehabilitate 3,280 hectares (15%) of an estimated total of 22,000 hectares of degraded land. Mercury Elimination: The formalization process supported by Oro Legal introduces technologies and practices that initially reduce and, if fully implemented, have the capacity to fully eliminate the use of mercury. Through this work with small and medium mines in the process of formalization, the program will remove 55 tons of mercury from the gold production chain. In addition, Oro Legal worked to measure airborne mercury contamination, a far greater risk to human health and located right in the middle of mining towns. USAID is collaborating with several diplomatic Missions in a joint effort to implement a communications campaign to support the elimination of mercury. Past mid-2019 it was approved a cost and time extension of $2.1million dollars and six months to focus on (Component 1) piloting 3-6 zero-mercury devices building on the gains of formalization. SUMMARY OF RELEVANT DOCUMENTS, INFORMATION, AND DATA OF THE ORO LEGAL ACTIVITY 307 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV The following documents contain information regarding the design, implementation, reporting and performance monitoring of the Oro Legal Activity. The Contractor must do a careful and detailed review and analysis of the documents, information, and data listed below as follows: Mandatory: • USAID/Colombia Country Development Cooperation Strategy (CDCS)132 • Section C and F of the Base Contract, and Contract Modifications on these sections • Program Annual Work plans • Monitoring, Evaluation and Learning Plan (MEL Plan) • Performance monitoring data logged into the USAID/Colombia “Monitor System” • Baseline data for performance indicators (those requiring it) • Annual Performance Reports • Gender Analysis • Stakeholders Analysis • Brief Updates • Lists of beneficiaries by type, sex, organization, value chain, geolocation in order to design surveys or other instruments. As Needed: • Mid-term Performance Evaluation to the Oro Legal Activity - Final Report • Journey to Self-Reliance - Principles/Policy • USAID Briefer: Oro Legal - Artisanal Gold Mining • Quarterly Performance Reports • Annual and/or quarterly financial reports • Airborne Mercury Monitoring in Mining Municipalities in Colombia (2019, Oro Legal) • Biweekly Highlights, videos, press materials, and social media posts • BioREDD/Mining documentation as available. Check the Development Experience Clearinghouse (DEC)133. • UNODC y GOC. Colombia. Explotación de Oro de Aluvión: Evidencia a partir de percepción remota. Reportes: 2016, 2017, and 2018. EVALUATION QUESTIONS Considering the Oro Legal Activity purpose stated above, the Contractor will conduct the systematic analysis around the performance and development of the Activity in its main goals and lines of work in order to answer the questions stated below. The questions might be reviewed and refined in any technical aspect prior to the finalization of the evaluation design in collaboration with USAID and the IP, following general co-creation principles. 1. Mining Formalization: To what extent do the application by Mining Production Units of the formalization standards, improved operational efficiency, and access to legal markets (MPU) gained through Oro Legal´s (OL’s) intervention provide sufficient incentives for miners involved in artisanal and small gold mining (ASGM) to remain legal/formal and improve the environmental performance of their mining operations? 132 https://www.usaid.gov/sites/default/files/documents/1862/USAID-Colombia-CDCS.pdf 133 https://dec.usaid.gov/dec/content/search.aspx USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 308 Context and how this question are expected to be answered: This is a follow-on to question #3 from the Mid-term Evaluation, adjusted to capture the outcome of the contribution of mining formalization towards one of the main goals of the Activity. Given a national context marked by a steady increase in international gold prices, local pressure from illegal actors, and a widespread culture of unlawfulness, have formalized gold miners been able to change their behavior in ways that demonstrate USAID´s contribution in this sector to the J2SR? If the benefits of formalization cited above have not provided sufficient incentives for ASGM miners to change mining and environmental behavior in a majority of cases, what underlying factors are preventing uptake of formalization and what other approaches can be identified/suggested to increase interest/uptake and under what requisite conditions? 2. Mining Governance and Policy: What have been the main factors (internal to Oro Legal or external arising from the sector context, stakeholder attitudes/behaviors, or existing government policy and capacity) that have contributed to or hindered OL’s progress in strengthening Colombian mining and environmental governance, and policies at national, regional, and municipal levels? Context and how this question is expected to be answered: This is a follow-on to question #2 from the Mid-term Evaluation. OL’s Theory of Change (ToC) proposes that the Activity will strengthen mining governance and policy, including at the municipal level. However, mining policy is a national purview, which is a flaw in the ToC not identified during the Mid-term Evaluation. Municipal authorities are only responsible for administration of the mining registry (RUCOM), have limited environmental authority, and lack the budget and means to enforce the mining and other laws against illicit armed groups linked to gold mining. Also, Antioquia is the only department with a Mining Secretariat with delegated mining authority by the Ministry of Energy (ME), whereas the department of Chocó does not enjoy similar status. Additionally, central government capacity to facilitate formalization is limited because of the sheer number of ASGM and other informal/illegal mining operations in Colombia, difficulties experienced in transferring operational responsibility for ASGM formalization from the ME to the National Mining Agency (ANM), and hesitant political will for effective mining policy reform. Given this context, it is critical to understand how effective OL’s approaches (both formal and informal) have been to policy and institutional capacity strengthening and what factors supported or hindered the Activity’s effectiveness in this area. The evaluation will suggest or recommend other approaches where it is relevant to do so. 3. Value Chain Development and Alternative Livelihoods: How do beneficiaries involved in OL´s value chains activities perceive the impact of alternative livelihoods on their present and future well-being outside the ASGM activities? Context and how this question is expected to be answered: This is a follow-on to question from the Mid-term Evaluation. The economic (not simply financial) and/or environmental rationale for offering alternative livelihoods to ASGM families is to: (i) provide an alternative licit source of income; (ii) reduce the disproportionate environmental impact of ASGM in tropical ecosystems; (iii) reduce the pressure on vulnerable populations from illegal armed groups linked to illicit gold mining; and (iv) consolidate broader regional security. Economic reconversion includes a set of real and perceived values, benefits, and costs beyond investment in time and inputs and income generation from the economic activity per se; e.g., peace of mind, quality of life, more time to devote to the family (particularly relevant for single women heads of household), personal health and safety, higher productivity, participation in a legal marketplaces, etc., which are often hard to quantify, but are paramount for stakeholders and contribute to broader Government of Colombia (GOC and USAID/Colombia development goals, objectives and priorities and, therefore, should be considered in answering this question. 309 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 4. Rehabilitation of Areas Degraded by Illegal Mining. How appropriate and effective since the environmental, social, and economic point of view have been the differentiated rehabilitation models developed by Oro Legal in Antioquia and Chocó in areas previously degraded by illegal mining? Context and how this question is expected to be answered: Rehabilitation initiatives are one of the most visible and potentially valuable USAID contributions under Oro Legal that were not properly addressed during the Mid-term Evaluation. Large-scale degradation of land and water resources by unregulated alluvial mining and river dredging is one of the most significant environmental problems in Antioquia and Chocó, with particularly serious impacts to critical and fragile ecosystems along the Cauca and Atrato and Quito rivers and tributaries, respectively. Most rehabilitation or reclamation models are complex and costly, and not suitable or economically viable for remedying impacts in degraded areas considered “environmental orphans” by the state, communities, and landowners. The Oro Legal models were purposely designed to be low-cost, practical approaches that are appropriate for private landowners and collective communities, and particularly relevant for global and national initiatives in forest restoration and climate change mitigation. Are the models cost-effective in comparison to other approaches employed in Colombia and elsewhere? Are they good for each regional reality? How robust are the models in quickly reestablishing vegetative cover, reversing degradation, and recovering biodiversity? How committed are beneficiaries to future maintenance of the models over the medium to long term? What conditions are required and/or what changes could be made to the models to make them more effective and replicable? 5. Elimination/Reduction of Mercury from the ASGM Supply Chain What combination of instruments and approaches employed in the effort to reduce mercury usage in the ASGM have proven to be most effective and why? Context and how this question is expected to be answered: Mercury has historically been used in ASGM, particularly by small-scale alluvial miners and river dredgers, as a cheap and effective amalgam to recover fine gold. Exposure to mercury is extremely harmful to humans and one of the major public health and environmental risks both in and downstream from areas where gold mining and processing are concentrated. Colombia is a signatory to the MINAMATA Convention on Mercury, a global treaty to protect human health and the environment from the adverse effects of mercury. Colombia also promulgated its own Mercury Law that bans all use of mercury in mining as of July 2018 and mandates financial support to mining operators for the transition to non-mercury processing, a provision of the law that has not been implemented. Through rigorous monitoring methods and documentation, Oro Legal beneficiary MPUs have eliminated or significantly reduced the amount of mercury used per gram of gold produced. How was this achieved and what were the motivations of beneficiary MPUs? What combination of instruments/approaches employed was most impactful? To what extent did external factors contribute to or hinder the Activity’s initiatives to reduce or eliminate the use of mercury? 6. Perceived Relevance and Impact of the Activity: What are the main perceptions of the relevance and effectiveness of the USAID - Oro Legal´s intervention among direct and indirect stakeholders - ASGM operators, gold mining private sector, academia, international donors, USG agencies, GOC agencies, etc.? Context and how this question is expected to be answered: The end of Oro Legal is an appropriate time for objective reflection and to seek honest feedback from all key stakeholders on the Activity’s contribution to ASGM, impact on the environment, and the socio-economic development in its two main geographies and in Colombia more generally. Stakeholders’ perceptions are influenced by three factors: (i) their personal circumstances and world view; (ii) their interests and expectations; and (iii) how well informed and knowledgeable they are about the Activity’s objectives, implementation, and outcomes. Assessment of key stakeholder opinions and feedback on specific components or activities, as well as incorporating broader cross-cutting considerations may uncover needs, expectations, and ideas USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 310 that are otherwise hard to capture, which, in turn, could trigger awareness, adaption, recognition, innovation, and learning for improvement in the future. The evaluation must offer information, analyses and findings, allowing USAID to: (i) have an objective and complementary perspective of the performance achieved by the Oro Legal Activity; (ii) make decisions for improving or refining future program designs. Those analyses and findings must consider or be disaggregated by department and by municipal level as well as categorized by the actors as working dimensions of the Activity’s developments: GOC, gold miners, livelihood and watershed beneficiaries, private sector, etc. Information and analysis should also be disaggregated by sex and ethnicity as much as possible. According to the findings obtained from the analyses to answer the questions, the evaluation must provide conclusions, actionable and context￾specific recommendations, and lessons learned for improving future designs, including potential means for replication and scalability. In general, the evaluation must offer a body of relevant information for future mining programming of USAID/Colombia and sharing with its stakeholders. Its design and methodology must be replicable in order to have technical validity and to be heuristic for other performance evaluation in the portfolio of USAID. EVALUATION DESIGN AND METHODOLOGY The Contractor through the Evaluation Team (ET) must propose the design and mixed methods methodology that will generate the highest-quality and most credible evidence in order to correspond to the evaluation purpose and questions. The final evaluation design and methodology will be approved by USAID. The design and methodology must be formal, documented, and replicable; if any novel or non-traditional method of sampling, data collection or analysis is employed, it must be technically justified, properly documented in the reports, and supported by indexed references. As a general guiding principle and pursuant of USAID policy, all the procedures and methods used must be documented and the sources properly cited or credited, regardless of whether any was partially or fully employed. Once the Contractor through the ET defines and agrees upon the design and methodology with USAID/Colombia, they must proceed to collect, organize, filter, compile, analyze, summarize and present the field data transformed into results and information according to the Evaluation Plan. The analysis will offer findings, conclusions, actionable and context-specific recommendations, and lessons learned that will concretely respond to the evaluation questions. The Contractor through the ET must take into account the following methodological approach and technical principles for the evaluation design and methodology: DESIGN MATRIX The Contractor must construct, propose and use a Design Matrix based in the USAID Evaluation Design Matrix Template134and corresponding guidance. The matrix must have at least the following columns: 134 https://usaidlearninglab.org/sites/default/files/resource/files/evaluation_design_matrix_templates.pdf 311 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV 1. Researchable Question(s) and Subquestion(s) 2. Dimensions and variables identified 3. Information Required and Source(s) 4. Type of data required: quantitative & qualitative; primary and secondary 5. Scope, Methodology, and Analytical Tools 6. Unit(s) of Analysis 7. Limitations and Key Assumptions 8. What the answer to the researchable question will likely allow the Evaluation to Say The final version of the design matrix will be approved by USAID / Colombia, and its latest version will be an annex to the final report. IDENTIFICATION OF BENEFICIARIES AND STAKEHOLDERS A central aspect of the evaluation design will be the definition of the universes of beneficiaries and stakeholders, and the identification of the sample (see next subtitle) of those who will be surveyed or interviewed; the Contractor will propose the criteria to identify the universe and must work closely with USAID/Colombia and the Implementer (IP) to determine this identification. The Contractor will revisit and update the map of stakeholders linked directly to Oro Legal and other key agents relevant for the analyses by using the objectives of the Activity, the Theory of Change, the Brief Updates, and establishing communications with Oro Legal. IDENTIFICATION OF THE DIMENSIONS, VARIABLES, INDICATORS, UNITS OF ANALYSIS, AND SAMPLE FOR THE EVALUATION The Contractor will devise any instrument necessary to collect the quantitative and qualitative data necessary to respond to the various questions of the evaluation by using mixed methods. The Contractor, in agreement with USAID, may explore alternative but valid ways to collect the necessary data. The Contractor will identify the dimensions, variables and indicators necessary to include in the design aimed to respond to the evaluation questions, based on the evaluation question laid out, the Results Framework, the Lines of Intervention of the Activity previously set forth, and the monitoring and performance information available. The Contractor will work together with the Implementer (IP) to define in the design matrix the units of intervention linked to each component of Oro Legal and to the evaluation questions; according to these, the Contractor will specify the unit of analysis for each dimension and variable selected for the evaluation. The Evaluation must use a mix of secondary and primary data sources at the national, departmental, municipal, organizational, and individual (or household) level. Also, the evaluation must obtain information of individuals from all relevant demographic groups to report on differential perception and experience among them, including men, women and LGBTI; youth; afro-colombian and Indigenous populations. It is not necessary to build a statistically representative sample for each demographic category. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 312 The Contractor will define and devise the sampling methods (when required)135, as well as the appropriate overall sample size to produce significant and reliable results. Previous to the collection of data, the ET will perform a Power Analysis in order to determine whether the minimum overall sample size is enough to detect an existing effect with at least 90% of chance, in those statistical analysis identified to respond partially or in full to any questions of the evaluation. If the calculated overall sample size results very large to be practicable, the ET will adjust and document the practical power level in agreement with USAID before proceeding to the collection of data. IDENTIFICATION OF THE ANALYTICAL TOOLS AND THEIR RANGES The Contractor will define clearly in methods presented and in the design matrix the mixed analytical and statistical tools deemed as the most appropriate to respond to each of the different evaluation questions. The selection of these analytical tools and statistical instruments will be as specific as possible based on the information available and the moment of the design, the type and amount of data planned to collect during the field phase, and in any key assumption or operational limitations foreseeable during the planning phase. The final selection of the mixed analytical and statistical tools will be agreed with USAID. The contractor will follow the formal customary practice of selecting specific analytical tools, mixed methods and statistical instruments before proceeding with the collection of data and information compilation, unless the Contractor agrees with USAID in what cases a backward approach is the most appropriate given constraints. The Contractor will analyze in advance the requirements, limitations, and appropriateness of each tool(s) to test or respond to the respective evaluation question(s). The limitations and weaknesses in the analyses, deviations from the key assumptions, missing data, biases, marginal testing levels, likelihood, level of significance, sample size, degrees of freedom, etc., will be succinctly included into the notes to the results and findings presented, and documented in the final design matrix annexed to the final report. SOURCES OF INFORMATION AND DATA MANAGEMENT The Contractor will annex to the Final Report a matrix of sources of information used during the evaluation. All the primary and secondary sources of information used must be traceable, and properly cited or credited in the drafts and final report, including personal oral or written communications. For the purpose of this evaluation, primary data are directly produced or attributed to the source. Secondary data are used by other sources; secondary sources citing or using other secondary sources should be carefully employed, and the citing secondary source should be renowned or credible. When using primary or monitoring data by USAID or the USG136, the ET must assess the quality of the data before using it; when using primary or secondary data from other sources the Contractor must note in the analysis and the results any limitations and data quality issues identified. 135 In cases of using a sample, the contractor must clearly specify the techniques to use (randomized, purposively, etc.). Samples should allow representativeness for the most relevant variables at the clusters of interventions with a margin of error equal or less than 5 percent in every possible case. 136 USG: The United States Government 313 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV The ET must put in place protocols for ensuring respondents of primary sources instruments are not harmed (especially if there are sensitive questions involved) and for the collection and safety handling of private data (PII). All the data collection instruments devised and used must be annexed to the final report in a widely used readable digital format. The main outputs from analytical software packages such as statistical or qualitative analyses containing further details should be annexed in full to the final report. All the surveys, queries, recordings, and data retrieved to respond to the evaluation must be digitized and annexed in known formats to the final report; the datasets collected and built, including GIS137 information must include either annexed or embedded metadata USAID / Colombia will provide metadata template suggested). Filenames, tabs or tables within databases or datasets containing Personal Identifiable Information (PII), should be labeled or named with the prefix “PII_”. The Contractor will follow the guidelines of USAID for the handling and transmission of PII provided by the Contractor´s COR. All quantitative data and coded qualitative data collected or retrieved by the ET must be provided in digital machine-readable, non-proprietary formats as required by USAID’s Open Data Policy (see ADS 579). The data should be organized and fully documented (i.e. containing metadata) for use by anyone not fully familiar with the Activity or the evaluation. USAID will retain ownership of the survey and all datasets developed. The contractor will prepare in a timely manner the datasets to be uploaded to the USAID Development Data Library (DDL). RECORD KEEPING Record keeping is paramount to allow USAID to search and retrieve easily any supportive documentation developed at any time along the evaluation process. It reinforces the replicability and freedom of access to information. The Contractor will keep, organize, store, and retrieve in a digital readable known format all the primary and secondary sources records, data retrieved or built, and information used for the evaluation, and will annex it the final report. The Contractor will keep, organize, store, and retrieve in a digital readable known format all the digital outputs of the statistical and qualitative analysis performed and other methods used for the evaluation. The Software packages used must be documented in the final version of the design matrix and will contain: make, package version, and digital platform used (Windows, Linux, etc). The contractor will develop a hierarchical filing system and convention to keep, organize, store, and retrieve the above-mentioned records. Acronyms and abbreviations should be listed and included in a code dictionary at the root folder in order to understand and consult the filing system. It may be used the following naming convention suggested as guidance: 1. File naming convention: always avoid proprietary, unmeaning, and short file names like “survey.doc”, “final.pdf”, “field.xlsx”, “solo-Antioquia.dbs”, “Kamilo5.ppt” 2. Date prefix (date of file creation or approval): yymmdd_ 3. Phase on the timeline: it is recommended to create folders and subfolders according to the next structure: 137 Geographic Information Systems USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 314 Data Collection (raw data): DC_ Data Processing: DP_ Analysis: Anls_ Question #: Q1_ Q2_ Q3B_, etc Results: Res_ Reporting: Rep_ 4. Subject naming: use meaningful and specific names. For example, to name the output of a regression use “Regress” instead of “rg”. 5. Suffix for consecutive version coding (integers only): V1, V2, V3, ..., etc. Avoid using “final” unless it is fully approved by another person than the producer of the document. For instance, a t-student test output from Stata package could be named like: 200812_Anls_Q2_T￾studentEthnic_V1.txt; a database could be named like 200730_DC_PII_RockMinersSurvey_Q3B_V1.sql, while the first draft could be: 201024_Rep_Draft-Report-FindingsOnly_V2.ppt EXECUTION, GEOGRAPHY AND PERIOD OF PERFORMANCE The evaluation will be conducted in Colombia. Specific geographic areas the Contractor’s ET will visit are the Oro Legal geographic coverage (see Annex A), which includes Bogotá where national institutions are based. Although the list of departments and municipalities covered by Oro Legal is at Annex A, the definite list of municipalities visited by the ET to retrieve the necessary information will be defined during the design stage of the evaluation. It is estimated that this evaluation will be carried out over a seven month period, approximately from May through November of 2020. The Contractor is expected to begin work immediately upon official acceptance of the Contractor´s COR. The contractor will present a succinct but comprehensive Design and Work Plan with the following consecutive stages or parts: 1. Design and Methodology 2. Planning and Fieldwork 3. Data Processing and Analysis 4. Results and Reporting 5. Dissemination Plan 6. Annex 1: Evaluation Team The plan must include at minimum: a) a graphic calendar of the phases with suggested dates matching the timeline set forth below; b) Main lead or responsible at each phase -if applicable-; c) Milestones and/or deliverables; d) Geographic location and/or venues; e) Main stakeholders and parties involved; f) Main limitations, risks and assumptions considered. USAID will approve the Design and Work Plan presented by the contractor. The dissemination plan will include activities, deliverables and means of 315 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV communication to share with selected stakeholders the main findings, actionable and context-specific recommendations, conclusions, and lessons learned of this final performance evaluation. a) The Evaluation in this SOW will be considered successfully executed and completed once: Every phase of the Evaluation Plan has been successfully executed and completed by the contractor upon official notice of the Contractor´s COR. b) All the deliverables in this SOW have been submitted by the contractor and have been accepted and approved by USAID/Colombia, according to the Acceptance Criteria set forth below. c) The contractor has uploaded to the USAID Development Clearinghouse (DEC) the approved version by USAID/Colombia of the final report; the contractor has uploaded the datasets to the USAID Development Data Library (DDL); and the CORs of both Activities are notified about it. REPORTING REQUIREMENTS AND DELIVERABLES USAID/Colombia through the EAM may request to the Contractor a summarized report about the progress at any time during the length of the contract for this evaluation. The contractor through the ET must submit to the USAID the deliverables detailed in the plan below, taking into account the timeframe and order set forth to be finalized. A. Planning Phase, weeks 1 – 4 1. Evaluation Planning Meeting and Meeting Minutes, week 1: At the outset of the work, USAID and the ET will hold an initial meeting in Bogotá to review expectations and brief the ET on the context and the Activity to inform the preparation of the evaluation. The contractor will prepare and submit to USAID a succinct meeting minutes in bullets with actions, guidances, steps and other matters discussed or agreed upon. 2. Context Session and Minutes, week 1: At the outset of the work, the ET will hold an initial meeting in Medellin to get familiar with IP´s team and to have a wide grasp and understanding of the Activity and the current gold mining context before proceeding with the Desk Review, interviews or any collection of data. In this one-day session, other relevant stakeholders may participate to help outline the context for the ET. The Contractor will prepare and submit to USAID a succinct meeting minutes in bullets with the topics addressed. It is recommended the contractor to start reviewing the relevant documentation (see Section 3.3). 3. Desk Review, week 2: The ET will conduct a desk study that covers all relevant documents (see Section 3.3) and identifies likely dimensions, variables, revisit the stakeholders´ analysis and in general, to acquire a more in-depth understanding of the Activity and the Context. The ET will hold initial conference calls with USAID and Oro Legal to seek their guidance and feedback on a draft design matrix and the list of stakeholders to be interviewed. 4. Drafts of the Design Matrix and summarized Work Plan, week 3: The ET will present and submit the draft Evaluation Design Matrix and a summarized draft version of the work plan. based on the feedback from USAID, the Contractor will prepare the Evaluation Design and Work Plan document. The Contractor should start planning the logistics for the field phase USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 316 and contact USAID´s security liaison for further security guidance. The IP to Oro Legal might also provide some guidance on the security and logistic aspects to take into account at each field site. 5. Evaluation Design and Work Plan, week 4: The revised draft evaluation design and work plan document will be submitted to USAID for final review and approval, highlighting any agreed￾upon changes from the draft submitted to USAID. The evaluation design and work plan must include at minimum: a fully developed design matrix; the methodology; data collection instruments; the proposed sampling design and sites; the proposed list of survey and interview participants; the schedule, including locations and dates for sampling; logistical arrangements; an outline of the final report; and the draft dissemination plan. B. Fieldwork and Analytical Phase, weeks 5 - 15 6. Fieldwork conducted and weekly bullets submitted, weeks 5 - 14: The contractor will commence the fieldwork by week 5 of the contractors´ COR approval of this SOW and upon USAID approval of the Evaluation Design and Work Plan. The fieldwork implies the implementation of the instruments and other methods to collect qualitative and quantitative data and revisiting relevant documents. The analytical part implies the application of the analytical tools and methods to build information. The ET will report each week to the EAM or his designate. Weekly bulleted lists of key achievements and/or issues -no more than 12 bullets- must be delivered to USAID/Colombia; each consecutive report issued and submitted should be numbered. 7. Pre-drafting debriefing meeting, week 15: Prior to commence drafting the evaluation report, the ET will hold a debriefing with the EAM, and any others that the ET and EAM consider should attend, to present orally their preliminary insights on findings and results from the analysis to that point. Some of these preliminary notions will change as analysis and drafting continue. Draft recommendations and lessons learned should be discussed with USAID, the IP, and stakeholders to check their feasibility. C. Reporting Phase, weeks 16- 24 8. CLA meetings for final report drafting, week 18: Three weeks after the debriefing meeting (by week 18), The contractor through the ET will make detailed presentations (Power point format) of the findings, conclusions and recommendations to USAID and the IP. These spaces will aim to present the collected evidence in depth and to receive feedback in order to strengthen the evaluation exercise within the framework of a collaborative scheme and active participation of the parties. USAID and the IP will submit comments on the presentations to the ET, within two weeks after CLA meetings. Recognizing that the review of the presentation by USAID and the IP may not agree with all comments, and to be sure that each and every comment is considered carefully by the ET, the Contractor will build a table that lists each comment, the response of the ET to the comment, and where the changes resulting from the comment – if any – will be found in the final report. This same cell will also indicate if no changes were made and why not. 9. Final Evaluation Report, weeks 20-22: Two weeks after receiving USAID’s comments, the ET will submit to USAID/Colombia the final report electronically; USAID/Colombia, including the 317 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV aforementioned table to track comments. After the submission of the final report, USAID can still send comments that may entail final adjustments before approval and acceptance of the final report. These comments will be based on the criteria for ensuring the quality of the Evaluation as listed in the Section “Acceptance Criteria of this Evaluation”. 10. Dissemination, weeks 22-24: The ET is expected to present the findings and recommendations of the final. At least 2 (and no more than 4) final presentations may be required in person to discuss the summary of findings and recommendations with USAID, the Evaluation contractor and other key stakeholders identified by USAID. These presentations will be scheduled as agreed upon with USAID. Some of the presentations may be orally in English. FINAL REPORT The evaluation final report should include an abstract of no more than 250 words in English and Spanish; an executive summary in English and Spanish; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see “USAID How-To Note: Preparing Evaluation Reports” and ADS138 201mah, USAID Evaluation Report Requirements. A suggested evaluation report template is available in the Evaluation Toolkit. The executive summary should be 2–5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations, and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail in annexes. l. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The body of the report (apart from the abstract, the executive summary and the annexes) should be no more than 60 pages. All other detailed analysis or data must be included in annexes. The annexes to the report will include: • The most recent version of this Evaluation Statement of Work (SOW); • All data collected, methodologies, and analytic tools used in conducting the evaluation, such as questionnaires, checklists, maps and GIS data, analytical outputs, technical procedures, etc; • All sources of information properly cited and listed using a widely accepted standard format. 138 USAID Automated Directive System (ADS) or Operational Policy. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 318 • Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest. • Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. • Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with USAID ADS 201, the contractor will make the final evaluation report publicly available through the Development Experience Clearinghouse (DEC) repository within one month of the approval of the final report by USAID; also, during that time, the Contractor will upload the datasets to the USAID Development Data Library (DDL) within the same period of time. ACCEPTANCE CRITERIA OF THIS EVALUATION The acceptance criteria or definition of “job done” of this evaluation is based on (a) the submission and receipt of all the deliverables in this Statement of Work; (b) each deliverable fulfills the terms to be accepted with minimum grade of “satisfactory”; and (c) the evaluation report meets each and every quality criteria established at the USAID ADS 201maa and the USAID Evaluation Policy with a minimum grade of “satisfactory”. Pursuant to the ADS 201 (Part 201.3.5.17) the Office of Environment of USAID / Colombia will employ a peer technical committee to review and rate the draft reports and technical deliverables, as follows: 1. MEL specialist to the Office of Environment. 2. COR of the Activity Evaluated. 3. Technical member of any office within the USAID / Colombia; Technical Bureaus or the Regional Bureau. In order to rate other deliverables of this SOW such as meetings, procedures performed, etc by the ET, the contractor´s COR may serve as the third member of the committee. Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.139 • Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. • Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. 139 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance 319 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV • The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. • Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. • Evaluation methodology should be explained in detail and sources of information properly identified. • Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. • Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. • If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. • Recommendations included should be supported by a specific set of findings and should be action￾oriented, practical, specific. The evaluation criteria to rate the deliverables in this SOW will follow the same criteria defined in the FAR 42.1503 (Table 35 summarized below: TABLE 35: EVALUATION CRITERIA RATING REQUIREMENTS PROBLEMS CORRECTIVE ACTIONS Exceptional Exceeds Many – Gov’t Benefit Few Minor Highly Effective Very Good Exceeds Some - Gov’t Benefit Some Minor Effective Satisfactory Meets All Some Minor Satisfactory Marginal Does Not Meet Some – Gov’t Impact Serious; Recovery Still Possible Marginally Effective; Not Fully Implemented Unsatisfactory Does Not Meet Most – Gov’t Impact Serious; Recovery Not Likely Ineffective USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 320 OTHER PROVISIONS OVERSIGHT AND MANAGEMENT The ET will report to USAID’s Evaluation Activity Manager, as named above, or subsequently delegated. The Evaluation Activity Manager will work in close collaboration with the COR of the Artisanal Mining Program, as named above. ACCOUNTABILITY AND SCOPE OF THE ANALYSIS The Contractor will be responsible to ensure that the ET complies to the objectives of this evaluation, respond technically to the evaluation questions, and have evidence-supported and verifiable argumentation to present the findings, and elaborate the lessons learned, recommendations and conclusions. Although the USAID Evaluation Policy140 embraces the principle of the “mutual accountability” with the stakeholders, the current evaluation is not meant to be an auditory of the implementation. The analyses, recommendations and conclusion must respond objectively to the questions laid out regarding the purpose of the evaluation, not to what ought to be respect to personal stands of the ET members. Every possible achievement and/or shortcoming of the implementation must be measured and documented and will respond to the best empirical evidence and most suitable objective analysis available. MANAGEMENT OF SERIOUS REMARKS AND DISPUTE RESOLUTION Should any irregularity or serious remark arise, neither contractor will reach directly to the other to address the matter without the consent and/or presence of the other party´s Contracting Officer Representative (COR). The decorum, professionalism, excellent communication, and proactive attitude will be the governing principles for any dispute resolution. If the Contractor through the ET discovers irregularities or has serious remarks about the performance of the implementation during the evaluation process, it will communicate in a separate summary written report those findings to its own COR, who will transmit it to the COR of the other Activity, before submitting those in any draft report and/or communication to the IP. Likewise, the same mechanism will be applied if the IP discovers irregularities or has serious remarks on any part of the evaluation developed by the Evaluation Contractor. The defendant will have the right to reply and will present its arguments and counter-evidence only to its respective COR in the first instance. Given that USAID has established other mechanisms to assess the contractual performance of each contractor, the Contracting Officer Representatives (CORs) and/or the Contracting Officers (COs) to both Oro Legal and to the Contractor´s Evaluation, will decide based on the evidence and allegations presented by both parties in which extent the irregularities or remarks pertain to the technical scope of the current evaluation or should be addressed by a different mechanism established by the USAID. The CORs of both contractors may arrange moderated meetings and other mechanisms to convene the contractors and other relevant parties to address the contentious matter(s). Each COR or CO will communicate in writing the decision of USAID and/or the technical guidance arrived at to each respective contractor to settle the matter(s). 140 https://www.usaid.gov/sites/default/files/documents/1870/USAIDEvaluationPolicy.pdf 321 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV When applicable, the evaluation report will include “Statements of Differences” regarding any significant unresolved differences of opinion on the part of funders, implementers, and/or members of the evaluation team. LOGISTICAL SUPPORT The Evaluation Contractor is responsible for arranging all logistical support for the evaluation. However, given the difficulties of travel within project intervention sites, Oro Legal will be available to provide advice on security issues to the ET. SECURITY The evaluation contractor will be responsible for its own security coordinations as per stated in its contract and will follow the channels and procedures mandated or established by USAID/Colombia. As part of the overall security requirements, the Evaluation Contractor must report any security threats and/or incidents immediately to USAID/Colombia. All subcontractors will be required to report any threats or incidents to the prime Contractor, who will immediately after, notify the USAID/Colombia. EVALUATION TEAM COMPOSITION An experienced team performing the task must implement the field data collection. This work can be done through a subcontract and may need to be coordinated or supported by a survey team with proven successful experience. The personnel that the Evaluation Contractor must furnish for the performance of this Evaluation are as follows and will constitute the ET: a) Evaluation Team Leader b) Evaluation Specialist c) Mining Affairs Specialist d) Environmental Specialist e) Value Chain Specialist f) Social Specialist g) Quantitative Expert h) Data Collection Firm The following are the recommended responsibilities and qualifications the respective Evaluation Team members should have: a) Evaluation Team Leader Responsibilities: the Team Leader will have responsibility for all technical and operational development of the evaluation. S/he will be primarily responsible for maintaining the technical communication with USAID and the Implementing Partner, implementing the detailed methodology, managing and implementing the work plan and all the related evaluation team activities, and leading the writing of the main technical deliverables and technical communications. S/he will also be responsible for presenting findings during briefings, out-briefings, final presentations, and final report writing. Required Qualifications: - Masters or other advanced degrees in any relevant related area to the evaluation. - Strong analytical skills. - Skills or knowledge on project administration, economics, and team management. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 322 - At least five (5) years of experience in managerial positions at international cooperation projects, field-based academic projects, and/or field-based public or private social investment projects. Minimum of three (3) years of that experience conducting or leading the evaluation of projects. - Relevant subject matter expertise demonstrated by published evaluations, peer reviewed publications; and/or professional trajectory of at least five constant years (5) in the academy, environmental, economic, development or the extractive sector. - Good knowledge of mining or extractive issues in Colombia, Latin America or the tropics; highly desirable to have knowledge regarding regulatory and environmental aspects of mining. - Good knowledge of the rural societal and economic contexts in Colombia. - Understanding of USAID or Public International Organizations. - Excellent written and oral communication skills in English and Spanish. b) Evaluation Specialist Responsibilities: the Evaluation Specialist will lead the design and methodology, sampling, and will conduct or lead the technical analyses. S/he will lead the literature and desk reviews, will guide the collection of data, surveys and/or interviews; S/he will be responsible for the information management and record keeping, and will assist the Team Leader in the compilation and presentation of results, findings, recommendations, and lessons learned. Required Qualifications: - Professional in social, natural, earth or data sciences. - Masters or other advanced degrees in evaluation, economy, econometry, statistics, data sciences, business intelligence, or related area. - Working knowledge in geographic information systems (GIS). - At least five (5) years of proven experience in experimental design-testing, conducting and leading field-based evaluations of projects or public policy. in Two (2) of the five (5) years can be replaced by experience in monitoring and evaluation of development projects. - Proven knowledge and experience in evaluation by mixed (qualitative and quantitative) methods, preferably in the context of rural development. - Demonstrated capability in designing evaluation tools, conducting evaluation interviews, focus groups, managing and interpreting qualitative and quantitative data collection, analysis processes and utilizing secondary data. - Experience working with survey statistics, integrating and analyzing different datasets, and conducting structured and non-structured interviews, and focus groups. - Ability to manage logistics, including arranging key informant interviews and group interviews/focus groups, accommodations, and travel. - Experience or knowledge in team management. - Excellent oral and writing skills in Spanish and English. c) Mining Affairs Specialist Responsibilities: The Mining Affairs Specialist will serve on the evaluation team and will support the Team Leader with research, subject matter consultations, data collection, analysis, and subject writing. Required Qualifications: - Professional in engineering, natural or earth sciences. - Graduated degree in mining, geology, industrial engineering, civil engineering, environmental engineering, production engineering, landscapes, industrial ecology or geography. - Desirable working knowledge and skills in geographic information systems (GIS). - At least six (6) years of experience in the mining sector of metals in Colombia. 323 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV - Good knowledge of mining or extractive issues in Colombia, Latin America or the tropics. - Good knowledge of the regulatory and operational aspects of gold mining. - Desirable working-level written and verbal communication skills in English and Spanish and ability to conduct interviews and focus groups in Spanish and provide presentations and documents in English. d) Environmental Specialist Responsibilities: the Environmental Specialist will serve on the evaluation team and will support the Team Leader with research, subject matter consultations, data collection, analysis, and subject writing. Required Qualifications: - Professional in engineering, natural or earth sciences. - Graduated degrees in environmental sciences, geography, environmental engineering, landscapes, industrial ecology, forestry, resources management, or related areas. - Desirable working knowledge and skills in geographic information systems (GIS). - At least six (6) years of experience working in Colombia on environment issues. - Relevant expertise and experience conducting program evaluation or similar research and analytical work. - Strong knowledge regarding environmental issues as related to mining, forestry, land reclamation, resources management, environmental toxicology, landscape ecology, environmental biology and/or conservation. - Good knowledge of the regulatory and environmental aspects of mining or gold mining. - Working-level written and verbal communication skills in English and Spanish and ability to conduct interviews and focus groups in Spanish and provide presentations and documents in English. e) Value Chain Specialist Responsibilities: the Value Chain Specialist will serve on the evaluation team and will support the Team Leader with research, subject matter consultations, data collection, analysis, and subject writing. Required Qualifications: - Professional in engineering, social, natural or earth sciences. - Graduated degree in economy, business, social projects, production, agronomy, agroecology, agriculture, development, industrial ecology, environmental engineering or related areas. - At least six (6) years of experience working in Colombia on agricultural value chain issues. - Relevant expertise and experience conducting program evaluation or similar research and analytical work. - Strong knowledge regarding value chains development and commercialization in rural Colombia. - Working-level written and verbal communication skills in English and Spanish and ability to conduct interviews and focus groups in Spanish and provide presentations and documents in English. f) Social Specialist Responsibilities: the Social Specialist will serve on the evaluation team and will support the Team Leader with research, subject matter consultations, data collection, analysis, and subject writing. Required Qualifications: - Professional in social sciences. - Graduated degree in anthropology, economy, development studies, gender or ethnic studies, cultural studies, governance, sociology or related areas. - At least five (5) years of experience working or researching in Colombia on rural social or development issues. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 324 - Relevant expertise and experience conducting program evaluation or similar research and analytical work. - Strong knowledge regarding culture of unlawfulness/illicit economies, extractive industries, socio-economic issues, or rural development in Colombia. - Strong background and training in mixed methods research. - Experience and knowledge of qualitative data analysis and qualitative data management techniques using NVivo. - Working-level written and verbal communication skills in English and Spanish and ability to conduct interviews and focus groups in Spanish and provide presentations and documents in English. g) Quantitative Expert Responsibilities: The Quantitative Expert will serve on the evaluation team and will support the Team Leader with quantitative methodology for the Work plan drafting including sampling methodology, design instruments for quantitative data collection, analysis, and drafting of quantitative final report Required Qualifications: - Professional in economics, statistics, social sciences and related fields. - Graduated degree in economics, statistics and related fields. - A minimum of seven years (7) of experience conducting quantitative research. - Proven experience in conducting high quality quantitative analysis. - Education in specific subject area, with country experience would be helpful. - Strong background and training in mixed methods research. - Experience and knowledge of data analysis and data management techniques. - Working-level written and verbal communication skills in English and Spanish h) Data Collection Firm The Data Collection Firm will perform the data collection following the technical direction of the Contractor based on the requirements of this SOW and the needs identified during the design and planning phases. Required Qualifications: - Proven experience in household level surveys in conflict areas of Colombia. This team may also be supported by assistants with at least three (3) years of experience in its respective field. a) All the members of the ET are considered essential to the work being performed under this SOW. The Evaluation Contractor will get personnel for the ET with a dedication of at least 50% for the term of this SOW unless otherwise approved by USAID. Each ET person will be under the direct supervision of the prime Contractor (not under a subcontractor), unless otherwise approved by USAID. b) Failure to provide ET personnel may be considered non-performance unless such failure is beyond the control of the Evaluation Contractor. All individuals listed as ET Personnel must reside in Colombia for the duration of the SOW. c) The Contractor must immediately notify USAID / Colombia of any ET personnel’s departure and the reasons therefore. The Contractor must promptly propose a replacement for each vacated position. The proposed replacement should possess the qualifications listed above for the corresponding 325 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV personnel. The Contractor must not replace any of the ET Personnel without the prior written approval of USAID / Colombia. d) ET members should not have any past or current business relationships with the USAID Oro Legal Activity or the USAID BioREDD+ project / Mining component. The proposed personnel should sign an Organizational Conflict of Interest (OCI) letter submitted as an annex within the Work Plan. e) The team members should not have any known adverse remarks concerning past performance or communication in Activities or projects funded by USAID. f) USAID / Colombia reserves the right to vet the ET personnel proposed by the Contractor as well as to adjust the number of the ET Personnel during the performance of this SOW. MODIFICATIONS TO THIS SOW All modifications to the required elements of this SOW in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, or deliverables, must be agreed upon in writing by the Contractor´s COR and by USAID/Colombia. Any revisions should be updated in the SOW that will be included as an annex to the final evaluation report. THE ARTISANAL GOLD MINING (ORO LEGAL) GEOGRAPHY BY DEPARTMENT AND MUNICIPALITY Department of Antioquia 1. Barbosa 2. Buriticá 3. Cáceres 4. Caucasia 5. Don Matias 6. El Bagre 7. Nechí 8. Remedios 9. San Roque 10. Segovia 11. Tarazá 12. Zaragoza Department of Choco 1. Atrato (Yuto) 2. Cértegui 3. Condoto 4. El Cantón San Pablo 5. Istmina 6. Nóvita 7. Quibdó 8. Rio Quito, 9. Tadó 10. Unión Panamericana USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 326 Exhibit 73: GIS Representation of the The Artisanal Gold Mining (Oro Legal) Geography 327 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV ANNEX VIII: SUMMARY INFORAMTION ABOUT EVALUATION TEAM MEMBERSBERS TABLE 36: EVALUATION TEAM MEMBERS POSITION NAME PROFESSIONAL PROFILE Evaluation Team Leader Manuel Castro MA in Public Policy and Public Administration, BA in economics, BA in political science. Experience as government executive, multilateral agency official, publicsector specialist, international consultant, evaluation director and university teacher in public policy and public administration with emphasis in public sector reform, territorial investment and capacity strengthening policies, public investment, and results-based management. Expert in the design and implementation of territorial development policies and tools, planning, implementation capacity, monitoring and evaluation systems linked to the budget process. Specialist in the design and implementation of results-based budgeting reforms aimed at improving quality of service delivery and public expenditure. Support in implementing and advising such reforms nationally and sub nationally in a range of countries, including in Latin America (Mexico, Colombia, Uruguay, Panama, Dominican Republic, Bolivia, Honduras, Trinidad & Tobago, Jamaica), Asia (China, Indonesia, Filipinas), and Africa (South Africa, Mauritius, Seychelle Islands). Evaluation Specialist Mauricio Aguilar Economist and Master in Economics from the National University of Colombia and Master in Evidence￾Based Social Intervention and Policy Evaluation from the University of Oxford in the United Kingdom. He has more than six (6) years of experience in the application of statistical research methodologies. He has a history of 15 years in the design, implementation, monitoring and evaluation of public policies, where his work stands out in different entities such as the DNP, the ministry of health and social protection, social prosperity, USAID, IOM, UNHCR, FINDETER, IDB, the development research center (CID) of the national university of Colombia, holding positions of project director, advisor on evaluation issues, quantitative expert, among others. Additionally, he has been a postgraduate professor at the Universidad de Atlántico de Barranquilla, the Higher School of Public Administration (ESAP), as well as an undergraduate professor at the Universidad de los Andes and the Universidad Nacional de Colombia. Mining Affairs Specialist Jairo Herrera Geologist engineer and master in resource management from Universidad Nacional de Colombia, specialist in mining and energy law from Universidad Externado de Colombia. 20 years of integrated experience in mining sector planning, mining exploration (coal, gold, barite and emerald), environmental, hydrology, hydraulics and fluvial geomorphology. Director of Mines of the National Mine Planning Agency (UPME) for three years; exploration manager for Zamin Group (2 years) and Trident Gold (1 year). Geologist consultant for companies as Volador, Carbones del Cerrejón, Canacol Energy, RCS, Austmine, Sikuri Mining, San Pancrecio Mining, Co-author in investigations for national authorities: National Mining USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 328 TABLE 36: EVALUATION TEAM MEMBERS POSITION NAME PROFESSIONAL PROFILE Development Plan (2006-2010, View to 2019 and 2011-2014), Mercury chain in Colombia with emphasis in gold mining, Plan for climate change in the mining and energy sector of Colombia, Effects caused by gold and coal mining on the availability of water resources. Environmental Specialist María Andrea Rueda Biologist from Universidad Javeriana, with a master's degree in environmental law from Universidad Externado de Colombia. 12 years of experience in the environmental sector, working mainly in central government institutions. Leader in evaluations of environmental interventions within the DNP, mining planning and environmental licensing process, additionally with experience in the formulation of policies for biodiversity conservation, sustainable forests and climate change. Principal investigator of different international environmental agendas such as the Sustainable Development Goals, the Paris Agreement and the Convention on Biological Diversity. Experience in promoting sustainable development in rural communities, with emphasis on forest management. Value Chain Specialist Luis Fernando Monroy Industrial Engineer (Universidad Javeriana 1991) and MBA (Universidad de los Andes 1998). Experience as specialist for more than 12 years in design, implementation and monitoring of rural development projects in Colombia aimed at socio-business, financial and commercial strengthening of rural productive organizations focused on improving competitiveness, promoting sustainable development and social inclusion. This expertise has been developed with public institutions such as the productive alliances program of the ministry of agriculture, with the commercialization direction of rural development agency (ADR) and the productive projects program of land restitution unit (URT), as well as with the NGO PBA applying the rural participative innovation methodology and as consultant of Cadena de Valor, a rural development advisory firm. Quantitative Expert Oscar Quiroz Bachelor of Economics (Universidad Externado de Colombia, 2009), Specialization of Statistics (Universidad Nacional, 2012) and MSc of Economics (Universidad de los Andes, 2018). Currently he has 10 years of work experience related with economic research, quantitative and qualitative analysis. From 2010 to 2011 he worked in market research firms such as IPSOS and YANHAAS leading quantitative research and samples designs. From 2011 to 2013 he worked in the Chamber of Colombian Construction – CAMACOL in the economic research team. In 2014 he joined the public sector in the Institute of Urban Development of Bogotá. From 2015 to 2018 he worked at the National Planning Department of Colombia advising the Territorial Affairs Office, Sustainable Rural Development Office and Territorial and Public Investment Sub-Directorate. In 2019 he returned to the Chamber of 329 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV TABLE 36: EVALUATION TEAM MEMBERS POSITION NAME PROFESSIONAL PROFILE Colombian Construction as Chief Economist and led a group of economists to do macroeconomic and sectoral research. In the most recent years, he worked as a consultant at UNDP managing and leading the design of a Governance Index for Colombian municipalities Social Specialist Juliana Moreno Bachelor of Economics (Universidad de los Andes Colombia, 2011) and MA in Critical and Creative Analysis (Goldsmiths, University of London, 2019). Over nine years of work experience as a public policy advisor in social security and poverty alleviation programs within the Colombian National Government. Experience in the implementation of policies targeted towards marginalized populations such as victims of internal forced displacement, children, indigenous and black people, youth, woman and LGBTQ. Involved in the roles of design, planning, implementation, monitoring and evaluation of social strategies, including the leading supervision role in the impact evaluation of "Youth in Action" (Jóvenes en Acción), the second-largest conditional cash transfer program in Colombia, during 2017. Qualitative Specialist Camila Chavarría Economist from Jorge Tadeo Lozano University and current student in the master program in social and cultural studies at El Bosque University. More than 5 years of experience as a research assistant and consultant in projects with qualitative, quantitative and mixed methods for private and state-owned organizations, including evaluating programs and public policies. Skilled in research, communitarian work, leadership ability, design and application of information collection instruments, including surveys, interviews, focus groups, ethnographies and case studies, also processing and analysis of large volumes of information through software such as AtlasTI, NVivo, Stata, SPSS and Tableau. USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 330 ANNEX IX: SIGNED DISCLOSURES OF CONFLICTS OF INTEREST FROM EVALUATIONTEAM MEMBERS 331 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 332 333 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 334 335 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 336 337 | ORO LEGAL EVALUATION FINAL REPORT ANNEXES USAID.GOV USAID.GOV ORO LEGAL EVALUATION FINAL REPORT ANNEXES | 338