MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY PERFORMANCE EVALUATION OF THE GOVERNANCE IN ECOSYSTEMS, LIVELIHOODS, AND WATER ACTIVITY IN HONDURAS SUBMISSION DATE: February 27, 2019 Contract Number: AID-OAA-I-15-00019 – Task Order Number: AID-522-TO-16-00002 This publication was produced for review by the United States Agency for International Development. It was prepared by International Advisory Products and Systems (i-APS), for the MESCLA Activity, implemented by Dexis Consulting Group. Monitoring and Evaluation Support for Collaborative Learning and Adapting i MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY PERFORMANCE EVALUATION OF THE GOVERNANCE IN ECOSYSTEMS, LIVELIHOODS, AND WATER ACTIVITY IN HONDURAS VERSION: Final - February 27, 2019 SUBMITTED TO: USAID/Honduras Prepared by: International Advisory Products and Systems (i-APS) for Dexis Consulting Group DISCLAIMER: The authors’ views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Monitoring and Evaluation Support for Collaborative Learning and Adapting ii TABLE OF CONTENTS Executive Summary.......................................................................................................................................................................1 1 Introduction ...........................................................................................................................................................................7 1.1 GEMA Background .....................................................................................................................................................7 1.2 GEMA Performance Evaluation ...............................................................................................................................8 1.2.1 Purpose and Audience...........................................................................................................................................8 1.2.2 Evaluation Questions.............................................................................................................................................8 1.2.3 Evaluation Team and Schedule ............................................................................................................................9 2 Methodology..........................................................................................................................................................................9 2.1 Overall Methodology .................................................................................................................................................9 2.2 Data Collection ........................................................................................................................................................ 10 2.3 Data Analysis............................................................................................................................................................. 11 2.4 Limitations.................................................................................................................................................................. 12 3 Findings and Conclusions ................................................................................................................................................ 13 3.1 Conservation Enterprises (CE) ............................................................................................................................. 13 3.1.1 Findings................................................................................................................................................................... 13 3.1.2 Conclusions ........................................................................................................................................................... 18 3.2 Resilience ................................................................................................................................................................... 19 3.2.1 Findings................................................................................................................................................................... 19 3.2.2 Conclusions ........................................................................................................................................................... 23 3.3 Gender........................................................................................................................................................................ 24 3.3.1 Findings................................................................................................................................................................... 24 3.3.2 Conclusions ........................................................................................................................................................... 26 3.4 GEMA Resources..................................................................................................................................................... 27 3.4.1 Findings................................................................................................................................................................... 27 3.4.2 Conclusions ........................................................................................................................................................... 28 3.5 Integration.................................................................................................................................................................. 28 3.5.1 Findings................................................................................................................................................................... 29 3.5.2 Conclusions ........................................................................................................................................................... 30 3.6 Sustainability and Governance .............................................................................................................................. 30 3.6.1 Findings................................................................................................................................................................... 30 3.6.2 Conclusions ........................................................................................................................................................... 34 3.7 Constraints and Opportunities ............................................................................................................................. 34 3.7.1 Findings................................................................................................................................................................... 34 3.7.2 Conclusions ........................................................................................................................................................... 35 3.8 Water Quality........................................................................................................................................................... 35 3.8.1 Findings................................................................................................................................................................... 35 3.8.2 Conclusions ........................................................................................................................................................... 38 4 Recommendations............................................................................................................................................................. 38 Bibliography.................................................................................................................................................................................. 42 Appendix A: Statement of Work For The GEMA Pe & Ie............................................................................................... 45 Appendix B: Type and Status of GEMA Performance Indicators (End Sept. 2018).................................................... 47 Appendix C: GEMA’s Results Framework ........................................................................................................................... 52 Appendix D: Maps...................................................................................................................................................................... 53 Appendix E: Methodology Supplement................................................................................................................................. 60 Monitoring and Evaluation Support for Collaborative Learning and Adapting iii ACRONYMS ADEC NGO Agua y Desarrollo AHAC Agencia Hondureña de Aeronáutica Civil AI Areas of Influence BOD Biological Oxygen Demand CCA Climate Change Adaptation CDCS Country Development Cooperation Strategy CE Conservation Enterprises CENAOS National Center of Atmospheric, Oceanographic and Seismic Studies CIAT International Center for Tropical Agriculture CICOH Climate Information System for Western Honduras (Centro de Información Climática en el Occidente de Honduras) COD Chemical Oxygen Demand CODEL Local Emergency Committee CODEM Municipal Emergency Committee COPECO Permanent Emergency Commission CUROC Centro Universitario Regional del Occidente DAI Development Alternatives Incorporated DO Development Objective DRM Disaster Risk Management ENEE Empresa Nacional de Energía Eléctrica EQ Evaluation Question ESA European Space Agency FGD Focus Group Discussion GEMA Gobernanza en Ecosistemas, Medios de Vida y Agua Activity GIS Geographical Information System GOAL Irish non-governmental organization GoH Government of Honduras HH Household HHS Household Survey i-APS International Advisory, Products and Systems ICF National Institute for Forestry, Protected Areas and Wildlife IE Impact Evaluation IR Intermediate Result JAA or JA Junta Administradora de Agua (Water Administrative Boards) KI Key Informant KII Key Informant Interview MAMCEPAZ Mancomunidades of the Central Area of La Paz MCH Meteorology, Climatology and Hydrology Information System MCSE Mecanismo de Compensación para Servicios Ecosistémicos (Ecosystem Services Compensation Mechanism) MESCLA Monitoring & Evaluation Support for Collaborative Learning and Adapting MiAMBIENTE Ministry of Energy, Natural Resources, Environment and Mines MIPYME Micro-Small and Medium Enterprises NGO Non-Governmental Organization NRM Natural Resource Management Monitoring and Evaluation Support for Collaborative Learning and Adapting iv PA Protected Area PE Performance Evaluation PI Performance Indicator PIRS Performance Indicator Reference Sheet PMCCA Programa de Monitoreo de Calidad y Cantidad de Agua (Program for Monitoring the Quality and Quantity of Water) PPP Public-Private Partnership RE Renewable Energy RF Results Framework RNHM National Meteorological Network SINAPH Hondurean National System of Protected Areas (Sistema Nacional de Áreas Protegidas de Honduras) SW Sampling Weights TA Technical Assistance TAP Technical Assistance Package ToC Theory of Change UMA Municipal Environmental Units UNAH Universidad Nacional Autónoma de Honduras USAID United States Agency for International Development USG United States Government WA Work Areas WH Western Honduras Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1 EXECUTIVE SUMMARY Building on the results of the prior USAID ProParque Activity, the Gobernanza en Ecosistemas, Medios de Vida y Agua (GEMA) Activity is a 4-year project targeting Western Honduras to improve natural resource management (NRM) approaches. The Activity focuses upon priority watersheds that are linked to the Honduran National System of Protected Areas (SINAPH). GEMA builds capabilities, promotes sustainable practices, develops opportunities for inclusive, environmentally-sustainable economic growth and supports a climate-informed and resilient civil society. Implemented by DAI Global, LLC, GEMA began in September 2016 and is due to end in December 2020. GEMA works via service providers and Governance Anchor Institutions1 to implement Technical Assistance Packages (TAPs) in 54 Priority Micro-Watersheds2 and 586 communities within 15 Work Areas(WA)3 . GEMA’s results statements are: (i) Reduce threats in areas of biological importance and/or critical ecosystems to preserve biodiversity and protect water supplies; (ii) Increase income-generating actions related to conservation; and (iii) Increase the capacity of vulnerable populations to adapt to climate change and variability. GEMA is expected to effect change in 11 Areas of Influence (AI)4 covering a total area of 614,600 hectares in Western Honduras. PURPOSE OF THE EVALUATION USAID commissioned MESCLA to implement a Performance Evaluation (PE) to: • establish a baseline against which to measure GEMA’s achievements when it ends in 2020; • set out findings and conclusions as the basis for recommendations to improve GEMA’s efficiency and effectiveness during the remaining implementation timeframe; and • provide evidence for the GEMA Impact Evaluation (IE), which is implemented concurrently. METHODOLOGY MESCLA subcontractor, International Advisory Products and Services (i-APS) undertook the PE from June to December 2018. The team, composed of local and international specialists, applied a range of techniques, including document review, a Household Survey (HHS), field observation, Key Informant Interviews (KII), and Focus Group Discussions (FGD) to gather the evidence required to inform findings and conclusions. The team used analytical techniques to collate and interpret the data collected. When feasible, the team triangulated sources for each of the findings and then drew conclusions and formed recommendations. 1 Governance Anchors Institutions are the primary institutions and organizations that play a key role in the governance and socio-economic development within the intervention areas, such as municipalities, JAAs, local and municipal emergency committees, co-managers of public protected areas, producer associations and cooperatives, local implementing partners, business service providers and micro and small business. 2 Priority Micro-watersheds are selected because they represent the greatest opportunity for an integrated intervention of all the GEMA’s results. 3 Work Areas are the territorial framework where GEMA concentrating investments to produce a greater effect for reducing the threats to the ecosystems, the improvement of alternative sources of income and the strengthening of the communities’ watershed management capacity 4 AI are geographical framework to assess threats to and dynamics of the biodiversity and the ecosystem services. The definition of these areas follows the criteria of landscape connectivity, biodiversity spots, water production areas, concentration of key actors for water governance and the existence of other interventions. Method Source HHS GEMA beneficiaries in 12 treatment watersheds KII GEMA staff, service and partner organizations, participants in Water Administrative Boards (JAAs), Local and Municipal Emergency Committees (CODELs and CODEMs), conservation enterprise (CE) Micro-Small and Medium Enterprises (MIPYMEs) owners FGD Document review GEMA activity documents Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 2 PERFORMANCE EVALUATION QUESTIONS (EQS) The evaluation is structured around 8 evaluation questions, focusing on areas that include: 1) Conservation Enterprises, 2) Resilience, 3) Gender, 4) GEMA resources, 5) Integration, 6) Sustainability and Governance, 7) Constraints and Opportunities, and 8) Water Quality. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS CONSERVATION ENTERPRISES Findings: • CEs and associated households (HH) have benefited from GEMA support to clean production methods and renewable energy (RE) systems. The support resulted in financial benefits to HH. • GEMA’s assistance to agricultural enterprises are beginning to increase yields and quality of crops within existing production areas and this is likely to contribute to conserving forest areas and their biodiversity in the medium-term. • Benefits to HH have resulted from improved links between agricultural enterprises and markets. Limitations include the fact that some of the non-agricultural enterprises GEMA assists are based in locations too distant from protected areas and improvements in their practices could not significantly influence natural resource conservation and the fact that more efficient equipment will only marginally reduce deforestation or forest degradation. Conclusions: Seventy-five percent of GEMA’s support is directed toward coffee CEs that are likely to have the most impact on biodiversity and employment/income in the WAs. Improved production practices and adoption of RE by GEMA-supported agricultural CEs are likely to benefit biodiversity within GEMA’s AI, but data are lacking to confirm these benefits. The impact on HH that are engaged in non-agricultural CEs mostly occurs outside of GEMA’s WAs, and therefore the support is contributing only marginally to conserving biodiversity. Recommendations: R1. Focus GEMA’s assistance on agricultural and non-agricultural CEs located around its target micro-watersheds. R2. Strategically focus the RE TAP on provision of RE systems to HHs geographically located near Protected Areas core zones, and within communities that receive water from target micro-watersheds. R3. Prioritize assistance to agricultural enterprises within and around its target micro-watersheds. R4. With service providers, assess the profitability and sustainability of both agricultural CES and non-agricultural MIPYMEs. RESILIENCE Findings: GEMA’s actions to reduce threats to ecosystems, diversify livelihoods and improve disaster risk management (DRM) capacities are contributing to the target population’s resilience. • In 25 communities where GEMA has supported CODELs, GEMA has improved the community awareness of the importance of resilience and the benefits of DRM and Climate Change Adaptation (CCA) tools and plans, but the level of baseline capabilities, and the weak role of municipal actors are significant challenges for effective GEMA support. (GEMA strategic area 1) • 53,263 ha. of forest and agroforestry land cover have been maintained under stable status and are estimated to be effective over a total land area of 112,182 ha. (GEMA strategic area 2) • Supporting the establishment of the Climate Information System for Western Honduras and progress has been made at national level, but the system is not still rolled out to local level. (GEMA strategic area 3) Conclusions: GEMA has increased resilience by addressing core concerns of its target population. GEMA’s assistance to agricultural CEs, especially coffee producers, is influencing farmers to improve on-farm NRM which is beginning to support the resilience of farmers’ livelihoods, and these actions show considerable potential for linkages Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 3 to the Ecosystem Services Compensation Mechanism (MCSE) schemes that the activity has fostered. GEMA has selected CODELs for upcoming capacity building but needs to speed up their promised provision of needed equipment. GEMA’s actions to establish the CICOH and increase sharing of meteorological data have the potential to deliver early warnings alerts, and once established locally, will permit organizations to collaborate effectively to increase the resilience of its target population. Overall, on the utilization of climate information and implementing risk-reducing actions, GEMA is progressing well. Recommendations: R5. GEMA should ensure that training support to CODELs is matched with rapid follow-up for development of resilience-building plans and the physical equipping of CODELs with DRM response assets. R6. GEMA should ensure technical coherence in training content between information about current short-term risks from known hazards and the slower-onset, longer-term risks, stemming from climate change impacts. R7. GEMA should develop actions that specifically support CODELs in their interactions with key local and national institutions so as to support the improved local governance and resilience of DRR. R8. Appraise the capabilities of Centro Universitario Regional del Occidente (CUROC) to manage CICOH and investigate the opportunity to provide more training in the use of Meteorological Climatology and Hydrological Information System. GENDER Findings: By the end of Year 2: • Women made up 44% of 3,045 participants in agricultural CEs and HH beneficiaries of RE technologies • 782 individuals (43% women) gained new income in non-traditional, nonagricultural or niche CEs • Women have strongly participated in local-scale DRM, and their specific vulnerabilities are becoming more recognized GEMA training (including gender-aware training and technical assistance plans) has contributed to: • An increase in women’s participation in JAAs and community water assemblies • Economic welfare, with 71% of employed women receiving salaries as their economic compensation in agricultural enterprises (Year2 Annual Report, KIs) • Improved health, through the adoption of RE technologies Limitations to women’s participation: 1) In local land ownership customs in relation to JAA roles and 2) As decision￾makers in new Ecosystem Services Compensation Mechanisms (MCSEs) schemes. Conclusions: Overall, across key indicators, an average of approximately 47% of GEMA’s participants are women. GEMA’s actions are appropriate to increasing the participation of women in water governance, thereby contributing to the economic empowerment of women, especially in non-agricultural CEs. Still, an increase in women’s participation in decision-making and leadership roles is not yet apparent. Recommendations: R9. Generate updated training content focused on women’s roles in agricultural and non-agricultural CEs. R10. Increase the scope and quality of women’s participation in both JAAs and CODELs. R11. Engage service providers staff in updated gender training, to enable these organizations to apply effective practices that foster high-quality gender participation in their contracted activities. GEMA RESOURCES: Findings: GEMA is two years into implementation with many activities still at an early stage. GEMA has suffered delays in a range of aspects (e.g. defining WAs and planning challenges) due to a range of constrains (e.g. complexity of Activity). GEMA is on target to meet 9 of its Performance Indicators (PIs) (45%) and on about half of the PIs of Results 2 and 3. However, 11 PIs (55%) are currently behind schedule and Result 1, focusing on improving bio￾diversity, is the area most in need of extra time. FGDs and KIIs with community respondents in 12 watersheds indicated they welcome closer ties with GEMA staff and service providers. JAAs want more in-depth support to Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 4 move forward the legal process for official declarations of water recharge zones, including improved linkages with UMA. Conclusions: Consulted communities state that they would like GEMA to provide a longer total duration of its support and to provide greater volume and quality in GEMA communication about the Activity’s approach, plans, and participation opportunities. This will require increased resources or a more focused geographical intervention. GEMA’s outcomes (especially for Result 1) are unlikely to be achieved on time. Feedback from beneficiaries suggests that GEMA interventions need more intense follow-up and technical assistance that may require more resources, therefore, reallocation of resources towards key priorities needs to be assessed by GEMA and USAID Honduras. Some targets may be too ambitious such as delivery of targets in Results Area 2 due to the hard-to-resolve weaknesses in the institutional capabilities of JAAs CODELs and local government counterparts. Recommendations: R12. With USAID, GEMA should agree priorities for remaining time period, using findings from this evaluation and other USAID inputs, to refocus efforts and apply finance to those priorities and adjust GEMA indicators accordingly. INTEGRATION Findings: It is clear from KIIs and field observation that GEMA’s individual TAPs are undertaken in areas where they have the highest degree of potential benefit, rather than integrated in a holistic manner with other TAPs that would provide a menu of options to target communities. The Theory of Change (ToC) does not successfully articulate how GEMA’s three different results statements should be integrated. Through the course of the PE it became increasingly clear that the issue of successful water governance was a common factor influencing the success of GEMA’s results statements. There are also existing tools and data, such as The Honduran Water Platform, Terra￾i and the Monitoring the Quality and Quantity of Water (PMCCA) that could be utilized to successfully integrate the water governance focus. Conclusions: GEMA’s TAPs have not been integrated with each other but rather undertaken to achieve targets and maximum benefit in the individual results areas. Governance over water resources could serve to integrate GEMA’s actions but would require a modified ToC that prioritized core watersheds, as the basis for an updated Results Framework (RF). Recommendations: R13. Integrate GEMA’s actions, focusing on more limited land-area units such as targeted watersheds. Around these, achieve effective CEs, DRR, NRM, MCSE schemes; and improved water governance to ensure long-term access to water resources R14. Closely integrate GEMA’s gender specialists to act as advisors to the technical results area teams to address gender equity in all the activity’s work. SUSTAINABILITY AND GOVERNANCE Findings: GEMA’s core strategy for achieving environmental governance involves strengthening the capacity of JAAs to govern their water resources. The PE found GEMA’s effort to create alliances to establish accountable local institutions, such as JAAs, was the “foundation of effective governance”. This was underpinned by the feedback from communities which displayed high levels of engagement in water governance. In fact, 75% of the HHS respondents had participated in activities to protect the sources of their water and 92% of respondents thought the community bears the responsibility to care for their water sources. GEMAs support of linkages to international coffee markets that require eco-certification is key to conserving water resources and ensure long term markets for coffee. While there is evidence of the application of improved business practices by CEs as a result of GEMA’s activities, there are insufficient data to be able to confirm the longer-term benefits for their sustainability, especially regarding non-agricultural CEs that are not linked to wider market value chains. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 5 Conclusions: Ensuring water quantity and quality is the key measure to achieve sustainable adaptation to climate change. With firm support from communities, GEMA has a significant opportunity to foster effective water governance, which will have a long-term and sustainable impact upon regional biodiversity. GEMA’s support to agricultural CE’s, specifically working with the coffee sector to access international markets, has increased sustainability, especially as alliances with public-private partnerships deepen. Recommendations: R15. Increase the effectiveness of MCSE schemes in GEMA’s governance strategy by providing structured support to local stakeholders to ensure effective implementation. R16. Ensure the adoption and application of Irish NGO GOAL’s governance measurement tools by key stakeholders. R17. Continue to strengthen coffee production to international eco-standards and links to international coffee markets. CONSTRAINTS & OPPORTUNITIES Findings: At project conclusion in 2020, the PE suggests that GEMA will meet its quantitative performance indicator targets; however, it may not yet have achieved the full set of expected outcomes and the prospects for the sustainability of the resulting impacts is likely to still be uncertain. The PE also outlines how the protection and governance of water resources is central for protecting water, soil, and vegetation over a geographically-significant area, as well as, contribute to both conserving biodiversity and reducing poverty in Western Honduras. GEMA must demonstrate a measurable array of outcomes for support for management of natural resources and protection of biodiversity. The level of understanding and willingness of the local populations to engage in watershed governance is a clear opportunity to foster a conducive environment to deliver sustainable watershed management. Conclusions: The GEMA is constrained by the available time remaining as activities start to close in mid-2020. It is unlikely that activities aiming at enhancing governance will have the opportunity to begin producing meaningful impact in the areas of watershed management or DRM and CCA. Overall recommendation is that GEMA re-focus its activities to achieve measurable, sustainable, effective governance of water resources. If GEMA achieves this outcome, it will contribute substantially to both conserving biodiversity and reducing poverty in Western Honduras. Recommendations: R18. USAID should examine the feasibility of an extension to the GEMA Activity to enable activities to yield significant outcomes. R19. USAID should begin post-GEMA planning to ensure that an effective Sustainability and Exit Strategy is in place at least 12 months before GEMA’s confirmed close-down date. WATER QUALITY Findings: GEMA’s sub-contract with the University of Zamorano to implement the Program for Monitoring the Quality and Quantity of Water (PMCCA) is designed to strengthen collaboration and has begun developing useful approaches and tools for water quality monitoring; however, it does not measure the effect of TAPs’ effects on water quality within its targeted micro-watersheds. JAA (main target) participation has been limited and water samples are collected in a diffuse manner. The PMCCA plans to analyze waste water from coffee processing. GEMA intends for the PMCCA stakeholders to strengthen their return of water quality laboratory results to JAAs and to increase institutional collaboration for support of the JAAs, especially through stronger linkages with the Public Health Ministry and ICF. The Honduras Water Platform (USAID-funded) hosts geographical information on delimited watersheds that has potential to combine information outputs from the PMCCA and watershed modeling. Conclusions: GEMA has widely engaged key water quality stakeholders and this is beginning to influence decision￾making and governance through improved access to water quality information. Informants proposed that greater attention could then be given to water treatment for the control of pathogens, financed by local contributions by water consumers. The sustainability of the PMCCA requires greater support if JAAs are to persuade water users Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 6 to finance water quality sampling and analyses. GEMA actors suggest that actions could be improved in four ways: (i) include data about the biophysical characteristics of land uses above the water intakes; (ii) ensure sample collection methods adhere to a quality protocols and that training is provided; (iii) apply modeling of hydrological responses to declared micro-watersheds and use the Honduras Water Platform to host information; and (iv) pilot the use of simple water testing kits to be used by mature JAAs to monitor their water supplies. Recommendations: R20. Focus service provision on a coherent and comprehensive suite of actions that enable JAAs to increase not only their administrative capabilities and legal status but also ensure that GEMA’s actions in the field deliver integrated support for a range of planning, and concrete and practical outcomes. R21. Enhance the impact of PMCCA actions by undertaking the technical improvements mentioned above. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 7 1 INTRODUCTION 1.1 GEMA BACKGROUND Across the mountainous terrain of Western Honduras, a complex set of forest ecosystems support a diverse array of plant and animal species; however, these forests and their soils are under increasing human pressure. These landscapes supply vital services, upon which populations depend on for human health and economic welfare. To counter threats to the socio-economic and environmental fabric in Honduras, USAID/Honduras has established a Country Development Cooperation Strategy (CDCS) with the goal to enhance “a more prosperous and safer Honduras that advances inclusive social and economic development among vulnerable populations.” The CDCS Development Objective 2 (DO2) is to reduce the poverty of vulnerable populations in six departments in Western Honduras. The DO2’s development hypothesis is: “if natural resources and biodiversity are protected and enhanced, resilience of livelihoods to climatic and economic shocks is strengthened (IR 2.1), families can increase their incomes (IR 2.2), and human capital is improved by focusing on improving education and health for these communities (IR 2.3), then poverty in Western Honduras will be sustainably reduced.” The development hypotheses for IR 2.1’s Sub-IR’s are: “If Honduras’ management of ecosystems in Western Honduras is sustainably improved (Sub-IR 2.1.1) and climate change adaptation actions are implemented (Sub-IR 2.1.2), then the poorest households who depend on the natural resources provided by ecosystems in Western Honduras will have more resilient livelihoods.” To contribute to the achievement of DO2, USAID/Honduras has financed the GEMA. GEMA is a four-year project targeting Western Honduras to improve natural conservation policies and practices; develop opportunities for inclusive, environmentally-sustainable economic growth; and promote a more climate-informed and resilient civil society. Building on the accomplishments of the previous USAID-funded ProParque activity, GEMA has a projected investment of $23.8 million, and is implemented by DAI Global, LLC, with partners including GOAL, and the University of Zamorano. The Activity began in September 2016 and is scheduled to end December 2020. GEMA works via service providers and Governance Anchor Institutions to implement TAPs in 54 Priority Micro￾Watersheds and 586 communities within 12 Work Areas (WAs) and AI covering 177,127 hectares. GEMA combines a focus on protected areas identified in partnership with Government of Honduras (GoH). These include critical watersheds and a mosaic of other land-use categories; sustainable economic growth actions; climate change information and adaptation strategies; and conservation initiatives, with the aim of reducing poverty, while contributing to NRM. The project targets the economically-disadvantaged through livelihoods development and access to water. GEMA’s 3 results statements are: Result 1: Threats in areas of biological significance and/or natural resources will be reduced, through renewable energy adoption, enhanced water governance, and implementation of compensation for ecosystem services schemes; Result 2: Conservation-related, income-generating actions will increase; and Result 3: Vulnerable population’s capacity to adapt to climate change will be increased, through building of disaster response capacity and climate resilience. GEMA’s integrated strategy focuses on gender equality, organizational strengthening and a project management approach that aims to be both adaptive and proactive, adjusting in response to information and changes in context, ensuring learning. Institutional capacity development is fostered through competitive processes, whereby grants and contracts are distributed to service providers to increase their effectiveness, and to promote local management. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 8 1.2 GEMA PERFORMANCE EVALUATION 1.2.1 PURPOSE AND AUDIENCE The purpose of this performance evaluation is to assess whether GEMA is on target to meet its outcomes and results and to establish how effectively the program is delivering interventions and impacting beneficiaries. The results of this evaluation will be used by USAID to inform any necessary changes to improve GEMA’s implementation, so mid-course adjustments for the remaining life of the activity can be made as needed. The final performance evaluation of GEMA should determine whether the findings, recommendations, and lessons learned from the performance evaluation were considered during the remainder of the implementation period, and if so, what the impact was of any changes in program implementation. In May 2018, Dexis Consulting Group, through the MESCLA Activity, contracted i-APS to conduct a performance evaluation (PE) and impact evaluation (IE) of the GEMA project (refer to Appendix A: Statement of Work). The audience for the PE is, the USAID/Honduras Mission, DAI and implementing partners, USAID/Washington bureaus and offices, including the Bureau for Latin America and the Caribbean, the Office of Forestry and Biodiversity, the Bureau for Economic Growth, Education, and Environment; and stakeholders in Honduras, including GEMA partners, and national and regional government ministries and offices. 1.2.2 EVALUATION QUESTIONS USAID, MESCLA, GEMA and i-APS started off by refining the EQs in i-APS’ Scope of Work (Appendix 1: SOW). The questions were written so as to reflect GEMA’s changing nature. The PE seeks to answer the EQs outlined in Figure 1 below. Figure 1: Performance EQs CEs: 1A. Are conservation enterprises GEMA has supported generating benefits to households in GEMA area of influence?; 1B. If so, what are the types and magnitude of benefits to households? IC. If so, what are the types and magnitude of benefits to biodiversity?; ID. How sustainable from a business perspective are these enterprises and their practices? IE. Why have conservation enterprises been able to generate those benefits to households? 1F Why have conservation enterprises generated those benefits for biodiversity? 2. 1. Resilience: 2A. Have GEMA’s actions to reduce threats to ecosystems, diversify livelihoods and improve capacity for risk mitigation and disaster response contributed to building the resilience of its target population? 2B. If so, what is the magnitude of its contribution, and why has it been able to make this contribution? 3. Gender: 3A. Has GEMA successfully increased the participation of women in decision-making positions in water governance and MCSE schemes? 3B. If so, how and to what extent? 3C. Has GEMA successfully increased the economic empowerment of its target women population? 3D. If so, how and to what extent? 3E. Have GEMA actions strengthened the participation of women in disaster relief management? 3F. If so, how, and to what extent? 4. GEMA Resources: 4A. Which of GEMA’s actions need more time, technical assistance (TA) training or funding to produce their target outcomes? 4B. What additional resources, if any, would be required produce the target outcomes? 5. Integration: Have GEMA’s different actions been integrated effectively with each other? 6. Sustainability and Governance: 6A. Has GEMA produced sustainable environmental governance, watershed management, conservation enterprises and adaptation to climate change? 6B. If so, to what extent and how? 7. Constraints and Opportunities: Based on its performance indicators, what constraints and opportunities have affected GEMA’s progress towards achieving its planned outcomes? 8. Water Quality: 8A. How is GEMA measuring the effect of GEMA’s TAPs on water quality within its targeted micro￾watersheds? 8B. How has the information gathered, and support provided, influenced decision-making and improved the management of governance and water quality data systems? 8C. Could the PMCCA actions be improved to yield more useful, higher quality data, and to strengthen local capacities more effectively? GEMA Performance Evaluation Timeline June July August September October November December 11 18 25 2 9 16 23 30 6 13 20 27 3 10 17 24 1 8 15 22 29 5 12 19 26 3 10 17 24 31 1 Inception 2 Reconnaissance & Preparation 3 Data Collection 4 Validation & Reporting Kick off Meeting Team Engagement Document Review Field Initiation Meetings Evaluation Design Drafted Evaluation Design Submitted Design Review Sample Site Reconnaissance Tool Development Survey Team Training Tools Refined Tools Complete HH Survey Data Collection Beneficiary KII & FGDs Specialist KII & FGDs Data Collection Complete GEMA Validation Workshop Feedback Incorporated Presentation to USAID Evaluation Draft Submitted Final Report submitted ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 9 1.2.3 EVALUATION TEAM AND SCHEDULE To conduct the GEMA performance evaluation, i-APS assembled a team of knowledgeable and skilled local and international social and environment experts, who are well-versed with the latest conditions in Western Honduras. See Figure 2 for the team composition. The PE was completed between June and December 2018. Figure 3: GEMA PE High-Level Schedule 2 METHODOLOGY 2.1 OVERALL METHODOLOGY The evaluation methodology was designed to answer the EQs using a mixed-methods, gender-oriented approach that comprised: (i) extensive document review; (ii) field observation; (iii) beneficiary HHS across 12 sample watersheds; (iv) beneficiary KIIs and FGDs; and (iv) KIIs with GEMA Staff, GEMA partners and local institutions. The team started off the evaluation by a comprehensive inception phase, including field reconnaissance and document review to inform the development of an evaluation matrix, which in turn guided the overall approach used to address the evaluation questions. During the inception phase, the team developed tools for the HHS, as well as KII and FGD interview guides. The team triangulated quantitative and qualitative data collected through KIIs with GEMA and partner staff, and through direct field observation. See Appendix E for an outline of the PE Methodology. Figure 2: Evaluation Team Members Team Members Position / Area of expertise Bruce Kernan Team Leader Peter Schlesinger Remote Sensing Jose Ney Rios Watershed Management Aroldo Santos Forestry & Renewable Energy Montserrat Blanco Gender Sandra Buitrago Resilience and climate adaptation Manuel Martinez Enterprises & Governance Ted Tuthill i-APS: evaluation operations management Mark Kowal i-APS: technical support & quality control Performance Evaluation Independent, constructively-critical and robust findings concerning GEMA’s effectiveness Performance Evaluation Sectors Constraints & Opportunities (EQ5) Resources (EQ7) Integration (EQ4) Conservation Enterprises (EQ1) Resilience (EQ2) Gender (EQ3) Water Quality (EQ8) Participatory Data Collection Household Survey 427 GEMA Beneficiaries 20 Focus Group Discussions 147 men, 79 women Beneficiaries JAA Members CODEL/CODEM Members 294 Key Informant Interviews 183 GEMA Beneficiaries 111 GEMA Staff GEMA Partners Local Institutions NGOs Field Observation 12 Sample Locations Waterpoints Community enterprises Document Review Area conservation plans Annual Reports Quarterly Reports Workplans Sample: 12 Level 1 Microwatersheds within GEMA work areas with concentration of GEMA activities Governance & Sustainability (EQ6) Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 10 For each evaluation question, the Team gathered evidence and findings, that were used as the basis for conclusions and recommendations to increase the effectiveness of GEMA’s TAP. Figure 4 below sets out the overarching methodology used for the PE. 2.2 DATA COLLECTION Preparation and implementation of the HHS: In consultation with USAID/Honduras and MESCLA staff, the team designed and implemented an HHS to gather data on GEMA beneficiaries, within 12 watersheds and surrounding areas. Systematic sampling was used to select the sample household within the populations of GEMA beneficiaries linked to each Level 1 sample micro watershed. Respondents comprised 70% men and 30% women. Document Review: The team conducted a systematic review of relevant data. This included literature and program documents from GEMA such as Quarterly and Annual Reports, service provider databases, Work Plans, technical reports, Requests for Applications, and Requests for Proposals. The document review was conducted to assess implementation progress towards GEMA’s project objectives and outcomes and is listed in the bibliography. Key Information Interviews (KIIs): KIIs were held with two categories of key informants: Figure 4: Overarching methodology used for the PE Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 11 (A) administrators, staff, professionals and technicians associated with GEMA, implementing partners, and national and local institutions; and (B) beneficiaries of GEMA’s field activities, including local leaders of Patronatos, conservation enterprise owners, and members of CODELs and JAAs. These interviews were critical to help the PE team understand the attitudes and impressions of GEMA’s processes and performance. Focus group discussions (FGDs): the team held FGDs with GEMA beneficiaries and administrators of Water Administrative Board (JAAs), CODEL/ CODEMs CEs and (MIPYMEs). FGDs were conducted using a standardized guide for each type of FGD. These were conducted both within 12 sample treatment watersheds, and in other GEMA locations. The aim was to explore issues and to enable qualitative analysis of GEMA’s reach, successes and challenges. Field Observations: The team conducted site visits and field observations in the 12 sample Level-1 watersheds and associated communities. The visits combined the aforementioned KII with local businesses to better understand their operating processes and their potential sustainability and direct observation. Direct observation included: • Data related to the membership and structure of the JAA and CODELs • information on key value chains • Both agricultural and non-agricultural CEs 2.3 DATA ANALYSIS The evaluation team used both quantitative and qualitative analytical techniques to code, collate, and interpret data captured through interviews, discussion groups, and direct field observations. Each evaluation team sectorial expert analyzed streams of data derived from data collected in document review , the household survey, KII and FGDs. Data were disaggregated by variables as watershed location and sex. Sample factors were used to weight the survey data. The team prepared graphical representations of the analysis results and an overall HHS report was finalized, available separately. For qualitative KII and FGD information, the evaluation team used response themes and relational content analysis via Excel spreadsheets to identify common themes of response categories and patterns, as well as to elucidate emergent themes, contextual factors, and trends. Summary reports were prepared of the data collection by watershed and of key findings by themes, and these were disaggregated by sex of participants. Data were compared from the HHS data, document review, KIIs, FGDs, and field observations against one another to determine whether Figure 5A: Engagement with institutional-level Key informants INSTITUTIONAL RESPONDENTS NUMBER OF KIIs MEN WOMEN GEMA STAFF 17 14 GEMA SERVICE PROVIDERS & PARTNERS 12 9 PUBLIC INSTITUTIONS 16 6 TOTAL INSTITUTIONAL-LEVEL KIIs 74 Figure 5B: Engagement with beneficiary-level Key informants BENEFICIARY RESPONDENTS NUMBER OF KIIS MEN WOMEN JAA 12 10 CODEL 10 8 Non-Agricultural Enterprises 3 8 Agricultural Enterprises 9 6 Range of KIs from 12 Treatment Watersheds 64 3 TOTAL BENEFICIARY-LEVEL KIIs 133 Fig 5C: Engaged beneficiary-level Focus Group Informants NUMBER OF FGDS BENEFICIARIES & LOCATIONS MEN WOMEN CODELs 23 18 12 Treatment Watersheds 124 61 TOTAL BENEFICIARIES FGDs 147 79 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 12 findings were divergent or convergent. When multiple informational streams provided consistent information, these were included as findings. The team drew conclusions by reflecting on the key findings for each EQ. Analysis accounted for gender and social dimensions, when relevant. 2.4 LIMITATIONS This PE was undertaken at the mid-point of program implementation, resulting in limited findings on the impact of GEMA activities. As some GEMA activities are in the early stages of implementation or limited to certain watersheds, it has not been possible to produce conclusions on their impact. This is to be expected at GEMA’s mid-point but does limit the PE’s ability to fully address all of the EQs. Data are lacking about GEMA’s work with irrigation groups. When the PE was planned, GEMA’s irrigation component appeared to be such a small part of GEMA’s activities that it appeared to be unnecessary to collect related data. GEMA’s Y2 Annual Report indicates only 10 out of the 450 watersheds the Activity is working with provide water for irrigation (rather than for household use), thus the PE did not look at use of water for irrigation and irrigation water boards. While a reasonable decision at the time of this performance evaluation, in future assessments management of water resources for irrigation as well as consumption must be included. Substantial challenges were encountered in collecting data on both the fuelwood saving and the health impacts of eco-stoves. The HHS has collected data regarding fuelwood collection and usage and the perception of health benefits as a result of adoption of eco-stoves. However, at this stage in 2018 it was not possible through the HHS data to accurately attribute health benefits and the impact upon deforestation of the eco-stoves, as no prior baseline was established by GEMA of fuelwood use by households (HHs) and CEs, or on technologies, respiratory illness incidence, and the volumes used, prices and sources of fuelwood. Field-level data collection limitations. Some of the challenges when accessing the beneficiary sample and engaging with survey, Key Informants (KIs) and FGD informants were: (i) informants were at time less aware than might be expected of the support given through GEMA to JAAs and other organizations; (ii) as many of GEMA’s activities are still at a relatively-early development stage, this led to a greater level of knowledge of GEMA, compared with the proportion of informants stating that they actively participated in, or benefited, from GEMA actions; (iii) absence of the beneficiary, especially of coffee producers, due to the time of approach, length of their working day, and the nature of their employment; (iv) imprecise names of GEMA beneficiaries and their GPS locations, leading to difficulty locating the microentrepreneur; and (v) informants no longer resided where they had been registered. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 13 3 FINDINGS AND CONCLUSIONS 3.1 CONSERVATION ENTERPRISES (CE) 3.1.1 FINDINGS GEMA provides different kinds of support to promote improved production practices to conserve natural resources and biodiversity. GEMA supports conservation enterprises (CE) that adopt renewable energy (RE) systems, as well as CEs that engage in public-private partnership for more eco-friendly production of coffee, and those supporting livelihood diversification efforts including non-agricultural enterprises and the development of niche agricultural products and livestock. GEMA does this through a contract with Centers for Enterprise Development who support non-agricultural conservation enterprises and by supporting Non-Governmental Organizations (NGOs) and other service providers to promote diversification of agricultural livelihoods and more efficient coffee production. CEs and associated households have benefited from GEMA support to clean production methods and renewable energy systems. GEMA has provided direct support to three CDEs to build their capabilities through diagnosis of their weaknesses, and subsequent training and systems development, which not only contributes to their sustainability, but has led to similar CDE contracts (KIIs GEMA and CDEs, Annual Report,) funded by the EU, UN Development Program, and UN Women. As the CDEs are public-private institutions and GoH’s policy is to support them, these enhanced capacities are likely to enable continued assistance to CEs after GEMA ends. As of the end of Year 2, GEMA had supported both CEs and individual households by financing the installation of 361 photovoltaic systems and 395 solar driers for seed producers of coffee, corn and beans. One cattle biodigester had been installed in Copan (GEMA KIIs). In total, GEMA’s Year 2 Report indicates that 8,998 people were provided with better access to renewable energy, either through low-cost photovoltaic (PV) systems, or fuel-efficient stoves. Commitments for PV systems for Year 3 totaled 2,800 new systems with 12 municipalities. More efficient wood-burning cook stoves (for homes) and ovens (for CEs) that GEMA finances are beginning to produce benefits to households. Through September 30, 2018 GEMA had financed the installation of 1,570 improved cook stoves. Letters of Understanding between GEMA and Municipality (GEMA, 2018) indicate a total of 1,931 households with stoves installed or committed to be installed. HHS data showed that eco-stoves and ovens benefit households in several ways. Respondents were asked about the reasons for changing to efficient stoves to cook their food and identified as key benefits: reducing the consumption of firewood (56%), reducing smoke (51%) and time saving (23%) and money saving (28%) in firewood collection and purchase. EQ 1A: Are conservation enterprises GEMA has supported generating benefits to households in GEMA’s Areas of influence? EQ 1B: If so, what are the types and magnitude of benefits to households? IE. Why have conservation enterprises been able to generate those benefits to households? 14 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) Through the three CDEs, GEMA has supported over 500 individual or enterprise level environmental mitigation plan, cleaner production, and business training, including administration, accounting, packaging and marketing. Forty-four percent of the respondents in the HHS said that with GEMA assistance they had changed their production practices to become more efficient. As shown in Figure 6, HHS data indicates that GEMA has assisted 25% of those respondents to reduce their water consumption during non-agricultural production processes, and 22% to reduce their use of materials, contributing to the households associated with these CEs reducing the quantity or costs of energy or materials. Additionally, 250 handicraft CEs are supported with PV systems in Intibucá; 220 basic grains producers around Lake Yojoa with seed dryers; and 80 coffee farmers around Ocotepeque are also using solar dryers. Sixty Lake Yojoa CE restaurants had begun to treat their gray waste water and engage in biogas production, though this activity was still at validation stage (the team, supported by the National University of Forestry Sciences restaurant surveys, flow and lab analysis were completed,). GEMA Service Providers have supported 200 non-agricultural CEs with frying stoves, bread and handicraft ovens, and kilns for tile and brick production (KIIs, Annual Report Year 2). Approximately half of CE KIs said the training CDEs have delivered has improved their ability to calculate production costs accurately, to set prices, to match production to demand, and to effectively advertise and package products. CDE KIs reported that GEMA alliances and training are contributing to the CEs’ ability to tackle the procedures required to certify products and services as “special” or “organic” and to comply with government regulations, such as taxes and sanitary certificates. GEMA documents and KIIs provided evidence for: (i) Universidad Nacional Autónoma de Honduras (UNAH) certification for nature adventure tour guides; (ii) production systems of differentiated coffees (certified by quality seal; integrated with other USAID initiatives in Lempira, Intibucá and La Paz Departments); (iii) certified-varieties of cocoa plants; and (iv) for SAG-SENASA to begin actions for a sugarcane processing system certification system (KIIs, Annual Report Year 2; no further specific evidence from KIIs). GEMA has generated some financial benefits to households as result of its support to CEs. A sample CDE database shows in 2018 at baseline for 60 non-agricultural CEs, the total number of owners was 95 (of whom 63% are women), for a total of 270 employees (50% women), indicating that each CE benefits a total of 1.5 owners and 4.5 employees, with only 18% of CEs having more than 10 participants, and 60% having fewer than the average of 6 participants, with a Modal Value of 2, and a Median Value of 4.5 participants. CDE reports and KIs suggest that, while similarly detailed data from the Lempa and Santa Barbara CDEs were not available, since the types of non-agricultural products and services are similar across the three CDE portfolios, the average total participants per CE is unlikely to vary greatly between the regions. GEMA’s Annual Report for Year 2 indicates that 705 permanent jobs had been generated (of which 32% are held by women), while 767 temporary jobs were created. Service provider KIs consistently held the view that assistance benefited households by increasing employment within CEs, potentially reducing migration, and increasing resilience. Across the sample of 12 watersheds, the HHS reported that out of 72 households that had received support for improved CE practices, 12 of those respondents reported an increase in employment, while 6 in this sample reported a decrease. There is some indication that support to CDEs is enabling micro, small and incipient conservation enterprises to improve their business competencies: 50% of CEs that qualified as C and D performers at baseline had moved up to the next scale, while only 1 of the 16 B performers had moved up to an A rating. However comparable data from the other two CDE in Santa Barbara and Lempa were unavailable. In addition, it is unclear how many of the supported CEs have failed. 0% 10% 20% 30% 40% 50% Reduced Water Consumption Reduced cost of inputs Reduced use of firewood Other Figure 6. Effects perceived by MIPYMEs due to GEMA support Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 15 GEMA’s assistance to agricultural enterprises has included support to 2,500 coffee farmers through 11 public￾private partnerships delivered through simplified grants to service provider grouping involving private companies and municipal association (KIIs, Annual Report). Coffee farm CEs sizes vary considerably in size, with analysis of sample service-provider portfolios showing average areas under coffee production of 3.7 ha. in the HQC-supported Aruco/Higuito region, and 1.7 ha. in the Honducafe-supported Lake Yojoa/Gualcarque region. Overall, this support has contributed to the sale of 9,075 quintals of coffee beans in 2017-2018, at robust prices. KIIs with service providers and farmer CEs indicate that coffee-growing households are benefiting from receiving more net income, especially as they are beginning to increase their yields. However, obtaining robust data for the PE on improvements in coffee farm yields, sales and income at the level of the individual enterprise proved difficult, as KI respondents were reluctant to state these figures (KIIs with agricultural CEs, GEMA staff and CDE technicians). All the same, KIIs indicate that successful, stable coffee farms are now contributing to greater employment of workers from outside of their household. CE key informants indicate that agricultural households that grow other export and commercial crops, such as cacao and sugar cane, are beginning to benefit from the value-chain based model that GEMA uses with coffee￾farming households, but still at a very limited scale. GEMA-supported avocado producers, for example, sold approx.137,000 lbs. at an average price of Lps. 13 per lb. in Year 2 of the GEMA Activity and the gross annual sales of for the non-agricultural CE portfolios of the 3 CDEs showed a 7% increase, from $5.08 Mn to $5.43 Mn. Nonetheless, across the entire CE portfolio sales declared by 2,841 CEs in GEMA’s databases added up to approximately $15,206,300, with 64% of that total due to coffee, and 33% due to the non-agricultural enterprises, that are supported via 3 CDE contracts. Thirty-one percent of those sales correspond to micro-enterprises led by women, and 69% to those by men. Sales of other products and services such as cacao, avocado, sugar cane and livestock are still marginal, as a proportion of total CE sales, together summing to reach a value of $438,088, which accounts for less than 3% of the sales made by the entire CE beneficiary set. This illustrates both the considerable importance of non-agricultural small business livelihoods, as well as the limited initial status and prospects for significant gains, in diversifying agricultural enterprises away from reliance on coffee. Comparable FY2018 information for gross annual sales, and for sales increases, was only partially available, due to the early stage of implementation of these contracts. Some of the non-agricultural enterprises GEMA assists are based in locations too distant from protected areas for improvements in their practices to significantly influence natural resource conservation. USAID’s reasoning for providing support to CEs is that, if incomes are increased, unsustainable activities will be altered, reduced or discontinued. However, GEMA KIs explained that GEMA could identify only a few, remote non-agricultural CEs located close to protected areas. Assisting them was expensive and did not help GEMA achieve the targets of new sales and jobs. GEMA, with USAID’s agreement, shifted its geographical focus to more semi-urban areas, where CEs are abundant. GEMA KIs recognize that changed practices within these CEs may not directly contribute to conserving protected biodiversity, as per this Result’s hypothesis. Instead, GEMA encouraged CEs to adopt cleaner production practices, which can reduce negative local environmental impacts. This is confirmed by FGDs across 12 watersheds, indicating non-agricultural CEs business activities have little impact on more distant vegetation or water sources. FGDs found that water-user communities near PAs are engaged in activities such as recharge area surveillance, reforestation, fencing, and land purchase, but the only activity that non￾agricultural CEs might be able to carry out, is contributing to financing of MCSEs for watershed protection. More efficient equipment will only marginally reduce deforestation or forest degradation. GEMA’s support for eco-stove and ovens; pressure cookers; fryers; photoelectric systems; and biogas digesters is based on the reasoning that distribution to households and CEs will result in less firewood consumption which in turn will reduce deforestation within buffer zones of PAs. GEMA has not yet established a monitoring method in its RE EQ IC: If so, what are the types and magnitude of benefits to biodiversity? EQ 1F: Why have conservation enterprises generated those benefits for biodiversity? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 16 component to measure both consumption and savings of firewood that would make it possible to determine whether there has been a measurable reduction to the rates of deforestation around assisted communities. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 17 Data from the HHS, observations and KIIs confirm that most households collect their fuelwood around their homes, from pruning on-farm shade trees, or from areas of fallow land with regenerating vegetation. Fuelwood is sourced to a much lesser extent either via market purchases or from local forests. Obtaining fuelwood from these local and on-farm areas only incurs labor and collection-time costs and for this reason a PE cost/saving comparison was not feasible. In the HHS, 83% of the respondents in the HHS said that, rather than purchasing fuelwood or from local forests, they collect their firewood from on-farm locations. Insufficient data were available to provide a specific proportional distribution of the sources of firewood of eco￾stove (for households) or eco-oven (CE) beneficiaries or to provide precise data on time savings, which would require a tailored study. GEMA’s study of the brick/tile value-chain found these enterprises do purchase their fuelwood, but that document does not provide fuelwood volumes used per unit time, nor sources or species used. Documents, observations and KIIs suggest that conversion of forest to agriculture, rather than excess fuelwood consumption, largely drives deforestation, and KIIs and the literature review indicate that a robust cause-effect linkage cannot be made between reduced use of fuelwood due to stove adoption on the one hand and reduced deforestation on the other hand. GEMA’s assistance to agricultural enterprises are beginning to increase yields and quality of crops within existing production areas and this is likely to contribute to conserving forest areas and their biodiversity in the medium-term. GEMA hypothesizes that, if support helps farmers to increase profits from their existing plot areas, they have less incentive to eliminate forests within protected areas to establish more coffee. GEMA is contributing to environmental awareness through the provision of training and assistance plans. Field observations and KIls with farmers and extension agents indicated that GEMA support is leading to the adoption, especially for coffee, of more effective planting and pruning techniques, use of improved varieties, control of insects and pathogens, and application of compost. According to the Annual Report Year 2, approximately 2,439 producers have improved their technological and market practices, with the implementation of training and assistance plan through support from Service Providers, totaling 1,854 in coffee, 120 in cocoa, 151 in livestock, 150 in sugarcane and 164 in avocado producers. In addition, 160 vegetable producer CEs in Ocotepeque have started producing compost. According to GEMA KIIs, prior experiences in ProParque and other projects indicated that these practices will increase crop yields and quality, while reducing costs. PE analysis of GEMA documents, KIIs and FGD information, shows that GEMA is beginning to contribute to increased yields; improved production practices and product quality; and to net income. However, it is too early for to be able to show to what extent GEMA TAPs directed at agricultural CEs has reduced deforestation and forest degradation, either outside or within PAs. Benefits to households have resulted from improved links between agricultural enterprises and markets. Honduran specialty coffees show high demand, but production is limited and prices fluctuate dramatically. To address this, GEMA implements a value chain strategy, leveraging funds from private companies to link producers to markets, while aiding NRM. GEMA KIIs note that producing for world markets requires coffee farmers to comply with international quality and environmental standards which, when met, can lead to access to more stable and remunerative markets. Public Private Initiatives offer a solution that GEMA has fostered, and these are now beginning to link groups of farmers, private entities, and groups of municipalities (KIIs; Annual Report Year 2), however KIIs with coffee producers were unable to confirm this. In addition, coffee exporters are increasingly concerned about the welfare of growers and their compliance with social standards, which require practices that Figure 7 Firewood Collection, Intibucá Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 18 benefit employees through improved work conditions (GEMA KIIs). KIIs indicate that 39 model coffee farms are now demonstrating more sustainable management of coffee farms in the Gualcarque River, Lago Yojoa and Mejocote River watersheds (Annual Report Year 2), and KIIs mention that these are beginning to affect wider coffee production practices. 3.1.2 CONCLUSIONS The agricultural CEs GEMA has supported are benefiting households within GEMA’s AI. The findings indicate that GEMA is assisting small agricultural CEs that are mostly located within GEMA’s WAs. CEs do employ people from outside the household, usually seasonally, thereby providing financial benefits to households other than their own; however, analysis of service provider databases shows that the numbers of workers and the frequency and length of their employment varies greatly between each service provider portfolio, with larger coffee producers more frequently reporting the hiring of seasonal workers. There is insufficient evidence to confirm if the agricultural CEs GEMA has supported are benefiting biodiversity. HHS feedback indicate GEMA has assisted CEs to improve their production practices, particularly through the provision of training to reduce soil erosion and protect vegetation. GEMA ToC for this component states that improved practices will: (i) result in benefits to biodiversity by retaining more vegetation and providing more varied and larger amounts of on￾farm habitat; and (ii) increase crop yields and reduce pressure to clear forests. Based on analysis of their geospatial datasets, GEMA states that such benefits have already occurred over an area of 112,183 ha. including 53,264 ha. of forest in GEMA’s WAs that have maintained their coverage relative to GEMA’s baseline. GEMA indicates that despite threats and community economic, there has been limited change in coverage (Annual Report Year 2). While KII evidence suggests that GEMA’s agriculture extension programs have the potential to increase yields, quality and net income, while contributing to conserving forests, further PE data are not available to link improved management practices with specific biodiversity benefits. It is too early in the program timeline to determine the extent to which GEMA has reduced deforestation or forest degradation through assisting agricultural CEs. The non-agricultural CEs GEMA has supported are contributing only marginally to conserving biodiversity within AI. The findings indicate the enterprises GEMA has created and supported do not yet significantly benefit biodiversity by drawing people away from activities that harm biodiversity within protected areas. These enterprises are located mostly outside of GEMA’s WAs, and frequently do not engage in utilization of natural resources at a scale that could affect key portions of priority watershed and protected areas. The findings do not support GEMA’s assumed linkage between its provision of more efficient wood-burning stoves and reduced deforestation, inside or outside protected areas, or GEMA’s WAs. Figure 8. Coffee Cultivation Within Intibucá Work Area Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 19 3.2 RESILIENCE 3.2.1 FINDINGS Taken together, GEMA’s actions to reduce threats to ecosystems, diversify livelihoods and improve DRM capacities are contributing to the target population’s resilience. GEMA promotes resilience through three main strategies: (i) community-level strengthening of DRM organizational capabilities, through training focused upon CODELs and CODEMs, and implementation of their DRM contingency plans; (ii) wider training in climate change variability and adaptation, directed at JAAs, and coffee farmers and their associated service providers; and application of coffee farm vulnerability assessment tools and mapping techniques, that are aimed at building up the resilience of farms and their associated households; (iii) establishing climate information systems focused upon the Climate Information System for Western Honduras (CICOH), and strengthening capabilities in support of resilience-improvements and risk-reduction actions, such as agroclimatic information and warning system products, working with organizations with both national, and regional, mandates and coverage. In the 25 communities where GEMA has supported CODELs, GEMA has improved the community awareness of the importance of resilience and the benefits of DRM and CCA tools and plans, but the level of baseline capabilities and the weak role of municipal actors are significant challenges for GEMA’s support to lead to substantive outcomes. GEMA’s resilience-building actions have strengthened the DRM capacities of CODELs; however, while their equipment needs have been identified, few have been supplied due to procurement delays. GEMA’s support to Municipal Environment Units (UMAs) and to CODEMs is less advanced than to CODELs, but local governments are strengthening their capabilities which provides some prospect of sustainability. Currently, GEMA is assisting, at various stages, 96 different CODELs (64% of the total goal of 150 CODELs) through organization and training actions, 48 in the La Esperanza region and 48 in the Santa Rosa de Copán region. Of these 96 CODELs, GEMA has judged 25 as having “adequate disaster capacity.” GEMA’s methods for measuring capacity for DRM in CODELs, JAAs or in livelihoods development are unclear. The Performance Indicator Reference Sheet (PIRS) for PI 3.1 defines “adequate disaster capacity” as being achieved when CODELs and CODEMs “can carry-out more than 75% of the technical assistance package by themselves.”5 The PIRS also states that “prior to beginning activities with a set of CODEMs and CODELs, the team will carry out a baseline survey through the application of validated participatory tools focused on community resilience capacities; the study will include the capacities of the CODEM and CODELs in DRR, CCA, and gender aspects of both.” As the majority of GEMA’s activities in support of these participant organizations is recent, it is too early to establish the outcomes of GEMA-support in building DRM capacities. However, communities state that there is good progress in community hazard-mapping and threat monitoring, and 28 Community DRM and CCA plans have been developed with participation of over 363 people, of which 36% are women (GEMA Annual Report Yr. 2, KIs, FGDs), at least 17 further CODEL and 7 CODEM DRM Plans are under preparation. Even so, GEMA’s method for measuring changes in DRM capabilities is unclear. While the Second Annual Report regarding GEMA’s cumulative achievement for Indicator 3.1 states that GEMA is supporting communities to develop plans to attain adequate disaster capacity, it does not indicate more precisely how the capacity of these communities is to be measured against a baseline survey, nor against the target of having 75% capacity to carry out core activities by themselves. GEMA’s resilience activities have largely focused upon training: firstly, to build the DRM capacity of CODELs; secondly for a wider set of organizations in climate variability and vulnerability assessment of value-chains and of JAAs; and lastly livelihoods diversification. Figure 10 indicates the courses and 5 The seven modules are: (1) roles y functions (2) gender issues and disasters (3) evaluation of damages & analysis of needs; (4) concepts of risk; (5) climate variability; (7) first aid and rescue and early alert systems. EQ 2A: Have GEMA’s actions to reduce threats to ecosystems, diversify livelihoods & improve capacity for risk mitigation/disaster response contributed to building resilience? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 20 number of men and women who have been trained in disaster preparedness, response and recovery. Across the supported CODELs, 1,323 men and 713 women have participated in this process, however the initial selection process for CODELs and criteria employed, are unclear (KIIs). GEMA KIs explained their resilience-building strategy is to first strengthen the capacity of CODELS and CODEMs to help their communities prepare for, react to, and recover from the disasters from natural events that are most likely to affect them. Once CODELS and CODEMS have those capabilities, GEMA can then focus training more deliberately on resilience, vulnerabilities and climate change adaption. Adaptation to CCA content has been given to 15 CODELs (15%), the gender focus to 37 CODEL (39%), and risk concepts to 38 CODEL (39%). In terms of the supply of equipment, all CODELs have defined their needs, but according to FGDs and GEMA reports, none have yet been supplied with equipment (Annual Report 2, Q9) due to procurement delays. FGD participants reflect this emphasis. As well, KIIs and FGDs indicated that, in communities where CODELs have been organized, support from the local UMA was not yet reliably supplied and linkages with municipal-level CODEMs were often incipient. FGDs and the HHS reflect GEMA’s strategy of building resilience first through disaster training. Individual informants observed that “we’re ready for any eventuality. We have a shelter that is safe from floods or landslide and is stocked with tools” and “We’re trained to react, resist and recover from threats.” Both FGDs and HHS data indicate in those villages where GEMA has given training, people know more about how to respond to disasters than in those communities where it has not given training. For example, half the FGD informants in Río del Coyol-Suptal and Río Conchagual, where GEMA has provided disaster training, stated that they understood information about hazards. By contrast, in Varsovia, where GEMA has not given disaster training, only 20% of the people knew anything about how to handle natural disasters. GEMA is beginning to improve the awareness of households of the importance of resilience and to build DRM and CCA capacities, but the level of baseline capabilities; the limited convergence of stakeholders on clear definitions and understandings of resilience-building; and the weak role of municipal actors, constitute significant challenges for effective GEMA support. Interpretation of KIIs and statements by FGD participants show that local baseline understandings of resilience are diffuse and incomplete and many CODELs lack awareness and capacities. Women associate resilience with institutions and projects that work on management of local water resources in relation to disaster risk, while a majority of men interviewed are not aware of the meaning of the term “resilience.” Coffee producers relate resilience with support for counteracting rust disease, coffee systems diversification (such as planting citrus, or shifting to cocoa where conditions permit), and other measures to adapt to climate change (HHS, KIIs, FGDs). For CODEL participants, resilience is understood in terms of strengthening disaster responses (emergency alerts, escape routes and evacuation shelters), as well as community recovery actions and readiness. In the 12 sample watersheds, FGDs found that many of the CODELs are still weak, as noted in the fact that only a minority of communities are organized in commissions that distribute DRM work areas. Communities are interested in establishing emergency alert systems, evacuation routes/shelters, and in slope stabilization, improved drainage, and more effective solid waste management. However, GEMA’s targeting and training strategy is unclear, and GEMA-supported CODELs are concerned about a lack of GEMA follow-up, slow implementation of agreements following training events, insufficient support for community DRM plans, and delays in equipment supply. Importantly, FGDs and KIIs also converge on complaints about the weak support provided by Municipal Authorities, and the urgent need to improve CODEL-CODEM and JAA-UMA linkages. However, CODELs are beginning to Figure 10: Topics covered in training given to CODELs Training Topic Men Women Total Gender 474 743 1,217 General Concepts in Risk Management 267 125 392 Damage Assessment / Needs Analysis 52 26 78 Roles and Functions 180 130 310 Simulation Exercises 180 139 310 Source: GEMA 2018a GEMA Second Annual Report, p 66, Table 34 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 21 promote women’s participation and livelihoods diversification. Joint action between CODELs and JAAs are becoming more frequent as memberships of the two bodies often overlap, such as supporting water system improvements, obtaining legal status, delimitation of the local forests; training; and use of improved seeds and irrigation systems. Wider support and training on climate change adaptation; farm and watershed resilience; and vulnerability assessment and tools is beginning to produce outcomes. According to GEMA’s Annual Report for Year 2, 53,263 ha. of forest and agroforestry land cover has been maintained under stable status, relative to GEMA’s baseline assessment, and these reductions in threats to ecosystems are estimated to be effective over a total land area of 112,182 ha., based on GEMA’s spatial analysis of its WAs and AI. GEMA has trained 721 members of CODEL and JAAs to build awareness of climate variability and its effects on resilience, however at household level there is limited understanding of the relationship between DRM and climate threats. Through training of service providers and a subsequent total of 700 farm-level vulnerability assessments, improvements to on-farm natural resource management are beginning to support the resilience of farmers’ livelihoods. These actions show considerable potential for linkages to the MCSE schemes that GEMA has fostered (KIIs, GEMA Annual Report Yr2). GEMA is just beginning to promote joint actions between CODELs and JAAs towards improving local watershed management through delimitation of micro-catchment areas, and the establishment of MCSE, often with some support from municipal entities and other partners, however these processes and their outcomes suffer from weaknesses, and this work requires further support. KIIs and FGDs indicate that DRM plans must better emphasize potential linkages between CODEL actions, and the resilience-building of farms, ecosystem conservation and water management systems. GEMA has promoted improved understandings of gender vulnerabilities by undertaking a resilience baseline and value-chain studies that assist risk identification and livelihoods diversification, and these activities are contributing to GEMA stakeholders (particularly CODELs) starting to address women’s specific vulnerabilities. With support from GEMA, JAAs have taken actions to improve water quality. FGDs and HHS confirm JAAs have undertaken water analysis, improvement of intake structures and distribution networks, expanding access via increased household coverage, cleaning of tanks, and application of agreements to avoid contamination. Successes so far include some purchases of land in recharge areas, obtaining legal status, applying training, chlorination of water, and delimitation of the micro-catchment area. FGDs clarified that conservation measures include reforestation, forest surveillance, land use inspections, improved fencing, and increasing awareness. FGD participants and HHS informants also reported that MCSEs are being established to reduce the sources of contamination in watersheds, at times with the support of UMA officials and other partners. These involve land purchases from farmers to avoid water resource pollution and to enable reforestation. GEMA has provided training and studies to assist people to diversify their livelihoods. KIs noted that GEMA has found it difficult to diversify sources of income of its target population, given the limited markets in Western Honduras for much else than coffee and cacao. The value chain studies for non-agricultural products such as fibers offer a useful basis for solutions, however the relevant CDEs still need to adapt their services to the specific needs of each value chain. As the sustainability of small businesses of all types is tightly-linked to their market access and negotiating strength, these aspects remain a significant challenge for a short-duration intervention such as GEMA. Nonetheless, GEMA reports increased incomes for 782 people in six different types of activities that would diversify their livelihoods away from traditional economic activities, such as coffee and towards non-agricultural or niche livelihood activities. Figure 11 highlights the number of women and men who have benefited from support in non-traditional/niche agricultural and non-agricultural support activities. Figure 11: HHs with new income in niche/non-traditional agricultural & non-agricultural livelihoods Activities that Diversify Livelihoods Men Women TOTAL Households participating in conservation enterprises 92 179 271 Sustainable management of avocado farms in the Higuito River watershed 141 23 164 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 22 The level of implementation of risk reduction measures in the participating communities is still at an early stage. GEMA reports indicate that 2,036 individuals have received support to strengthen disaster response, DRM, and CCA with a gender approach, of which 1,323 (65%) are men and 713 (35%) are women (Annual Report 2, Q9). For the HHS sample area, only 1 out of every 10 respondents said they had received training in disaster risk management and climate change, and these figures only reached 5 out of 10 respondents in the watersheds of Río del Coyol-Suptal and Rio Conchagual. Queries on the effectiveness of the training process showed that 82% of those trained said they applied their new capacities (83% men and 77% women, according to the HHS). Training topics applied included community hazard mapping, monitoring of key hazards facing communities (64% of respondents), followed by assistance in forest care. Areas where capabilities had not been applied were group work in fire prevention (7%). Men were more involved in activities such as monitoring (especially of drought risk) and approaches to agriculture, while women were active in preventing tree felling (HHS). GEMA is supporting the establishment of the CICOH and progress has been made at national level but the system is not still rolled out to local level. To deliver this TAP, GEMA has strengthened the participant institutions of the Meteorology, Climatology and Hydrology Information System (MCH) of the RNHM. MCH databases have been installed by the Dirección General de Recursos Hídricos/MiAMBIENTE, CENAOS/Permanent Emergency Commission (COPECO), AHAC, UNAH and ENEE, to benefit key Honduran institutions that use meteorological information, and staff members have been given basic training in its use. GEMA itself, however, is not responsible for ensuring that the MCH system becomes operational. According to KIIs and the Annual Report of Year 2, the institutions that make up the system now have MCH databases functioning; these bodies are now developing their joint system to automatically capture meteorological observations through existing and new hydro￾meteorological stations in Western Honduras. GEMA’s Result Area 3 partners have begun to build the information web portal for the CICOH. KIIs from both these institutions and GEMA agreed that it has been proven difficult to share data through the MCH because participant institutions lack a common format for recording meteorological data, and because their server capacity is limited. Several of the institutional KIIs said that their institutions’ staff require more training to be able to use MCH. GEMA plans to finalize the design for the Climate Information Center of the Western Region of Honduras during early 2019; however, at local level, CICOH is not yet organized nor has started operations. While plans are established for CUROC-UNAH to administer CICOH locally (linked to the national MCH), KIs identified that CUROC has limited technical capacities to deliver the required scope of CICOH products and services. Based on consultations with water management boards and the coffee cooperative CAPUCAS, GEMA has begun to design agroclimatic products that would deliver disease control and other alerts for coffee farmers. Overall, on the utilization of climate information and implementing risk-reducing actions, GEMA is progressing well and has increased the number of people benefitting from DRM and CCA. EQ 2B: If so, what is the magnitude of its contribution; why has it been able to make this contribution? GEMA is making good progress in supporting people to adapt to the effects of climate change. According to the GEMA Annual Report 2, a total of 13,640 people (91% of the total goal of 15,000 people) have received support to adapt to climate variability, made up during Year 1 of 3,000 individuals, and during the second year reaching 11,699. Of these, 3,852 have been supported in renewable technologies, 416 are CODEL members Ecological production of sugar cane products in Mejocote River watershed 144 6 150 Renewable Energy Micro Enterprises 27 0 27 Entrepreneurship in nature tourism products 23 9 32 Conservation enterprises in sugarcane, avocado and non-agricultural products 45 93 138 TOTAL 445 337 782 Source: GEMA 2018a Second Annual Report, Table 2.3 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 23 and 7,431 are livelihoods conservation enterprises. Farmers in the FGDs are responding to concerns about adaptation to climate change by crop or farming systems diversification (through use of crops like citrus and cocoa), shifting to low-input agriculture, micro-irrigation systems, and the use improved seeds. Furthermore, eight FGD participants in San Pedro Copán reported they have been given information on risks and their causes, on vulnerabilities and actions to be carried out, such as reduced deforestation, improvement of drains and reduction of garbage, but definition is still poor of which actions or practices most strongly contribute to improved DRM, as opposed to enhanced CCA, and how and why those differences in the role of potential practices occur. Further conceptual clarity remains a challenge within GEMA and in outreach and training to CODELs. GEMA has fostered initial improvements in institutional relations between UMAs, CODEMs and CODELs, JAAs, producer groups and other bodies, that may contribute towards enhanced levels of DRM and CCA, but this is incipient and requires support. In FGDS, a major concern that was expressed is the weakness of local government institutions and poor cooperation from those agencies that should provide CODELs with finance, training, equipment and technical support. Weak articulation of municipal-level actions by CODEMs or UMAs was frequently observed in FGDs: participants complained that little has been established in terms of collaborative institutions, programs and projects that involve various levels of government and sectors, within locally-useful systems of disaster risk management. A further worry was weak local capacity for resolution of conflicts that arise when trying to tackle poor NRM and when challenging or seeking to control those actors and their activities that are contributing to greater levels of disaster risk. A limited number of KIIs and FGDs report that GEMA’s activities in DRM through UMAs, CODELS, JAAs and CODEMs may have positively involved local government, and in those cases that may lead to a higher level of sustainability once GEMA has concluded. 3.2.2 CONCLUSIONS Taken together, GEMA’s actions to reduce threats to ecosystems, diversify livelihoods and improve risk mitigation and disaster response capacities, are contributing to resilience-building of its target population. GEMA’s assistance to agricultural enterprises, especially coffee, is influencing farmers to improve NRM on their own farms, and to not expand agriculture into protected areas through deforestation, thereby reducing threats to forest and fresh-water ecosystems. GEMA’s actions to establish water governance will increase ecological resilience through the reduction of these threats. The sum of the areas of the 102 Level-1 micro-watersheds that are implementing MCSEs would be the magnitude of this reduction in threat levels, although the total area covered is not yet available. GEMA has provided training for CODELs that has improved their capacity for risk mitigation, and that this is a reasonable first step in increasing the resilience of GEMA’s target population, however both CODEL selection criteria, and conceptual content in training for CODELs, are still unclear. The magnitude of this contribution is 25 CODELs and further CODEMs, with capacity to mitigate risk and respond to disasters, but their equipment needs require follow-up. The findings indicate that GEMA has worked to increase resilience by addressing core concerns of its target population: livelihoods; availability of supplies of clean water; and reduced risk from hazards, such as drought and floods. In the medium term, GEMA’s action to increase sharing of meteorological data among Honduran institutions has the potential to deliver early warnings of extreme events and will permit organizations to collaborate to increase the resilience of the region’s population. Further attention is needed concerning the selection criteria for CODELs and CODEMs, on the training model these, and for how GEMA should measure institutional disaster risk management capabilities and change. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 24 3.3 GENDER 3.3.1 FINDINGS EQ 3A: Has GEMA increased women’s participation in decision-making in water governance and Mechanisms for Compensation for Ecosystem Services? EQ 3B: If so, how and to what extent? GEMA’s gender training strategy and materials emphasize women’s roles in decision-making. Through September 30, 2018, GEMA had organized 210 gender workshops with the participation of 1,615 men and 1,648 women (GEMA 2018a, Second Annual Report). Training materials GEMA has prepared for gender training enable the training of women and men together and emphasize gender roles in water governance and small businesses. GEMA training materials clearly convey that effective governance over water requires women’s participation in decision-making (GEMA, 2018h. Género en la Gestión de Agua). FGDs across 12 watersheds show that women are largely satisfied with the training given to date, but emphasize the need for further topics outside water governance for both agricultural and non-agricultural CEs. Requested topics include: how to improve product packaging, shifts to more efficient modern equipment, overcoming marketing bottlenecks, understanding climate adaptation in relation to MCSE schemes, financial management and how to access credit so as to boost entrepreneurial capabilities. GEMA training has contributed to a slight increase in women’s participation in JAAs and community water assemblies. Community assemblies and JAAs make the decisions about water governance, with Assemblies convening only occasionally and JAAs meeting more frequently. KIs commented that more women than men tend to attend the community assemblies because their work often prevents men from attending. GEMA KIIs and documents indicate that about 22% of the members of JAAs are women, which is a 6% increase since GEMA started two years ago. (GEMA Annual Report Year 2). Data are unavailable that indicate clearly the increase can be attribute directly to GEMA’s actions. GEMA beneficiary participants in FGDs and KIs, however, noted that men have become more open to women’s participation in JAAs, which may be partially a result of GEMA’s gender training. GEMA KIs reported that most JAAs have one or two female members, but that they are more frequently in the position of secretaries or treasurer than president or vice president. The HHS indicated that only 35% of those interviewed thought the participation of women in relevant roles on JAAs has increased. Female heads of household reported their participation in decisions and positions in JAAs has increased by around 23%. These results indicate a gradual shift towards women being involved in community water issues. FGDs across 12 watersheds indicated women are taking on more roles on JAAs, and are more vigorously seeking out training, equipment and financing for JAAs operations, as well as greater collaboration with municipal authorities. The evolution of Water Assembly elections (held every two years) should provide more evidence of any changes fostered by GEMA that lead to enhancements in the participation of women in decision making roles, rather than in administrative or observer functions. GEMA is beginning to contribute to women’s economic empowerment by assisting conservation enterprises. Evidence is limited that, as a result of GEMA’s interventions, women’s incomes have significantly increased within conservation companies. KIIs with the Santa Rosa CDE indicate positive progress, but the other two CDEs do not yet report this clearly. GEMA’s Second Annual Report does not disaggregate by sex the participants in these income generating activities but does state that 31% of them were women. The Second Annual Report does not establish how many new jobs can be attributed to GEMA activities, but does indicate that 31% of them were filled by women. GEMA’s Annual Report for Year 2 observes that the contribution of non-agricultural enterprises towards GEMA’s meeting its target for the economic empowerment of women has been only about 12%, which is small relative to the contribution of household renewable energy systems and agricultural livelihoods. EQ 3C: Has GEMA successfully increased the economic empowerment of its target women population? EQ 3D: If so, how, and to what extent? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 25 HHS data indicate that of the people who have been able to access financial resources due to GEMA, 45% are women, while the Second Annual Report states that the sales declared by the 2,841 enterprises totaled USD $15,200,000 and that 31% of these sales correspond to enterprises led by women. Sales declared in GEMA’s databases at the end of Year 2 are distributed with 64% of that total due to coffee, and 34% due to non-agricultural enterprises in which female owners and employees are more prominent. The contribution of GEMA to these increased sales and how those are contributing to improved women’s livelihoods is still unclear. GEMA has contributed to women’s economic empowerment by assisting agricultural enterprises. Thirty-seven percent of GEMA’s progress towards its indicator 2.3 for the economic empowerment of women comes from its support to agricultural activities, which reports the percentage of female participants in USG-assisted programs designed to increase access to productive economic resources. GEMA has specified in its Terms of Reference for activities to assist agricultural value-chains that 35% of the participants must be women. Overall, 31% of GEMA beneficiaries are women. Figure 13 outlines the activities GEMA has financed that have contributed to the economic empowerment of women by assisting agricultural enterprises. While GEMA has contributed to the economic empowerment of women through its training and TA activities for agricultural and non-agricultural enterprises, neither the absolute (to date), nor the potential, values of the degree of economic empowerment are yet clear. GEMA has not conducted a comparative gender-focused study of the relative sustainability and profitability of agricultural and non-agricultural products CEs, but research into this would help to ensure that GEMA’s support to both classes of CEs favors greater gender equity. Figure 13: Support to agricultural CEs that contribute to women’s economic empowerment Training and grants that contribute to the indicator Men Women Total Field schools to market and produce milk and beef: Higuito/Mejocote 138 19 157 Grant for sustainable avocado farm management in the Higuito River 141 23 164 Grant for sustainable coffee farm management in the sub watershed of the Gualcarque River and Lago Yojoa (CO, HONDUCAFE) 299 102 401 Coffee farms in the Aruco, Higuito and El Suptal sub watersheds (HQC) 256 44 300 Coffee farms in the Sazagua and Puringla river watersheds. MAMCEPAZ 233 67 300 Coffee farms in the sub watersheds of the Sazagua river. (COMUCAP) 0 236 236 Eco-production of sugar cane loaves in the Rio Mejocote sub watershed 144 6 150 Total 1211 497 1708 GEMA Annual Report Year 2 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 26 Women make up over 40% of participants in GEMA’s Disaster Relief Management training. GEMA’s principal activity related to DRM has concerned organizing, training and equipping CODELs. Figure 14 indicates that GEMA has trained 318 women (44% of all trainees) in adaption to climate change, which includes DRM. In only one of the eight municipalities that received training was the percentage of women trainees less than 37% and in most of the municipalities the percentage of women was near or above 50%. By contrast, among the respondents to the HHS which represented 12 watersheds, some of which hadn’t received training, only about 15% of women were familiar with the functions of CODELs (compared to 31% of the male respondents). FGDs and KIIs report that CODEL members perceive that GEMA’s focus on improving the capacities of participating women is highly relevant, as women are often among the most vulnerable. This fact is now increasingly recognized by local actors, however women’s leadership roles in building external alliances and taking forward CODEL Work Plans is still underappreciated. GEMA has strengthened women’s participation in DRM. The findings indicate that women have participated in nearly equal numbers as men in GEMA’s training activities for DRM. No GEMA document that was available to elucidate the cause and effect link between such training and strengthened women’s participation in DRM. Presumably, however, the training has provided female participants with knowledge and skills that will enable them to play more active, useful roles in DRM. Data were not available to link the relatively high percentage of women who are members of CODELs, with GEMA-supported activities. GEMA’s actions are not yet contributing strongly to increasing the participation of women in decision-making positions in water governance. FGD and KII findings indicate that the number of women who are members of the JAAs has increased slightly since GEMA started, but they do not indicate this increase is necessarily due to GEMA’s actions. Based on Training Materials such as its “Cartilla de Género en la Gestión de Agua”, GEMA has given training to both female and male JAA members. There are no data available to indicate whether or not this training has increased the participation of women as JAA decision-makers. While women do participate in the assemblies of the JAA, GEMA has not targeted female members of the assemblies and it is not yet possible to attribute training of women to the increased participation in decision-making roles and positions in either the JAAs or community water assemblies. 3.3.2 CONCLUSIONS Overall, 31% of GEMA beneficiaries are women and GEMA’s actions are appropriate to increasing the participation of women in water governance; contributing to the economic empowerment of women, especially in non￾agricultural CEs; and have strengthened women’s participation in Disaster Relief Management. Even so, significant challenges remain to further increasing the participation of women in decision-making roles. Figure 14: DRM Training Attendees by Gender in CODELs Municipality M F Total % F Márcala, La Paz 124 128 252 51% Cucuyagua, Copán 32 35 67 52% Talgua, Lempira 46 34 80 43% Las Flores, Lempira 52 31 83 37% Gracias, Lempira 73 49 122 40% Chinacla, La Paz 33 21 54 39% Jesús de Otoro 14 16 30 53% Intibucá, Intibucá 29 4 33 12% Total 403 318 721 44% EQ 3E: Have GEMA actions strengthened the participation of women in Disaster Relief Management? EQ 3F: If so, how, and to what extent? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 27 3.4 GEMA RESOURCES 3.4.1 FINDINGS Community informants hold a range of views on proposed improvements to GEMA support. KIIs and FGDs were undertaken to better understand community perceptions of GEMA and their priorities for GEMA support. Overall, communities seek closer ties with GEMA staff and service providers and request consistent follow￾of the activities GEMA has started, including with MCSEs, CODELs and the JAAs. In particular, there were requests for practical steps to take forward MCSE scheme implementation, especially seed capital and support to obtain more effective municipal support. Consulted communities state that they would like GEMA to provide a longer total duration of its support, with more frequent service provider and GEMA visits and to provide a greater volume and quality in GEMA communication about the Activity’s approach, plans, participation opportunities and benefits. For specific Results Areas: coffee farmers requested further support for climate-resilient coffee varieties, greater access to effective technologies such as post-harvest coffee pulping and drying equipment and greater support for accessing reliable markets. Farmers in general, emphasized that they would like to be supported to install small irrigation systems with GEMA’s technical support and financing and to be aided to construct climate-adapted water reservoirs. JAAs want more in-depth support to enable the legal process for official declarations of water recharge zones, including improved linkages with UMAs and to obtain external financing for reforestation and for land purchases in watershed recharge zones. CEs are keen to receive further organizational and legal technical assistance, support for producer joint-organizations, contributions of seed capital for women’s CEs and training in marketing and accounting. CODELs seek GEMA’s support for equipment provision to implement community DRM plans, more frequent presence of institutions such as COPECO, support for control of infrastructure construction in relation to hazards and improved understanding of women’s vulnerabilities to threats. Some, but not all, of GEMA’s Indicators are on track. More time and resources may be required to meet end-of activity targets. Figure 15 highlights GEMA’s progress towards its 20 performance indicators as of September 2018. This indicates that nine PIs (45%) are on or ahead of schedule and 11 PIs (55%) are behind schedule. R1 has the most PIs behind schedule, while about half of PIs of Results 2 and 3 are on schedule. The cross￾cutting gender PI is 100% on schedule. GEMA has suffered inevitable delays in a range of aspects, such as defining its WAs, strategies, implementation methods, challenges in planning, recruitment and services procurement. Many of these are due to the very large size of the intervention area, the number of actors and processes, and the scale and complexity of the Activity. An analysis of GEMA’s RF and 20 PIs is set out in Appendix B. This provides the justification for the proposals in Figure 16 below. Figure 15: GEMA Performance Indicators Progress (Sept. 2018) RESULT # OF INDICATORS ON TARGET BEHIND TARGET Qty % complete Qty % Result 1: Reduced threats to biodiversity 8 2 25 6 75 Result 2: Increased conservation–related income 7 4 57 3 43 Result 3: Increased capacity to adapt 4 1 50 3 50 Crossing-Cutting 1 1 100 0 0 Total 20 8 12 Source: Analysis based on Second Annual Report: See Appendix B EQ 4A: Which of GEMA’s actions need more time, technical assistance, training or funding to produce their target outcomes? EQ 4B: What additional resources, if any, would be required to produce the target outcomes? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 28 Figure 16: Resources Required to Meet PI Quantitative Targets PI # Activities to realize PI Quantitative Targets 1.1 • Clarify uncertainties in targets and in data presented in 2nd Annual Report 1.1.1 • Train JAA assemblies, not only JAA board members 1.2.1 • USAID support for simplifying ICF process for protection watersheds 1.2.2 • Assign more resources to field implementation of MCSE schemes, establish methods to support JAA land purchases, especially for Lenca indigenous communities in Opalaca and Montana Verde PAs. • Analyze MCSE sustainability concerns, define expected institutional linkages, and prepare support plans • Provide sustained and effective GEMA support for simplifying ICF procedures for baseline studies and approval to declare protection watersheds • Assign time/resources to strengthening institutional support to JAAs and linkages with municipalities 1.3.1 • Focus target number of households onto core locations linked to protected watersheds • Undertake baseline study on firewood sources, and on consumption changes due to RE adoption. 1.3.2 • Reduce target number of enterprises and focus GEMA assistance on agricultural CEs and non-agricultural CES located within and around its target micro-watersheds • Assess the feasibility of PI targets for new net sales and new employment of CEs • Increase support to improved coffee production techniques and agroforestry diversification • Review relative profitability/sustainability of agricultural and non-agricultural CEs • Ensure that Value Chain studies lead onto design and implementation of focused Action Plans 2.1/2.2 • Clarify GEMA activities in tourism 2.4 • Clarify PI targets for new income for main commodities and for the types of CEs as these may be set too high. • Establish further targets and approaches for CE employee maintenance and for business sustainability 2.1.2 • Clarify scale of public-private alliances • Review feasibility of public-private alliances leveraging substantial inward investments • Identify and establish clear pathways for CE groups to attain certified status 3.1 • Clarify the achievement of PI target; and analyze feasibility of attaining “adequate disaster capacity” • Assign resources to provision of CODEL equipment; respond more rapidly to CODEL requirements • Ensure appropriate balance between support to CODELs and to UMAs/CODEMs; and foster linkages 3.1.1 • Ensure the Meteorological System becomes functional. Provide training to CUROC & partners • Ensure that CICOH system results in provision of early warning alert systems 3.1.2 • Accelerate training programs in climate change adaptation Cross￾cutting • Ensure women are fully involved in vulnerability assessments and CCA plans • Increase the scope/quality of women’s participation in both JAAs and CODELs; involve younger women • Assess expected changes in gender equity& women's income via participation in agri & non-agri enterprises Source: Appendix B derived from GEMA MEAL Framework and B. Kernan Analysis 3.4.2 CONCLUSIONS Many of GEMA’s actions need more time, technical assistance, training and funding. While GEMA is only half-way through its period of implementation, progress to-date shows its outcomes require more than four years to achieve and many of GEMA’s actions need more time, TA and funding. GEMA has suffered inevitable delays in a range of aspects, such as defining its WAs, strategies, implementation methods, and services procurement, due to the size of the intervention area, the number of actors and processes, and the scale and complexity of the Activity. In some cases, such as delivery of targets in Results Area 2, the targets may be too ambitious, due to the hard-to-resolve weaknesses in the institutional capabilities of intermediary service providers. As a result, several of GEMA performance indicators need to be clarified or modified and it may not be feasible to achieve others within GEMA’s program completion timeframe. 3.5 INTEGRATION Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 29 3.5.1 FINDINGS EQ 5: Have GEMA’s different actions been integrated effectively with each other? GEMA distributes rather than integrates its field activities, but lacks an updated ToC resources management, and specifically support greater access to adequate water resources. USAID/Honduras contracted GEMA to implement actions over a large geographic area in Western Honduras. GEMA’s potential for integrating its field activities in geographic locations is therefore limited. A GEMA KI explained: “GEMA is not attempting to integrate field actions, rather, it implements different activities in the same area, betting that each one will increase conservation, even though not integrated with other activities. If GEMA were to integrate its activities with each other, it could not implement activities across its current large geographic area.” Thus, for example, in the Santa Barbara/Lake Yojoa Area of Influence area, GEMA is financing efficient eco-stoves and planning to support MIPYMEs that make products from natural fibers, as shown in Appendix D (Maps) which indicates how GEMA has distributed its activities in that case. Other informants agree that GEMA’s activities have been implemented across a wide region and have involved complex processes undertaken by a large number of service providers and partners, with varied types of beneficiaries. This leads to the dispersion of GEMA’s TAPs, rather than the provision of a holistic set of actions, delivered together to each targeted location. In summation, for a given community, GEMA’s TAPs are not as focused in the targeting of their delivery, as would be required to effectively ensure close linkages between the available TAPs. This limits the potential for integration and synergies between TAPs. Integration of TAPs is also required when targeting support for training and equipping CODELs, as the Activity should ensure it both targets the most vulnerable populations, as well as including the communities receiving support to their CODELs, in those TAPs that are related to both Results 1 and 2. Concerns about water, PAs and forests are central to GEMA’s actions. A GEMA KI said “water is always priority of the people. It glues everything GEMA does together.” GEMA documents and KIs and JAA KIs, for example, frequently note that the risk of water scarcity links GEMA’s Results 1 (biodiversity conservation and water management) and 3 (resilience). A GEMA KI said “We are linking the two results by the threat of drought which is the biggest threat from climate change.” The HHS found that 92% of the respondents understand where their source of water is located and the importance of forest conservation in its environs, while nine out of 10 expressed concern about the effect of climate change on their water supplies and therefore on their livelihoods. GEMA’s ToC to the extent expressed in its RF and reports, does not reflect the real strategy of distributing out its TAPs across a wide geographical area, nor the shared view that access to water is a key integrating factor. GEMA does not have a written ToC, other than the introductory paragraphs of its reports and the RF that is set out in its Monitoring, Evaluation and Learning Plan of April 2018 (see Appendix C). These frameworks do not reflect GEMA’s actual ToC as expressed by its leadership. GEMA may benefit from concentrating and integrating its TAPs within more closely-limited land-units. Water, forests and watershed land-use together integrate GEMA’s technical assistance to increase the benefits of agricultural and non-agricultural, services and products and reduce their externalities. As GEMA does not have an integration strategy, since water is a shared concern for all, water is proposed as a potential key integrating factor for a more precise strategic and geographical focus of GEMA’s TAPs. KIIs with GEMA Results Areas leaders clearly indicated that GEMA’s actual ToC integrates most of its actions around the outcomes of improving management of protected areas, their forests and water resources, in order to ensure effective, long-term access to enough water of adequate quality, while improving the governance of risks and reducing threats in its priority watersheds. In this articulation, GEMA aims to achieve improved natural resources management and governance by strengthening the capacities of networks of social agents including the Public-Private Partnerships (PPPs), service providers, co-managers and, in particular, those JAAs that derive water from these micro-watersheds. GEMA’s current RF does not yet fully convey these concepts within its ToC. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 30 Hydrological modeling, Terra-i and PMCCA could be integrated to support water governance. The use of existing platforms or products generated at a territorial scale such as those fostered by other USAID programs, can facilitate the understanding of hydrological behavior and its interactions at watershed scale. The Honduras Water Platform, Terra-i, and the CICOH all provide data that are useful, first for strengthening technical understanding of a given watershed, and secondly to then enable management planning that increases resilience to climate change and supports water governance. 3.5.2 CONCLUSIONS GEMA’s TAPs have not been integrated with each other but rather undertaken to support individual Results Areas. Governance over water resources could serve to integrate GEMA’s actions but would require a modified ToC that prioritized core watersheds, as the basis for an updated RF. 3.6 SUSTAINABILITY AND GOVERNANCE 3.6.1 FINDINGS EQ 6A: Has GEMA produced sustainable environmental governance, watershed management, CEs and adaptation to climate change? In the following sections, key factors to be addressed in water governance and CE sustainability are set out in order to support the PE learning process. Environmental Governance and Watershed Management With support from GOAL, GEMA has prepared a tool to assess water governance; and stakeholders are increasingly aware of the relevant factors to be fostered and how to measure these. GEMA has financed GOAL to prepare a methodology for measuring water governance. Using the Conflict, Legitimacy, Interests and Power (CLIP) Model, GOAL/GEMA has developed the Actor Mapping and Evaluation Framework, which contains a set of governance factors and a scoring system. In this, a set of indicators are applied to each actor across the key watershed functions of Provision; Distribution and Use and Compensation for Services. GOAL’s and GEMA’s KIIs indicated that Water Governance should support an interrelated set of actions that improve the technical and administrative capabilities of organizations to improve delivery of their products and services and comply with legal requirements and mandates and that consolidate inter-institutional relationships for the exchange of information between different public and private actors. The Water Governance tool was validated through four micro-watersheds case studies (or Estudios de Gobernanza Hídrica) that GEMA/GOAL undertook. Based on these, the following principles and indicators were incorporated into the framework: (i) data and information that are consistent and relevant to water issues are shared and used to improve water resource management; (ii) financial resources are managed efficiently; (iii) regulatory frameworks are applied effectively and policy coherence is fostered; (iv) mechanisms of compensation for ecosystem services are implemented; and their benefits for participants and conserved areas are evaluated; (v) organizations are capable of adapting to evolving management challenges; (vi) participation accounts for gender equity and conflict resolution is applied; (vii) accountability practices ensure stakeholders’ confidence in decision-making; (viii) resilience is accounted for by ensuring organizations are aware of the main threats and have the capacity to respond to adverse events. The GOAL/GEMA tool evaluates all the above components of watershed governance in relation to actors, levels of collaboration, and threats, and can be used to identify feasible actions to protect water. All the KIs consulted about this methodology agreed the tool responds to an important need for measuring governance; and explained that in GEMA’s year 3 and 4, they will apply the tool to establish a baseline of their level of water/environmental governance. Application of the tool will require co-managers and associations to take on leadership governance roles for protected areas and watersheds. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 31 GEMA is fostering co-manager-community collaboration. Under the SINAPH, Honduras’ protected areas are co-managed by the government and conservation NGOs. Co-managers, according to two GEMA KIIs, have tended to see the people living near protected areas as threat to biodiversity. Based on the experience of ProParque, GEMA encourages NGOs to view these people as collaborators, rather than as threats. As fostered by numerous programs, ICF and GEMA terms this approach “community co-management.” While GEMA KIs stated this model was becoming widely applied, and independent confirmation was not possible, expert consultation indicates that many NGOs are committed to integrating participatory approaches. GEMA fosters watershed management through compensation for ecosystem services schemes. GEMA has established 102 operational MCSEs and is progressing with the establishment of an additional 271 MCSEs, as shown in Figure 18. To be assessed as operational, the MCSEs must have statutes approved by the respective JAA assembly, be linked to a JAA that maintains its legal status, and prepare and implement annual work plans that protect micro-watersheds. Figure 18 shows the status of the MCSEs that GEMA is promoting, and the lead organizations that are providing support such as the Asociación de Juntas Administradoras de Agua and Irrigator Groups (IG). GEMA KIs explained that GEMA is fostering “voluntary compensation for ecosystem services” which involve voluntary actions by water users to protect their water. As one KI said, “through voluntary work the people who use the water appropriate the protection of the watersheds.” GEMA is developing ways to track and account for the financial value of such voluntary work so that, as a GEMA KI said, “water users will become more aware of the value of the watershed.” Hydrological modeling could contribute to sustainable micro-watershed management. Hydrological monitoring provides a strong basis for land-use planning within watersheds. The USAID-financed Honduras Water Platform provides a possible basis for undertaking hydrological modeling of JAA watersheds. It provides a hosting platform for data on surface water flows and provides indicators for measuring water quality on the scale of Level￾1 micro-watersheds. Terra-i/Honduras, financed by GEMA, provides data on land-use changes. These data could be used to model the links between land use and hydrology in micro-watersheds. The CICOH which GEMA finances, would also provide hydrological models with useful data. While hydrological modeling may be capable of supporting watershed management for GEMA and other implementing partners, the equipment, data and capacity development needs will need to be carefully assessed, especially data management and analysis capabilities, so that modeling can contribute to better decision-making. GEMA fosters institutional alliances that will assist the JAAs to govern and manage their micro￾watersheds. GEMA’s Second Annual Report describes how it “anchors its actions in micro-watersheds and municipalities” while facilitating “alliances between public entities, the private sector and civil society to create a local platform for effective natural resources governance” which coincides with the observations and needs expressed in FGDs by beneficiaries. Figure 18: Status of the MCSEs MCSE Status JAA IG JAA Total Identified 16 5 1 22 Documented 46 46 Legalized 46 5 1 52 Implemented 151 151 Operating 102 102 Total 361 10 2 373 Source GEMA. 2018a. Annual Report Y2 Figure 17. A well-managed water source in Chinacla, La Paz Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 32 KIs with JAAs, GEMA staff, Ministry of Health and municipal officials confirmed that GEMA’s effort to create alliances to establish accountable local institutions was widely accepted as the “foundation of effective water resources governance”. A GEMA KI said, “The micro- watershed is the administrative unit that interests GEMA. We want to link the communities in and around protected areas with their sources of water.” Complying with the governance factors set out above, Municipal and JAA KIs agreed with GEMA’s strategy and described their respective roles: municipalities can formulate, approve, enforce, and implement ordinances pertaining to water use and watershed management, and their UMA sometimes help to resolve conflicts of water and land use. The JAAs can formulate specific regulations for water and land use in the watersheds that provide their water.6 Local interest and knowledge underlie GEMA’s focus on JAAs. KIs with GEMA staff, JAAs, Ministry of Health and municipal government staff indicated that many JAA members understand how the welfare of their communities requires reliable, sufficient and clean water supplies. JAA members frequently expressed their commitment to conserving their water and demonstrated detailed knowledge about the biophysical and social characteristics of their micro-watersheds. In one FGD, a participant shared the opinions that “we all have to unite in an alliance between the JAAs and the CODELs to protect our water sources” and similar views were expressed in several further FGDs. Data from the HHS also indicate the interest of local people in conserving the quality and quantity of water flowing from watersheds. During the last 12 months, 75% of the respondents had participated in activities to protect the sources of their water. Overall, 92% of the respondents to the HHS considered that it is the communities’ responsibility to take care of their water sources, while only 7% considered it to be the responsibility of the central or municipal governments. Sixty-three percent of the respondents to the HHS indicated they would be willing to pay additional fees to be used to protect their water sources. Qualitative data from the FGDs confirmed the results of the HHS. “In our communities we are really all part of the JAA” and “we make collective decisions about water in the community assembly” were consistent comments made by their participants. Conservation Enterprises The long-term sustainability of the non-agricultural enterprises supported by GEMA is vital to their owners, but it is as yet unclear the extent to which smaller CEs are sustainable. According to USAID (2017), six sustainability conditions are necessary for conservation enterprises, including the following criteria: (1) sources and methods of income-generation are effective in meeting participant’s livelihoods needs; (2) business and financial aspects are in place such as reserves and access to credit; (3) strong internal governance exists; (4) supportive policies and partnerships are in place; (5) participants have necessary skills and capacity; (6) production and survey chains in place and favorable. KI and documentary evidence indicate that few of the non-agricultural conservation enterprises GEMA has supported meet all of these conditions. A sample CDE database (for the Sta Rosa CDE) shows that in 2018, at baseline, for 60 non-agricultural CEs, the total number of owners was 95 (63% women), and these CEs had a total of 270 employees (of which 50% are women), indicating that each CE benefits a total of 1.5 owners and 4.5 employees. Only 18% of CEs have more than 10 participants, while 60% of them had fewer than the average of 6 participants. While larger enterprises may comply with many of these criteria, many of the subsistence-level CEs 6 Postel and Richter (2003) states “environmental governance involves making alliances and identifying common objectives and goals”, and presents case studies of how such alliances made sustainable governance of water resources feasible. GEMA’s strategy of supporting alliances thus appears to adhere to best practices for water governance. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 33 with fewer than average participants, and low levels of capital, may not achieve the required benchmark levels. Several studies have found that in Western Honduras owners of non-agricultural CEs are frequently single women, and they may be the majority of the owners of smaller-sized CEs. Analysis shows that in the sample CDE portfolio, 40 women-owned CEs have an average of fewer than 4.8 total participants. These sustainability concerns that face smaller CEs require a specific, gender-sensitive, strategy. The agricultural enterprises GEMA has assisted are likely to be sustainable. Documentary and KII evidence indicate that GEMA has assisted agricultural enterprises that do meet the six conditions for sustainability previously stated. The agricultural enterprises, however, differ from the type of CE considered in the USAID document. GEMA’s strategy is to assist established agricultural enterprises to form links with large exporters, and it has focused particularly on coffee enterprises. Well-established enterprises have been growing coffee in Western Honduras since the mid-1800’s. Individual coffee enterprises may sometimes fail, but overall, coffee has been a sustainable enterprise. By assisting coffee CEs and helping them to establish links to exporters, GEMA is likely to ensure that its assistance to these CEs will produce sustainable results. Adaptation to Climate Change Assuring water quantity and quality is the key measure in Western Honduras to achieve sustainable human adaptation to climate change. Documentary, KII, FGD, observational and HHS data, as discussed previously, provide strong evidence for the importance of water quality and quantity for people within GEMA’s AI, to adapt to climate change. GEMA documents and KIs emphasize that sustainability of supplies of clean water to communities in GEMA’s Area of Influence is at GEMA’s core. EQ 6B: If so, to what extent and how? It is not possible to fully confirm that GEMA activities have improved water governance in its watersheds. While findings indicate that GEMA has established operating MCSE schemes in 102 watersheds, by GEMA’s own criteria it therefore follows that GEMA has produced – for that governance factor –– a significant improvement in governance over those 102 watersheds. Data were unavailable to confirm this conclusion based on independent findings, as the Second Annual Report does not provide enough data to be able to judge the degree of sustainability of the MCSEs, and the GOAL tool has yet to be applied to generate a governance baseline. Insufficient KIIs were carried out to verify this and the HHS did not cover this issue. The findings, however, do indicate that, to continue to be considered as an “operating MCSE” scheme, a JAA must maintain its legal status; and must implement annual work plans that incorporate protection activities. In particular, literature and GEMA KIs indicate that, in Western Honduras, typically it is difficult for JAAs to maintain their legal status. GEMA’s interventions are too recent to establish the proportion (if any) of GEMA-supported JAAs that have lost that status. GEMA’s documents as well as KIIs with GEMA staff and JAAs indicate that establishing operational MCSEs is a key method of implementing both watershed management and governance; however, data are insufficient to evaluate the degree to which the reported 102 JAAs are indeed implementing MCSE activities and, therefore, how sustainable the associated watershed management may have become within the watersheds where GEMA has recently established MCSEs. GEMA has assisted agricultural CEs to apply a conservation focus through adoption of practices aimed at reduced environmental impacts. The agricultural enterprises GEMA has assisted already existed before GEMA started; therefore, it would be inaccurate to state that GEMA has itself “produced sustainable agricultural enterprises.” Most of the factors that make these enterprises sustainable, or not, are outside of GEMA’s control. GEMA has, however, given them technical assistance, which may make the production aspects of the agricultural enterprises more sustainable. It is not yet possible to determine if GEMA has produced sustainable adaptation to climate change. Findings from KIIs and documentary review indicate that the principal way in which GEMA will produce sustainable adaptation to climate change within its Area of Influence is by establishing sustainable governance over water resources and sustainable management of Level-1 micro-watersheds by JAAs. As discussed in the previous Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 34 conclusions, it is not yet possible to determine the degree to which GEMA has produced either sustainable water resources governance, or effective management of Level-1 micro-watersheds. Therefore, it is not yet possible to determine if GEMA has produced sustainable adaptation to climate change. 3.6.2 CONCLUSIONS The PE observes that ensuring water quantity and quality is the key measure in Western Honduras to achieve sustainable human adaptation to climate change. With firm support from communities, GEMA has a significant opportunity to foster both effective water governance and watershed management, which will have a long-term and sustainable impact upon protected areas and their biodiversity. GEMA’s support to agricultural CE’s, specifically working with the coffee sector to access international markets, has increased sustainability, especially as alliances with public-private partnerships deepen. 3.7 CONSTRAINTS AND OPPORTUNITIES 3.7.1 FINDINGS EQ 7: Based upon its performance indicators, what constraints and opportunities have affected GEMA’s progress towards achieving its planned outcomes? The main constraints identified are gender inequality, complicated water legalization regulation, limited education of the JAA, weak institutions and unintegrated Activity logic. Opportunities exist to build on gender equality efforts concentrating on water sector, availing the profound concern that women display for conserving the quality of their water supplies. There are opportunities to adapt materials for low education levels and to use the supportive legal framework to advocate for simplification of the complicated regulations. In general, there is an opportunity for GEMA to focus its interventions more narrowly to make its contribution to sustainable natural resources and biodiversity more effective. GEMA’s actions to increase gender equality have the potential to reinforce actions to establish water governance. KI, FGD, HHS and documentary data indicate that a high proportion of women in GEMA’s WAs have a profound concern for conserving the quality of their water supplies but current land ownership customs (regulations?) limit their role in water board governance. Working on gender equality within the realm of water management can help ensure a sustained committed to conserving water resources. GEMA’s actions to eliminate and/or simplify excessively costly and complicated regulations, if successful, will greatly expand opportunities for effective governance over water resources. KIs repeatedly emphasized that complex government regulations limit effective governance of watersheds in GEMA’s WAs. For their micro watersheds to be declared officially protected, JAAs must comply with the ICF’s demanding requirements for technical reports and topographic surveys. For JAAs to be legally established they must meet the Ministry of Health’s rigorous requirements, and then travel yearly to Tegucigalpa to submit detailed accounting and other reports. Similarly, the ICF’s complex regulations discourage sustainable forest management and limit fuelwood markets7. A recent GEMA study recommends that ICF “streamline and simplify the bureaucratic procedures that govern forestry in Honduras”8 , however the PE was unable to obtain an ICF statement or KII. The limitation of weak formal education, more than constituting a barrier by itself, does indicate that GEMA’s methodologies and materials must be tailored to these characteristics, and requires effective service delivery and communication strategies. The HHS found that 79% of the sampled 7 GEMA. no date (b). Estudio de la Cadena de Valor de Teja y Ladrillo de Barro. The study states, “Due to more strict regulations and controls on the consumption of fuelwood, when the producers cannot find fuelwood to buy, they obtain fuelwood on their own. This practice could affect the environment since the fuelwood could come from within protected areas or from near to a micro-watershed.” 8 FAO and VIDA. 2018. Oferta, Demanda, Barreras y Plan para Incrementar el Uso de Madera Legal, Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 35 population has only a grade school education or less. Less education often correlates with a reduced ability to resolve complex problems, such as the governance and conflict management issues. Yet KIIs and FGDs indicated that many members of the JAAs have years of experience with ensuring supplies of clean, abundant water to their communities. Numerous JAA members were able to explain clearly the links between land uses, contamination sources, and water quality and quantity. Many expressed their dedication to protecting water supplies9. The JAAs are still mostly weak institutions, but there is a supportive regulatory framework for watershed governance. GEMA is mostly assisting JAAs that have been legalized only recently, and whose watersheds have also only recently been declared, or are in the process of being declared as protected. While local government and private institutions that should provide JAAs with support often suffer from ineffective administration, Honduras has established a regulatory and institutional structure for assisting the JAAs to govern their watersheds. GEMA, however, is not applying a clear set of indicators or a framework to assess and to respond to JAA institutional capabilities. While GEMA lacks a clear, written integrated TOC, GEMA’s leadership can articulate one clearly. GEMA’s documentation and RF do not clearly explain GEMA’s ToC. Pages 2 to 16 of GEMA’s Second Annual Report, for example, state about 20 different objectives for GEMA. Yet GEMA leadership can clearly express GEMA’s ToC: “GEMA’s aim is to reduce threats to biodiversity through establishing the basis for governance over water resources” (GEMA KII). 3.7.2 CONCLUSIONS Clear opportunities exist to reduce or remove the constraints that limit GEMA’s ability to produce significant, sustainable outcomes for biodiversity conservation, resilience to climate change and poverty alleviation. GEMA faces the constraints of gender inequality, unsupportive government regulations, poorly educated target populations, weak institutions, an excessively short time period for implementation, and an unclear ToC. There are feasible actions USAID/Honduras and GEMA could take to reduce or eliminate these constraints, as described in Section 4, below. Without these constraints, GEMA should be able to achieve substantial, sustainable outcomes providing a sound basis for future USAID/Honduras activities to reduce poverty in Western Honduras through improved natural resources conservation. 3.8 WATER QUALITY 3.8.1 FINDINGS EQ 8A: How is GEMA measuring the effect of GEMA’s TAPs on water quality within its targeted micro-watersheds? GEMA’s sub-contract with the University of Zamorano to implement the PMCCA within its targeted micro-watersheds is beginning to develop approaches and tools for water quality monitoring.Through September 2018 the PMCCA has been implemented in 97 communities, which are distributed in eight zones: Celaque-Puca-Camapara, Anillo Verde, San Jose, Santiago de Puringla, Márcala, Mixcure, Sirima and Cumes. The PMCCA trains staff of co-manager institutions and municipal governments to measure water quality and quantity of community water systems, at water source intakes and at other water distribution structures. The water quality parameters that PMCCA measures are: pH, turbidity, total dissolved solids, temperature and free chlorine and bacteriological content. Appendix D4 provides a map that shows the location of the sampling sites. KIIs and PMCCA documents indicate that the PMCCA program does not refer to quality control protocols for water quality and quantity measurements (GEMA, 2018g, Strategic Elements to Formulate Water Quality Monitoring Programs). 9 GEMA.2018h. Género en la Gestión de Agua: states that JAA members: “their leader normally receives no pay for their work, but they do it due to their interest and social commitment. However, mostly their work is anonymous and invisible with little recognition.” Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 36 According to GEMA KIs, water quality and quantity data have been collected in Western Honduras by a range of agencies over recent decades; however, the results of tests and assessments have rarely been used in an effective manner, despite the key role of information-sharing between organizations as a factor in water governance. A GEMA KI said: “We do not measure the impact of GEMA TAPs on water quality. The purpose of the PMCCA is to use the (existing) data to educate the members of selected JAAs about the links between land uses and water quality and flow.” GEMA KIs pointed out that GEMA lacks sufficient funds to invest directly into watershed management and its limited funds are therefore being used to strengthen JAAs via the GEMA training program, under the assumption that this will lead to improved watershed management. GEMA staff argue that if JAA members learn to take water quality and volume data, they will better understand they need to manage these watersheds. As one GEMA KI said: “We want to use water quality data so that the members of the JAA become interested in initiating their own actions to improve water quality. We are changing the culture of the JAAs.” The PE was unable to evaluate the effectiveness of the training provided by GEMA to JAAs; however a limited number of KIs observed that the focus and benefits were very satisfactory. The PMCCA does not measure the effect of TAPs’ effects on water quality within its targeted micro￾watersheds. From KIIs and a technical analysis of the approach, it is clear that the PMCCA tool is designed to strengthen institutional collaboration and cooperation in support of JAAs, and to educate JAAs about how to use water quality and quantity data in making decisions about their water systems. The tool has not been designed to link to water quality linked to GEMA’s specific TAPs, nor to the status of, and changes in, land-use in recharge area. EQ 8B: How has the information gathered, and support provided, influenced decision-making and improved the management of governance and water quality data systems? PMCCA does not collect biophysical data on the Level-1 micro-watersheds that would be useful for making decisions about watershed governance and management. KIIs revealed that water samples and measurements of water flow are only at the JAAs’ water intake structures, not from other sites within the Level-1 micro-watersheds. The PMCCA approach does not require sampling and measurement of the biophysical characteristics and the land uses of areas above the water intake or the sample water’s Biological Oxygen Demand (BOD) or Chemical Oxygen Demand (COD). These are the two water parameters that most closely correlate with contamination due to inappropriate land use. GEMA KIs stated that the high expense of measuring BOD and COD prohibited their systematic measurement via PMCCA systems. On the assumption that access to data on these parameters could be increased, informants proposed that greater attention could then be given to water treatment for the control of pathogens; financed by local contributions by water consumers. Figure 19 illustrates the key locations and relationships between a typical Honduran PA, its surrounding buffer zone, a Level 1 watershed (colored orange) and associated water intake, and water distribution networks. Figure 19: Relationships between communities, watersheds, sources and Protected Areas Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 37 A USAID-financed study by the Honduran Institute for Earth Sciences (IHCIT-UNAH 2018;) that evaluated 152 protocols for measuring water quality/quantity, including measuring the effects of land use on variables such as soil erosion and infiltration, found that Honduran institutions are implementing these protocols. However, based upon the review of the PMCCA implementation, it does not appear to have used the content of this document. The PMCCA is planning to analyze waste water from coffee processing as a means to make decisions about controlling this source of water pollution. KIs agreed that during the coffee harvest season (November to March) the greatest sources of water contamination in many of WAs is the waste water from processing coffee (aguas mieles). According to the Marcala-based NGO Agua y Desarrollo (ADEC) analyzing this waste water requires specialized laboratory equipment. GEMA’s 2018 budget includes funds for purchasing such equipment, that will be donated to ADEC, which will be able to provide JAAs with the service of testing potable water, and to analyze coffee production waste water that would enable coffee organizations to obtain some certifications. GEMA intends for the PMCCA to strengthen inter-institutional collaboration for support of the JAAs. GEMA documents and KIIs with partners noted that most of the JAAs in its target micro-watersheds will remain weak when GEMA ends. KIs indicate that amongst many pressing needs, JAAs require support in obtaining and maintaining their legal status; in preparing, implementing and reporting on their Annual Plans; in structuring water use fees; in managing conflicts and preventing adverse land-use around recharge areas; and ensuring financial accountability. As part of its response, GEMA intends to use the PMCCA to strengthen institutional collaboration and coordination between the Ministry of Health, the ICF, municipalities and NGOs in support of the JAAs, after GEMA ends. This support can overcome JAA limitations in capabilities, training needs, resources and funds, however a common issue is the ineffectiveness of counterpart support provided by local authorities. FGDs observed consistently that community members are frequently dissatisfied with Municipal support, complaining of the politization in decision-making and information-sharing. Although there are many cases of robust Municipality-JAA collaboration, overall the strengthening of UMA to improve their capabilities requires GEMA attention. GEMA is beginning to influence decision-making, governance or water quality data systems. The findings indicate that JAA training, MCSEs and the PMCCA are GEMA’s main methods for ensuring water data is available and made relevant to improved decision-making. All these instruments are incipient; therefore, their influence on decision-making is still in its early stages. Nonetheless, stakeholders have shared water quality laboratory test results with members of the JAAs. The findings do not indicate that there are data to measure the influence that these discussions have had on the decisions that the JAA members have taken. EQ 8C: Could PMCCA actions be improved to yield more useful, higher quality data and to strengthen local capacities more effectively? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 38 PMCCA actions could be improved in several ways: The data PMCCA collects would be improved by: (i) including data about the biophysical characteristics of land uses above the water intakes; (ii) ensuring water sampling and laboratory testing adheres to a consistent quality control protocol for the entire PMCCA; (iii) modeling the hydrology of some of the officially declared protected micro-watersheds, which could be shared via the Honduras Water Platform10 to provide data useful for the PMCCA; and (iv) promoting the use of simple water quality testing kits that, on a pilot scale, could be provided to JAAs to encourage clearer understanding of water quality parameters and of the relationships between quality levels and contamination sources (KIs; USAID informants; specialist assessment). These options should be developed and assessed for feasibility. 3.8.2 CONCLUSIONS While GEMA does not measure its TAPs’ effects on water quality, GEMA is beginning to influence decision-making, governance and water quality data systems. The PMCCA should be focused upon the capabilities of the JAAs to interpret water quality and quantity data, and to increase their comprehension of the links between watershed management and water quality. The PMCCA should be integrated with the work underway to train JAAs in administration, obtaining legal status, delimiting recharge areas, and establishing MCSEs. More attention should be given to water treatment for the control of pathogens; financed if possible, by local contributions by water consumers. Both MCSEs and water quality measurements require that most of the financing needed to sustain these, must come from increases in the Water Use Fees that are levied by JAAs on communities. If JAAs are to persuade water users to finance water quality sampling and analyses, PMCCA actions should be improved in the following ways: (i) including data about the biophysical characteristics of land uses above the water intakes; (ii) ensuring sample collection methods adhere to a quality protocols, and providing related training to JAAs on quality parameters and their implications for land use practices; (iii) modeling the hydrology of declared watersheds; (iv) and piloting water quality testing kits. 4 RECOMMENDATIONS The PE acknowledges that GEMA’s TAPs have advanced considerably: significant financial and human resources investments have been made and there has been considerable progress. Even so, at project conclusion in 2020, while GEMA may meet most of its quantitative performance indicator targets, GEMA may not yet have achieved the full set of expected outcomes, and the prospects for the sustainability of GEMA’s impacts is likely to still be uncertain. If GEMA is to demonstrate an unambiguous, measurable array of outcomes as the basis for continued support for NRM and protection of biodiversity in Western Honduras, our overall recommendation is for GEMA to re-focus its activities to achieve measurable, sustainable, effective governance of water resources in and around a more tightly limited set of watersheds. If GEMA achieves this outcome, the Activity will have established the essential basis for protecting water, soil and vegetation over a geographically-significant area, and will contribute substantially to both conserving biodiversity and reducing poverty in Western Honduras. EQ 1: Conservation Enterprises R1. Prioritize and focus GEMA’s assistance on agricultural and non-agricultural CEs located within and around its target micro-watersheds. As improving the operations of CEs in semi-urban contributes relatively little to the outcomes of the conservation of protected watersheds and given that one of GEMA’s key aims is to conserve water quality and quantity, investing a greater proportion of TAP implementation within these areas would help achieve that outcome. R2. Focus the renewable energy TAP strategically on the provision of RE systems to households that are located geographically near to PAs core zones and within communities that receive water 10 USAID/Honduras, under an activity different from GEMA, financed the construction of the Water Platform which hosts Western Honduras geo-spatial and water quality measurement information. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 39 from the target micro-watersheds. At the same time, in liaison with improved eco-stove and oven supplier institutions and collaborating municipalities, GEMA should contract a baseline study on firewood sources and on consumption changes due to RE adoption. R3. Focus GEMA’s assistance to agricultural enterprises within and around its target micro￾watersheds. The expansion of coffee cultivation into the protected areas is the greatest threat to the biodiversity within GEMA’s Area of Influence. By focusing GEMA’s assistance for improving coffee production techniques, GEMA therefore will provide the largest benefits for conserving biodiversity. R4. In liaison with Service Providers, conduct a study to assess the profitability and sustainability of both agricultural conservation enterprises and non-agricultural MIPYMEs, to thereby enable more effective targeting of GEMA resources on the enterprises that are most likely to succeed. EQ 2: Resilience R5. Ensure technical coherence in training content between information about current short-term risks from known hazards and the less-immediate, slower-onset, longer-term risks, stemming from climate-change impacts. In addition, GEMA should refine the theoretical foundations of its approach to resilience by defining terms more coherently and applying a more precise definition to its TAPs, to be able to foster greater awareness of the dimensions of resilience. This would enable the definition of clear selection criteria for CODELs and updated training content, so the benefits of this TAP is maximized. R6. Ensure that training support to CODELs is matched with rapid follow-up for development of resilience-building plans, and the physical equipping of CODELs with DRM response assets that are indicated as needed within CODEL plans. This recommendation is essential to ensuring CODELs can practically carry out the activities as proposed to them during the training processes. R7. Support the improved local governance of DRM and resilience by developing GEMA actions that specifically support CODELs in their interactions with key local and national institutions, especially to strengthen inter- and intra-municipal relationships between each CODEL with CODEMs, UMAs and Mancommunities (where these exist and are effective), and as well with COPECO. R8. Appraise the capabilities of CUROC to manage CICOH and investigate the opportunity to provide more training in the use of Meteorological Climatology and Hydrological Information System. The institutions that now have the MCH installed may need more training for their staff members in its use and therefore GEMA should evaluate needs for more training in MCH and provide it if possible. EQ 3: Gender R9. Engage Service Provider staff in updated gender training, to enable these organizations to apply and monitor effective practices that foster high-quality gender participation in their contracted activities. This will enable GEMA’s partners to foster and to monitor gender issues, establish M&E reporting mechanisms and encourage adaptive learning on gender results. At the same time, engage partner Municipalities in actions that lead to updated municipal gender policies and support from Municipal Women’s Offices for GEMA’s TAPs. Assess whether a series of one-day advanced training modules could be developed for GEMA’s own staff for quarterly repeated events that would serve to review progress as well as consolidate and apply innovation in GEMA’s gender approaches. R10. Increase the scope and quality of women’s participation in both JAAs and CODELs, by providing detailed training in: (i) legal and administrative aspects of operating a CODEL/ JAA; and (ii) technical aspects of watershed management, DRM and MCSEs to potential female leaders. This recommendation could lead to review Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 40 of land ownership requirements that inhibit female participation, and the reform of regulations for the participation of women in JAAs and might establish a timeline by which a certain percentage of JAA members should include women in leading roles, especially younger cohorts. R11. Generate updated training content specially focused upon women’s roles in both agricultural and non-agricultural CEs, including how to improve product packaging, shifts to more efficient modern equipment, overcoming marketing bottlenecks, understanding climate change adaptation in relation to CEs, financial management, how to access credit and approaches for boosting their entrepreneurial capabilities. EQ 4: GEMA Resources R12. With USAID, agree on priorities for the remaining time period, using findings from this evaluation and other USAID inputs, to refocus efforts and apply finance to those priorities. As GEMA is only half-way through implementation and its outcomes require extensive time to mature, GEMA’s plans should be refocused. An assessment of its annual planning and TAPs should identify those actions which have enough time/funding, and available service provider assistance; and those that may not. Based on analysis of which outputs are realistically achievable, GEMA should consider TAPs should be not implemented, and reconsider its geographical area priorities. EQ 5: Integration R13. Integrate GEMA’s actions by focusing on more limited land-area units such as targeted watersheds. Within and around these, achieve effective conservation enterprises, DRR, NRM, MCSE schemes; and improved water governance. Establish and test a structured ToC and prepare a new RF that outlines outcomes and TAP contributions, to establish the basis for achieving sustainable access to quality water resources in the targeted areas via: (i) poverty alleviation through increased participation of women in decision￾making, diversification of livelihoods, and promotion of improved practices; (ii) improved governance and management of water resources; (iii) conservation of biodiversity; and (iv) resilience of communities. R14. Closely integrate GEMA’s gender specialists to act as advisors to the technical results area teams to address gender equity in all their work, by investing resources in periodic staff gender training and by generating monitoring tools and establishing specific requirements for service providers to ensure that they implement gender approaches. This will enable GEMA to monitor Service Provider contracts and for them to apply greater gender awareness. EQ 6: Sustainability and Governance R15. Increase the effectiveness of MCSE schemes in GEMA’s governance strategy by providing structured support to local stakeholders to ensure effective implementation. GEMA should monitor progress in MCSE implementation, develop guidelines for roles of institutions involved, and define their tasks and expectations. R16. Ensure the adoption and application of GOAL’s governance measurement tools to help contribute to a greater understanding of the meaning and importance of water governance among key stakeholders. Capacity-building on the tool for agencies such as Mancommunities and PA co-managers should raise awareness concerning roles within the watershed institutional fabric, to ensure that governance is applied in practical terms. EQ 7: Constraints and Opportunities R17. USAID should examine the feasibility of a program extension to enable GEMA’s current activities to yield significant outcomes. As of September 2018, only 18 months remain for GEMA’s field Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 41 implementation. Significant investments have been made for project start-up, and many field activities are only now beginning and within an 18-month timeframe, it is unlikely to produce GEMA’s stated outcomes. R18. USAID should begin post-GEMA planning to ensure that an effective Sustainability and Exit Strategy is in place at least 12 months before GEMA’s confirmed close-down date. The strategy should propose actions by stakeholders that address the continued support needs for the sustainability of end-of-activity outcomes. EQ 8: Water Quality R19. Focus service provision on a coherent and comprehensive suite of actions that enable JAAs to increase not only their administrative capabilities and legal status, but ensure that GEMA’s actions in the field deliver integrated support for: (i) compliance with ICF requirements for PA Micro-Watershed Declarations; (ii) developing simple management plans with community consent; (iii) developing charges applied to water users; and obtaining finance and equipment to implement MCSEs and management plans; (iv) physical delimitation and fencing of recharge areas; and (v) MCSE via purchase of land in the recharge area, reforestation, forest surveillance and land use inspections. R20. Seek to enhance the impact of the PMCCA actions by: (i) modifying the PMCCA so that it collects data about the biophysical characteristics of land-uses around and above the water intakes of JAAs; (ii) designing and implementing a water quality control protocol for the data PMCCA collects and related training; (iii) ensuring that contracted water analysis laboratories (such as ADEC) can measure more complex variables, such as BOD, COD and agrochemical contamination; and (iv) promoting simple water quality testing kits that, first on a pilot scale, are provided to JAAs to foment a clearer understanding of water quality parameters and of the relationships between quality levels and contamination sources. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 42 BIBLIOGRAPHY ANED Consultores. 2018. Reporte Final, Levantamiento de la Encuesta de Evaluación Proyecto i-APS, 64 pp. + índices, Tegucigalpa ANED. 2018. Informe Gira de Reconocimiento Microcuencas en Muestra, 22 pp. ANED, Tegucigalpa Business Dictionary. Accessed Nov 1, 2018. http://www.businessdictionary.com/definition/indicator.html CDE (Centro de Desarrollo Empresarial). 2016. Diagnósticos y Línea de Base de PYMES, XX pp, Tegucigalpa CDEs de Santa Rosa, Santa Barbara y La Esperanza, octubre, 2018. Informes trimestrales. CDM (Centro de Derechos de Mujeres) 2014. Special Rapporteur on Violence Against Women, its Causes and Consequences, 10 pp. Honduras Center for Theory of Change. Accessed 11/2018. https://www.theoryofchange.org/what-is-theory-of-change CEPAL. 2018. Informe nacional de monitoreo de la eficiencia energética de Honduras, 2018. Ciudad de México. 78 p. DAI Global, LLC (Development Alternatives Global). 2018. Biodiversity Integration in Practice – A case Study in Western Honduras, 15 pp. Washington, D.C. Drigo Rudi, Robert Bailis, Adrian Ghilardi and Oma Masera. 2015. Analysis of Fuelwood Supply, Demand and Sustainability in Honduras, 12 pp., Yale-UNAM, Project Geospatial Analysis and Modeling of Non-Renewable Biomass: WISDOM and beyond Research Grant provided by the Global Alliance for Clarn Cookstoves EPICOLECT5. Accessed November 28, 2018. http://www.epicollect.net/ Ente Regulador de los Servicio de Agua Potable y Saneamiento. 2016. Reglamento de Juntas de Administradores de Agua, 32 pp., Tegucigalpa Excel files with water quality and quantity monitoring data Garay, Luigi. 2017. La Masculinidad y la Corresponsabilidad, 31 pp., GEMA, Tegucigalpa GEMA (Gobernanza en Ecosistemas, Medios de Vida y Agua). 2017a. Monitoring, Evaluation and Learning Plan, 64 pp. Tegucigalpa GEMA (Gobernanza en Ecosistemas, Medios de Vida y Agua). 2018h. Plan de Mitigación y Monitoreo Ambiental, Proyectos para la Protección de Microcuencas y Áreas de Importancia Biológica para la Conservación en el Occidente de Honduras, 81 pp., Tegucigalpa GEMA (Gobernanza en Ecosistemas, Medios de Vida y Agua). 2017b. Análisis Rápido de los Ecosistemas Criticas, GEMA, 2018. Once Planes de Conservación GEMA, 2018. Estudio de Gobernanza Hídrica, Microcuencas el Chiflador y Guaralape, Municipios de Marcala y Santa Ana, GEMA, 2018a. Second Annual Report and Quarterly Report 9, 109 pp. + Annexes, Tegucigalpa GEMA, 2018b. Plan de Mitigación – Proyectos para la Protección de Microcuencas y Áreas de Importancia Biológica para la Conservación en el Occidente de Honduras Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 43 GEMA, Plan de Mitigación y Monitoreo Ambiental - Programa de emprendimiento en artesanías de fibra natural en Santa Bárbara, 52 pp, Tegucigalpa GEMA, 2018e. Expected Results of the PMCCA 2018. 2 p., Tegucigalpa GEMA, 2018e. Plan de Mitigación y Monitoreo Ambiental, Conservación de Biodiversidad de la Calidad del Agua Mediante la Producción de Abonos Orgánicos a Partir de Pulpa y Aguas Mieles en Café, 69 pp, Tegucigalpa GEMA, 2018f. Power Point Presentation: Strategy Proposal for the Development of Water Quality Monitoring Programs. 24 p., Tegucigalpa GEMA, 2018g. Elementos Estratégicos para Formular Programas de Monitoreo de Agua, 23 pp., Tegucigalpa GEMA, no date (b). Estudio de la Cadena de Valor de Teja y Ladrillo de Barro, 66 pp., Tegucigalpa GEMA, 2017- 2018, Líneas de base y diagnósticos de las empresas de conservación agrícolas y de las MIPYMES. Cuadros Excel. Tegucigalpa, Honduras. GEMA, No date (a). Calidad de Agua – Gotitas Educativas, 22 pp. Tegucigalpa GEMA, 2017. Estrategia de Genero e Inclusión Social, 37 pp., GEMA, Tegucigalpa GEMA, No date. Programa Monitoreo Calidad y Cantidad de Agua - Microcuenca Anillo Verde, Tegucigalpa, 42 pp. GEMA, 2017. Guion Metodológico para formación en Masculinidad, 11 pp. GEMA, Tegucigalpa GEMA, 2017. Herramientas para la Equidad de Género, 25 pp. GEMA, Tegucigalpa GEMA, 2017. Lineamientos claves para la transversalización de género, 11 pp. GEMA, Tegucigalpa GEMA, 2018h. Género en la Gestión de Agua, 9 pp. + Anexo, GEMA, Tegucigalpa GEMA, 2018i. Género, Reducción de Riesgos de Desastres y Variabilidad Climática, 18 pp. GEMA, Tegucigalpa GOAL 2018. Estudios de cadenas de valor de Fibras naturales y Teja de arcilla y ladrillo de barro. Tegucigalpa, Honduras. GEMA 2016 Informe Consolidado de los Resultados del Estudio de Línea Base Sobre Resiliencia Comunitaria Gobierno Nacional de Honduras, 2015. Estrategia Nacional para la Producción y Uso Sostenible de Lena y Carbón Vegetal en Honduras 2015– 2022. 36 pp. Tegucigalpa Houdreta, Annabell, Ines Dombrowskya, Lena Horlemannb. 2013. Accessed November 23, 2018. The institutionalization of River Basin Management as politics of scale – Insights from Mongolia IN Journal of Hydrology, Honduran National University of Honduras Honduran Institute of Earth Sciences, 2018. State of the art methodologies for the estimation of water balances, including good practices. 87 pp., Tegucigalpa Honduran National University of Honduras Honduran Institute of Earth Sciences, 2018. Estado del Arte de las Metodologías para la Estimación de Balances Hídricos, incluyendo las Buenas Prácticas, 82 pp. Tegucigalpa Mancomunidad de Municipios del Parque Nacional Montaña de Celaque (MAPANCE), 2018. Elementos Estratégicos para Formular Programas de Monitoreo de Agua, 26 pp., Tegucigalpa MAPANCE, 2018. Sistema de Monitoreo de Microcuencas para la Región de la Mancomunidad de MAPANCE, 22 pp., Tegucigalpa Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 44 Mazier, Dilcia 2017. Género y Liderazgo, 28 pp., GEMA, Tegucigalpa Mazier, Dilcia. 2017. Guion Metodológico para formación en Género y Liderazgo, 7 pp. GEMA, Tegucigalpa Parker, John et al. 2014. Vulnerability and Resilience to Climate Change in Western Honduras, 108 pp., USAID/Honduras, Tegucigalpa United Nations, 2015. Marco de Sendai para la Reducción del Riesgo de Desastres 2015-2030, 40 pp, Japan MESCLA, 2017. GEMA’s Theory of Change by Focus Areas, 6 pp., Tegucigalpa MESCLA, 2017. Request for Proposals, Performance and Impact Evaluations of GEMA, Annex C, Statement of Work Patton, Michael Quinn. 1990. Qualitative Evaluation and Research Methods, Second Edition, Sage Publications,506. Pp. Newberry Park, London, New Delhi Rodas, José G. Flores and M A. Meijer. 2015. Evaluación del Sector Forestal de Honduras, 30 pp., Tegucigalpa Suazo, Laura. 2018. Participación de Mujeres en el Uso, Manejo y Conservación de la Biodiversidad y Gobernanza del Agua en el Occidente de Honduras, 56 pp., Zamorano, Honduras TecnoServe. Accessed Nov 20, 2018. The MAS Project: Sustainable Agricultural Improvement www.technoserve.org/our-work/projects/sustainable-agricultural-improvement. USAID, Vision for Ending Extreme Poverty. Accessed Nov. 2018. https://www.usaid.gov/ending-extreme-poverty USAID, 2017. Building a Conservation Enterprise – Keys for Success, Washington, D.C. 26 pp. United States Department of Agriculture, Accessed November 20, 2018. Gain Report, Honduras Coffee Annual. https://gain.fas.usda.gov/Recent%20GAIN%20Publications/Coffee%20Annual_Tegucigalpa_Honduras_5-24-2017.pdf Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 45 APPENDICES APPENDIX A: STATEMENT OF WORK FOR THE GEMA PE & IE Background: Dexis Consulting Group (Dexis) has been contracted by the United States Agency for International Development (USAID) to provide management and technical services for the implementation of USAID/Honduras’ Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) activity aimed at supporting and strengthening monitoring and evaluation of USAID financed development assistance in Honduras. MESCLA also supports the Mission measure the effectiveness of its efforts to promote gender equality, support vulnerable populations, reduce migration flows, and prevent violence through its place-based strategy for violence prevention. In 2011, USAID/Honduras contracted DAI to implement its ProParque activity, which worked with government stakeholders to strengthen the nation’s protected area management system and promote rural enterprise growth, climate change adaptation, disaster risk reduction, and clean energy development. To build upon these accomplishments, USAID/Honduras has designed the Gobernanza en Ecosistemas, Medios de Vida, y Agua, or Governance in Ecosystems, Livelihoods, and Water (GEMA) project as a follow-on activity to scale up and replicate the successes of ProParque. GEMA is working in Western Honduras to further improve natural conservation policies; develop opportunities for inclusive, environmentally sustainable economic growth; and promote a more climate-informed and resilient civil society in Honduras. SOW Objective: The objective of this SOW is to complete data collection and analysis for a PE and an IE (baseline and mid-line measurements) of the GEMA Activity. The purpose of these evaluations is to provide rigorous evidence on whether and how the expected results of the GEMA Activity have been achieved and provide insights on how to improve its effectiveness. The evaluations will answer the EQs put forth by USAID. Scope of Work: Impact Evaluation The IE will provide statistically significant evidence of the impact of the GEMA Activity, comparing treatment and non￾treatment areas, in particular as related to the impact of GEMA’s Technical Activity Packages. To answer the EQs, the subcontractor will be required to design the evaluation, complete briefings on the proposed evaluation plan, and provide MESCLA with reports as described in the Deliverables section. As part of designing the evaluation, the Subcontractor will submit the evaluation methodology and workplan for approval by MESCLA. The Subcontractor will implement the baseline impact evaluation by obtaining and analyzing existing remote sensing data for land cover classification and conduct select ground-truthing of that data. For the end-line impact evaluation the Subcontractor will (1) acquire and process a combination of free and accessible 2020 European Space Agency (ESA) Sentinel 1 radar data, optical ESA Sentinel 2 data, optical NASA Landsat data and MODIS data; (2) make a best-case comparison of land use and vegetation in the sample watershed in 2010, 2014, and 2020. Impact EQs 1. What is the impact of GEMA’s TAPs on the rate of change in forest cover in its micro watersheds? 2. What is the impact of GEMA’s TAPs on the rate of change in land cover types in GEMA’s micro watersheds: (1) within 50 meters of water bodies’ borders; (2) within 250 meters around Juntas de Agua water sources, and (3) within and outside of protected areas? Scope of Work: Performance Evaluation The PEs purpose is to provide quantitative and qualitative information that to answer eight PE Learning Questions (detailed below). The PE assesses GEMA's work in a number of thematic areas, such as water quality and governance, livelihoods, resilience and risk mitigation, as well as crosscutting issues such as gender. To answer all eight Evaluation Learning Questions, the Subcontractor will be required to design the evaluation, complete briefings on the proposed evaluation plan, and provide MESCLA with reports as described in the Deliverables section. The Subcontractor will need to submit the evaluation methodology and workplan for approval by MESCLA, and then implement the evaluation according to the Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 46 approved methodology. The methodology will be based upon the Subcontractor’s Technical Proposal and is anticipated to include the following data collection components, among others: • Baseline and end-term surveys • Key informant interviews and focus group discussions • Geographic analysis • Reports and document review • Water sampling and testing • Remote sensing • Baseline diagnostic assessment of micro-, small- and medium enterprises • Creekside vegetation sampling Performance EQs: 1) Are the conservation enterprises GEMA has supported generating benefits to households within the GEMA area of influence? If so, what are the types and magnitude of benefits to households and to biodiversity? How sustainable from a business perspective is this practice? Why have conservation enterprises been able to generate those benefits? 2) Have GEMA’s actions to reduce threats to ecosystems, diversify livelihoods, improve capacity for risk mitigation and disaster response, contributed to building resilience of its target population when measured with GOAL’s6 three resilience-measuring tools? If so, what is the magnitude of its contribution and why has it been able to make this contribution? 3) Has GEMA successfully increased the participation of women in decision-making positions in water governance and MCSE schemes? If so, how and to what extent? Has GEMA successfully increased the economic empowerment of its target women population? If so, how and to what extent? Have GEMA actions strengthened the participation of women in Disaster Relief Management. If so, how, and to what extent? 4) Which of GEMA’s actions need more time, technical assistance, training or funding to produce its target outcomes? What additional resources, if any, would be required produce the target outcomes? 5) Integration: a. Have GEMA’s different actions been integrated effectively with each other? b. Have they been integrated with USAID’s other activities that have been or are being implemented in GEMA’s area of influence? If so how? c. Have the two types of integration contributed to the target outcomes related ecosystems and livelihoods? If so, in what way? 6) Has GEMA produced sustainable environmental governance, watershed management, conservation enterprises and adaptation to climate change? If so, to what extent and how? 7) Based on its performance indicators, what constraints and opportunities have affected GEMA’s progress towards achieving its planned outcomes 8) What is the effect of GEMA’s TAPs on water quality within its targeted micro watersheds? What explains the outcome? What is the correlation between rate of change by land cover types and water quality? Tasks & Responsibilities: For the PE and IE of the GEMA Activity, the Subcontractor will: • Work with USAID/Honduras and MESCLA to finalize the GEMA IE and PE evaluation designs and timelines, including data collection and analysis methods • Develop and review data collection instruments and data analysis tools for all data collection efforts • Travel to Honduras to oversee data collection phases, at a minimum; additional travel as appropriate • Conduct data analysis • Hold briefings on each evaluation in accordance with the approved evaluation methodology/workplan • For both the PE and IE, develop the draft baseline evaluation reports and submit for comments • Complete a final end-term evaluation report for both the PE and IE • Complete other tasks related to the IE and PE as discussed with MESCLA technical team and USAID/Honduras, if applicable Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 47 APPENDIX B: TYPE AND STATUS OF GEMA PERFORMANCE INDICATORS (END SEPT. 2018) N° TYPE OF INDICATOR INDICATOR BY YR. 2 LOP TARGET ACTUAL DIFF. LOP TARGET DIFF. NEEDS TO MEET PI QUANTITATIVE TARGETS Result 1 – Reduced Threats in Areas of Biological Significance and/or Natural Resources to Conserve Biodiversity and Protect Water Delivery 1.1 PPRI Result Progress Indirect No causal link Number of hectares of biological significance areas showing improved biophysical conditions as a result of USG assistance. 50,000 48,400 or 112,179 -1,600 150,000 - 101,600 -Clarification. Second Annual Report gives two different areas for this PI through Yr. 2. Need to clarify. -Clarification. Second Annual Report states the target for Yr. 2 is 40,000 but GEMA KI states USAID did not accept this target and kept it at 50,000. Need to clarify. -Clarification. If the target is 50,000, then 1,600 is so close to the target as to be an insignificant difference, given that the method of measuring PI 1.1 is subject to a large error 1.2 PPRI Outcome Progress Indirect No causal link Number of hectares of biologically significant areas under improved natural resource management as a result of USG assistance. 100,000 112,182 + 12,182 240,000 - 127.818 ACTUAL EXCEEDS TARGET – NO NEEDS Sub Result 1.1 – Strengthened Local Institutions to Manage Natural Resources 1.1.1 PPRI Input Absolute Direct Causal link Number of people trained in sustainable natural resources management and/or biodiversity conservation as a result of USG assistance 3,750 2,290 -1.460 8,000 -5,710 NEEDS: -Training Currently, GEMA provides training only to the 7 members of the board of the JAAs, not to the water users. GEMA will begin to give training to the participants in the JAA assemblies rather than only to JAA board members. The average attendance at the JAA assemblies is about 50 people. If GEMA give a three-hour course for groups of 50 people, it will be able to train many more people more quickly than if it gives training only to the JAA board members. Such training will also increase the percentage & absolute number of women GEMA trains. 1.1.2 PPRI Outcome Progress Indirect Causal link Number of institutions with improved capacity to assess or address climate change risks supported by USG assistance 325 457 +132 700 - 243 EXCEEDS TARGET – NO NEEDS Sub Result 1.2 – Ecosystem Services Compensation Mechanism Implemented 1.2.1 Custom Input Progress Number of micro-watersheds in process of declaration. 25 23 -2 100 NEEDS -Time. Minus 2 is insignificant & KII data indicates that GEMA will meet the 25 targets by end of calendar year 2018. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 48 N° TYPE OF INDICATOR INDICATOR BY YR. 2 LOP Direct Causal link -USAID support. Criteria is six steps, but Step 6 is “submission completed and on file with ICF”. Although perhaps practical, given how slowly ICF operates, it is not a satisfactory endpoint for the PI, since ICF might never approve the application. Need to get ICF to approve the applications & need to add Step 7, ICF approval of submissions. 1.2.2 Custom Outcome Absolute Indirect Causal link Number of ecosystem services compensation mechanisms operational as a result of USG assistance 250 100 -150 450 -350 NEEDS - Clarification. Opalaca and Montana Verde are areas where people are mostly Lempa. It is difficult to establish MCSE with them & there are fewer communities. Also, these areas do not have co-managers as the other areas do. To achieve the PI in these areas would require more time, field staff, training courses and the funding for them. -USAID support. Current procedures require 13 steps and presentation of complicated technical reports by licensed foresters and topographers. The reports are time-consuming & expensive. GEMA is trying to simplify the procedures. ICF staff is resisting simplification, even though its director has asked the director of USAID/Honduras for assistance Although MCSE schemes may remain operational with GEMA support, they may not be robust enough by GEMA’s end to continue subsequently. It may take many years of support for them to become sustainable. - Time. For watershed management to occur through MCSE, JAAs will need external support for a long time. GEMA is trying to organize such support though associations of JAAs, municipalities (UMAs) and/or Mancomunidades. GEMA, however, probably will be able to provide for adequate support to the JAAs by the time it ends. Sub Result 1.3 – Renewable and Clean Energy adopted 1.3.1 Custom Output Progress Direct Weak causal link Number of households with improved access to renewable or clean energy due to USG assistance 3,000 1,931 -1,069 6,000 -4,069 Reduce target number of households 1.3.2 Custom Output Absolute Direct Weak causal link Number of conservation enterprises adopting clean/renewable energy technologies and/or best practices 500 337 -163 1,000 -663 Reduce target number of enterprises Result 2 – Increased Conservation-Related Income-Generating Activities Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 49 N° TYPE OF INDICATOR INDICATOR BY YR. 2 LOP 2.1 PPRI Output Absolute Indirect Weak causal link New net sales of participating conservation enterprises as a result of USG assistance. 3.0 15.2 +12.2 10 +5.2 NEEDS -Clarification. Second Annual Report reports GEMA support to tourism but GEMA KIs said that GEMA does not work in tourism -Clarification. Second Annual Report reports Due to low prices agricultural products can no longer contribute to increasing the dollar value of sales of agricultural products, including coffee, avocado, sugar cane and cattle; Niche business are micro enterprises that operate on a very small, local, sometimes intermittent basis. They are unlikely to generate higher sales They are family enterprises. The target amount of $10 million was predicted and established in the DAI contract with USAID/Honduras based on projections based on data of sales that ProParque was able to achieve. ProParque, however, operated when coffee prices were high and ProParque included tourism, an activity that can increase sales and produce jobs. Coffee prices are now lower and GEMA was not permitted to included tourism as one of its economic growth activities; Higher sales would result from: productivity, more area, higher prices 2.2 PPRI Output Absolute Indirect Weak causal link New employment created in participating rural conservation enterprises (full time equivalents - FTEs) as a result of USG assistance 1,250 3,593 +2,343 3,000 -593 2.3 PPRI Input Absolute Indirect No causal link Percentage of female participants in USG- assisted programs designed to increase access to productive economic resources 20 44 +22 35 +9 ACTUAL EXCEEDS TARGET – NO NEEDS 2.4 Custom Result Absolute Direct Weak causal link Number of households with new income from non￾traditional, non-agricultural or niche livelihoods 1,600 782 -818 3,850 -2,182 NEEDS Clarification. Most rural enterprises are family owned and run & are very small. The aim should be to keep them existence rather than to expand them. Their markets are very local and small-scale, so it is unlikely that they will produce new income. Reasonable PI is to maintain current income. 2.5 PPRI Result Progress Indirect Weak causal link Number of people with improved economic benefits derived from sustainable natural resource management and/or biodiversity conservation as a result of USG assistance 55,000 65,690 +10,690 120,000 -54,310 ACTUAL EXCEEDS TARGET – NO NEEDS Sub Result 2.1 - Business Alliances and Partnerships Created /PPPs 2.1.1 Custom Output Absolute No causal link Number of public-private partnerships formed as a result of USG assistance 15 33 +18 35 -2 ACTUAL EXCEEDS TARGET – NO NEEDS Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 50 N° TYPE OF INDICATOR INDICATOR BY YR. 2 LOP 2.1.2 Custom Input Absolute Indirect Weak causal link Value private sector investment leveraged for conservation US$ millions 0.250.0 0.217 - 0.033 3.00 -2.967 NEEDS -Clarification. Although 2.1.1 exceeds target, the private￾public alliances involve small-scale enterprises. There are no opportunities to create public-private alliances in which private enterprises would contribute large amounts of US$. GEMA is very unlikely, therefore, to be able to meet the target Result 3 – Increased Vulnerable Population's Capacity to Adapt to Climate Variability and Change 3.1 Custom Outcome Absolute Indirect Weak causal link Number of communities with adequate disaster capacity. 25 25 0 150 125 -Clarification. Second Annual Report states the PI target has been met. However, it also states that the equipment component has not been met. -Time. Degree of achievement of PI 3.1 measured by: (1) CODELS legal; (2) Plan of Adaptation; (3) CODEL members have received 5 training modules. (4) CODELs do not yet have appropriate equipment for emergency responses to threats. First three are on track. Provision of equipment has been delayed because its procurement is through subventions. 3.1.1 PPRI Output Indirect Weak ink Number of people using climate information or implementing risk-reducing actions to improve resilience to climate change 6,000 0 25,000 -25,000 Make the MCH functional 3.1.2 PPRI / Input Absolute Weak ink Number of people trained in climate change adaptation 2,250 1,885 -365 5,000 3,115 Accelerate training program in climate change adaptation 3.2 PPRI Output Absolute Direct Weak causal link Number of people supported by the USG to adapt to the effects of climate change 6,000 13,640 +7,640 15,000 -1,360 ACTUAL EXCEED TARGET – NO NEED Sub Result 3.1- CICOH Established 3.1.1 PPRI Outcome Progress Indirect Causal link Number of people using climate information or implementing risk-reducing actions to improve resilience to climate change as supported by USG assistance 6,000 0 -6,000 25,000 -25,000 3.1.2 PPRI Input / Absolute Direct Weak link Number of people trained in climate change adaptation supported by USG assistance 2,250 1,885 -365 5,000 -3,115 Cross-cutting Indicators Custom Input Number of persons trained with USG assistance to 2,250 3,263 +1,013 5,000 -1,737 ACTUAL EXCEEDS TARGET – NO NEEDS Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 51 N° TYPE OF INDICATOR INDICATOR BY YR. 2 LOP Absolute Direct Weak link advance outcomes consistent with gender equality 52 APPENDIX C: GEMA’S RESULTS FRAMEWORK DEVELOPMENT OBJECTIVE 2: Extreme poverty sustainable reduced for vulnerable population in Western Honduras Result I: Reduced Threats en Areas of Biological Significance and/or Natural Resources to Conserve Biodiversity and Protect Water Delivery Result III: Increased Vulnerable Population′ns Capacity to Adapt to Climatic Variability and Change Result II: Increased Conservation – Related Income – Generation Activities I R 1.1 Strengthened Local Institutions to Manage Natural Resources I R 1.3 Renewable Energy Adopted I R 1.2 Ecosystem Services Compensation Mechanisms (ESCMs) Implemented Has of biologically significant areas showing improved biophysical conditions Has of biologically significant are under improved Natural Resources Management 2.1 New net sales of participating CE 2.3.% female participants increasing access productive economics resources 2.2 New FTEs 2.4 Nº household with new income from ag, non-agricultural – niche livelihoods 2.5 Nº people with improved economic benefits derived from Natural Resources Management Sub R 2.1. Business Alliances and Partnerships Created/PPPs I R 3.1 Climate information System for Western Honduras Established 2.1.1 Number of public￾private partnerships formed as a result of USG assistance 2.1.2 Value private sector investment leveraged for conservation 3.1. Number of communities with adequate disaster capacity. 3.2. Number of people supported by the USG to adapt to the effects of climate change. Number of persons trained with USG assistance to advance outcomes consistent with gender equality or female empowerment through their roles in public or private sector institutions or organizations % of integration actions implemented between USAID activities 1.1.1Number of people trained in sustainable natural resources management and/or USG assistance biodiversity conservation as a result of USG assistance 1.1.2. Number of institutions with improved capacity to assess or address climate change risks supported by USG assistance 1.2.2 Number of ecosystem services compensation mechanisms operational as a result of USG assistance 1.2.1 Number of micro-watersheds in process of declaration. 1.3.2. Number of conservation enterprises adopting clean/renewabl e energy technologies and/or best practices 1.3.1. Number of households with improved access to renewable or clean energy due to USG assistance 3.1.2 Number of people trained in climate change adaptation supported by USG assistance 3.1.1 Number of people using climate information or implementing risk-reducing actions to improve resilience to climate change as supported by USG assistance 53 APPENDIX D: MAPS D1: GEMA’S AREA OF INFLUENCE WORK AREAS AND PRIORITY MICRO-WATERSHEDS 54 D2: LOCATION OF AGRICULTURAL AND NON-AGRICULTURAL ENTERPRISES 55 D3: GEMA’S WATER SAMPLING POINTS 56 D4: MICRO-WATERSHEDS IN PROCESS OF DECLARATION 57 D5: MAP OF PROTECTED AREAS WITHIN GEMA’S AREAS OF INFLUENCE 58 D6: THREATS TO BIODIVERSITY IN GEMA’S AREAS OF INFLUENCE 59 D7: LOCATION OF COFFEE PRODUCERS IN GEMA’S AREA OF INFLUENCE 60 APPENDIX E: METHODOLOGY SUPPLEMENT Data Collection i-APS established an evaluation team of subject matter experts in each of the key learning sectors, including resilience, gender, governance and sustainability, water quality, watershed management, micro-enterprises and renewable energy. The evaluation team developed and produced a full Inception Report that outlined the proposed evaluation approach and worked with GEMA and MESCLA to develop a comprehensive evaluation matrix to identify the key indicators to support the structuring of analysis and tools development. The evaluation team developed implemented a mixed evaluation methodology quantitative and qualitative data collection that included: • Extensive document review • Field observation • Beneficiary Household Survey • Beneficiary KII and FGDs • KIIs with GEMA Staff, GEMA Partners, local institutions Data Collection: Micro Watersheds To provide an overall geographical framework for both the Impact and PEs, 12 sample and 12 control watersheds were identified. Sample Level 1 Micro-Watersheds: The following criteria were applied to select twelve sample Level 1 micro-watersheds: • Inside of GEMA WAs with a concentration of GEMA activities; • Elevations at around 1800 meters above sea level; • Locations in or near the nucleus zone of a national protected area; • Representative distribution of sample micro watersheds across GEMA’s work Areas; • Existence of a significant number of Water Boards (Junta de Administrator de Agua) in the watershed. • Potentially some of the sample sites may have protected zones with Declaration of Micro-watershed status and associated management plans. The average area of the sample Level 1 micro-watersheds is 782 hectares. Control Level 1 Micro-Watersheds: Control micro-watersheds were selected that met similar criteria as used for the selection of the sample micro watersheds but without any GEMA activities. The following criteria were used to select twelve control Level 1 micro-watersheds: • No GEMA-financed activities; • Comparable in size to a matching sample Level 1 micro-watershed; • Located outside the GEMA work Areas but inside the GEMA AI; • Minimum elevation highest elevation of micro watersheds • Adjacent as feasible to a protected area if matched sample was also adjacent to a protected area; • If possible, with similar vegetation types as the sample, using the CIAT 10x10m digital map of land cover and land use Control watersheds could not be selected outside of the Area of Influence, because no land cover or land use data are available outside of the AI. In order to ensure the suitability of each sample location the i-APS evaluation team undertook field reconnaissance of each sample location and created a profile report for each sample detailing local actors and institutions and mapping key factors that could influence the outcome of our findings. 61 The initial approach was to take a random sample of that population and thus establish baselines for each of the populations that draw water from one of the 12 sample watersheds. However, the field reconnaissance showed that in several of the selected areas, a low proportion of the water user population was participating in GEMA activities. Pilot reconnaissance studies shown a limited knowledge about GEMA projects, and limited participation in these, amongst the water systems’ beneficiaries (abonados de las Juntas de Agua). As well, the original focus only on JAA-related households was a limitation, as these households were found to be unlikely to overlap with the other classes of TAPs. A sample population based only on these water users, would have resulted in a large percentage of null replies to queries on benefits planned or received from GEMA. This would have provided only a portrayal of general baseline socioeconomic conditions, and not a baseline of GEMA’s actions and participants. In addition, a further issue that arose during reconnaissance, was that the area of each micro-watershed, selected as described above, varied in size between 73 ha, to 2664 ha, with an average of 782ha. These areas varied greatly in the proportion of forest and agricultural cover, and in several cases, the watershed areas had few established communities within their limits. As well, in some cases GEMA participants and the communities served by the water sources were located outside the boundaries of the micro-watershed. As a result of these two factors, the decision was made to require as high as possible a proportion of GEMA beneficiaries in the sample, and this required extending the limits of the sample frame outside the confines of the 12 sample watersheds. Respondent households were therefore drawn as far as possible on GEMA’s Geographical Information System (GIS) layers and database lists of HH that are benefitting from GEMA TAPs, of the components that have a field presence such as eco-stoves, business support/clean production, JAA support, CODELs, and value chain support subcontracts. In summary: i-APS found it necessary to amplify the geographical range of the sample universe, to reach outside the administrative geographic boundaries of the micro-watershed, and applied the concept of the Social Watershed, which included communities supported by JAA, and the location of GEMA activities. i-APS used data about beneficiaries in each of the GEMA projects based on GIS layers and Excel service provider databases, and cartography from the last census implemented by National Institute of Statistics in 2013. Based on analysis of those data sets provided by GEMA, i-APS was able to provide the available information, on locations, informant types, and contact details, for arranging visits for the surveys, KIIs and FGDs. Based on these, the team prepared a set of Watershed Profile Spreadsheets, detailing the number of targeted households in each location, according to the information available from GEMA databases, GIS layers and the criteria specified. This will enable the return data collection period in 2020 to gather data effectively, based on the same geographical areas and key information sources. In 2020, the household survey will sample same or similar beneficiaries to see how their conditions have changed. Household Survey The HHS design has considered the elements to accomplish the PE objectives, to build a 2018 baseline, and to provide a foundation for comparisons in 2020. The survey assumes that the selected sites exhibit a diversity of characteristics that encompasses the heterogeneity of all sites and assume that represents them. 62 Preparation of the Household Survey: USAID/Honduras and MESCLA staff provided comments on a draft household survey. The i-APS gender and resilience experts worked with ANED staff to respond to these comments and adjust the language of the household survey to common Honduran usage. MESCLA, ANED and i-APS staff then collaboratively reviewed the household survey several times. ANED ran a field test of household survey. Based on the field-testing results, further adjustments were made to the household survey and the target population was changed from the lists of people directly related to Juntas de Administracion de Agua to instead target beneficiaries of GEMA activities within the area served by the JAAs.11 Reconnaissance of Sample Micro-watersheds: During September 2018, two teams each visited the 12 sample micro-watersheds to gather data about their JAAs, water systems, water-users, crops and road conditions that were necessary to design, plan and implement the household survey, and KIIs and FGDs that were to be carried out12. Field observations in the sample Level-1 micro-watersheds and areas where the populations that use the water from these watersheds that were recorded using the EpiCollect5 data collection tool.13 Selection of the Population to be Sampled: The universe of the sample population was the beneficiaries of GEMA activities among the population of those people who obtain water from the 12 treatment Level 1 micro watersheds which were selected for sampling for the IE. Considering the geographical dispersion of the sample, i-APS identified those communities, using a probability proportional to size (PPS) and systematic approach. Sampling Frame: The sample frame is made up of 12 micro-watersheds in the area of influence of the GEMA program, located in the departments of Comayagua, Intibucá, La Paz, Lempira, Santa Bárbara, Copán, and Ocotepeque. The sampling frame includes 7,563 beneficiaries (5,543 from the water system project and 2,018 from other projects). The distribution of beneficiaries and programs implemented vary across micro-watersheds. Table 1. Sampling Frame by Micro-Watershed MC Micro-Watershed GEMA JAA Support Other GEMA TAPs Total MC-1 Quebrada Güergüerence 1,011 149 1,160 87.2% 12.8% MC-2 Río Hondo 515 45 560 92% 8% MC-3 Quebrada Palania 1,275 13 1,288 99% 1% MC-4 Quebraditas 460 46 506 91% 9.1% MC-5 Río del Coyol-Suptal 196 383 579 33.9% 66.2% MC-6 Río Conchagual 47 179 226 20.8% 79.2% MC-7 La Florida/El Naranjo 240 119 359 67% 33.2% MC-8 Río Urungo 72 91 163 44% 55.8% MC-9 Río Sirima 780 226 1,006 78% 22.5% 11 ANED Consultores. 2018. Levantamiento de la Encuesta de Evaluación Proyecto IAPS, 64 pp. + índices, Tegucigalpa 12 ANED. 2018. Informe Gira de Reconocimiento Microcuencas en Muestra, 22 pp. ANED, Tegucigalpa 13 EPICOLECT5. Accessed November 28, 2018. http://www.epicollect.net/ EpiCollect5 is a generic data collection tool that allows you to collect and submit geotagged data forms (along with photos) to a central project website 63 MC Micro-Watershed GEMA JAA Support Other GEMA TAPs Total MC-10 Río Varsovia 276 132 408 68% 32.4% MC-11 Quebrada de Balas 382 405 787 48.5% 51.5% MC-12 El Tunco 291 230 521 56% 44.2% TOTAL 5,545 2,018 7,563 Sample Size: Based on beneficiary data provided by GEMA, the population to be sampled was determined to be 7,563. The sample size was estimated using a formula to estimate proportions and a normal distribution of the indicator to be assessed. The following assumptions were made: a) Large population assumption in the 12 micro-watersheds (≥ 10,000), and therefore no finite population factor was utilized. b) Basal level of the indicator to be assessed (P=50%) c) Confidence level of 95% (α = 0.05) d) Margin error of 5% e) Non-response rate of 10% Where: n = Sample size Required Z = 1.96 (Critical Z score corresponding to the .975 percentile of the normal distribution) P = Proportion (basal level) of the total population has the attribute to estimate Q = 1-P (the complement of P) E = Margin Error λ = Non-response rate The sample size required to achieve a confidence level of 95% was determined to be 427 households. Sample distribution: Equal and proportional allocations were used to distribute the sample between the 12 populations, with a minimum number of 30 surveys per each population that uses the water from a sample Level 1 micro watershed. The sample was initially distributed using proportional allocation which considers the total number of beneficiaries in each micro-watershed. However, following field testing few sites were assigned minimal numbers of surveys and the team opted to use a mixed approach of the equal and proportional allocations. This approach set a minimum number of surveys (n=30) in order to make the inferences per site more efficient. In order to facilitate the mixed approach, it was agreed to allocate the sample in each micro￾watershed differentially by type of GEMA beneficiary: 15% for water systems project beneficiaries and 85% for other GEMA projects. When calculating the sample size with the above described parameters, the sample size resulted in 384 households; and after adjusting for the non-response rate, is 427 households. Selection of Households: Systematic sampling was used to select the sample household within the populations of GEMA beneficiaries linked to each Level 1 sample micro watershed. To achieve representativeness of male and female respondents 70% of them were men and 30% were women. Sampling weights: The sampling weights represent the capacity of each sampling unit (household) to represent its universe. The household sampling weight was calculated by first calculating the weight of each sampled population and subsequently calculating the weight of each household. Given the nature of the sampling strategy used, two-stage sampling, each household will have a sampling weight which will characterize its community and ) ( λ) E P( P) n=(z − − / 1 1 2 2 64 the number of communities in each micro-watershed. Sampling weights (SW) for k units in a simple random sample design are defined by: π = NK nK Where: NK corresponds to the total number of elements in the sampling unit. nK corresponds to the number of elements to be selected in the sampling unit. Stage 1: Selection of communities cSWi = Ni ni for i=1, 2, 3,4 Where: Ni: Total number of communities by micro-watershed i ni: Number of communities to be selected in micro-watershed i Stage 2: Selection of households hhSWj = Nj nj for j=1,…..,ni Where: Nj: Total number of beneficiaries in micro-watershed j nj: Number of beneficiaries to be selected in micro-watershed j Final sampling weight: The final household SW was calculated by multiplying all stage sampling weights. SW = (cSWj)(hhSWj) Implementation of the Household Survey: ANED/i-APS staff then administered the HHS in mid-October 2018. Responses to the household survey questions were recorded in Kobo software utilizing smartphones. Survey Field Strategy: The field strategy used for the baseline survey program is described below: a) The supervisor identified the community sample areas, and assigned routes to follow to enumerators. b) Team supervisors contacted the community leader (presidents of water boards and / or Parish chairpersons), to help locate the beneficiaries in the sample. They also contacted GEMA technical staff. c) The supervisor explained the purpose of the survey. d) Enumerators spoke with informants from the sample list, with to locate their homes. e) The supervisor assigns to each enumerator the names of the beneficiaries to be surveyed. f) The enumerators visited each home of the people from the sample assigned, and the survey was applied. It was established that, as far as feasible, the informant must be aware of, and/or a participant or beneficiary of the GEMA program. In relation to Beneficiaries of the GEMA JAA support: The household had to be replaced if: a) If the identified person was absent. b) If the house is unoccupied. c) The beneficiary had changed their residence, or had migrated. d) The person does not agree to participate in the survey and rejected taking part. In all these situations, the beneficiary was replaced, and the interview was then applied to the next house, depending on the time schedule and replacement household locations. 65 In relation to Beneficiaries of other GEMA TAPs/sub-programs: As above, based on the data sets available, the team supervisors contacted and requested the collaboration of leaders and GEMA technicians to identify and locate the homes of the people in the sample. Where beneficiary lists were without names or inadequate: Given that for some TAPs, the list of GEMA beneficiaries was not available, for example, for participants of the resilience program or a CODEL, then the supervisors were recommended to: a) Contact the community leader and ask for the names of people participating in the resilience programs, CODEL and then apply the survey. b) Use snowball sampling, by asking the person surveyed to supply the name of other people taking part in those GEMA programs, until the sample was completed. Beneficiaries with name (list available): In relation to the sample of beneficiaries of micro-enterprises (producers of coffee, micro-enterprises, improved stoves, etc.), for which names were available, household residences were identified to apply the survey. If they could not be found, those informants were replaced from an additional list, or failing that were then suggested by GEMA staff. Key Informant Interviews Within the 12 sample watersheds, interviews were held with two categories of key informants (KIs): (1) administrators, professionals and technicians associated with GEMA or its implementing partners; and (2) 80 actual or potential beneficiaries of GEMA’s field activities who use water from the 12 sample Level 1 micro￾watersheds. Consultants interviewed category 1 KIs. ANED field staff interviewed Category 2 KIs. i-APS developed questionnaires, and KII for five different types of respondent, according to specifically relevant topics or sectors. These were designed to be conducted in about 45 to 60 minutes with a list of targeted respondents. The beneficiaries-level KIIs were implemented by field enumerators in 12 community locations, each in a distinct watershed. Each KII was geo-located using interviewer’s smartphone and recorded to enable transcripts to be developed. Data Quality checks were conducted. KII responses were also collected using paper before both paper and audio transcripts. Key observations were entered into a matrix to enable effective analysis. Category 2 KIs were selected for their knowledge of one of GEMA’s five categories of field activities: i.e. agricultural conservation enterprises; non-agricultural conservation enterprises; resilience; gender; water governance. One female and one male KIs were selected to be interviewed about each GEMA category. The GPS location of where the KIIs took place was noted, the KIIs were recorded on smartphones and written notes were also taken. Through spot checks, ANED controlled the quality of the interviews. In addition to KIIs carried out within the sample watersheds, a set of KIIs was prepared by each sectoral specialist. These questions were then adapted into two overall KII Formats: Format 1 was designed to contain questions relevant to the institutional level, while Format 2 is designed to hold the overall set of questions and issues relevant to beneficiary informants. These were then applied by each sector specialist to informants of both types, during the institutional-level and field data collection period. To identify informants for these two levels, the assessment specialist team worked in close liaison with GEMA Tegucigalpa offices and GEMA field office staff in Gracias, La Esperanza, Santa Rosa de Copan, and Santa Barbara. Format 1 was designed to be applicable generically to informants from some of the following groups: • Managers / GEMA staff working from the Tegucigalpa Office • Regional Directors / Technicians in La Esperanza, Gracias, Sta Rosa and Sta Barbara • GEMA collaborators such as GOAL and UZamorano • Business Development Center • Subcontractors and support providers • Private businesses 66 • Co-Managers of PAs • UMA, Municipalities, Mancomunidades, CODEM • Institutes, Foundations, Federations, Commissions and Related Councils • Staff of the Regional Offices of related Ministries (SANAA, Public Health, ICF). TGU staff • Cooperatives, Associations, Projects and related NGOs • Cooperation Agencies in Tegucigalpa • Other local partners The institutional-level KIIs that were carried out by the Specialists, were not restricted to the Social Watersheds that were the focus for the household survey and the local-level KIIs and FGDs that were carried out in and around the target sample watersheds. For Format 2, local and watershed-level informant and respondent types for KIIs and FGDs included: Juntas de Agua and Irrigation User Groups; CODELs; small agricultural-based enterprises; non-agricultural SMSEs; and other local informants. Focus Group Discussions i-APS prepared an FGD tool that included a discussion guide for a session designed to last approximately an hour to an hour and half and include between 6-8 respondents and applied across the 12 watershed-based sample sites. FGDs were conducted by an enumerator and a note-taker. FGDs were held with: (1) GEMA beneficiaries; and (2) administrators and professionals associated with the implementation of GEMA activities. Responses were collected using paper and audio recordings, smartphone or tablet. Locations were geolocated. A record of each FGD was prepared based on the audio recordings and paper notes. Data Analysis Each Evaluation Team sector specialist has analyzed streams of data derived from: • Assessment of GEMA literature • Household Survey conducted in the 12 target Social Watersheds • KIIs and FGDs carried out with local level informants in the social watersheds • Institutional-level KIIs carried out by specialists focused on service providers and other key stakeholders • Field-level KIIs and FGDs carried out by specialists for further investigation into sector-specific topics • Other meetings and reviews of specialist literature Indicator Matrix: An Indicator Matrix was used to show the type of data that would be required to answer each evaluation question and the sources and tolls required to obtain those data. The indicators served to guide preparation of the KII and FGD interview guides the consultants used and to guide the formulation of findings based on the evidence that had been collected from the various sources of data. Appendix 1 that provides a list of Indicator included in that Matrix. Household Survey Data: Excel was used to analyze household survey data. Frequencies and percentages were calculated. Data were disaggregated by such variables as sex, age, occupation, and location. Sample factors were used to weight the survey data. Graphical representations of the analysis results were prepared. Documents: GEMA provided the consultants with the administrative and technical report it has prepared. From these consultants selected and consulted those that were most up-to-date and relevant for providing reliable data related to their assigned evaluation questions. As they thought advisable, the consultants also consulted other relevant reports and publications. Documents were selected from among the documents available as providing the most up-to-date and relevant documentary data. Key Informant Interviews: ANED summarized data from KIIs with GEMA beneficiaries so as to clarify their principal points and conclusions. KII data was disaggregated by relevant categories such as sex, and association with GEMA. The summaries were entered into a qualitative analysis process. ANED then identified and 67 formulated written statements of the themes that emerged from the beneficiary qualitative data.14 ANED and i￾APS consultants analyzed the data they collected through KIIs and FGDs, using standard qualitative evaluation methodologies to identify themes relative to their assigned evaluation questions. Summary reports were prepared of the data collection by watershed and of key findings by themes, and these were disaggregated by sex of participants. Focus Group Discussions: The same procedures and methodologies were used to analyze the FGD qualitative data as described for the analysis of KII qualitative data. The themes developed from the KIIs and FGDs were triangulated with the other data sources, including the GEMA work plan and project M&E data, data from MIPYME databases, other documents, as well as the analyzed survey data and biophysical data. Field Observations: Field observation data related to each of the 12 sample Level-1 micro-watersheds were compiled in 12 Level-1 Micro Watershed Profiles. Limitations In relation to the HHS objectives, some of the key constraints observed in accessing the targeted sample and engaging with survey, KII and FGD informants were: • Informants were at time less aware than might be expected, of GEMA support being given to the JAA and local organizations, at the institutional level • As GEMA’s activities were initiating or under early development, this had led to a relatively greater level of knowledge of GEMA, compared with the proportion of informants stating that they actively participated in or benefited from GEMA actions. • Absence of the beneficiary, due to the time of day and the nature of their employment • Absence of the coffee producer, due to the length of the working day and the informant being away working in distant fields. • Infrequently it was difficult to locate the CE owner, as the individual was absent on other business; or the name available did not match local perceptions. • Potential informants no longer resided in the area of the social watershed In addition, data were collected from the population that uses the water from the 12 Level-1 micro-watersheds that selected as samples for the IE. Equivalent data were not collected from the populations that use the water from the Control Level-1 micro-watersheds. Without a baseline for the populations that use the resources of the control Level-1 micro-watersheds, we note that this does limit future determinations whether changes observed in land use and forest cover between the sample and the control micro-watersheds can be attributed or not to GEMA’s actions. 14 Patton, Michael Quinn. 1990. Qualitative Evaluation and Research Methods, Second Edition, Sage Publications,506. Pp. Newberry Park, London, New Delhi U.S. Agency for International Development/ Honduras Avenida La Paz, P.O. Box 3453 Frente a la Embajada Americana Tegucigalpa, Honduras www.usaid.gov