USAID/HAITI MID-TERM EVALUATION OF THE REFORESTATION PROJECT FINAL REPORT Haiti Evaluation and Survey Services Project May 2021 PHOTO CREDIT: USAID / HAITI USAID/HAITI MID-TERM EVALUATION OF THE REFORESTATION PROJECT FINAL REPORT Date: May 21, 2021 Submitted to USAID/EGAD Contract Number: CA AID-521-A-17-00002 This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development Authors: Cynthia Berning, Team Leader Philippe Phanol, Deputy Team Leader Contact: Salima Mutima, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ABSTRACT The United States Agency for International Development (USAID) Haiti requested Social Impact’s (SI) Evaluation and Survey Services (ESS) project to design and implement an independent mid-term evaluation of the USAID Reforestation Project (URP) in Haiti, a Project implemented by Chemonics International in partnership with The National Cooperative Business Association Cooperative League of USA International (NCBA CLUSA) and the International Center for Tropical Agriculture (CIAT). This evaluation aims to inform USAID/Haiti about possible mid-course corrections and future program orientation of the URP, whose objective is to reduce the threat to targeted forests and increase tree cover. This midterm evaluation answers evaluation questions (EQs) related to (1) the effectiveness of two implementation mechanisms (direct investment vs. grants), (2) if the Project is improving beneficiaries’ resilience1 in the face of natural and economic shocks, (3) if the Project is integrating youth and gender, and (4) whether the Project has laid the foundation for sustainability. The Project's two implementation mechanisms each have their advantages and disadvantages that should be strategically weighed in future project development, the resilience activities have increased income but likely only in the short term, that gender and youth integration has not been a strong focus of the Project, and that few if any of the activities will lead to a sustainable reduction in deforestation 1 USAID defines resilience as the ability of people, households, communities, countries, and systems (social, economic, ecological) to mitigate, adapt to, and recover from shocks (including co-variates such as drought and floods…) and stresses (climate changes, population pressure…) in a manner that reduces chronic vulnerability and facilitates inclusive growth. TABLE OF CONTENTS ABSTRACT I ABBREVIATIONS AND ACRONYMS IV EXECUTIVE SUMMARY 1 I. INTRODUCTION 7 I1. EVALUATION DESIGN 12 III. FINDINGS AND CONCLUSIONS 23 IV. RECOMMENDATIONS 43 V. EVALUATION MATRIX 46 VI. EVALUATION MANAGEMENT 50 VII. EVALUATION DELIVERABLES AND TIMELINE 53 ANNEX A. SCOPE OF WORK 55 ANNEX B. DATA COLLECTION PROTOCOLS 61 ANNEX C. DATA COLLECTION TOOLS 62 ANNEX D. DOCUMENTS REVIEWED 81 ANNEX E. LIST OF DATA COLLECTION EVENTS 83 ANNEX F. URP SUB-AWARDEES 87 ANNEX G. TREE SEEDLINGS PRODUCED AND TRANSPLANTED 91 ANNEX H. TRAININGS CONDUCTED IN 2018-2020 92 ANNEX I. WOMEN’S ORGANIZATIONS ASSESSED IN FY2018 96 ANNEX J. EXAMPLE OF INFORMATION NEEDED REGARDING SEX￾DISAGGREGATED NUMBER OF PARTICIPANTS IN TRAININGS 97 ANNEX K. SIGNED DISCLOSURE OF CONFLICT OF INTEREST FORMS FOR ALL EVALUATION TEAM MEMBERS 98 iii | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV TABLES AND FIGURES FIGURE 1. URP RESULTS FRAMWORK (2018).....................................................................................................10 FIGURE 2. USAID REFORESTATION PROJECT THEORY OF CHANGE .....................................................11 TABLE 1. LIST OF FGDS CONDUCTED.................................................................................................................14 TABLE 2. STAKEHOLDERS INTERVIEWED DURING FIELDWORK .............................................................16 TABLE 3. NUMBER AND DISTRIBUTION OF KIIS/GIS BY STAKEHOLDER GROUP AND EQS ADDRESSED.....................................................................................................................................................................17 TABLE 4. URP'S OUTPUTS OF ENVIRONMENTAL SERVICES DELIVERED PER TYPE OF IMPLEMENTATION MECHANISM............................................................................................................................25 TABLE 5. URP'S OUTPUTS OF SEEDLINGS PRODUCED AND TRANSPLANTED PER TYPE OF IMPLEMENTED MECHANISM.....................................................................................................................................26 TABLE 6. PROS AND CONS OF DIFFERENT IMPLEMENTATION MECHANISMS...................................26 TABLE 7. DECISION MATRIX FOR GRANTS VS. DIRECT INVESTMENT...................................................44 TABLE 8. EVALUATION MATRIX.............................................................................................................................46 TABLE 9. EVALUATION TEAM MEMBER ROLES AND RESPONSIBILITIES.................................................51 TABLE 10. DELIVERY SCHEDULE .............................................................................................................................54 TABLE 11. EVALUATION SCHEDULE.....................................................................................................................59 TABLE 12. EVALUATION DESIGN MATRIX .........................................................................................................61 TABLE 13. KIIS.................................................................................................................................................................83 TABLE 14. GIS..................................................................................................................................................................84 TABLE 15. FGDS .............................................................................................................................................................85 TABLE 16. URP SUB-AWARDEES..............................................................................................................................87 TABLE 17. SEEDLINGS PRODUCED AND TRANSPLANTED AS OF THE END OF FY20......................91 TABLE 18. TRAININGS CONDUCTED IN 2018-2020 .......................................................................................92 TABLE 19. WOMEN'S ORGANIZATION ASSESSED IN FY2018.....................................................................96 TABLE 20. TRAININGS CONDUCTED DURING FY2020 Q3.........................................................................97 ABBREVIATIONS AND ACRONYMS 3BNMP 3-Bays National Marine Park AAF Association des Apiculteurs de Ferrier ADS Automated Directive System ANCASA Antillean Canning Societé Anonyme AOR Agreement Officer’s Representative APDK Asosiyasyon Pechè pou Devlòpman Karakòl APM Asosyasyon Pechè Madras ASEC Assemblée des Sections Communales ATL Assistant Team Leader CACOPA Cadre de Concertation pour la Promotion de Apiculture CART Coopérative Agricole Rabouré de Terrier Rouge CASB Centre pour l’agriculture Saint Barnabas CASEC Conseil d’Administration de la Section Communale CBO Community-Based Organization CFAIM Centre de Formation en Aménagement Intégré des Mornes CHASE Collectif Haitien pour l’Avancement du Suivi et de l’Evaluation CIAT International Center for Tropical Agriculture CMBP Caribbean Marine Biodiversity Program COAIT Community Options Analysis and Investment Toolkit COP Chief of Party COR Contracting Officer’s Representative CRDD Centre Rural de Développement Durable DAI Development Alternatives Inc. DCOP Deputy Chief of Party DDA Direction Départementale Agricole DDE Direction Départementale de l’Environnement DEC Development Experience Clearinghouse DEED Développement Economique pour un Environnement durable DI Direct Investment DO Development Objective EGAD Economic Growth and Agricultural Development EQ Evaluation Question EQUI Evaluation Quality, Use, and Impact ESS Evaluation and Survey Services ET Evaluation Team FBO Faith-Based Organization FECCANO Fédération des Coopératives Cacaoyères du Nord FCR Findings, Conclusions, and Recommendations FGD Focus Group Discussion FoPRoBim Fondation Pour la Protection de la Biodiversité Marine FY Fiscal Year GAO Groupement Agricole Ouvray GESI Gender and Social Inclusion GI Group Interview GOH Government of Haiti HQ Headquarters IDDH Initiative pour le Développement Durable d’Haiti IPTT Indicator Performance Tracking Table kha Kilohectare v | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV KII Key Informant Interview LOE Level of Effort LPG Liquefied Propane Gas M&E Monitoring and Evaluation MARNDR Ministry of Agriculture and Natural Resources MCAD Mouvement Coopératif Agricole de Dubourg MEL Monitoring, Evaluation, and Learning MMK Mobile Restaurant MOU Memorandum of Understanding MSME Micro, Small, and Medium Enterprise MSPJ Marais Salant Partout de Jacquesil MT Metric Ton NCBA/CLUSA National Cooperative Business Association/Cooperative League of USA NGO Non-Governmental Organization NRM Natural Resources Management NUPAS Non-U.S. Organization Pre-Award Survey ODL Organisation pour le Développement de Labadie OPPBL Oganizasyon Pechè pou Pwoteksyon Bodmè Limonade PA Project Assistant PAC/VIAHSA Paul Agribusiness and Culinatity/Village Apicole Haitienne S.A PAP Port-au-Prince PD Project Director PEA Political Economy Analysis PM Project Manager QA Quality Assurance RCCL Royal Caribbean Cruise Line RECOCARNO Réseau des Coopératives Caféières de la Région Nord RFEO Rassemblement des Femmes Engagées de Ouanaminthe SI Social Impact SOW Scope of Work SWMC Sub-Watershed Management Committee TL Team Leader TOC Theory of Change UNEPH Université Episcopale d’Haiti URP USAID Reforestation Project USAID United States Agency for International Development 1 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV EXECUTIVE SUMMARY The United States Agency for International Development (USAID) Mission in Haiti requested Social Impact’s (SI) Evaluation and Survey Services (ESS) project to design and implement an independent mid￾term evaluation of the USAID Reforestation Project (URP) in Haiti. The Project is implemented by Chemonics International in partnership with the National Cooperative Business Association Cooperative League of USA International (NCBA CLUSA) and the International Center for Tropical Agriculture (CIAT). The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted populations to economic and natural shocks and long-term stresses for their improved well-being in Haiti’s North and North-East regions. In order to fulfill this vision, a five-year, US$39,305,099 project was designed and approved with a start date of September 1, 2017. The URP is scheduled to close in August 2022. The URP aims to achieve four results, each with four intermediate results: 1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved. This Final Report outlines the evaluation team’s (ET) understanding of the evaluation scope of work (SOW), provides background on the Project, explains the evaluation’s methodology (adapted to current COVID-19-related emergency measures to protect public health), presents findings based on the ET’s review of Project documentation and fieldwork efforts, and highlights recommendations for the final two years of the project and future USAID programming. EVALUATION PURPOSE AND QUESTIONS The purpose of this mid-term performance evaluation is to inform USAID/Haiti about possible mid￾course corrections and future program orientation of the URP. The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, Chemonics (the implementing partner), the grantees and sub-grantees, and the Government of Haiti (GOH). This midterm evaluation answers the following evaluation questions (EQs): 1. Effectiveness: What are the strengths and weaknesses of the implementation mechanisms (direct investment vs. grants) as utilized separately and together to promote improved natural resource management?2 2. Resilience: To what extent and in what ways is the Project improving beneficiaries’ resilience3 in the face of natural and economic shocks? 3. Youth and Gender: To what extent and in what ways is the Project integrating youth and gender? 4. Sustainability: After two and half years of implementation, to what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries’ improved approaches toward natural resources management through the adoption of principles and practices promoted by the Project? 2 Natural resources management (NRM) is the management of resources such as land, water, soil, plants, and animals, with a particular focus on how management affects the quality of life for both present and future generations (stewardship). 3 USAID defines resilience as the ability of people, households, communities, countries, and systems (social, economic, ecological) to mitigate, adapt to, and recover from shocks (including co-variates such as drought and floods) and stresses (climate changes, population pressure) in a manner that reduces chronic vulnerability and facilitates inclusive growth. DETAILED METHODOLOGY The ET used a primarily qualitative evaluation design, consisting of document review, 27 key informant interviews (KIIs), 5 Group Interviews (GIs) and 20 focus group discussions (FGDs). The ET applied several analysis methods (content, trend, gap, and gender) to provide evidence for the evaluation’s findings and conclusions. The team disaggregated data collected through KIIs, GIs, and FGDs by sex and analyzed for effects on different participant segments. Evaluation challenges included Internet connectivity, difficulty in reaching some key contacts, and low levels of knowledge of the project and its achievements among many respondents due to delays in project implementation that resulted in many activities that were only recently begun. The ET conducted fieldwork from April 12-30, 2021. Upon final approval of the report, ESS will submit the report to the Development Experience Clearinghouse (DEC) for dissemination. FINDINGS RELATED TO THE EVALUATION QUESTIONS EQ-1: WHAT ARE THE STRENGTHS AND WEAKNESSES OF THE IMPLEMENTATION MECHANISMS (DIRECT INVESTMENT VS. GRANTS) AS UTILIZED SEPARATELY AND TOGETHER TO PROMOTE IMPROVED NATURAL RESOURCE MANAGEMENT (EFFECTIVENESS)? FINDINGS The URP’s focus in the beginning on implementation through grants elicited a limited response from eligible local organizations. Concerned that this approach would not achieve the Project’s targets, the focus shifted to more direct investment starting in FY19. In FY2018, according to the FY2018 annual report, to implement the Project’s resilience-focused and reforestation activities the URP only used a grant mechanism. Despite the project officially launching in September 2017, the Project signed only one grant agreement in 2018 and was seriously behind in reaching its targets for Years 1 and 2. With the shift to direct investment, the establishment and/or rehabilitation of forests and woodlots, agroforestry production, the number of people receiving the Project’s livelihood-related co-benefits and seedling production was able to ramp up substantially even though by the end of FY20, the actual number of seedlings planted still did not meet the Project’s targets. Project and USAID staff stated that there were pros and cons to both mechanisms, with no method clearly preferred across the board. There were differing opinions about whether one or the other mechanism was better for the Project, with many respondents saying that both were important and appropriate within the Project because they accomplished slightly different goals – Direct Investment has allowed for a more systematic and large-scale roll-out of activities to reach targets and has resulted on more trees in the ground in a shorter amount of time, while grants facilitated capacity building among community organizations and thus could contribute more to sustainability. Most sub-awardee respondents stated that the grant process was too long and difficult, with organizations selected through a call for proposals and with reimbursements taking too long, which made it difficult to reach targets. A common related experience of sub-awardees has been that the Project promised equipment, materials, or other assistance that came very late or still has not come. 3 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV CONCLUSIONS Given the short timeline, substantial delays, and the aggressive targets of the project, the move to direct investment was logical. Grant management, including capacity building, will take too long and likely face too many delays to be able to successfully complete sub-grant activities before the end of the Project. Given the time required for processing grant agreements and working with organizations, the project does not have time to pursue further grants. Even with fewer delays at the beginning of the project, the grants mechanism as it was implemented was unlikely to help the Project reach its targets. The arrangement of reimbursing organizations after activities are completed may not be a feasible approach in a resource￾poor place like these targeted regions of Haiti. A focus on grants, with organizations selected through a call for proposals, led to the scattered and sometimes not-logically connected assortment of activities. This strategy ensures that the Project is not attempting to implement an activity that is not of interest in a particular community. On the other hand, it leads to a scattered approach of disconnected individual activities that do not always have a logical connection to the project’s objectives. With the move to direct investment, the project became more able to focus on the specific activities that would lead to the achievement of the main objectives of the project – namely the planting of 4 million trees. Delays on the part of the Project in providing promised materials have had negative impacts on activities. The ET did not learn why these delays occurred, but they were a frequent enough experience among grantees to suggest a systemic issue with Project procurement procedures rather than a one-time or occasional problem due to specific unforeseen circumstances. EQ-2: TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT IMPROVING BENEFICIARIES’ RESILIENCE IN THE FACE OF NATURAL AND ECONOMIC SHOCKS (RESILIENCE)? FINDINGS The focus of the Project shifted from primarily reforestation activities to a broader collection of activities intended to improve the resilience of beneficiaries by increasing their incomes and improving their management of livestock, but the Project’s indicators do not reflect that focus. Nearly every stakeholder who was interviewed agreed that resilience activities were a necessary part of a reforestation project, but many people stated that the resilience activities that were part of the project may have benefited people, but that the activities were too few and spread too thinly among communities to make a big difference. The resilience activities that were most frequently mentioned as effective were the livestock-focused activities, followed by the bee-keeping activities. The Project implemented instant infusion of cash, free tools, plants, and seeds. The most￾cited benefit of the Project was the cash for work strategy it employed through both the grants and direct investment mechanisms. CONCLUSIONS While the various resilience activities individually could be beneficial, the project lacked a cohesive strategy (or struggled to effectively implement a strategy) that would ensure enough of an income increase to prevent further deforestation. Deforestation is a very complex issue with a lot of interrelated causes, and people will continue to cut down trees as long as they are poor and have no other alternatives to earning enough income. The activities most likely to lead to improved resilience are the livestock-focused ones. Cash for work and provision of free tools, plants, and seeds affected beneficiaries’ livelihoods immediately (though temporarily). While it is unknown whether the longer-term income generating activities will permanently raise incomes for beneficiaries, it is clear that this instant infusion of cash, free tools, plants, and seeds did have an immediate impact. EQ-3: TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT INTEGRATING YOUTH AND GENDER? FINDINGS It is unclear whether and to what extent gender/youth were considered during URP’s initial/design phase. URP has sex- and age-disaggregated indicators, but its gender/youth reporting is inconsistent. Beyond participation numbers for women, no partners are tracking any other indicators of inclusion. CONCLUSIONS Gender targets and quotas have been easily met by partners, and women appear to be active in decision-making, not just token members. To minimize potential self-censorship in mixed-gender FGDs, the ET conducted three all-female FGDs, and the same message was relayed in these: that women are active participants and involved in decision-making and were active before the Project as well. It is difficult to assess whether the Project’s approach to reaching women within the context of traditional gender roles in the sector was a positive way to ensure the equal participation of women or in fact reinforced potentially harmful, traditional social norms or inequalities. EQ-4: TO WHAT EXTENT AND IN WHAT WAYS HAS THE PROJECT LAID THE FOUNDATION FOR SUSTAINABILITY IN TERMS OF IMPROVED APPROACHES TO NATURAL RESOURCES MANAGEMENT THROUGH THE ADOPTION OF PRINCIPLES AND PRACTICES PROMOTED BY THE PROJECT (SUSTAINABILITY)? FINDINGS The Resilience Activity benefits that people most frequently cited were short-term. FGD participants spoke more about the benefits of paid labor through the Project, which will end when the Project ends, than about benefits that will continue post-Project. While Project staff spoke positively about the sustainability of resilience activities such as beekeeping and livestock forage growing, USAID and GOH staff were less optimistic. URP explored partnerships with the private sector to increase income-generating activities for its stakeholders but most of these conversations have stalled. This was the case for both Royal Caribbean Cruise Lines (RCCL), which was unable to be reached, and Antillean Canning S.A. (ANCASA), which was reached through a KII. Sub-Watershed Management Committees are currently functioning but only with Project support. 5 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV The trees that have been planted on people’s private property that will contribute to their income are likely to be protected, but trees planted on public land where animals roam freely are not likely to survive or be cared for. The URP supported the finalization of municipal decrees designating reforestation areas, but these decrees are unlikely to have much effect on tree cutting. CONCLUSIONS Most resilience activities are not likely to be sustainable, given the short project timeline, lack of resources among farmers, and many project delays. Beneficiaries learned new practices and incorporated those practices into their agricultural activities but did so primarily because seeds, plants, and training were given to them for free and they were paid a daily wage to implement these practices on their own parcels. Sustainable behavior change takes time, with subsistence farmers especially tending to be risk-averse and often requiring several seasons of tangible evidence of benefits in order to enact new techniques. At most, beneficiaries will have experienced one or two harvests of new crops, or in the case of slower-growing trees, no visible benefits of the Project’s activities for several years. Pursuing new and unproven techniques is a lot to expect from people who are already food insecure and cannot take the risk of a potential lost harvest, even if there is the possibility of increased income using new techniques. While the exercise of creating a watershed management plan may have some small benefits to the communities, the committees are unlikely to continue to function without support. Without an entity financing activity of the SWMCs, organizing meetings, and paying travel/per diem costs, the committee members themselves stated that they would not be able to continue to do anything after the project ends. The Project’s limited work with clean cookstove promotion is limited and is unlikely to significantly lessen the demand for charcoal. The Project’s efforts in this domain are not going to significantly reduce the demand for charcoal in these two regions to spark the kind of landscape-level change that the Project aimed to achieve. RECOMMENDATIONS In the final two years of the Reforestation Project, Chemonics should as a first priority: ● Not sign any additional grant agreements at this point, focusing on completing what is underway. ● Understand why delivery of materials has been so slow and prioritize the fulfillment of outstanding promises. ● Prioritize finding a way to fence off areas from animals where trees are to be planted in order to protect the trees. ● Continue to employ as many women as possible in the direct investment activities and pay them at least a standard daily wage for their time. Once these top priorities are met, Chemonics should aim to: ● Focus remaining efforts on the activities that have shown fast uptake, particularly the livestock￾related ones. ● Focus on developing the private sector cooperatives and trade associations like FECCANO and RECOCARNO because they have the best chance at providing market opportunities for agroforestry crops. In future reforestation-focused projects, USAID/Haiti should: ● Strategically evaluate how to effectively include both direct investment and sub-grants mechanisms in implementation, likely with direct investment for the majority of the main project, but using grants where local organization exist to support promising local projects. ● Require a more logical and cohesive strategy for increasing income to a level high enough to decrease deforestation in these poor areas. ● More specifically define the project’s objectives and intended outcomes around youth and gender beyond targeting female participants. ● Understand the different motivations for deforestation-causing behaviors among various sub￾groups of the population and attempt to focus messaging and activities more specifically to these different demographics. ● Balance a results-based payments approach with realities on the ground. ● Focus tree planting efforts on places and species that people have strong incentives to protect. 7 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV I. INTRODUCTION The United States Agency for International Development (USAID) Mission in Haiti requested Social Impact’s (SI) Evaluation and Survey Services (ESS) project design and implement an independent mid￾term evaluation of the USAID Reforestation Project (URP) in Haiti. The Project is implemented by Chemonics International in partnership with the National Cooperative Business Association Cooperative League of USA International (NCBA CLUSA) and the International Center for Tropical Agriculture (CIAT). The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted population to economic and natural shocks and long-term stresses for their improved well-being in Haiti’s North and North-East regions. In order to fulfill this vision, a five-year, US$39,305,099 project was designed and approved with a start date of September 1, 2017. The URP is scheduled to close in August 2022. The URP aims to achieve four results, each with four intermediate results: 1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved. This Final Report outlines the evaluation team’s (ET) understanding of the evaluation scope of work (SOW), provides background on the Project, explains the evaluation’s methodology (adapted to current COVID-19-related emergency measures to protect public health), presents findings based on the ET’s review of Project documentation and fieldwork efforts, and highlights recommendations for the final two years of the project and future USAID programming. This mid-term evaluation is taking place during the COVID-19 pandemic. In addition to its devastating public health consequences, the pandemic is causing major disruptions in daily life in nearly all countries across the globe, including Haiti. This final report describes how the ET took into consideration the pandemic when developing the evaluation planning and methodology and carrying out data collection, by conducting all KIIs, GIs, and FGDs remotely with minimal travel and face-to-face contact. The scope of this evaluation covers the period from the start of the URP until end of FY 2020 (September 30, 2020). The URP is being implemented in the north/north-east of Haiti, and the evaluation will look at the different activities implemented in both Region 1 and Region 2.4 Region 1 ● The sub-watersheds of Joli-Trou/Cormier and Bahon (Grande Riviere du Nord watershed) for which management plans were completed, and in the sub-watershed of Milot (Haut du Cap watershed) for which a management plan will be developed this year. The latter will integrate the direct investment activities ongoing in the communal sections of Perche-de-Bonnet and Bonnet à l’Evêque; ● The mangroves area of the coastal zones of the Bay of Acul du Nord, the 3-Bays National Marine Park (3BNMP) and their surrounding communities; ● The communal section of Bande du Nord (home to village Labadie and RCCL Private Destination – Tourist Resort Labadee), as part of the Project’ strategic partnership with Royal Caribbean Cruise Line; and 4 Workplan 2020 (priority zones, p. 18) ● The geographic areas covered by the cooperative members of Fédération des Coopératives Cacaoyères du Nord (FECCANO) and Réseau des Coopératives Caféières de la Région Nord (RECOCARNO) for the cocoa- and coffee-based agroforestry production systems. Region 2 ● The sub-watersheds of Perches (Marion watershed), Gens de Nantes (Jassa watershed), and Capotille (border area – Massacre watershed) for which management plans were completed, and in the sub-watershed of Haut Ouanaminthe – communal sections of Savane Longue and Savane au Lait (Jassa watershed) for which a management plan will be developed this year; and ● The mangroves area of the Bay of Fort-Liberté and their surrounding communities. EVALUATION PURPOSE AND AUDIENCE The evaluation examines how new processes, mechanisms, and systems established by the URP have contributed to restoring environmental services, improving livelihoods, and building the resilience capacity of beneficiary communities through integrated reforestation interventions. Lessons learned and recommendations provided through the evaluation report will also guide how the Project is strategically, operationally, and sustainably progressing toward its objectives, including youth and gender inclusion in reforestation programming and governance. The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, Chemonics (the implementing partner), the grantees and sub-grantees, and the Government of Haiti (GOH). This evaluation focused on the Evaluation Questions (EQs) below: 1. Effectiveness: What are the strengths and weaknesses of the implementation mechanisms (direct investment vs. grants) as utilized separately and together to promote improved natural resource management? 2. Resilience: To what extent and in what ways is the Project improving beneficiaries’ resilience in the face of natural and economic shocks? 3. Youth and Gender: To what extent and in what ways is the Project integrating youth and gender? 4. Sustainability: After two and half years of implementation, to what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries improving approaches to natural resources management through the adoption of principles and practices promoted by the Project? The full evaluation SOW is included in Annex A and detailed evaluation protocols in Annex B. 9 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV BACKGROUND DEFORESTATION IN HAITI Deforestation has been a major concern in Haiti, adversely affecting land use practices, destroying biodiversity and ecological habitats, and making long-term investments in the lowlands vulnerable to natural threats (hurricanes, soil erosion, floods, and drought). In the North and North-East regions of Haiti, trees are removed for agriculture, construction, and charcoal production, primarily to ensure day￾to-day economic livelihoods and food subsistence in the face of major stressors and shocks, such as drought and lack of alternative income sources. According to Global Forest Watch, from 2002 to 2020, Haiti lost 72.1kha of tree cover, including 2.87 kha of humid primary forest, or 8.4 percent of the 2000 total tree cover. The total area of humid primary forest in Haiti decreased by 33 percent in this time period.5 URP PROJECT GOAL AND OBJECTIVES The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted populations to economic and natural shocks and long-term stresses. The five-year, US$39,305,099 project was designed and approved with a start date of September 1, 2017 and is scheduled to close in August 2022. The URP’s Results Framework includes one overarching goal to increase the foundations for resilience, stability, and inclusive growth. To contribute to this goal, URP aims to achieve four results—1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved—each with four intermediate results. Figure 1 presents the objective of the USAID Reforestation Project “restore the provision of environmental services in targeted forest and agroforestry areas in support of food security and/or resilience to economic and natural resources shocks” with a direct link to the USAID/Haiti Strategic Framework for 2018 – 2020 and Results Framework goal “foundation for resilience, stability, and inclusive growth reinforced.” 5 https://www.globalforestwatch.org/dashboards/country/HTI FIGURE 1. URP RESULTS FRAMWORK (2018) Source: Annual Report FY 2018, p.47 The Project’s direct beneficiaries include charcoal value chain actors, marketers, consumers of agroforestry products, smallholder farmers, and fisherfolk, as well as watershed residents and fuel-wood users. Their participation and behavior change are necessary to achieve results. In addition, the Project plans to build the capacities of sub-watershed management committees as well as provide training to municipalities. The Project’s indirect beneficiaries include urban, coastal, and downstream residents; plains farmers; hotel and tourism operators; and enterprises whose daily needs are met by forest and watershed environmental services. The Project aims to especially target women and youth as participants and beneficiaries. URP’S THEORY OF CHANGE URP’s theory of change (TOC) can be summarized as “tree cover will increase if the threat of deforestation is reduced, the incentive to plant and maintain new trees is increased, community resilience to economic and natural shocks is improved, and environmental governance and coordination is strengthened.” Figure 2 presents the Project’s TOC. 11 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV FIGURE 2. USAID REFORESTATION PROJECT THEORY OF CHANGE Source: Annual Report FY 2018, Project Document, page 6 The assumptions underlying this TOC are: ● Political stability and effectiveness of Project’s partner ministries: The partnership with and support of Haitian ministries and government is critical to the Project’s success and sustainability. The Project is aware of potential political instability and the possible effects not only on its institutional champions, partnerships, and buy-in, but also on Project operations and safety. ● No natural or ecological disasters: The Project’s sub-watershed management and agroforestry efforts have the potential to be greatly affected by natural and ecological events. The plan for sustainable impact is closely tied to the physical state of the target area. ● Security: The Project’s function and activities are dependent on security in the region. Haiti, including the northern region, is regularly affected by violent protests and other insecurity, so it is critical to consider security in developing and monitoring the exit strategy. ● Continuous leadership of grantees and local partners: Grantees and local partners play a critical role in the Project’s sustainability and impact. As such, their continuous leadership, buy-in, and increased capacity are important factors in the Project’s exit strategy and long-term impact. URP SUMMARY PROJECT INTERVENTIONS To achieve enduring results and outcomes, the Project tries to understand and address the primary drivers of deforestation in the region and builds on the findings from assessing the region’s economic, institutional, and cultural characteristics. This approach helps to identify the opportunities and challenges inherent in the Project’s implementation process in the area. The Project involves and works with government entities at the central and regional level to ensure institutional and policy ownership of its activities. It partners with local organizations (community-based, faith-based, non-governmental) and the private sector to implement initiatives that sustain the local economy while improving the livelihoods of targeted groups. To implement its activities, the Project uses two mechanisms: partnerships with the above-mentioned organizations through sub-awards and in-kind grants; and direct investments in collaboration with municipalities and other actors involved in community development. The main activities of the Project are: ● Promoting use of efficient cookstoves and liquefied petroleum gas as a strategy to reduce pressure on tree stands and increase tree cover in the targeted sites. ● Establishing community nurseries to produce forest, fruit, and mangrove seedlings for reforestation and rehabilitation of agroforestry systems. ● Establishing forest woodlots, fruit orchards, and agroforestry systems (mixed of fruit and forest trees and cash crops). ● Rehabilitation of degraded mangrove forests to protect the coastal communities and ensure conservation of mangrove ecosystems. ● Strengthening the productivity and resilience of livestock systems, through drought adaptation measures, deworming, forage production, and storage. ● Developing livelihoods and income diversification sources (beekeeping, short-term employment, agroforestry production, sustainable fishing) to help households build economic assets and cope with major stresses and shocks affecting food security in the region. ● Building the capacity of targeted groups and local governments to develop and implement sub￾watershed management plans that promote productive land use, reduce disaster risks, and improve natural resource management governance. I1. EVALUATION DESIGN PHASE ONE PLANNING AND PREPARATION This mid-term evaluation formally began with a client kick-off meeting on December 2, 2020. During the meeting, the TL, ATL, ESS Field Office staff, ESS Contracting Officer’s Representative (COR), USAID/Haiti technical team members who manage URP, and Chemonics team members, including the Chief of Party (COP) and Deputy COP (DCOP) identified key evaluation stakeholders, reviewed the EQs, and discussed evaluation use. It was agreed that USAID would provide comments in writing regarding the ET’s understanding of the EQs, especially EQ-4, and will also provide feedback regarding the list of stakeholders that the ET identified. The ET also requested several additional documents (Annual Reports, sub-awardees, contracts, IPTT 2018 and 2019, the PEA, performance report, etc.) in order to proceed with preparing the Inception/Design Report. Categories of documents are fully referenced in Annex D. The ET continued its review of URP Project documents, contacted Chemonics, the URP’s prime grantee, for additional documents and stakeholder contact information, and prepared the Inception and Evaluation Design Report, which was submitted on December 23, 2020, and approved by USAID on March 31, 2021. PHASE TWO DATA COLLECTION Following USAID/Haiti’s approval of the Inception and Evaluation Design Report, the ET launched the evaluation data collection phase on April 12, 2021 and collected data from April 12 to 30, 2021 (three weeks). The ET conducted the first days of data collection together to test and adjust the data collection protocols. Given the number of planned KIIs, the ET conducted several interviews with USAID and 13 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Chemonics together, but then divided into teams during the second and third weeks, to accommodate key respondents’ scheduling constraints and language preferences. The ET frontloaded the schedule with the most knowledgeable respondents, for example, USAID and Project (Chemonics, NSBA/CLSA, CIAT), who were well placed to provide context and highlight key issues that the ET could probe in later interviews with GOH officials, sub-awardees, municipalities, the private sector, academic institutions, and NGOs/CSOs. To accommodate potential technical challenges and rescheduling needs, the ET planned approximately three to four interviews per day, but the realities of respondents’ busy schedules meant these numbers ranged from one to seven interviews per day. The ET used non￾interview time during the data collection phase to finalize notes, review FGD notes provided by CHASE, send follow-up emails to respondents, and create and update the emerging themes matrix throughout fieldwork. The ET primarily used a qualitative evaluation design. The team extracted quantitative data from the URP’s monitoring and evaluation (M&E) reports, including disaggregated data on URP-relevant indicators by partner institutions (when available) and gender to differentiate results. The ET triangulated the Project M&E data with the qualitative data it collected through document review, KIIs and GIs with Project stakeholders (sub-awardees, sub-watershed management committees, private sector, academic institutions, NGOs/CBOs, cooperatives, and other stakeholders), and FGDs with URP beneficiaries. GUIDING PRINCIPLES The ET conducted fieldwork with four guiding principles in mind: 1. Informed Consent: The ET administered informed consent scripts and asked for respondents’ verbal consent to ensure that respondents understood the purpose and voluntary nature of the KIIs/FGDs, as well as the risks and mitigation measures associated with the COVID-19 pandemic in the case of FGDs. 2. Confidentiality: The confidentiality of key informants is a priority for ESS. The ET has produced a report that aggregates data and omits identifiers. Although the ET may use quotes, none of the individuals it interviewed are named in this report. 3. Gender Integration: USAID’s Automated Directive System (ADS) 205 requires gender integration into the evaluation design and implementation. ESS takes this guidance several steps further through SI’s Evaluation Quality, Use, and Impact (EQUI)® framework, which uses gender analysis frameworks to inform sampling strategies, gender-sensitive data collection protocols, sex￾disaggregated data collection, and the consideration of gender and social dimensions in data analysis and reporting. A dedicated Gender Specialist, through sub-contractor EnCompass, supports the ET by reviewing deliverables at each stage and scoring against a Gender Scorecard, as well as contributing to team meetings throughout evaluation planning, data collection, and results debriefings. This scorecard synchronizes with EQUI’s quality assurance (QA) checkpoints to ensure that the ET integrates gender and social dimensions into each stage of the evaluation process, and as they relate to each evaluation topic. Furthermore, it paid particular attention to EQ-3, which specifically addresses integration of gender and youth into the URP. Data collection instruments included questions about the Project interventions’ effects on various gender groups, impacts on gender relations and equality, and intended and unintended impacts on women and men. Integration of gender into this evaluation includes review of the extent to which the activity (not just the evaluation) design, implementation, and MEL is consistent with the USAID Gender Equality and Female Empowerment Policy and overall ADS 205 policy objectives and requirements for gender integration. 4. COVID-19 Safety Protocols: As highlighted above, because of COVID-19-related travel and meeting restrictions, the ET conducted all KIIs and GIs remotely. The FGD approach involved gathering farmers in a common location where they could sit socially distanced and participate in the FGD via a tablet that was provided to them by the CHASE logistician. The reason for gathering farmers in one location for the focus groups is that farmers tend to lose focus when they are participating in FGDs at home and/or are not actually available at the agreed upon time, according to our recent experience. The ET believed it would not be possible to collect quality FGD data from farmers without gathering participants in a common location at a specified time. For this to happen, one team member needed to be in place to organize the location, provide the Internet and tablets, disinfect the space, and enforce distancing. This approach minimized physical interaction to the greatest extent possible, while simultaneously enhancing the possibilities of collecting quality data. DATA COLLECTION METHODS FGDs. The ET engaged the local firm, Collectif Haitien pour l’Avancement du Suivi et de l’Evaluation (CHASE) to conduct FGDs with farmers throughout Haiti’s North and North-East departments. The ET conducted 20 FGDs in the five different sub-watershed areas that the URP is supporting. In each sub￾watershed area, four FGDs were organized. The ET fielded two teams in order to carry out two FGDs simultaneously. This allowed the team to organize and complete all 20 FGDs in three weeks. The FGDs included women’s groups, youth associations, sub-watershed committee members, and farmers that benefitted from the Project. Each FGD included between five and fifteen participants. To maximize participation, the facilitator conducted the discussion in Creole, and called upon individual participants in the group to promote their active participation. Information gathered from FGDs informed the ET’s answers to all four EQs. TABLE 1. LIST OF FGDS CONDUCTED SUB-WATERSHED ASSOCIATION/ GROUP MALES FEMALES ASSOCIATION ACTIVITIES Limonade APPWOLIM/ Asosyasyon Pwodikte Let nan Limonad/ Association d’eleveurs 7 1 Agroforestry, livestock and dairy Ouanaminthe GEDWA/Association d’Eleveurs pour le Développement de Ouanaminthe 4 3 Livestock Perches Bénéficiaires 3 7 Agroforestry, Livestock Terrier Rouge KOET Koperativ Elvè Terrier Rouge 8 7 Livestock Trou du Nord/Roche Plate AFDRP Assosyasyon Fanm pou Devlopman Roche Plate 0 12 Livestock Ferrier AAF/ Association des Apiculteurs de Ferrier 9 5 Agroforestry, Bee￾keeping Caracol FoProBiM/ APDK Asosyasyon Péchè pou Devlopman Karakòl 8 1 Mangrove Reforestation, Bee￾keeping 15 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV SUB-WATERSHED ASSOCIATION/ GROUP MALES FEMALES ASSOCIATION ACTIVITIES Fort-Liberte CACOPA 4 3 Agroforestry, Bee￾keeping Madras APM/ Asosyasyon Pechè Madras 7 2 Mangrove Reforestation Limonade OPPBL Òganizasyon Pechè pou Pwoteksyon Bòdmè Limonade 5 1 Mangrove Reforestation Grande Rivière du Nord CAJBC Coopérative Agricole Jean Baptiste Chavannes 6 2 Agroforestry, Reforestation and Cacao Quartier Morin OTG /Oganizasyon ti Plantè Gran Pré 8 1 Agroforestry Bois de Lance/ Limonade MPDB Mouvman Pwogresis pou Devlopman Bois de Lance 6 2 Reforestation Plaine du Nord FECCANO 6 2 Agroforestry, Cacao Limonade PLOMB 8 1 Agroforestry Vallières KODV Komite Oganizasyon pou Devlopman Valyè 6 2 Reforestation Vallières MPFV Mouvman Pwogresis Fanm Valyè 0 9 Reforestation Vallieres ADTP 4 2 Agroforestry Jaczil AFVMJ / Asosyasyon Fanm Vanyan Marais Salant Jaczil 0 5 Agroforestry Ouanaminthe OTDO (Organisation des Travailleurs pour le Développement de Ouanaminthe) 1 5 Agroforestry, Livestock Key Informant and Group Interviews. The ET conducted 28 KIIs and 5 GIs (with 19 participants) with representatives of key stakeholder groups, including USAID, grantees, and sub-awardees, GOH, sub-watershed committees, private sector, academic institutions, CSOs, and farmers’ associations. The ET selected KII and GI participants using purposeful sampling techniques, supplemented by snowball sampling. Findings from KIIs and GIs contributed to the ET’s responses to all four EQs. Despite incorporating feedback from the Gender Scorecard to ensure sampling would be gender-inclusive within purposeful sampling, in the end the ET was not able to achieve a gender balance among KIIs and GIs; less than 25 percent of KII and GI respondents were women. In many cases, this was due to gender imbalances within the institutions, particularly among leadership positions. The main stakeholder categories the ET interviewed are outlined in Table 2. TABLE 2. STAKEHOLDERS INTERVIEWED DURING FIELDWORK STAKEHOLDER CATEGORY ORGANIZATIONS REPRESENTED # MALES # FEMALES USAID/Haiti AOR, Environmental Officer 1 1 Prime and Sub￾primes Chemonics, NCBA CLUSA/CIAT 5 2 Sub-Awardees Saint Barnabas Agriculture Center (CASB); APAPANNE Dosmond Germplasm Center; FoProBiM; Kiskeya Vèt; Institution Univers, FECCANO, Village Planète, AAF, ODL, RECOCARNO, and MSPJ 10 2 Government Directions Départementales du Nord of the Ministries of Environment and of Agriculture 3 0 Sub-Watershed Management Committees Joli-Trou/Cormier and Bahon (both in Grande Riviere du Nord watershed), Perches (Marion watershed), Gens de Nantes/Savane Longue (Jassa watershed), and Capotille (border area – Massacre watershed) 15 4 Private Sector Antillean Canning S.A. (ANCASA), One Tree Planted 1 1 Academic/Training Institutions Centre de Formation en Aménagement Intégré des Mornes (CFAIM) and Université Henry Christophe de Limonade 2 0 Total 37 10 USAID personnel KIIs. The ET interviewed two USAID personnel: the USAID Activity Agreement Officers’ Representative (AOR), who was responsible for overall Project oversight and coordination, and the Environment Officer, who provides technical oversight for several environmentally focused USAID projects in Haiti, including the URP. URP personnel KIIs. The ET interviewed key prime and sub-prime Project personnel, including the COP, DCOP, and Gender and Social Inclusion (GESI) and MEL Coordinators. Furthermore, the ET contacted the Project’s regional office in Ouanaminthe to get their field perspective on activities implemented in Region 2. The team interviewed seven Project staff to respond to the four EQs. Because the current COP and DCOP joined the Project in Q3 FY 2020, the ET attempted to contact the previous COP to understand the two first years of Project implementation but was unable to reach him. Sub-awardees personnel - KIIs. The ET interviewed 12 sub-awardees personnel, including representatives of CASB, Village Planète, APAPANNE, FoProBiM, FECCANO, Institution Univers, and Kiskeya Vèt. The ET used information learned from sub-awardee personnel to answer all four EQs. Sub￾awardee personnel interviews provided additional data and filled gaps in documentation on the grants implementation mechanism, its processes, accomplishments, internal and external challenges, and lessons learned (EQ-2). Interviews also gathered information on how women, men, and youth have increased their resilience (EQs 2 and 3) and provided insight on how the Project has increased awareness and achieved behavioral change as well as to what extent the Project has supported income￾generating activities (EQ-4). Sub-Watershed Management Committee GIs. The ET organized GIs with 19 representatives of the five Sub-Watershed Management Committees. The purpose of these meetings was two-fold: they allowed the ET to assess how the committees are functioning and to what extent (from their 17 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV perspective) the Project is building the resilience of farmers within their geographical area. Female members of the committees were present in all five GIs. These GIs and FGDs helped the ET respond to EQs 2, 3, and 4. Ministries of Environment and Agriculture and Municipalities. The Project has partnered with the Ministry of Environment and the Ministry of Agriculture and Natural Resources (MARNDR) as well as five municipalities: Bahon, Perches, Capotille, Grande Rivière du Nord, and Ouanaminthe. The ET understands that it is through these public institutions that the Project did direct investment. The team held KIIs with the Project’s focal point in these two ministries to better assess the DI mechanism and respond to EQ-1. Furthermore, the ministry and municipal officers would be able to provide their perspective on the effectiveness of resilience activities that the Project supported (EQ-2). Finally, the local public officials would be able to share information regarding the potential for sustainability of Project activities and will discuss with the Municipality of Bahon the process of developing the municipal decree (EQ-4). Private Sector – KIIs. The ET organized KIIs with two representatives of the private sector companies that have partnered with the URP (ANCASA and One Tree Planted) to understand how the partnerships were established and the private sector perspective on the effectiveness and sustainability of the Project’s activities. These KIIs provided additional information on the types and effectiveness of resilience activities the Project implemented (EQ-2). Finally, meeting with the private sector helped the ET address the question of sustainability, in particular their interest in supporting income-generating activities (EQ-4). Academic Institutions – KIIs. The Project partnered with the Faculté des Sciences Agronomiques, Campus Henry Christophe de Limonade, and the Centre de Formation en Aménagement Intégré des Mornes (CFAIM) to train Mayors and General Directors of municipal administrations, and Conseil d’Administration de la Section Communale (CASEC) and Assemblée de la Section Communale (ASEC) members on laws and regulations pertaining to forest/tree cover, water resources, and livestock management. By meeting with these two academic institutions, the ET was able to respond to EQ-3 (different participation and needs of women and men) and benefits of capacity building activities, which contribute to sustainability (EQ-4). Table 3 summarizes the ET’s KII stakeholder groups, its actual number of respondents, and the priority topics and EQs that were addressed with each stakeholder group. TABLE 3. NUMBER AND DISTRIBUTION OF KIIs/GIs BY STAKEHOLDER GROUP AND EQS ADDRESSED STAKEHOLDER GROUP NUMBER PRIORITY TOPICS AND EQS ADDRESSED EQ1 – Grant vs. DI mechanisms EQ2 - Effectiveness of resilience activities EQ3 – Gender and Social Inclusion EQ4 – Sustainability of Activities USAID/Haiti 2 URP Staff (Chemonics, NCBA/CLUSA, CIAT) 7 Sub-awardees 12 Government of Haiti (Ministries of Environment and Agriculture) 3 STAKEHOLDER GROUP NUMBER PRIORITY TOPICS AND EQS ADDRESSED EQ1 – Grant vs. DI mechanisms EQ2 - Effectiveness of resilience activities EQ3 – Gender and Social Inclusion EQ4 – Sustainability of Activities SWMCs 19 Private Sector 2 Academic Institutions 2 Total 47 The protocols that guided data collection can be found in Annex B. The purposes of these protocols were to: 1) ensure the ET covered all key issues during data collection; 2) help organize information in a form that the ET can usefully and efficiently analyze; and 3) ensure that the ET gathered information to assess the extent to which URP effectively addressed the specific concerns of both male and female stakeholders. The protocols were reviewed and scored by a Gender Specialist against the Haiti ESS Gender Scorecard to ensure they adequately incorporated USAID gender priorities and considered project outcomes related to gender equality. The protocols consisted of questions that addressed the EQs, as well as attempted to fill gaps from the ET’s document review and its initial discussions with USAID. PHASE THREE DATA ANALYSIS The TL oversaw and managed systematic analysis of the qualitative data. The evaluation design included several data collection methods (document review, KIIs, FGDs, and secondary data collection) that enabled the ET to conduct triangulation across different methods. In addition, several researchers were involved in the data collection and analysis (research triangulation), mitigating the risk of researcher bias. Finally, the ET collected information and perspectives from a variety of URP stakeholders, many of whom have different perspectives, interests, and experiences with the Project (data triangulation), mitigating respondent bias. The ET compared data from the varied streams described above to determine whether findings are divergent or convergent. The extent to which multiple informational streams provide consistent findings will inform the certitude and internal validity of evaluation conclusions. Where the different sources have diverged, the team has undertaken additional analysis, and has stated conclusions in a way that reflects the nuanced data. The ET has ensured that a large portion of data collected across methods addressed the same questions to enhance triangulation possibilities. DATA ANALYSIS METHODS The ET employed several data analysis methods to identify key findings from the collected data, as well as to draw conclusions and make recommendations for Project follow-up or future potential programming. The type of analyses depended on the specific data being assessed (e.g., content analysis for qualitative KIIs and FGDs) and included: 1. Content Analysis: Content analysis entailed the ET’s intensive review and systematic coding of KII and GI transcripts to identify and highlight the existence of certain themes, and their frequencies, 19 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV within the data. The ET then summarized these standardized data to better understand the contributions of URP interventions to its objectives and to triangulate findings. 2. Context Analysis: Context analysis took into consideration progress of the target vulnerable population and communities toward increased resilience to shocks and stresses linked to the environment, economic, social, and political issues. 3. Performance Trend Analysis: Trend analysis enabled the ET to examine different URP indicators over time to identify patterns of convergence (or divergence) in outputs and outcomes toward the stated objectives. 4. Gap Analysis: Gap analyses by the ET examined which aspects of the URP, if any, fell short of anticipated performance, and the likely factors contributing to these gaps. 5. Comparative Analysis: The ET undertook comparisons between DI and the grant mechanism. To the extent that data was available, it looked at the differences in number of women, men, and youth reached, delays in both types of implementation mechanisms, and strengths and weaknesses of the two implementation mechanisms. 6. Gender Analysis: EQ-3 was specifically dedicated to analyzing URP’s gender aspects across all of the Project’s aspects. In addition, the ET used a gender lens to answer the other three EQs. Supported by the EnCompass Gender Specialist, the team worked across all the EQs to capture and compare Project results as they specifically benefitted (or did not benefit) both women and men. The ET disaggregated all data collected through KIIs and GIs by sex and analyzed for effects on both male and female participants to show any significant differences. The ET utilized the ESS Gender Scorecard the EnCompass Gender Specialist completed for each evaluation phase to verify the analysis approaches successfully integrate gender considerations. 7. Sustainability Analysis: EQ-4 focused on determining whether the interventions implemented to date were likely to be sustainable. For this analysis, the ET reviewed available facts, data and evidence to assess whether the technical and technological NRM tools and practices promoted by URP were relevant, sufficient and effectively adopted by the target beneficiaries, to lay the foundation for long-term sustainability. This analysis required examining the technical, technological, human, contextual, financial, organizational, political and law enforcement capacities, opportunities, and constraints that affect sustainability of the NRM interventions implemented by the Reforestation Project. DATA ANALYSIS PROCESSES The team took detailed notes of KIIs and FGDs, cleaning and sharing electronic summaries on a rolling basis throughout fieldwork. Team members conducted internal debriefs at least every two days during fieldwork to discuss progress and any adjustments that were needed in the evaluation schedule. The ET discussed evidence collected to help answer the evaluation EQs, as well as identify any discrepancies. The team also began identifying any emerging patterns and themes that would prove helpful in developing the analysis coding scheme and recorded them in a collaborative emerging themes matrix that was updated on an ongoing basis. Following data collection, the ET developed a tally sheet matrix structured by EQ and EQ sub-questions, which included themes that arose from the interviews across all KIIs and FGD, and tallied the number of times the same theme occurred by stakeholder group and sex. The ET reviewed notes thoroughly and annotated them electronically to reflect the themes that were included in the tally sheet. This approach enabled the ET to look for trends within and across sub-groups, including trends by sex. The TL captured findings, conclusions, and recommendations (FCR) in an Excel-based matrix that categorized analysis by EQ. The matrix: 1) ensured that the ET prepared a systematic and thorough response to each EQ; 2) verified that analysis accounted for gender and social dimensions; 3) identified any gaps where additional clarification or analysis was necessary; 4) clarified connections between FCR; and 5) served as the basis for developing this evaluation report. The EnCompass Gender Specialist also participated in analysis debriefs and in reviewing the FCR to validate inclusion of sex-disaggregated data. Quantitative data were obtained from the FY2020 Annual Report and accompanying Indicator Tracking Table. These data were included in the analysis of EQ1 in an effort to compare the outputs of the two different methods of implementation. PHASE FOUR DISSEMINATION AND UTILIZATION Prior to submission of this final report, the report underwent internal quality assurance in line with EQUI and the ESS gender integration targets. Along with the Social Impact Project Director, the Gender Specialist reviewed the final report to assess the extent to which the final report addressed all gender issues/questions in the SOW as well as how Project interventions directly or indirectly supported USAID gender equality objectives. The ET will present its findings and conclusions in a virtual out-brief for the USAID/Haiti Technical Team. This will be followed by a Mission-wide Out-brief. The ET will revise the draft report to address USAID and grantee comments within ten business days of receiving reviewers’ feedback. The ET anticipates submitting the final evaluation report on June 30, 2021. Upon final approval, ESS will upload the report to the DEC. USAID/Haiti will disseminate the final evaluation report. In addition, ESS suggests that USAID consider disseminating evaluation findings in the “table sectorielle” meetings. METHODOLOGICAL LIMITATIONS AND BIASES The ET identified potential evaluation limitations and biases and developed mitigation strategies, which are discussed below. Health risks due to the COVID-19 pandemic. In Haiti, from January 3 to May 13, 2021, there have been 13,268 confirmed cases of COVID-19, with 269 deaths. With the exception of a small spike following the Christmas and New Year holidays, there has been a consistent decline in new cases daily since June 2020 (often fewer than 10 new cases per day), despite the airport and schools reopening.6 According to International SOS, the COVID-19 domestic operation impact is rated “Low,” but Haiti is still rated a high security risk. ESS submitted a workplan to USAID, including a detailed description of the proposed FGD methodology. While USAID preferred to avoid any in-person data collection, they approved the approach to conducting the FGDs proposed in the Inception and Design Report. The proposed methodology, which was described above, foresaw that two CHASE staff (a logistician and a driver) planned and organized the FGDs. The evaluation team recognizes that there was a risk of the spread of COVID-19 even though facilitation was done remotely. As a result, the ET followed the following measures to reduce the risk of contamination: ● One CHASE staff provided single-use face masks to participants who did not have one and are medically able to use them; 6 https://covid19.who.int/region/amro/country/ht 21 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ● CHASE team members, including facilitators and note-takers at CHASE headquarters, were also provided with masks for use during all data collection activities; ● CHASE made clear in setting up the FGDs that distancing guidelines are requested, and if upon arrival these conditions are not met, the team was empowered to not proceed with the FGDs; ● CHASE set up FGDs in large, well-ventilated rooms where participants could maintain a social distance while participating in the discussion; ● CHASE team members checked their temperatures daily and monitored their own health. No data collection staff became ill during data collection. However, the protocol was in place that should a member of the CHASE team have become ill and suspected to have COVID-19, in￾person data collection would have been paused until that person could be tested. Should that person have been positive, the CHASE management team would have communicated that fact back to all participants who had been in contact with that individual. For any further contact tracing, the Public Health Ministry of Haiti has been following its own protocols, which CHASE also followed. The CHASE team provided hand sanitizer to all meeting participants, and participants were instructed during the scheduling call to utilize sanitizer before, during, and after meetings. Technological challenges. Given the inability to conduct KIIs and FGDs in person and face-to-face due to the COVID-19 pandemic, the ET anticipated technological challenges. To overcome these difficulties, the ET had several alternatives to connect remotely (e.g., Zoom, WhatsApp, conference calls, Skype, etc.) and to use the one that was most relevant at the appropriate time. Similarly, in order to ensure good Internet connections, the ET chose between wireless routers of the two largest companies with national coverage, using the one that was most appropriate for a given location. Diminished openness in virtual meetings. The ET was concerned that people could be less open and willing to share personal experience in an interview or FGD that was conducted remotely. To mitigate this, facilitators spent more time than normal building relationships with target respondents. While some respondents were confused about why the discussions had to take place remotely, since they had been able to attend in-person trainings and other events without issue, in the end, it seemed as though people may have been even more open to discussing personal experiences when the interviewer was not present; it may have lent a feeling of anonymity and a lack of judgement. One upside of the KIIs being conducted remotely was that it was much easier to schedule and carry out KIIs on short notice because phone calls are less intrusive than in-person meetings, and the ET was able to carry out many more interviews per day than would have been possible if they had had to travel from office to office or town to town to conduct in-person interviews. Incomplete documentation. The Annual Reports are produced for purposes other than responding to evaluation questions; they are created independently of a research agenda. Therefore, they usually do not provide enough detail to answer specific EQs. The evaluation team noted that the documentation is incomplete and that there are changes in the format of the documents from one year to another. For example, Table 3 in Quarterly Report 2 of FY 2020 is key for responding to EQ-1, however this information is not available for every year. Similarly, the table on page 33 of the 2019 Annual Report was useful for addressing several EQs, but an updated version was not included in Annual Report 2020. Finally, Table 3 in Quarterly Report 3 for FY 2020 provided information on training delivered by topic and location and sex-disaggregated, but this information was not available for other quarters/years. It has therefore been difficult to aggregate information available from these sources. The ET requested the Ambassador’s deep dive report, which should have aggregated the results up to Q2 2020. In addition, the ET compiled draft (incomplete) tables that it will complete with information available in the Annual Report and IPTTs, and would request Chemonics to verify and complete. Lack of institutional memory. According to the 2020 quarterly reports, the MEL Coordinator left the Project in Q2 and the current COP and DCOP joined in Q3 2020. Since the evaluation spans all of FY 2018 and 2019 as well as 2020, the ET found that the staff who joined the Project in FY 2020 were not always able to provide all the necessary information regarding activities that took place in the first two years of the Project. The ET attempted to interview the previous COP to mitigate this risk but despite receiving his contact information from the current COP, they were not able to reach him. The effect of this lack of institutional memory is likely an incomplete picture of the planning and start-up phases of the Project. However, since most of the implementation did not begin until years two and three of the Project, the ET believes it did obtain a fairly complete picture of the implementation phase to date from the current staff members. Response Bias, Including Acquiescence Bias: Response bias is the risk that key informants may be motivated to provide responses that would be considered socially desirable or influential in obtaining donor support. Furthermore, some respondents have a tendency to agree with all the questions addressed to them. The ET mitigated this risk by making it clear in the informed consent introduction to the KIIs and FGDs that there will be no direct benefits associated with participating in the evaluation, Furthermore, the ET asked questions in such a way that did not elicit a “yes” or “no” answer, and requested the respondent to elaborate further on his/her reply. Gender Bias: Gender bias is a risk because most individuals have a sub-conscious sense of “appropriate” roles and behavior for women and men. The ET reviewed gender-sensitive approaches during the inception period by taking and discussing USAID’s Gender 101 training, focusing discussion on what possible gender preconceptions might come into play during this evaluation, and reviewing how to minimize these during data collection and analysis. In addition, ESS sub-contractor EnCompass reviewed evaluation deliverables and suggested how the ET could better integrate gender considerations at various stages, including design, data collection, analysis, and reporting. 23 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV III. FINDINGS AND CONCLUSIONS This section discusses findings from the ET’s document review and fieldwork that answer the evaluation questions (EQs). As discussed above, in the document review the ET reviewed URP annual work plans, annual reports, M&E plans, and sub-awardee contracts. The fieldwork consisted of KIIs, GIs, and FGDs. A list of documents the ET consulted can be found in Annex D, and a list of KIIs, GIs, and FGDs conducted in the fieldwork can be found in Annex E. EQ-1 WHAT ARE THE STRENGTHS AND WEAKNESSES OF THE IMPLEMENTATION MECHANISMS (DIRECT INVESTMENT VS. GRANTS) AS UTILIZED SEPARATELY AND TOGETHER TO PROMOTE IMPROVED NATURAL RESOURCE MANAGEMENT? The ET assessed the strengths and weaknesses of direct investment versus grants by analyzing the following differences between the two implementation mechanisms: 1) the outputs7 reached by each mechanism;8 2) the budget allocated for each mechanism; 3) the extent to which each mechanism was able to provide training/technical assistance; 4) the number of beneficiaries who have applied improved management practices or technologies per type of implementation mechanism (EG 3.2-24); 5) the ability of each mechanism to partner with CBOs; 6) any delays experienced due the different modes of implementation (including impact of COVID-19); and 7) any difference in capacities to carry out monitoring, evaluation, and learning (MEL). FINDINGS The URP’s focus in the beginning on implementation through grants elicited a limited response from eligible local organizations. Concerned that this approach would not achieve the Project’s targets, the focus shifted to more direct investment starting in FY19. In FY 2018, according to the FY18 annual report, the URP used only the grant mechanism for implementation. The Project’s resilience-focused and reforestation activities were chosen through a process whereby local organizations were invited to submit proposals for how they could implement key resilience activities within their communities. The Project received proposals on a rolling basis from organizations scattered across the North and North East regions, and to date have selected 15 of these organizations and signed grant agreements with10.9 Though these are evenly distributed between the two regions (seven operating in the North, six operating in the North East, and two operating in both regions), within the regions, the activities do not cover the entire targeted area, and some areas and beneficiaries may be involved in more than one grant activity while many other parts of the region are not covered at all. Because the organizations proposed activities that they were already involved in or otherwise able to do, not all beneficiaries and areas received consideration for all types of resilience activities. According to the FY19 Annual Report, the URP introduced direct investment (DI) as a new mechanism in FY19. Project and USAID staff recounted that together they determined that a parallel approach was needed to directly reach key beneficiaries and targets, and to provide timely responses to 7 Number of beneficiaries, number of trees planted, number of hectares habilitated, etc. 8 For example, in FY 2019, in nine months, APAPANNE trained more than 450 farmers in improved agroforestry and woodlots management, environmental education, climate change, and pastureland management, and accompanied them to apply the promoted technical packages in their plots. Despite recurring economic stresses in the country, APAPANNE, managed to achieve close to 150 percent of its tree planting target for the year (source: Annual Report 2019). 9 The FY20 Annual Report stated that 12 grant agreements had been signed by the end of FY20. In KIIs with Project staff, the ET learned that an additional three had been signed in FY21. If the ET receives information from the Project about these recently signed agreements, FY20 numbers throughout this report will be updated. critical needs in areas outside of the identified priority zones. The Project created a DI unit, and Project staff stated in KIIs that this contributed significantly to its reaching and/or exceeding many of its targets, especially those related to the establishment and/or rehabilitation of forests and woodlots, agroforestry production, and the number of people receiving livelihood Project’s related co-benefits. According to the 2019 Annual Report, that approach provided for rapid and direct partnership with local government authorities, CBOs, faith-based organizations (FBOs), and youth or women’s groups and associations, contributing to strengthening and empowering them for sound engagement in similar activities in the future. It helped engage, empower, and build the capacity of targeted partners, including local authorities, to guide and coordinate natural resource management works in their respective communities. Through the DI approach, the Project installed small community decentralized tree nurseries, and is relying on tree seedling procurements from private reputable vendors in or around the community until the community nurseries are fully operational. This method facilitated in-kind community contributions for the implementation of Project activities, whether through tree-planting or community volunteer days. In FY 2020, the Project continued with using both DI and grants as implementation mechanisms and signed an additional six grant agreements. In FY 2020, according to the draft Annual Report FY 2020, the Project’s strategy for increasing tree cover is to provide both a mix of sub-awards to organizations whose missions align with Project objectives so they can scale their programs, and DI that focuses on partnerships with public authorities and CBOs to collaborate on installing community nurseries and planting trees under Project supervision. Project staff stated in KIIs that sub-awards are key to long-term sustainability of tree planting efforts, but that in their experience, DI has been much more flexible for rapidly deploying resources in the face of unpredictable constraints. In addition to its continued collaboration with CASB, by the end of FY20 the URP had signed grants with 12 organizations: the Centre pour l’Agriculture Saint Barnabas (CASB),10 the Ministry of Environment’s Centre de Germplasm de Dosmond, 11 Fondation Pour la Protection de la Biodiversité Marine (FoProBIM),12 Kiskeya Vèt,13 Institution Univers,14 FECCANO, Village Planète, Association des Apiculteurs de Ferrier (AAF), Organisation pour le Développement de Labadie (ODL), RECOCARNO, and Marais Salant Partout de Jacquesil (MSPJ) (see Annex F). In the course of FY2020, Project reports and Project staff stated that the Project put a greater focus on DI to establish decentralized nurseries near planting sites and quickly mobilize manm konbit, residents of the target community who joined forces with their neighbors to transport, transplant, and monitor seedlings, among other work. Outputs differed between Direct Investment Activities and Grant Activities. The following table shows the breakdown in outputs between grants and direct investment for each part of the Project (according to the Project’s FY20 Annual Report): 10 Support for reforestation for sustainable environmental management. 11 Tree seedlings production. 12 Rehabilitation of 100 hectares (100 ha) of mangroves in the protected area. 13 Reinforcement and improvement of the tree cover. 14 Development of an agroforestry production on 100 ha. 25 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV TABLE 4. URP'S OUTPUTS OF ENVIRONMENTAL SERVICES DELIVERED PER TYPE OF IMPLEMENTATION MECHANISM ENVIRONMENTAL SERVICES DIRECT INVESTMENT SUB-GRANT 3.1.1 Woodlot management techniques 1,120,432 seedlings transplanted15 715,229 seedlings transplanted through sub-grants to: Kiskeya Vèt Institution Univers, CASB and DDE￾NE/Ouanaminthe 2.2.1 and 2.4.1 Agricultural and fruit production techniques in agroforestry systems 460 farmers trained in establishing agroforestry systems 26 coffee and 12 cacao nurseries have been established through Sub-grants to FECCANO and RECOCCARNO 2.3.1 Feeding and managing livestock 518 farmers trained in forage production in agrosylvopastoral techniques to help them apply these practices on their own plots to feed their livestock 1,656 farmers given access to forage crop seeds through their local demonstration plots, leading to the production of over 159MTs of dry fodder Nine hay production units created with the goal or producing, storing, and marketing hay to communities. These units have produced 76MTs of hay to date. 632 farmers have reported now having access to hay. CASB, produced, stored and sold 7,500 kg of hay MARNDR’s Gwoupman Sante Bè (GSB) treated over 51,000 herbivores owned by 13,000 farmers during a deworming campaign 2.3.2 Beekeeping 128 beekeepers of Mouvement Coopératif Agricole de Dubourg (MCAD) in Limonade, the Cadre de Concertation pour la Promotion de l’Apiculture (CACOPA) in Fort￾Liberté, and the Association des Apiculteurs de Ferrier (AAF) in Ferrier participated in training sessions. A model apiary was established in collaboration with the municipality of Sainte-Suzanne Through subgrant to CASB, 15 beekeeper members of the Groupement Agricole Ouvray (GAO), the Initiative pour le Développement Durable d’Haiti (IDDH), and the Coopérative Agricole Rabouré de Terrier Rouge (CART) received improved beekeeping materials and equipment as part of the process of modernizing their respective apiaries A model apiary was established through a sub-grant to the Association des Apiculteurs de Ferrier (AAF)16 3.2 Mangrove restoration techniques None 110 hectares of protected marine coastline rehabilitated through Sub￾grants to FoProBiM and Village Planète To more closely compare the specific results from the two mechanisms as they pertain to the tree￾planting activities, the following table compares the numbers of seedlings produced and transplanted (as of the end of FY20), the number of beneficiaries served, the number of hectares covered, and the costs associated with these outputs (total amount budgeted, including funds not yet disbursed). The below table shows that while the total number of seedlings produced and transplanted is greater under the sub-grant mechanism, the percent of those seedlings that have been transplanted into their permanent 15 These numbers are as of the end of FY20. 16 Implementation began in early 2021. locations is significantly higher for those produced under direct investment, as is the number of hectares covered. A greater number of beneficiaries have participated in the direct investment activities than in the sub-grant activities, at a lower cost per beneficiary and a lower cost per hectare (though this can likely be explained by the lower numbers of seedlings that have been transplanted to date). The cost per seedling produced appears to be much higher under direct investment than through sub-grants. A better comparison of the differences in cost efficiency and results will only be possible when the activities have been completed and the final numbers can be compared. TABLE 5. URP'S OUTPUTS OF SEEDLINGS PRODUCED AND TRANSPLANTED PER TYPE OF IMPLEMENTED MECHANISM INDICATOR DIRECT INVESTMENT SUB-GRANT Number of seedlings produced or in production (not yet transplanted) 351,366 1,449,500 Number of seedlings transplanted 1,120,432 715,229 Percent of produced seedlings that have been transplanted 76% 33% Number of beneficiaries 1,825 887 Number of hectares covered 1,233 978 Total budget for seedling production 1,174,483 $1,250,136 Cost per seedling produced or planted $0.80 $0.58 Cost per beneficiary to date $644 $1,409 Cost per hectare covered to date $952 $1,278 Project and USAID staff stated that there were pros and cons to both mechanisms, with no method clearly preferred across-the-board. There were differing opinions about whether one or the other mechanism was better for the Project, with many respondents saying that both were important and appropriate within the Project because they accomplished slightly different goals. USAID staff expressed concern that the sub-grant approach created an additional management layer that increased the separation between the Project and the targeted results, and a move to DI made the Project more accountable to delivering those results. TABLE 6. PROS AND CONS OF DIFFERENT IMPLEMENTATION MECHANISMS MECHANISM PROS CONS Grants Allows for more sustainable capacity development of organizations and provides an entry into the community. Assists existing associations that were already involved in activities within the community that were of interest to the community. Organizations knew the communities and their specific needs and limitations and were able to recruit and motivate beneficiaries to participate in Project activities. Difficulty on the part of many organizations to meet Project reporting requirements. Requires in many cases significant investment of resources into training these organizations to be able to produce the required deliverables Results in benefits that are not evenly distributed among target areas and beneficiaries 27 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV MECHANISM PROS CONS Excludes the participation of some organizations that lack sufficient management capacity Direct Investment Faster and more efficient way to reach targets Top-down approach allows for strategic and logical progress toward the Project’s Theory of Change Promotes greater accountability to the donor to deliver results Potentially less sustainable development of local capacity if outside management and trainers are primarily managing the activities Supply driven approach that rolls out uniform activities in many areas may result in communities being targeted for activities that do not interest them (this can be mitigated with significant community consultation). Most sub-awardee respondents stated that the grant process was too long and difficult, with reimbursements taking too long, which made it difficult to reach targets. Of the sub￾awardees interviewed that had already submitted at least one deliverable, most expressed frustration with the delays in receiving reimbursements. In an effort to adhere to a results-based payments system, the Project required that payments to sub-awardees occur only upon delivery of agreed-upon deliverables, usually a tangible delivery of planting material or completion of certain activities along with a written report. Many stated that they were not accustomed to working within this type of deliverables-based payment system and that it was difficult for them to produce required outputs before receiving any payments from the Project. A common related experience of sub-awardees has been that the Project promised equipment, materials, or other assistance that came very late or still has not come. Many leaders of recipient organizations said that they would have greatly preferred agreements where they received funds to pay for equipment, materials and labor up front, stating that they would have been able to begin and complete their activities much faster and more efficiently. Most sub-awardee KIIs stated that this was a major challenge for their organization, which had in all previous projects received funding up front that covered the costs of their activities. Reimbursements from the Project were very slow, given a series of bureaucratic procedures that took time, and as a result, organizations stated that they often could not begin work on the next activity or deliverable for many months while they waited for reimbursements that would allow them to pay for the next set of activities. CONCLUSIONS Given the short timeline, substantial delays, and the aggressive targets of the project, the move to direct investment was logical. Despite the project officially launching in September 2017, the Project signed only one grant agreement in 2018 and was seriously behind in reaching its targets for years one and two. This was due to several factors, including prolonged droughts in the northern parts of the country, country-wide lockdowns and political unrest, and difficulty with finding and keeping qualified staff. A series of meetings took place between Chemonics and USAID in which concern over these delays inspired a shift from the original strategy of implementing the bulk of the project through grants to local organizations, to a strategy of direct investment as described above. With this shift, seedling production was able to ramp up substantially even though the actual number of seedlings planted by the end of FY20 still did not meet the target. Even with fewer delays at the beginning of the project, the grants mechanism as it was implemented was unlikely to be able to reach the Project’s targets. The original Chemonics proposal and Year 1 workplan assumed that five years would be sufficient to employ a strategy of working mostly through grants. This strategy assumed that there would be enough local organizations that were interested in and able to produce the number of seedlings required by the Project. In an effort to adhere to a results-based payments system, the Project required that payments to sub-awardees occur only upon delivery of agreed-upon deliverables, usually a tangible delivery of planting material or completion of certain activities along with a written report. This method of payment forced organizations to front the costs of the activity, including payments to laborers and purchase of some supplies for sometimes months while waiting for a reimbursement from the URP. The reality of the situation in Haiti is that while there are plenty of local organizations that are excited about working on USAID projects, few organizations have the financial resources to be able to complete project activities without up-front payments. Chemonics tried to help this situation by promising to supply all necessary materials so organizations would only have to pay upfront for labor, but this has not worked as planned, with materials sometimes taking months to arrive, leaving organizations stranded and unable to start their activities. The arrangement of reimbursing organizations after activities are completed may not be a feasible approach in a resource-poor place like these targeted regions in Haiti - Fixed Amount Awards may in some cases need to include an initial disbursement to allow sub-grantees to begin their activities, in addition to deliverable-based payments. Given the time required for processing grant agreements and working with organizations, the project does not have time to pursue further grants. Grant management, including capacity building, will take too long and likely face too many delays to be able to successfully complete sub-grant activities before the end of the Project. A focus on grants, with organizations selected through a call for proposals, led to the scattered and sometimes not-logically connected assortment of activities. The demand￾driven approach of working through sub-grants ensures that the Project is not attempting to implement an activity that is not of interest in a particular community. On the other hand, it leads to a scattered approach of disconnected individual activities that do not always have a logical connection to the project’s objectives and do not evenly target all communities within the targeted area. Direct investment allows for a more structured, top-down approach that facilitates a logical theory of change. With the move to direct investment, the project became more able to focus on the specific activities that would lead to the achievement of the main objectives of the project, namely the planting of 4 million trees. The Project set up tree nurseries in strategic areas and staffed and resourced them to allow them to efficiently produce the seedlings the Project needs. Some resilience activities (such as beekeeping) may not lend themselves as well to direct investment, but seedling production was an area where DI has produced consistently positive results. While a similar uniform approach to all activities within the target areas would have also had its challenges, one advantage would have been a consistent Theory of Change, a logical connection to the Project’s objectives, and some economies of scale in implementation. Delays on the part of the Project in providing promised materials have had negative impacts on activities. As mentioned above, one of the tenets of the payment-for-deliverables arrangement between the Project and the sub-awardees was that the Project would procure and supply all necessary materials. While in theory this could make the work of the grantees more efficient and of uniform quality, in reality, the Project did not keep to its delivery schedule for many necessary tools and materials, causing the recipient organizations to have to wait in order to begin or continue their planned activities. The ET did not learn why these delays occurred, but they were a frequent enough experience among grantees to suggest a systemic issue with Project procurement procedures rather than a one￾time or occasional problem due to specific unforeseen circumstances. 29 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV EQ-2 TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT IMPROVING BENEFICIARIES’ RESILIENCE IN THE FACE OF NATURAL AND ECONOMIC SHOCKS? The ET examined three components of this EQ through: 1) an analysis of the interrelated activities to improve the resilience of beneficiaries (setting up agroforestry systems / giving rapid benefits to farmers / strengthening the livelihoods of farmers / strengthening small processing companies / facilitate financial inclusion, etc.); 2) a study of the extent to which these interventions have yielded the expected results at various levels: social, ecological, and economic; and 3) an analysis of how implementation is taking place in order to produce the expected results. FINDINGS The focus of the Project shifted from primarily reforestation activities to a broader collection of activities intended to improve the resilience of beneficiaries by increasing their incomes and improving their management of livestock, but the Project’s indicators do not reflect that focus. Nearly every stakeholder who was interviewed agreed that resilience activities were a necessary part of a reforestation project. Project staff and beneficiaries stated that this was a necessary shift, as resilience and income increases are the primary urgent needs of the people, even though most people know that reforestation has important long-term benefits. A common observation among project and USAID staff was that the indicators that are being tracked are more tied to the reforestation objectives than the resilience objectives, and that even though resilience activities have become a strong focus of the Project, the monitoring indicators do not reflect this shift and continue to focus on the tree-planting activities. For example, the Indicator Tracking Tables do not require reporting on questions of types of shocks and stresses that beneficiaries have experienced, their coping strategies, food security, working capital, productive assets, or any measures of poverty. These indicators are studied in the Resilience Baseline Study from January, 2020, and will presumably be asked again in and end-line survey, but are not part of the regular progress reporting.17 The same Project and USAID interviewees noted that the indicators that are reported against regularly are the ones that the Project will end up focusing time, budget and energy on achieving, and that not regularly measuring progress towards resilience removes pressure on the Project to achieve tangible resilience-focused results. Opinions differ on the effectiveness of different resilience activities. According to the Annual Reports, the production of cash crops and seasonal crops supported by the Project in agroforestry systems brought in additional income, which helps improve the farmers’ economic resilience. The Project is supporting high-value food and fruit crops (such as pigeon pea, yam, plantain, pineapple, 17 USAID defines resilience as “the ability of people, households, communities, countries and systems to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.” The ability to handle adversity and change without compromising future well-being depends on a number of capacities and how they are used in the face of shocks and stresses. Resilience is fundamentally about transformation – changing the very basis on which individuals and households can make decisions that influence their capacity to deal with stresses and shocks. Resilience can be measured through three capacities and by relating them to well-being outcomes: Absorptive capacity. Absorptive capacity is the ability to minimize exposure to shocks and stresses through preventative measures and appropriate coping strategies to avoid permanent, negative impacts. Adaptive capacity. Adaptive capacity is the ability to make proactive and informed choices about alternative livelihood strategies based on an understanding of changing conditions. Transformative capacity. Transformative capacity involves the governance mechanisms, policies/ regulations, infrastructure, community networks, and formal and informal social protection mechanisms that constitute the enabling environment for systemic change. (Resilience Baseline Study, January 2020) chayote, ginger, and passion fruit) in the coffee, cocoa, and cashew agro-forestry systems in order to increase the short-term profitability of the agroforestry plots. By diversifying livelihoods, the Project states that is also improving beneficiaries’ economic resilience. Many beneficiaries of both genders and project staff agreed that the activities with which they were involved were very effective at increasing income. These increases in income were due mainly to the Project’s payment of a daily wage for laborers who worked in the tree nurseries and demonstration plots (average 385 gourdes/day). Participants in trainings were also paid a wage to implement improved techniques on their own private parcels of land, usually with an arrangement that a beneficiary would contribute one day of unpaid labor to benefit the community or association for every five days of paid labor. In addition to wages for labor, smallholder farmers were given access to some staple and cash crop seeds, forage planting materials, and technical assistance to create sustenance and revenue while the trees mature. Staple and cash crop seeds like pineapple, yam, pigeon peas, and black-eyed peas were provided based on the season, soil type, and beneficiaries’ needs, as well as their nutritional and cash values. The Project and its local partners supported 6,596 individuals to increase production for improved food security, and beneficiaries stated that these donations of seeds, training, and materials were very beneficial to their productivity, though we have no concrete measures of the levels of benefits or productivity increases that they have seen. GOH and USAID respondents tended to see the resilience activities as not very effective at increasing income for either gender of participant, possibly because these respondents were focusing on the longer-term income-generating agroforestry activities (the future productive benefits of the trees that were planted via the Project or the improved agricultural techniques that were learned) rather than the immediately beneficial paid labor income increases. These future agroforestry-related income streams were less frequently cited by beneficiaries as benefits of the Project. In many cases, the crops had yet to produce a first harvest, and in a few cases, people expressed doubt that they would be able to find markets for significant increases in production. All respondents agreed that activities got off to a very slow start due to drought and other issues in Haiti, so there have not been many visible impacts to date. Many respondents of all categories stated that the resilience activities that were part of the project may have benefited people, but that the activities were too few and spread too thinly among communities to make a big difference. Opinions differed about whether there should have been more communities and beneficiaries targeted, or whether the project should have focused more narrowly on fewer places, people and activities in order to have more impact. A common view among leaders of the local associations was a desire to work with many more people than they had the resources from the Project to serve. Beneficiaries on the other hand, while appreciative of the interventions they were participating in, expressed desire for their own communities to be able to benefit from more activities and see more and deeper benefits rather than have so many communities benefit a little (of course, no beneficiaries wanted their communities to be excluded from the Project so that other communities could have more benefits). Project staff stated that the Project could have had more tangible and long-lasting results if fewer communities had been targeted with more resources devoted to each. The CRDD is not yet functioning. Regarding the improvement of access to technology, the annual plans and reports indicate that the Centre Rural de Développement Durable (CRDD) is not yet fully functional. The Project’s objective of establishing CASB as a CRDD was that it would help improve resilience in the communities it serves by being an anchor institution to continue providing much￾needed agriculture and agroforestry training and research services after the end of the Project. The Project expected to have a functional CRDD by the end of the second year. This did not happen for a number of both internal and external reasons. At the start of the project it was thought that the CRDD was to be an entity of the CASB, but after completing a Non-U.S. Organization Pre-Award Survey (NUPAS) of the CASB with the assistance of a consultant to gauge the CASB’s administrative, financial, and operational capacity, project managers changed their minds and looked for alternative management 31 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV arrangements. They contacted other actors such as the Departmental Directions of the Ministries of Agriculture and the Environment of the North and North-east, the Roi Christophe University, and the Episcopal University, to explore possibilities of having a more independent CRDD. In 2019, the Project reached an agreement with the Université Episcopale d’Haïti (UNEPH) – CASB’s parent institution – and the CASB for the allocation of five hectares on CASB’s campus on which to establish the CRDD. In 2020, the institutional infrastructure has been nearly completed, and the Project has reported that it has coordinated two general assemblies with 13 partner institutions and organizations (including the Ministries of Environment and Agriculture, and University of Limonade) to agree on governance. They now have proposed statutes, internal regulations, and a consensus management plan for operations. In addition to this change in strategy, the repeated socio-political crises and the country-wide lockdowns have prevented many organizational meetings and greatly delayed the CRDD. The CRDD installed agroforestry practical training sites, including a 3.5-hectare forage demonstration plot, a one-hectare woodlot, and a model apiary with melliferous tree species. In anticipation of demonstration plots on “jaden lakou” home gardens, the Project supported the newly hired CRDD technical team to develop a technical sheet on modern jaden lakou, now in use by Project beneficiaries. The Project reported in FY20 reports that it is making efforts to improve processing and market access for agroforestry products, though beneficiaries have not reported seeing significant improvements to date. To identify drivers of change, bottlenecks, and market opportunities for agroforestry value-added chains, a rapid assessment of non-timber agroforestry commodity value chains for coffee and cocoa was conducted in 2018 based on information drawn from meetings with various value chain actors, including management of FECCANO, RECOCARNO, CHOCOLAKAY, Manje Peyan’m Group, and Maison NOVELA. Some beneficiaries expressed concern about their ability to find markets for some of the new crops that the Project has promoted; not enough time has passed since these activities began to see a full harvest cycle and the results of the marketing of these products. In FY20 reports the Project reported having made progress in the high-value cacao and coffee value chains by launching sub-award partnerships with FECCANO and RECOCCARNO. The Project worked with FECCANO on the development of a sub-award to increase tree cover on 1,000 hectares in Grande Rivière du Nord, Milot, Bahon, and Plaine du Nord, and improve cocoa-based agroforestry production. It also worked with RECOCARNO on a sub-award to increase 723 hectares of tree cover in Vallières, Sainte Suzanne, Mont-Organisé, and Milot and improve coffee-based agroforestry production in the target area. In 2020, 26 coffee and 12 cacao nurseries were established. Other value chains have had more challenges and fewer tangible results; the periodic reports indicate that the ackee, moringa, and breadfruit value chains have not seen the progress that coffee and cacao have seen in this Project, likely because of the absence of existing cooperatives or enterprises. A KII with the Antillean Canning Company (ANCASA), a processing company that focuses on ackee, revealed that the Project’s initial discussions and agreements with the company fell through as a result of currency exchange issues, so this value chain remains virtually untouched by the Project. The resilience activities that were most frequently mentioned by all categories of respondents as effective were the livestock-focused activities, followed by the beekeeping activities. The URP conducted an assessment of animal value chains (goats and cattle specifically) at the beginning of FY18. According to data collected, free roaming livestock presents a big issue. Farmers expressed the need and their willingness to raise animals in pens or enclosures (especially goats and cattle) to reduce their negative impacts on crops and cope with thefts. They have also proposed to setup managed communal or family pastures and animal enclosures planted with more drought-resistant forage. To rapidly help address this feed problem, prevent overgrazing and destruction of tree cover, and build capacities for increased resilience to prolonged drought and erratic rainfall, the Project, through the DI mechanism, trained more than 250 livestock farmers (including 97 women) in forage, hay, and silage production, including choice of adapted plant species, harvesting, feeding, and storage. To help expand the forage production program, the Project worked in partnership with the directorates of agriculture for the North and the North-East, Université Henry Christophe de Limonade, and various agro-pastoralist associations to establish 15 hectares of forage production and demonstration plots across six communes (Ouanaminthe, Ferrier, Limonade, Terrier-Rouge, Trou du Nord, and Grande Riviere du Nord). In 2020, some livestock owners have partnered with other members of their associations to produce hay with Project support, creating new hay processing units that package and conserve fodder crops to be stored for the next dry season. FY2020 results include: ● Training of 518 farmers in forage production in agrosylvopastoral techniques to help them apply these practices on their own plots to feed their livestock; ● Giving 1,656 farmers access to forage crop seeds through their local demonstration plots, leading to the production of more than 159 metric tons (MTs) of dry fodder; ● Creating nine hay production units with the goal or producing, storing, and marketing hay to communities. These units have produced 76 MTs of hay to date; and ● 632 farmers have reported now having access to hay. The Project closely collaborates with the Directions Départementale Agricole (DDA) on livestock. They reported that no livestock were lost in FY2020 due to unavailability of forage. Three beekeeping associations were consulted through KIIs and FGDs. All of them expressed strong positive reactions to the Project and cited tangible benefits including a significant increase in the production of honey, which is easily sold on local markets. CONCLUSIONS While the various resilience activities individually could be beneficial, the project lacked a cohesive strategy (or struggled to effectively implement a strategy) that would ensure enough of an income increase to prevent further deforestation. A common response among GOH and USAID respondents to interview questions about whether the resilience activities will actually result in decreased deforestation was that deforestation is a very complex issue with a lot of interrelated causes. Beneficiaries and sub-awardees put it more directly, saying that people will continue to cut down trees as long as they are poor and have no other alternatives to earning enough income. While a few beneficiaries described potential for significant income increases as a result of project activities, most agreed that these benefits were not enough for them to meet all of their families’ needs, and that even if their family was not in dire enough need to cut down trees, there were an almost infinite number of other even poorer people who had no other choice but to continue to cut down trees. The activities most likely to lead to improved resilience are the livestock-focused ones. Livestock are an extremely important way for subsistence farmers to provide significant nutritional benefits to their families. The ability of livestock to reproduce can allow a family to generate and store wealth that can accumulate over time and be large revenue sources. Preventing the deaths of animals due to drought is a critical way to increase a family’s resilience and potential for long-term income gains. The Project introduced techniques of growing, harvesting, and storing forage crops that can feed livestock through dry seasons and even potentially sustained droughts. Several associations focused on livestock took part in FGDs and were very positive about the difference the livestock-focused activities had on their own families’ livestock management strategies. As livestock freely roaming in a search for food are also a primary cause of deforestation and the failure of other tree-planting efforts, encouraging people to keep their animals enclosed and fed with forage crops has the potential to both protect newly planted trees and make families more resilient in the face of drought. The limitation is that even just a few roaming livestock can still cause great harm to new tree plantations, so a strategy of livestock containment and forage production only works if everyone follows this strategy. 33 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Cash for work and provision of free tools, plants, and seeds did affect beneficiaries’ livelihoods immediately (though temporarily). The most-cited benefit of the project was the cash for work strategy employed by the Project through both the grants and DI mechanisms. While beneficiaries would have to wait one or more seasons for crops to grow and be harvested and sold, payments for labor were issued usually on a weekly basis and could immediately start meeting a family’s most urgent financial needs. The environmental and long-term production benefits of the project’s activities took a backseat to the immediate benefits of paid labor. The system of paying five days of wages for six days of work was seen favorably by beneficiaries and sub-awardees alike. While it is unknown whether the longer-term income generating activities will permanently raise incomes for beneficiaries, it is clear that this instant infusion of cash, free tools, plants, and seeds did have an immediate impact. With the Project purchasing as much as possible in terms of seeds and materials locally, and labor coming directly from the local communities, the Project can be assumed to have contributed positively to the local economies. EQ-3 TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT INTEGRATING YOUTH AND GENDER? The ET assessed the extent to which gender and youth were integrated into the Project by analyzing the following elements: 1) whether an assessment was undertaken to analyze the issues specific to youth and gender during the design/initial phase; 2) the extent to which youth and gender are taken into consideration during implementation (e.g., targeting specific youth/women ’s groups; design of activities that are relevant to youth/women; women’s access to credit; women’s leadership in the community, including targeting of women and youth in the sub-grantees’ agreements); 3) the extent to which youth/women have participated in capacity-building activities; 4) the extent to which youth/gender aspects are integrated into the URP’s MEL; and 5) whether there are sufficient human resources to ensure that youth/gender are integrated into the URP. These questions aim to assess the project’s technical approach and systems to ensure compliance with ADS 205, which operationalizes USAID’s Gender Equality and Female Empowerment Policy and calls for gender integration in all stages of activity design, implementation, and MEL. This includes the requirement, further codified under the 2018 Women’s Entrepreneurship and Economic Empowerment Act, that Agency activities are shaped by gender analysis. In addition to evaluating whether gender analysis data specifically informed project design and implementation, more specific/sample probes regarding implementation summarized here address key ADS 205 gender analysis domains that should be considered, such as “Access to and Control over Assets and Resources” and “Patterns of Power and Decision-making.” They also reflect that USAID’s Gender Policy identifies among its outcomes not only reduction of disparities in access to, but also “control over and benefit from resources, opportunities, and services” and “[i]ncreased realization of rights and decision making among women and girls.” These also aim to support the mission to (in compliance with ADS 205): · “Review Implementing Partners’ proposed Activity MEL Plans (see ADS 201.3.4.9) and advise on the inclusion of a gender-sensitive approach to monitoring and gender-sensitive indicators, as appropriate;” · “Review actual annual data against planned targets with attention to whether there are any gaps between the extent to which females and males are participating in and benefiting from projects and activities and discuss the findings with Implementing Partners (see ADS 201.3.4.10). Performance Plan and Reports (PPRs) must detail gender equality and female empowerment results achieved in a reporting fiscal year (see 205.3.9.2)…” FINDINGS It is unclear whether and to what extent gender/youth were considered during the URP’s initial/design phase. From the available documentation, it appears that the Political Economy Analysis (PEA) might have covered gender/youth-related aspects.18 However, since the PEA was not shared with the ET, it is difficult to assess to what extent gender/youth were taken into consideration during the initial phases of the Project. It does not seem that the URP has undertaken any specific youth/gender analysis during the design or initial phases of the Project to guide its implementation of activities. Furthermore, the Project documents do not include a definition of “youth” – though in the Indicator Performance Tracking Table (IPTT), in some instances, the beneficiaries are disaggregated by age (e.g., beneficiaries between 15 and 29 years old but not by both gender and age.19 Furthermore, it is unclear to what extent the recommendations (if any) regarding gender/youth of the completed PEA or other relevant gender analyses/assessments such as the USAID Haiti 2016 Gender Assessment were integrated into implementation.20 For example, according to an assessment done in 2020, one of the PEA’s conclusions was that “women participated in physical work but rarely were decision-makers when it came to issues related to the communities”;21 available documents do not provide information as to whether the Project used this conclusion to design and/or implement activities. URP has not partnered directly with any women’s associations, though several women’s CBOs are included in consortia that have received sub-grants. The Project conducted a rapid assessment of women’s and youth associations in the Project target area in 2018. These associations were encouraged to apply for sub-awards (see Annex I). The following women’s associations have been participating members of consortia that received sub-awards through the Project: ● AFDRP (Assosyasyon Fanm pou Devlopman Roche Plate) ● MPFV (Mouvman Pwogresis Fanm Valyè) ● AFVMJ (Asosyasyon Fanm Vanyan Marais salant Jacquezil) ● OTDO (Organisation des Travailleurs pour le Développement de Ouanaminthe) In 2020, the Project trained the members of the Rassemblement des Femmes Engagées de Ouanaminthe (RFEO) engaged in the processing and commercialization of honey, cashew, and moringa. Similarly, the Project identified a network of youth clubs associated with RECOCARNO. Each of the eight cooperative members of RECOCARNO had developed a youth club, forming a network of young coffee producers who promote agroforestry in their community and develop their own economic capacity. To carry out their production and marketing activities, these clubs all receive financial support from RECOCARNO. According to the 2018 Annual Report, the Project planned to support these youth clubs in FY 2019 since they could be models for other young people. However, neither the FY20 Annual Report nor any of the KIIs respondents mentioned any of these youth-focused activities, so it is unclear to what extent these have occurred. In 2019 and 2020, the URP achieved 244 percent of its initial target for the year in terms of percentage of female participants who increased their access to productive economic 18 According to the Annual Report 2018, in PEA “vulnerable populations like youth and women are omitted from sub￾watershed management decision making. The Project must engage and empower these demographics to be more involved in reforestation action and good environmental practices.” 19 For example, EG 3.2-24 “Number of individuals in the agriculture system who have applied improved management practices or technologies (FTF).” 20 The 2019 Annual Report states that the “Conclusions from the Project’s completed Political Economy Analysis were not integrated fully into implementation approaches. The structural barriers to improved natural resource governance identified in the PEA, and the organizational capacities identified through the Community Options Analysis and Investment Toolkit (COAIT) exercise, must be taken into consideration in the Project’s strategy and approach.” 21 JM Bonis Charancle and JM Bonis, 2020. The community options analysis and investment toolkit (COAIT): Its use and potential in the USAID Reforestation Project (URP), Final Report, January 21, 2020. 35 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV resources (assets, credit, income or employment) (GNDR-2). All respondents in all categories stated that women are actively included in all activities, with some activities specifically targeting women, but most activities are equally open to both genders. Women’s economic resources increased mainly through paid day laboring, and no respondent mentioned any difficulty in recruiting women to participate in Project activities, stating that women in these communities are active in the fields of agriculture and are very interested in earning income through paid work. However, the ET did not receive concrete estimates of the amount of economic benefit that these women are receiving, and it is difficult to assess whether this participation and economic benefit leads to “empowerment” as is mentioned in the Annual Reports.22 Most people agreed that there are fairly established gender roles in different sectors and the project has not changed those, but has worked within them. For example, women were employed in tree nurseries to fill containers with soil and establish and care for the seedlings, while men more frequently were employed to do manual labor related to clearing land and building enclosures. URP has implemented a series of activities targeting children and youth. These included: 1. Organizing an inter-school competition for environmental education on landscape management: In partnership with the Ministry of the Environment’s North Directorate and the Ministry of National Education, the Project organized an inter-school competition called TéléGénie to educate students on the importance of sound natural resource management. Sixteen schools23 (240 students) competed for four weeks on an eco-citizenship curriculum that covers the water cycle, climate change impacts, biodiversity protection, natural resources management, risk and disaster reduction, and the importance of reforestation. The ET is not aware of any gender-specific targeting in these student-focused activities. 2. Increasing community awareness of improved natural resources and tree cover managements: The Project supported the active involvement and participation of more than 3,500 school children in several schools24 to plant more than 10,000 trees. In some cases, these seedlings were produced by the students themselves in Project-supported tree nurseries in their school yards, such as the 3,225 seedlings produced in Capotille.25 3. Supporting graduating students from well-established universities in the preparation of their undergraduate dissertation on innovative topics, with a focus on the valorization of non-wood agroforestry products. This innovative activity is working with five students from two universities.26 The URP conducted several internal and external gender trainings and implemented awareness-raising activities to promote women’s leadership and economic empowerment. Internally, the URP’s Gender, Youth, and Social Inclusion (GESI) Coordinator conducted two sessions to raise awareness of Chemonics’ policies to protect the staff against all forms of exploitation, 22 USAID’s 2012 Gender Equality and Female Empowerment Policy (which was the guiding document for the time period evaluated) identifies among its outcomes not only reduction disparities in access to, but also “control over and benefit from resources, opportunities, and services” and “increased realization of rights and decision making among women and girls. This same policy indicates that “female empowerment is achieved when women and girls acquire the power to act freely, exercise their rights, and fulfill their potential as full and equal members of society. While empowerment often comes from within, and individuals empower themselves, cultures, societies, and institutions create conditions that facilitate or undermine the possibilities for empowerment. 23 From Quartier Morin, Milot, Plaine du Nord, and Limonade. 24 Institution Univers of Ouanaminthe, Institution Madame Rose of Morne Rouge, Ecole Nationale de Dericourt of Limonade, the two schools of Capotille, and five schools in Terrier Rouge (Collège Saint Bathelemy, Lycée de Terrier Rouge, Collège Eben-Ezer, Institut Baden Powell, and Ecole Nationale de Terrier Rouge). 25 Institution Mixte la Vie des Enfants and Ecole Nationale de Cana. 26 Université Américaine des Sciences Modernes d’Haïti (UNASMOH), Université d’Etat d’Haïti for Campus de Limonade, and Faculté d’Agronomie et de Médecine Vétérinaire (FAMV). discrimination, harassment, and sexual abuse in the workplace.27 Externally, in 2018, the GESI Coordinator conducted five sessions on gender and youth inclusion issues attended by 52 participants. However, the documentation does not specify whether participants were Project staff, implementing partners, or government staff. That said, the participants expressed their increased appreciation and understanding of the need for an equitable participation of women and youth in reforestation and agriculture activities. In 2020, the GESI Coordinator continued to provide orientation sessions for sub￾grantees to emphasize the need to involve women and youth at all levels in the implementation of their activities. Village Planète, FECCANO, and CASB participated in workshops on how to integrate gender and inclusion into their daily activities and also how to report on the impacts of their efforts.28 Finally, the URP used World Women’s Rights Day to raise awareness to promote leadership and economic empowerment for women.29 In interviews and FGDs, the ET frequently heard from the organizations that had received gender-related training that women were involved in association activities long before the Project required it, and that they had no real problem with women’s participation and empowerment. They were happy to participate in these trainings, but wanted to be sure the ET understood that women were empowered in their associations before as well. URP achieved its set target for percentage of women on sub-watershed management plans validation committees in leadership and decision-making roles in 2018 and 2019. The percentage of women on sub-watershed committees was 40 percent in 2018 and 30 percent in 201930. The achievement of this target was helped by the introduction, in the electoral law, of the quota of at least 30 percent of women in municipal councils, which has resulted in a significant increase in female presence in municipalities and rural sections. This is a substantial positive step, but only has a strong impact if women are also empowered to effectively exercise their roles. The 2019 Annual Report states that the agroforestry approaches have “improved women’s empowerment, specifically in terms of access to credit and leadership in the community.”31 The ET held FGDs that included female members of all of the Sub-Watershed Management Committees and heard in every case that the women’s participation is meaningful and women feel they are actively involved in decision-making and discussions. URP has a dedicated GESI Coordinator. However, because this person was unable to be reached for an interview during data collection, it is unclear when this person started in the position and if there has been any turnover. Furthermore, it is unclear whether the GESI position is full-time or if this person has other technical leadership responsibilities that could either complement or create competing priorities. URP has sex- and age-disaggregated indicators, but its gender/youth reporting is inconsistent. Consistently in interviews with Project staff and sub-awardees, respondents expressed some confusion about why we were asking so many questions about gender and youth inclusion when they had already told us that they were meeting all targets and women were active in all activities. It was clear that beyond participation numbers for women, no one was tracking any other indicators of 27 The first session was held on February 19, 2020 in the Cap-Haitien office and the second session on February 20, 2020 in the Ouanaminthe office. 28 Four workshops were conducted for this quarter: Village Planète on January 28, 2020, FECCANO on February 4 and March 6, 2020 and CASB on April 19, 2020. (Q2 2020). 29 For World Women’s Rights Day, on March 8, 2020, the Project organized two events to promote women’s leadership in the North and North-East departments. The theme of the events was: “Tout comme les hommes, les femmes peuvent produire des denrées agricoles ou diriger des espaces de décision” (Like men, women can produce agricultural commodities or run decision-making processes). Thus 94 female beneficiaries, including 54 women in Region 1 and 40 women in Region 2, were able to share their experiences in terms of capacity building in economics and leadership. 30 The 2020 Indicator Tracking Table that was provided to the ET does not include numbers for this indicator for 2020. The ET assumes that this is because no new Sub-Watershed Management Committees were formed in 2020 so the indicator was not tracked. All GIs with SWMC members stated that the percentage of female committee members was in line with the Project requirements, so the ET has no reason to believe that this target was not met in 2020. 31 MEL 2018 37 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV inclusion. In terms of M&E, initially in 2018, the URP had identified one indicator (PL-14) “Percentage of women on sub-watershed management plans validation committees in leadership and decision-making roles.” In 2019, a second indictor was added (GNDR-2): to monitor the “Percentage of female participants in USG-assisted programs designed to increase access to productive economic resources (assets, credit, income or employment)” offering the following rationale: “information generated by this indicator will be used to monitor and report on achievements linked to broader outcomes of gender equality and female empowerment. Additionally, the information will inform a wide range of gender￾related public reporting and communications products, and facilitate responses to gender-related inquiries from internal and external stakeholders.”32 In terms of reporting, whereas the 2018 Annual Report had a section dedicated to GESI, the 2019 and 2020 (draft) Annual Reports do not. Furthermore, it is difficult to assess the number of people who were trained disaggregated by sex. While the number of beneficiaries of some training activities is sex-disaggregated, this is not the case across the board.33 Finally, though the Indicator Performance Tracking Table (IPTT) tracks certain indicators by age, the MEL Plan does not include any indicator to monitor URP’s activities targeting youth. CONCLUSIONS Gender targets and quotas have been easily met by partners, and women appear to be active in decision-making, not just token members. This was stated unanimously by all organizations and groups of beneficiaries, with most stating that even before the Project required it, women were active participants, members and even leaders of associations. To minimize potential self￾censorship in mixed-gender FGDs, the ET conducted three all-female FGDs, and the same message was relayed in these: that women are active participants and involved in decision-making and were active before the Project as well. The project has not changed deep-seated gender roles but has ensured that women are welcome to participate in any activities. The most common response among sub-awardees and beneficiaries alike concerning women’s participation was that women were welcome to participate in any and all activities that men were. When pressed, they went on to say that of course, there are some parts of jobs that are more appropriate for men and others that are more appropriate for women, so people tended to work within those traditional groupings. The daily wage that was paid for labor was the same for men and women, and both genders were involved in every activity, though they often sorted themselves out by gender when it came to specific tasks, such as those involving heavy lifting. Respondents, including women’s groups, did not identify any gender-related barriers or issues related to Project activities. It is unclear whether the Project was hoping to change potentially inequitable gender norms through its activities, or was just hoping to ensure that women were benefiting from the Project as much as men. Without understanding whether the Project had the changing of gender norms as an objective, it is difficult to assess whether the Project’s approach to reaching women within the context of traditional gender roles in the sector was a positive way to ensure the equal participation of women or in fact reinforced potentially harmful, traditional social norms or inequalities. The targeting of youth has not been a priority for most Project activities and partners. Similar to the responses received about gender, most respondents stated that adults of all ages were welcome to participate in Project activities. There were no youth-related indicators that organizations were required to track, so when asked, most could not give a breakdown of activity participants by age group. There seemed to be confusion about the definition of “youth”; many people seemed to think we were asking about minors, and said that youth could not be part of associations. Others considered 32 MEL 2019 33 A good example is Table 3 in the Q3 2020 report, which provides information on the training activities by topic, commune, and total number of participants disaggregated by sex. youth to be young adults and said they could participate in all activities. When the Project disaggregates based on age, they use two categories: 15-29 and 30 plus years old. EQ-4 TO WHAT EXTENT AND IN WHAT WAYS HAS THE PROJECT LAID THE FOUNDATION FOR SUSTAINABILITY IN TERMS OF BENEFICIARIES’ IMPROVED APPROACHES TOWARD NATURAL RESOURCES MANAGEMENT THROUGH THE ADOPTION OF PRINCIPLES AND PRACTICES PROMOTED BY THE PROJECT? Based on clarification received from USAID on December 17, 2020, the ET focused on four key areas of sustainability: 1) promoting income-generating activities to provide farmers with alternative incomes in order to decrease cutting of trees; 2) increasing environmental awareness to produce behavioral change regarding environmental protection and conservation; 3) building the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub￾watershed plans beyond the life of the Project; and 4) supporting laws, policies, regulations, or standards (plans) addressing sustainable landscapes. FINDINGS According to the FY 2019 and 2020 Annual Reports and testimony from FGD participants, the URP has initiated the promotion of income-generating activities to provide alternative options to farmers to generate income instead of cutting down trees for charcoal production or other wood fuel, but most of these activities began too recently to reliably assess. The 2019 Annual Report mentions the following: 1) awareness raising for at least 1,500 young people per department on income opportunities in beekeeping; 2) inviting firms to submit concept notes on proposals for income-generating investments in agro-forestry related value chains;34 3) two anecdotal success stories of income-generation;35 and 4) in 2020, 544 farmers’ households have started selling their products for income generation or procurement of other goods, or using them to address their own food needs.36 Many of these activities started so recently that their impacts on income generation are still theoretical and their effects on the reduction of deforestation are speculative at best. The Resilience Activity benefits that people most frequently cited were short-term. FGD participants spoke more about the benefits of paid labor through the Project, which will end when the Project ends, than about benefits that will continue post-Project. Many of the beneficiary FGD participants who stated that they had learned new practices and incorporated those practices into their agricultural activities were quick to point out that they did so because seeds, plants, and training were given to them for free and in many cases they were paid a daily wage to implement these practices on their own parcels. The most common response given when asked whether they would continue to implement these practices after the Project ended was that they would implement those that showed clear benefits and that they could do with the tools and materials they had, but that they would be unable to purchase inputs or tools post-Project even if they knew there was a potential benefit. Too little time has passed to assess whether the improved practices have resulted in increased tangible benefits, or whether those benefits are sufficient to motivate people to implement them again the following season without being paid to do so by the Project. While Project staff spoke positively about 34 Though 19 concept notes were received, only two were promising ones (investments in moringa seed oil production and animal forage value chains). The Project initiated discussions with these two applicants in FY2020. 35 A woman who is a member of the Groupement Agricole d’Ouvray (GAO) and farmer from Limonade-based association APWOLIM. 36 Q1 2020. 39 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV the sustainability of resilience activities such as beekeeping and livestock forage growing, USAID and GOH staff were less optimistic. The URP explored partnerships with the private sector to increase income-generating activities for its stakeholders but most of these conversations have stalled. Starting in 2018, the Project started exploring ways to collaborate with Royal Caribbean Cruise Line (RCCL) and SISALCO.37 These discussions led RCCL to contribute US$50,000 in 2020 to support the Organisation pour le Développement de Labadie (ODL) to improve visitors’ experience and increase villagers’ income through agroforestry-based ecotourism. Furthermore, RCCL expressed interest in supporting income￾generating activities. As a result, in FY 2020, the Project submitted to RCCL two concept papers for the possible co-funding of sub-grants for agroforestry and forage production in Bande du Nord, and mangrove stand rehabilitation and income generation in Baie de l’Acul. When the COVID-19 pandemic suspended all cruise traffic, conversations with RCCL ceased and have not resumed. Despite numerous attempts, the ET was not able to reach a representative of RCCL during the fieldwork stage of this evaluation. Similarly, toward the end of the FY2019, the Project was in the final stage of discussions with Antillean Canning S.A. (ANCASA) to hold a marketing trial to source ackee from farmers in northern Haiti for processing in Port-au-Prince and exporting internationally. ANCASA revealed to the ET, however, that their agreement with the URP was cancelled after exchange rate fluctuations left the Project unable to meet the financial terms of the agreement. Also in 2020, the Project submitted an approval request to USAID to issue a sub-award to RWE Construction, New Roots Haiti, and BIEN Construction to invest in a refinery in Milot to produce and export moringa oil, sourcing raw moringa pods from smallholder farmers who live in the Project’s area of intervention. The Project is also considering how to engage the private sector and develop sound business models of hay or silage production without long-term Project financial support. In addition, it has identified opportunities for starting micro, small, and medium enterprises (MSMEs) to produce hay and silage as an income-generating activity.38 The ET did not hear anything about these activities during fieldwork interviews so is not aware of their progress. Awareness-raising campaigns are a major part of URP’s sustainability strategy but no respondents stated that they have changed their behavior as a result. URP awareness-raising and training activities are intended to increase know-how and ultimately change behavior regarding environmental protection and conservation. For example, in 2019, the Project organized several educational activities involving school children (e.g., school competitions, tree planting, etc.). The Project has directly assisted partners with their communications and awareness-building efforts. It supported APAPANNE to organize an awareness campaign targeting 500 children plus 132 representatives from their partner CBOs. The campaign aimed to inform community members about the 170,000 trees APAPANNE already planted and the hundreds of thousands more that they anticipate planting, and teaching children the importance of managing these trees for the long-term benefit of their generation. The Project is also helping sub-awardees to develop communications products for distribution, such as a leaflet on the importance of mangrove protection that was developed for FoProBiM. The Project assisted CASB to design an education and awareness plan around the promotion of sustainable wood and beekeeping production in the commune of Terrier Rouge. 37 SISALCO is a private company that holds a long-term lease of more than 3,300 hectares in the North-East for sisal production. As about 2,500 hectares of that concession fall within the 3-Bays National Park, the monocropping of sisal could potentially have significant ecological impacts. The Project team explored with the company’s senior management the potential for diversifying the monoculture into an agroforestry model that would mix sisal production with melliferous parks. However, these conversations did not result in a partnership agreement. 38 Silage requires investment in processing equipment and proper storage, and is highly technical. While simpler to produce, hay is only in demand during the dry season and agricultural campaigns (when plot area is dedicated to crops). In addition to the campaigns organized by its sub-awardees, the Project also delivered awareness-raising activities through its DI mechanism and conducted training on the healthy management of herds before and during the deworming campaign, reaching more than 30,000 farmers. Due to social distancing rules to minimize the risk of COVID-19, in 2020 the Project focused on developing radio spots that were broadcast through regional radio stations. The topics covered were 1) environmental awareness and education, 2) hay production to combat drought, 3) soil burning, and 4) deworming of livestock. None of the beneficiaries we spoke with during our FGDs mentioned learning about these improved practices through radio broadcasts. They did, however, say that they had benefited from in-person training sessions and were planning to implement the improved techniques that they learned about, to the extent that they had the resources to do so. One common sentiment among FGD participants and sub￾awardees was that everyone knew about the long-term importance of trees, but that short-term motivation to meet urgent individual needs was a stronger force than the desire for long-term environmental health. One positive effect of awareness-raising campaigns that was cited in an interview was that now people were more aware of differences between tree species and were more apt to cut down specific tree species that are invasive and fast-growing, letting other slower-growing and beneficial trees survive. The ET is not aware of any specific Project efforts to study the differences in current mindsets between genders or age groups and did not appear to tailor messaging to these various sub￾groups. Finally, through its partnership with FERRE Haiti SA, the Project’s FY20 Annual Report presents plans to support the establishment of an employer-backed credit scheme, and an aggressive marketing and awareness campaign to change the behavior of more than 2,000 households and encourage them to convert from charcoal use to LPG39. Despite many attempts, the ET was unable to reach FERRE Haiti for an interview so is not aware of any progress toward this goal. Even though clean cookstove conversions are a major objective of the Project and part of the Project's strategy to reduce demand for charcoal and therefore de-incentivize tree cutting for charcoal production, no KII or FGD respondents mentioned these campaigns around clean cookstoves. Respondents in one FGD did suggest that the Project distribute clean cookstoves to reduce the demand for charcoal, suggesting that the knowledge around cleaner fuels is there, but people lack resources to make the switch. Sub-Watershed Management Committees are currently functioning, but only with Project support. According to the 2018, 2019, and 2020 Annual Reports, the URP is building the capacity of local authorities and communities to manage and continue implementing sub-watershed plans beyond the life of the Project. The URP is doing this by: 1) providing training on how to prepare proposals and fill out application forms, which is key for increasing the number of sub-awardees; 2) offering several technical trainings to local authorities, Sub-Watershed Management Committees, and sub-awardees on themes ranging from forage management to natural agroforestry and from gender and youth inclusion to survey techniques and the Rural Code, etc. (Annex H provides a preliminary table outlining the different trainings as well as the number of beneficiaries disaggregated by sex – when available); 3) organizing field and exchange visits among the various communities in order to share good practices and learn from each other’s experiences;40 and 4) providing its in-kind grantees with not only the skills but also the equipment necessary to continue their reforestation efforts. Some preliminary results of these capacity building activities were observed by the fact that some meetings are now taking place directly between the Sub-Watershed Management Committees and the sub-grantees, without the Project’s direct participation or sponsorship to discuss and plan future actions within the sub-watershed.41 The ET 39 The FY20 Indicator Tracking Table showed no conversions to clean cookstoves (“0 beneficiaries with improved energy services”). Preliminary data from the first two quarters of FY21 show 1,102 beneficiaries with improved energy services, indicating that these cookstove conversion activities are now underway. 40 For example, in FY 2018, the Project organized and facilitated a two-day visit of CASB senior management to the Montrouis and Bas-Boën CRDDs. 41 Q2 2020 Report. 41 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV met with members of all five SWMCs, who appreciated the training they had received but ultimately doubted their ability to carry out any activities following the end of the Project without financial and training support from the government or another project. Several interviewees referenced a past USAID-financed project Developpement Economique pour un Environnement Durable (DEED), implemented by Development Alternatives Inc. (DAI), which also established similar committees only a decade ago. Respondents believed that these committees ceased to function immediately following the end of the project. The trees that have been planted on people’s private property that will contribute to their income are likely to be protected, but trees planted on public land where animals roam freely are not likely to survive or be cared for. Beneficiaries of both genders expressed an interest in continuing to care for the agroforestry trees that they were able to choose for their own properties, and expressed commitment to ensuring that they survive, and an understanding of how they could benefit from these trees in the future. They were less likely to speak about the trees that were planted on public or protected land, except to say that the free-roaming of livestock was a major issue that would negatively affect the survival rates of these trees. No one of either gender expressed sentiments of feeling responsibility toward these public trees, though most of the tree-planting efforts are still in the nursery phase with the planting efforts not yet begun in most places, so it is possible that these people have not yet been trained in proper long-term care of the planted trees. It is clear that providing trees that people want and are incentivized to protect because they have the potential to provide an income source is the best way to ensure that trees are cared for and not left to die or be cut down. With no one incentivized to protect the millions of trees that are not on private property or potentially income￾producing, it is unlikely that enough will survive to have a significant positive environmental impact. The ET did not hear of any pre-Project analysis of differences in motivation between the genders in regard to conservation or deforestation, and it does not seem as if the Project tailored its messages or efforts to men versus women. The URP supported the finalization of municipal decrees designating reforestation areas but these decrees are unlikely to have much effect on tree-cutting. According to the 2020 periodic reports, three municipalities—Bahon, Perches ,and Capotille—have, with the technical assistance of the project, finalized and signed municipal decrees designating reforestation areas. The Bahon decree was finalized and signed by the Municipal Council of Bahon and forwarded to the “Délégation Départementale du Nord” (representative of the executive branch of the GOH) for its approval, prior to its publication.42 The plans for Perches and Capotille were still pending signature per Q2 FY 2020 report. No update was given in the FY20 Annual Report or in KIIs as to any progress on this front. According to the 2019 Annual Report, findings from the Community Options Analysis and Investment Toolkit (COAIT) show that the Haitian Rural Code, although very old, remains the best compendium of the various laws and regulations pertaining to natural resources management that address sustainable landscape. However, after the training organized by the URP on the rural code, recurrent socio-political unrest prevented the realization of the forums planned to allow concertation between the municipalities and the other stakeholders involved or responsible for the application of the rural code. These forums would bring together actors such as judges/justices from peace courts, police commissioners, departmental representations of the Ministries of Agriculture and the Environment, and the Ministry of Interior delegations and vice-delegations. According to beneficiaries in FGDs, communities are in favor of increased enforcement of tree-cutting prohibitions, even as most participants agreed that these laws and regulations would continue to have little effect as long as people remain as poor as they are. 42 Q2 FY 2020 report. CONCLUSIONS Most resilience activities are not likely to be sustainable, given the short project timeline, lack of resources among farmers, and many project delays. Too little time has passed to assess whether the improved practices have resulted in increased tangible benefits, or whether those benefits are sufficient to motivate people to implement them again the following season without being paid to do so by the Project. Based on what the ET heard in interviews and focus groups, there is little indication that this Project has changed behaviors in the long term. Sustainable behavior change takes time, with subsistence farmers especially tending to be risk-averse and often requiring several seasons of tangible evidence of benefits in order to enact new techniques. At most, beneficiaries will have experienced one or two harvests of new crops, or in the case of slower-growing trees, no visible benefits of the Project’s activities for several years. Pursuing new and unproven techniques, especially if they need to seek out seeds or inputs that are not readily available, is a lot to expect from people who are already food insecure and cannot take the risk of a potential lost harvest, even if there is the possibility of increased income using new techniques or higher-value seeds. While the exercise of creating a watershed management plan may have some small benefits to the communities, the committees are unlikely to continue to function without support. Without an entity financing activities of the SWMCs, organizing meetings and paying travel/per diem costs, the committee members themselves stated that they would not be able to continue to do anything after the project ends. The government is similarly lacking in resources and motivation to carry on the organizational and support activities to keep the committees active. The ET saw and heard nothing that would suggest that the committees established through URP would be more sustainable than those established by DEED from 2008-2012. The Project’s work with clean cookstove promotion is limited and is unlikely to significantly lessen the demand for charcoal. To date, the Project is proceeding on two fronts to encourage the conversion from charcoal-burning stoves to clean energy-burning stoves. An activity with FERRE Haiti SA aims to convert 2,000 households, mainly employees of the Caracol Industrial Park, by providing a single community furnace running on liquified propane gas (LPG), encouraging people to abandon the use of charcoal for cooking. More than 1,100 households have already converted from charcoal to LPG cookstoves as part of this activity. The Project is also working with Palmis Enèji for the conversion of 150 mobile restaurants (MMK) from charcoal to LPG. Nearly 350 MMKs have been identified in the North and North East for assessment, and those meeting the predefined criteria will be the direct beneficiaries of the activity. It is too early to assess the success of this effort or whether it will have a significant impact on the demand for charcoal. However, 2,000 households represent less than 1 percent of the households in these two regions, and 150 MMKs are likewise a small percentage of the MMKs in the regions. It is safe to say that while every conversion counts, these efforts are not going to significantly reduce the demand for charcoal in these two regions enough to spark the kind of landscape￾level change that USAID aims to achieve. 43 None of the beneficiaries that we spoke with in FGDs had benefited from these clean cookstove activities, though several mentioned that they would be interested in having one if the Project were to distribute them. 43 ProLands, a study commissioned by USAID and published in June, 2021, reveals that the pace of deforestation as a result of the demand for charcoal has decreased and potentially plateaued as more farmers are planting trees on their land for the express purpose of converting them into charcoal. 43 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV IV. RECOMMENDATIONS In the final two years of the Reforestation Project, Chemonics should, as a first priority: ● Not sign any additional grant agreements at this point, focusing on completing what is underway. Grant agreements involve significant investment of time into bureaucratic and capacity-building efforts that the project does not have at this stage. Supporting the already￾underway grant agreements through to completion of the activities and developing capacity within these organizations to be able to continue their activities post-project should be the priority. ● Understand why delivery of materials has been so slow and prioritize the fulfillment of outstanding promises. Chemonics needs to fulfill its obligations to grantees including the delivery of all promised materials and tools, or, if this is not possible for whatever reason, the provision of funding so that the organizations can procure their own required materials and avoid unnecessary delays. ● Prioritize finding a way to fence off areas from animals where trees are to be planted in order to protect the trees. Given the substantial problem that free-roaming livestock are on newly planted tree saplings and other crops, if the Project hopes to see acceptable rates of tree survival and the positive environmental impacts that the Project has promised to deliver, plantation areas, particularly those that are not privately-owned and therefore not under any one person’s own responsibility, need to be protected from livestock. ● Continue to employ as many women as possible in the direct investment activities and pay them at least a standard daily wage for their time. While it is not a sustainable way to increase income and economic opportunities permanently for women, paid employment within the Project can have immediate positive impacts on the economic situations of women and families. Once these top priorities are met, Chemonics should aim to: ● Focus remaining efforts on the activities that have shown fast uptake, particularly the livestock-related ones. Instead of signing new grant agreements or starting any new activities, if the Project has time and resources to spare, Chemonics should expand the livestock-related efforts of the project to reach more beneficiaries by adding resources to existing agreements. ● Focus on developing the private sector cooperatives and trade associations like FECCANO and RECOCARNO because they have the best chance at providing market opportunities for agroforestry crops. While there is not time to begin new efforts to promote transformation of produce and seek out new marketing opportunities for new crops, the Project should focus attention on reducing any barriers that the existing private sector coffee and cocoa cooperatives face to ensure that these actors can continue to operate and serve as many producers as possible. In future reforestation-focused projects, USAID/Haiti should: ● Strategically evaluate how to effectively include both direct investment and sub￾grants mechanisms in implementation strategies. For a future reforestation project targeting the same areas as URP, it may make sense for the majority of the main project to be implemented through direct investment. During initial site visits, the Project could determine whether in each targeted community a local organization already has the capacity and interest in managing the prescribed activities through a sub-grant. Where these organizations exist, subgrants can be an effective way to quickly mobilize. In areas where these organizations aren’t active or don’t have the capacity to manage a sub-grant, direct investment can be used. In this way, the roll-out of activities does not depend on the existence of a local organization, but at the same time, the social capital and management structures of existing organizations are strengthened wherever possible and not reinvented. A smaller Grants Under Contract sub￾component could support a few promising local initiatives that may be complementary to the primary objectives of the project, and selected through an RFP process. This appears to be the best way to ensure accountability and a cohesive and logical Theory of Change while at the same time allowing for the capacity development and community benefits of existing local organizations and rising-star innovative initiatives. The following table summarizes the key issues to keep in mind when making decisions about the use of direct investment vs grants in future projects: TABLE 7. DECISION MATRIX FOR GRANTS VS. DIRECT INVESTMENT MECHANISM WHEN TO USE WAYS TO IMPROVE RESULTS Grants When there are local organizations able and eager to implement the Project’s planned activities in the targeted areas When Project timelines have some flexibility and room for potential bureaucratic delays When there is doubt about people’s receptiveness to messages or activities delivered or promoted by outsiders, or messages and activities are particularly sensitive in nature When a primary goal of the project is equipping beneficiaries to continue to carry out activities post-project Consider an initial payment upon signature of the grant agreement to allow for activities to commence, in addition to deliverable-based payments Provide some type of recourse or complaints channel from sub-grantees to USAID for use if the Project is not meeting its obligations to sub-grantees in terms of providing materials, processing payments, etc. Direct Investment When project targets require a large number of uniform outputs spread more or less evenly throughout a target area When it is important to reach an entire target area’s inhabitants and spur behavior change of large numbers of people When project timelines are tight and there is no room for administrative delays When specific, tangible and concrete tasks need to be completed during the project and the continuation of these activities post￾project is less of a priority When there aren’t enough local organizations capable of or interested in carrying out the project activities, or these organizations are not present in all targeted areas Hire locally for as many positions as possible, especially those that can provide training to others and serve in management roles Involve local organizations in decision making, planning and implementation of all activities as much as possible - even if they do not officially receive a sub-grant, the associations can feel ownership and play an important role in mobilizing the community and facilitating access ● Require a more logical and cohesive strategy for increasing income to a level high enough to decrease deforestation in these poor areas. Determine what level of income is likely to be sufficient to remove the pressure to cut down trees and seriously consider whether future project proposals have a logical strategy for reaching that level of household income, recognizing that there are significant barriers to the adoption of new crops or 45 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV technologies beyond a lack of knowledge or capacity, and that these barriers are often more difficult to address in a short-term project than is a lack of knowledge. ● More specifically define the project’s objectives and intended outcomes around youth and gender beyond targeting female participants. In this Project, gender participation metrics have been met but no other gender or youth-related indicators are being tracked, and metrics for economic empowerment outcomes are not clearly defined. Given USAID’s increasing prioritization at the global level of economic empowerment benefits to women and youth that go beyond participation in project activities, there is a need for further guidance from USAID, as well as opportunity during the remainder of this project, to specify those measures of success for the remainder of this and in future projects. ● Understand the different motivations for deforestation-causing behaviors among various sub-groups of the population and attempt to focus messaging and activities more specifically to these different demographics. It is likely that men and women, or older adults and youth, have different ideas and values that affect their behaviors toward the environment. Future projects could explore these differences and tailor their efforts in ways that will speak more effectively to each sub-population. For example, it could be helpful to know who is regularly cutting trees for charcoal and who is only occasionally cutting trees when absolutely necessary because they are experiencing a crisis, or how much tree-cutting is happening on sustainably-managed private woodlots vs. illegally on public land. Understanding these nuances could help the project better tailor its approaches. ● Balance a results-based payments approach with realities on the ground. If capacity building of local organizations is a priority for USAID, projects need to ensure that whatever results-based payment systems they have in place are appropriate for the situation of the local organizations. By paying only upon completion of an activity, smaller organizations without the capital necessary to front the costs of the activity will be necessarily excluded. ● Focus tree planting efforts on places and species that people have strong incentives to protect. Planting large numbers of trees is easier than ensuring that these trees survive to maturity. People must have direct and personal economic incentives to care for and protect trees; with limited time and resources, we cannot expect people to charitably continue to look after trees that may have only an indirect or long-term impact on something like rainfall levels, even if they understand the theory behind their importance. 46 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV V. EVALUATION MATRIX TABLE 8. EVALUATION MATRIX DATA COLLECTION DATA SOURCES ILLUSTRATIVE SUB-QUESTIONS DATA ANALYSIS METHOD METHOD EQ Context analysis Content analysis Performance trend analysis Gap analysis Gender analysis Thematic analysis of qualitative data Coding of documents by EQ To what extent have the different implementation mechanisms achieved the planned outputs? To what extent was each mechanism able to provide training/technical assistance, on what themes, to how many participants, including youth and women? To what extent have the different implementation mechanisms achieved the planned outcome (number of beneficiaries who have applied improved management practices or technologies per type of implementation mechanism)? To what extent are the different mechanisms able to partner with CBOs? To what extent did the different implementation mechanisms experience any operational challenge (e.g., delays, impact of COVID-19)? To what extent are the different implementation mechanisms able to carry out MEL? USAID Grantees and sub-grantees Other stakeholders Project M&E data Project documentation Other available research and technical reports Document Review KIIs FGDs EQ-1: What are the strengths and weaknesses of the implementation mechanisms (direct investment vs. grants) as utilized separately and together to promote improved natural resource management? Context analysis Content analysis Performance trend analysis Gap analysis Gender analysis To what extent were the Project’s objectives to improve social, ecological, and economic resilience achieved (through its different interventions), or are expected to be achieved? Are there any differential results across groups of beneficiaries? If so, where and why? What are the major influencing factors and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? What has been done so far by the Project to improve farmers’ access to credit? At the USAID Grantees and sub-grantees Project M&E data Project documentation Other available research and technical reports Document Review KIIs EQ-2: To what extent and in what ways is the Project improving beneficiaries’ resilience in the face of natural and economic shocks? 47 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV DATA COLLECTION DATA SOURCES ILLUSTRATIVE SUB-QUESTIONS DATA ANALYSIS METHOD METHOD EQ Thematic analysis of qualitative data Coding of documents by EQ household versus individual level? What is the farmers’ perception? Any difference between women’s and men’s perceptions? What is the perspective of other key government and non-government actors in the region on the effectiveness of the approach used by the Project to improve the resilience of beneficiaries? Context analysis Content analysis Performance trend analysis Gap analysis Gender analysis Thematic analysis of qualitative data Coding of documents by EQ Was an assessment undertaken to analyze the issues specific to youth and gender during the design/initial phase? If yes, how was the information used to inform he design? To what extent were youth and gender taken into consideration during implementation: How were youth/women’s groups targeted/selected? How were the activities designed to be relevant to youth and women? Did URP promote resilience activities that affect men and women or other potentially vulnerable groups differently? How and to what extent were youth/women able to access credit? To what extent was women’s leadership in the community enhanced? How and to what extent were gender and youth considerations included in the sub￾grantees’ agreements? To what extent youth/women have participated in capacity building activities? How and to what extent are youth/gender integrated into the URP’s MEL plans? Are there are sufficient human resources to ensure that youth/gender are integrated into the URP? USAID Grantees and sub-grantees Other stakeholders Project M&E data Project documentation Other available research and technical reports Document Review KIIs FGDs EQ-3: To what extent and in what ways is the Project integrating youth and gender? DATA COLLECTION DATA SOURCES ILLUSTRATIVE SUB-QUESTIONS DATA ANALYSIS METHOD METHOD EQ Context analysis Content analysis Performance trend analysis Gap analysis To what extent did the URP promote income￾generating reforestation or resilience activities? To what extent did the URP build the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub-watershed plans beyond the life of the Project? To what extent did the awareness raising/training activities lead to behavior change? To what extent did the URP support laws, policies, regulations, or standards (plans) addressing sustainable landscapes? Were these laws/policies etc. conducive for creating an enabling environment for increased women or youth engagement? USAID Grantees and Sub-grantees Other stakeholders Project M&E data Project documentation Other available research and technical reports Document Review KIIs FGDs EQ-4: To what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries’ improved approaches toward natural resources management through the adoption of principles and practices promoted by the Project? VI. EVALUATION MANAGEMENT This section summarizes the ET composition, as well as the roles and responsibilities of each team member, including those of the headquarters and/or field office management team. ORGANIZATIONAL STRUCTURE The two-person evaluation team is well-balanced and provides a complementary mix of evaluation and sectoral expertise. Team Leader (TL) Cynthia Berning has more than 10 years of experience leading agriculture programs and evaluations, including leading a Social Impact assessment for USAID/Haiti in 2018 and 2019. Team Leader (ATL) Philippe Phanol also has a strong sectoral background in environment and agriculture, as well as evaluation. In addition, two note-takers accompanied the team members during the data collection so that the team was able to work as two sub-teams, thereby maximizing the amount of data collected. The note-takers also contributed to the data coding. ESS engaged a local Haitian research firm to support the ET with data collection. The firm conducted FGDs with Project beneficiaries and GIs with Sub-Watershed Management Committees in remote areas where the Project is being implemented. EnCompass, a U.S.-based research and evaluation firm specializing in gender issues, provided the ET with remote technical assistance to ensure that gender was appropriately and adequately incorporated into the evaluation, particularly given that one of the evaluation questions specifically addressed gender. This team has been supported by an SI headquarters (HQ)-based Project Director (PD) Catharine Villada, and Project Assistant (PA) Alexandra McMullin, as well as ESS office-based Project Managers (PMs) Antoine Wesner and Kesly Felizor. Collectively, they have worked on many program evaluations, assessments, and capacity building activities. This management team ensured high-quality and compliant deliverables throughout the process. ROLES AND RESPONSIBILITIES Evaluation Team. The TL led the ET in designing the evaluation, conducting data collection and analysis, and writing the evaluation report. The TL used her evaluation expertise to ensure the evaluation objectives were met and that a quality, useful report was produced. The TL was responsible for delegating tasks to other team members and ensuring their inputs meet SI and USAID’s quality expectations. The ATL worked closely with the TL, assisting with the evaluation’s design and implementation, and providing sectoral and contextual expertise throughout. The note-takers supported the ET by taking detailed notes throughout data collection and assisting with coding the data. Local Research Firm. ESS completed a procurement process to engage a Haitian research firm to support the ET. The selected firm, CHASE, was responsible for conducting focus groups and GIs with farmers and Sub-Watershed Management Committees. CHASE worked closely with the ET to collect the data. EnCompass. EnCompass supported the ET through technical assistance to ensure that the evaluation adequately and appropriately accounts for gender. Working remotely, an EnCompass Gender Specialist reviewed the evaluation design and tools to make sure that they would gather any information necessary to ensure that gender issues were addressed. She checked in regularly with the team throughout the data collection to discuss any issues they were encountering in obtaining gender parity in respondents and provided suggestions about how to overcome these. She also provided support during the data analysis. Finally, she reviewed all deliverables, including the out-briefing presentation and evaluation 51 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV report, against a check-list/score card to ensure that gender was appropriately and adequately addressed. SI HQ and ESS Field Office Staff. A dedicated three-person management team composed of a PD, PM, and PA guided this mid-term evaluation, both technically and administratively (see Table 7 for more details). These staff used SI’s customized project management tools and QA checklists to implement each phase of the evaluation, including launch and preparation, data collection, and analysis and reporting. The PD was responsible for providing high-level technical guidance and overseeing the ET, PM, and PA. The PM managed the task order on a day-to-day basis, including arranging team check-ins and conducting in-depth reviews of all deliverables. Both the PD and PM worked closely with the TL to respond to all requests from USAID/Haiti. Finally, the PA reported to the PM and managed all administrative tasks, onboarding, mobilization, and invoicing for the ET. TABLE 9. EVALUATION TEAM MEMBER ROLES AND RESPONSIBILITIES POSITION RESPONSIBILITIES Cynthia Berning TL/Evaluation Specialist Lead the evaluation, including desk review, data collection and analysis, and report writing. Manage the team, including delegating responsibilities, training and guiding team members, monitoring progress, and providing feedback on inputs. Serve as the primary liaison with USAID/Haiti’s Democracy and Governance Office, facilitate briefings, and lead presentations. Philippe Phanol ATL / Sector Specialist Leverage sectoral and contextual expertise in the design of the evaluation and data collection instruments. Participate in desk review, data collection, and analysis. Lead a sub-team during fieldwork. Contribute to the production of high-quality deliverables. Note-takers (2) Coordinate in-country logistics, such as arranging KIIs and GIs. Support the ET in conducting data collection, primarily though taking detailed notes. Assist with coding and data analysis. CHASE Leverage local expertise in contacting key person resources to plan data collection events. Train data collectors. Conduct remote KIIs and FGDs. Provide detailed notes for each data collection event. Jennifer Pendleton EnCompass Senior Gender Specialist Review and provide feedback on evaluation design, data collection tools, and draft and final reports to ensure appropriate and adequate gender integration. Provide high-level guidance on methodology, major deliverables’, and any problems encountered during the evaluation with respect to gender integration. Support team through regular check-ins to provide technical assistance on appropriate and adequate gender integration. Catherine Villada Project Director Provide technical consultation and QA to the ET. Provide high-level guidance on methodology and major deliverables. Implement SI’s EQUI® approach in close coordination with TL, conduct in-depth review of deliverables using QA checkpoints, and review Utilization-Focused POSITION RESPONSIBILITIES Evaluation Checkpoints. Offer high-level supervision of contract financial compliance. Kesly Felizor Project Manager Promote client satisfaction throughout Reforestation Project evaluation. Ensure fulfillment of SI’s EQUI® approach, on-schedule completion of deliverables, and compliance with SI procedures and USAID regulations. Recruit and vet additional team members for evaluation, as required. Onboard the team and train them on SI procedures and quality standards. Facilitate pre-departure team planning meetings, check in regularly with ET, and manage personnel issues. Manage the process of responding to USAID comments on the draft report, using comments matrices to ensure that all feedback is addressed. Supervise PA. Oversee evaluation budget, produce projection forecasts, and maintain tracking system for LOE expenditures. Submit evaluation deliverables in compliance with USAID branding guidelines and SI EQUI standards. Approve invoices, and ensure expenses are allowable and consistent with contract requirements. Issue and ensure contractual adherence to partner sub-contract. Manage the budget by tracking LOE usage and other spending. Alexandra McMullin Project Assistant Organize the administrative and logistical procedures for rapid project start up, manage onboarding, mobilization, and deployment of the evaluation team. Mobilize travel arrangement, process invoices, expense reports, and other administrative documentation. Provide knowledge management support through SI’s online SharePoint intranet system. Copyedit and format deliverables. Onboard and orient consultants to SI standard operating procedures and security protocols. Coordinate with local logistician to ensure smooth operations. 53 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV VII. EVALUATION DELIVERABLES AND TIMELINE This mid-term evaluation was conducted from September 8, 2020 to approximately July 31, 2021. The period includes approximately four weeks for a desk review of Project documents, drafting of the Inception and Evaluation Design Report and other planning for mobilization, four weeks for data collection and analysis, and eight weeks for writing and reviewing the draft and final reports. 1. Evaluation Work Plan: The work plan included: 1) the anticipated schedule and logistical arrangements; 2) proposed ET members, delineated by roles and responsibilities with their level of effort (LOE); 3) identification of other required personnel and relevant local sub-contractors, their LOE, roles and responsibilities, and qualifications; and 4) tentative deliverable schedule. 2. Inception and Evaluation Design Report: The Inception and Evaluation Design Report presents the evaluation design. It outlines the ET’s understanding of the evaluation SOW, provides background on the Reforestation Project, presents preliminary findings based on the ET’s review of available Project documentation, and highlights gaps in information to inform the evaluation design. The report likewise presents the methodology the ET will use to conduct the fieldwork and data analysis portion of the evaluation. It includes data collection and analysis methods, known limitations to the evaluation design, and a dissemination plan. It also includes a detailed evaluation design matrix that links the EQs in the SOW to data sources, methods, and the data analysis plan, the list of potential institutions that will be interviewed, and data collection protocols. 3. In-Briefing: The ET conducted a kick-off meeting with the USAID/Haiti’s Economic Growth and Agricultural Development (EGAD) Office, the ESS COR, and the implementing partner on December 2, 2020 to discuss the ET’s understanding of the assignment, initial assumptions, EQs, methodology, and work plan, and clarify any questions or logistic needs. 4. Evaluation Out-Briefing: The ET will hold a final presentation to discuss its main FCR with USAID after concluding data collection and analysis. The TL will conduct the evaluation out-briefing via an online conferencing platform. The ET plans two presentations for USAID officials. The first presentation, date still to be confirmed, will be for the technical team. ESS will work with USAID to organize a second, Mission-wide presentation shortly after the first presentation. 5. Draft Evaluation Report: This draft evaluation report addresses each of the questions identified in the SOW, and any other issues the team considers bearing on the evaluation objectives.44 The ET submitted the draft evaluation report, presenting the ET’s main FCR, on May 21, 2021. 6. Final Evaluation Report: Once the initial draft evaluation report is submitted, USAID/Haiti will have ten working business days to review and comment on the initial draft, after which the ESS COR will provide ESS with consolidated comments for the ET to address. ESS will then submit a revised report ten business days thereafter, and again, USAID/Haiti will review and send comments on this final report within ten business days of its submission. Once the report is finalized, ESS will post the report to the DEC and submit all anonymized evaluation data and records (FGD and KII summary notes) electronically in an easily readable format, organized, and documented for use by those not fully familiar with the Project or evaluation, and owned by USAID. 44 Any such issues can be included in the report only after consultation with USAID. TABLE 10. DELIVERY SCHEDULE MAJOR DELIVERABLES ESTIMATED DEADLINES Inception/Evaluation Design Report December 31, 2020 In-Briefing January 5, 2021 Data Collection Phase April 12-30, 2021 Out-Briefing July 6, 2021 Draft Evaluation Report May 21, 2021 Final Evaluation Report July 27, 2021 55 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX A. SCOPE OF WORK PURPOSE OF THE EVALUATION The USAID Reforestation Project has been implemented for over 2.5 years, trying to restore environmental services, improve livelihoods and build the resilience capacity of beneficiary communities through integrated reforestation interventions. The purpose of this evaluation is to determine the extent to which the Project is strategically, operationally and sustainably progressing toward its objectives, including youth and gender inclusion in reforestation programming and governance. BACKGROUND Deforestation has been a major concern in Haiti, adversely affecting best land use practices, destroying biodiversity and ecological habitats, and making long-term investments in the lowlands vulnerable to natural threats (hurricanes, soil erosion, floods, drought). In other words, landscape degradation and the decline in environmental services from deforestation processes are a real impediment to the economic prosperity, disaster risk management capacity, and self-reliance of the communities. In the North and the North-East region of Haiti, trees are removed for agriculture, construction and charcoal production purposes, primarily to ensure day-to-day economic livelihoods and food subsistence in the face of major stressors and shocks, such as drought and lack of alternative income sources. Using an integrated approach, the Project addresses reforestation and tree cover management challenges through four main lenses: reducing the drivers of deforestation, improving communities’ resilience, increasing tree cover and improving environmental governance). Considering the potential roles and implications of women and youth in the forestry value chains, the Project also adds gender inclusion as a cross-cutting component of its exit strategy for the sustainability of reforestation interventions. The hypothesis and the Theory of Change are as follows: If the Project: ● Empowers men, women, and youth to engage in community-led sub-watershed analysis, planning, implementation and data-driven monitoring and decision-making grounded in lessons learned; ● Helps reduce wood-fuel production from standing forests, and strengthen supply chains for improved cookstoves and wood-fuel alternatives; ● Facilitates appropriate incentives for forest friendly asset building and income-generating activities; ● Fosters adoption of anti-erosion and tree-planting practices; ● Builds capacity of local structures for improved governance and disaster risk and natural resource management. Then, target communities will take ownership of initiatives to reduce the threat to targeted forests, and increase tree cover from ridge to reef. SUMMARY INFORMATION Strategy/Project/Project Name USAID Reforestation Project Implementer Chemonics International Cooperative Agreement/Contract # CA AID-521-A-17-0001 Total Estimated Ceiling of the Evaluated Project/Activity(TEC) $39,305,099 Life of Strategy, Project, or Activity Five years (September 1, 2017 - August 31, 2022) Active Geographic Regions North and North-East Departments, Haiti Development Objective(s) (DOs) DO2: Food and economic security advanced USAID Office Economic Growth and Agricultural Development (EGAD) EVALUATION QUESTIONS 1. Effectiveness: What are the strengths and weaknesses of the implementation mechanisms (Direct Investment vs. Grants) as utilized separately and together to promote improved natural resource management45? 2. Resilience: To what extent and in what ways is the Project improving beneficiaries’ resilience46 in the face of natural and economic shocks? 3. Youth and Gender: To what extent and in what ways is the Project integrating youth and gender? 4. Sustainability: After two and half years of implementation, to what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries’ improved approaches toward natural resources management through the adoption of principles and practices promoted by the Project? EVALUATION DESIGN AND METHODOLOGY It is expected that the evaluation team proposes an overall research design to address the evaluation questions and a plan for collecting and analyzing the data. Nevertheless, given the nature of the evaluation questions and limited time available to plan and implement the evaluation, the evaluation design should be based solely on a combination of qualitative techniques to address the evaluation questions. Key informant interview, focus group interview, and in depth review of projects reports are among the techniques that the evaluation team should consider when addressing the evaluation questions. ESS and the evaluation team should also ensure that gender is appropriately and adequately addressed. DELIVERABLES AND REPORTING REQUIREMENTS 1. Evaluation Work plan: Upon receipt of this Activity Request, Social Impact (SI) shall submit within four weeks a draft work plan to the Contracting Officer’s Representative (COR). The work plan will include: 1) the anticipated schedule and logistical arrangements; 2) a list of the members of the evaluation team, delineated by roles and responsibilities with their level of effort; 3) the identification of other required personnel and relevant local sub-contractors, their LOE, roles and responsibilities and qualifications; and 4) the deliverable schedule. 2. Inception Report/Evaluation Design: Within two weeks of approval of the work plan, SI must submit to Contracting Officer’s Representative (COR) a combined inception report and evaluation 45 Natural resources management, (NRM) is the management of natural resources such as land, water, soil, plants and animals, with a particular focus on how management affects the quality of life for both present and future generations (stewardship). 46 USAID defines resilience as: The ability of people, households, communities, countries and systems (social, economic, ecological) to mitigate, adapt to, recover from shocks (including co-variates such as drought and floods…) and stresses (climate changes, population pressure…) in a manner that reduces chronic vulnerability and facilitates inclusive growth 57 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV design document (which will become an annex to the Evaluation report). While the inception report will identify information gap to address the evaluation questions, the evaluation design will include: 1) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan; 2) draft questionnaires and other data collection instruments or their main features; 3) the list of potential interviewees and sites to be visited; 4) known limitations to the evaluation design; and 5) a dissemination plan. USAID/Haiti will take up to 10 business days to review and consolidate comments through the COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within 10 business days. 3. In-briefing: Prior undertaking field work, the evaluation team will have an in-briefing with the Economic Growth and Agriculture Development (EGAD) Team and the Evaluation and Survey Services (ESS) COR to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and to clarify any questions or logistic needs. 4. Evaluation Briefing/Presentation: The evaluation team is expected to hold a final presentation in person to discuss the summary of findings and recommendations to USAID within 10 business days after the conclusion of fieldwork. 5. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section IX: Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The sub-mission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the Governance Office will have 10 working business days in which to review and comment on the initial draft, after which point the ESS COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report 5 business days hence, and again the Governance Office will review and send comments on this final draft report within 5 business days of its sub-mission. 6. Final Evaluation Report: The evaluation team will be asked to take no more than 15 business days to respond/incorporate the final comments from the Governance Office. The evaluation team leader will then submit the final report to the COR. All project data and records (FGD and KII summary reports) shall be submitted in full and should be in electronic form in easily readable format, organized, and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. EVALUATION TEAM COMPOSITION The Evaluation Team shall be comprised of two Key Personnel positions: 1) a Team Leader, and 2) an Assistant Team Leader. SI is strongly encouraged to sub-partner with a local Haitian firm for data collection purposes. The selected Haitian firms should demonstrate proven capacity in collecting qualitative data in the fields of Agriculture. This approach is encouraged to build the local firm capacity and will also provide a Haitian perspective for the data collection and analysis. The Team Leader (TL) is ultimately responsible for the overall management of the evaluation team, coordinating the implementation of the evaluation, assigning evaluation responsibilities and tasks, and authoring the final evaluation report in conformity with this Statement of Work. The TL must be an experienced evaluation expert, with a documented track record of 10 years of experience in the field of evaluation. S/he should be fluent in French and English. S/he should have at least a Degree in agro￾forestry, natural resources management, international development, or a related field. The Assistant Team Leader (ATL) helps the TL in the overall management of the evaluation team and the final products, in conformity with this Statement of Work. The ATL should be familiar with the Agriculture sector. S/he must possess excellent writing and interpersonal skills and must be familiar with USAID programs, objectives, and reporting requirements. S/he should have experience in designing and implementing evaluations in the agriculture sector and in conducting FGDs. Fluency in French is required. English and Haitian Creole are highly desirable, as is significant prior work experience in Haiti. A master’s degree in Agricultural Economics, Political Science, Public Administration, or a related field is required to ensure that all areas of technical expertise required for the evaluation are effectively covered. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest. The evaluation team shall demonstrate familiarity with USAID’s evaluation policies and guidance included in the USAID Automated Directive System (ADS) in Chapter 200. 59 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV EVALUATION SCHEDULE TABLE 11. EVALUATION SCHEDULE TIMING (ANTICIPATED WEEKS OR DURATION) SCHEDULED ACTIVITIES 4 Weeks Preparation of the work plan and evaluation design 3 Weeks USAID Approval of the Work Plan and Inception Report 1 week Within 5 business days of the inception report 4 Weeks Data collection 2 Weeks Data analysis and Evaluation Briefing 2 Weeks Draft Report writing 2 Weeks USAID review of Draft Report 1 Week Incorporate USAID comments and prepare Final Report FINAL REPORT FORMAT The evaluation final report should include an abstract; executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see “How-To Note: Preparing Evaluation Reports” and ADS 201mah, USAID Evaluation Report Requirements. An optional evaluation report template is available in the Evaluation Toolkit. The executive summary should be 2–5 pages in length and summarize the purpose, background of the Project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The annexes to the report shall include: ● The Evaluation SOW; ● Any statements of difference regarding significant unresolved differences of opinion by funders, implement ● ers, and/or members of the evaluation team; ● All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; ● All sources of information, properly identified and listed; and ● Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of. ● Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. ● Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.47 ● Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ● Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ● The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ● Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions sub-sequently revised and documented in consultation and agreement with USAID. ● Evaluation methodology should be explained in detail and sources of information properly identified. ● Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ● Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. 47 See USAID Evaluation Report Requirements, A Mandatory Reference for ADS Chapter 201, 201mah_090716, September 7, 2016 and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. 61 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX B. DATA COLLECTION PROTOCOLS Below is the Evaluation Design Matrix that links the EQs with the data collection methods, data sources, and data analysis methods. TABLE 12. EVALUATION DESIGN MATRIX DATA ANALYSIS METHOD DATA COLLECTION DATA SOURCES METHOD EQ Thematic analysis of qualitative data, analysis of quantitative data (IPTT, MEL), coding of documents by EQ USAID, grantees and sub￾grantees, other stakeholders, project M&E data, project documentation, other available research and technical reports Document Review KIIs FGDs EQ-1: What are the strengths and weaknesses of the implementation mechanisms (direct investment vs. grants) as utilized separately and together to promote improved natural resource management? Thematic analysis of qualitative data, analysis of quantitative data (IPTT, MEL), coding of documents by EQ USAID, grantees and sub￾grantees, other stakeholders, project M&E data, project documentation, other available research and technical reports Document Review KIIs EQ-2: To what extent and in what ways is the Project improving beneficiaries’ resilience in the face of natural and economic shocks? Thematic analysis of qualitative data, analysis of quantitative data (IPTT, MEL), coding of documents by EQ USAID, grantees and sub￾grantees, other stakeholders, project M&E data, project documentation, other available research and technical reports Document Review KIIs FGDs EQ-3: To what extent and in what ways is the Project integrating youth and gender? Thematic analysis of qualitative data, analysis of quantitative data (IPTT, MEL), coding of documents by EQ USAID, grantees and sub￾grantees, other stakeholders, project M&E data, project documentation, other available research and technical reports Document Review KIIs FGDs EQ-4: To what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries’ improved approaches toward natural resources management through the adoption of principles and practices promoted by the Project? ANNEX C. DATA COLLECTION TOOLS Informed Consent for KIIs Must be read in its entirety to ALL respondents prior to commencement of any KII Consent Statement: Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from Social Impact, a company based in the United States. Our team is visiting people in Haiti to conduct a study about the Reforestation Project, which is funded by USAID. We would like to conduct a brief discussion with you today to learn about your experience with the Reforestation Project. Your responses, along with responses from other participants, will be compiled into findings for a report for USAID. The report will be publicly available once it is complete, but it will not include your name or other identifying information. Readers will not be able to identify the specific individuals for any specific quotes or data. The notes from this discussion will be kept in a format that does not include any names or other identifying information. The notes will only be provided in this format to USAID at the end of the evaluation. You will not receive any direct benefit or compensation for participating in this interview, either from Social Impact or from the Reforestation Project. It is important to understand that while we would like your help in this study, you do not have to participate if you do not want to, and you do not have to answer any questions if you feel uncomfortable doing so. The objective of this research is to improve the performance of the Reforestation Project as well as that of similar projects in the future. The information may be used by other organizations as well. If you have questions/concerns about that, please let me know. The interview is expected to take about 60 minutes. Do you have any questions? You may ask questions at any time. If you have questions or concerns about the research after we leave today, you can contact Wesner Antoine at wantoine@socialimpact.com, Cellular phone number: 36 98 3754 By saying “yes,” and participating in this study, you are indicating that you have heard this consent statement, had an opportunity to ask any questions about your participation, and voluntarily consent to participate. Will you participate in this interview? You may answer yes or no. Yes, I will participate No, I will not participate Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ 63 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – USAID staff General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs How would you assess URP’s work over its three-year period of performance? Very successful, successful, unsuccessful? Why? EQ-1 What are the advantages/disadvantages of the different implementation mechanisms (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? Probes: To what extent do the implementation mechanisms allow partnership with CBOs? (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities) To what extent can we say that the difference in the methods of implementation sometimes causes delays? To what extent do the differences in capacities to carry out the MEL cause delays depending on the type of mechanism? Are there any differences in the training delivered by the different implementation mechanisms used? (in￾kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? EQ- 2 What improved management practices or technologies did the Project promote in the northern region? Why were they chosen? What do you think is the primary value added for female and male farmers of these improved management practices or technologies? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project (per type of implementation mechanism)? To what extent were the Project’s objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Specifically, have behaviors leading to deforestation decreased? Are there any differential results across groups of beneficiaries (women, men, youth)? If so, where and why? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? Was an assessment undertaken to analyze youth and gender specific issues during the design / initial phase? If yes, to what extent does the Project take into account the main issues identified? Probe: Any relevant findings from the PEA? How were they used in the design/implementation? How and to what extent were youth and gender taken into consideration during implementation? Probes: How were youth/women’s groups targeted/selected? How were the activities designed to be relevant and accessible to youth and women? Did URP promote resilience activities that affect men and women differently? To what extent were youth/women able to access credit or other financial resources? To what extent was women’s leadership in the community enhanced? Were there other potentially vulnerable groups that were supported or that should have been reached? To what extent were gender and youth considerations included in the sub-grantees’ agreements? Are there sufficient financial and human resources to ensure that youth/women are integrated into the URP? Are there any additional resources required to improve how the Project is integrating youth and women into its activities? EQ-4 What would it take to make the Reforestation Project sustainable? Do you think the Project is on the right track to achieve sustainability? If yes, how? If not, what should be done? Based on the experience of other NRM Projects implemented in Haiti, what are the awareness-raising activities that need to be implemented, and how should they be implemented in order to lead to behavior change? To what extent did the Project foster partnerships with the private and public sectors to facilitate community access to credit, external funding or contributions as appropriate? Probe: What are the factors that facilitate (or make it difficult) to establish these partnerships? How does the Project work with central authorities and in the field to ensure the sustainability of interventions? What types of collaboration does the Ministry have with the Project to support local management of adaptation to the effects of climate change? What are the policies/laws that need to be put in place to achieve sustainability? 65 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – Reforestation Project PERSONNEL and Implementing Partners (Chemonics International, NCBA/CLUSA, and CIAT) General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs EQ-1 What are the advantages/disadvantages of the different implementation mechanisms (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? Please explain why the Project decided to introduce the direct investment mechanism in FY 2019. To what extent has the Project been able to provide training / technical assistance through each of the implementation mechanisms used? (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? Probes: What improved management practices or technologies did the Project promote per type of implementation mechanism in the northern region? Why were they chosen? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project (per type of implementation mechanism)? To what extent do the implementation mechanisms allow partnership with CBOs? (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? To what extent can we say that the difference in the methods of implementation sometimes causes delays? To what extent do the differences in capacities to carry out the MEL cause delays depending on the type of mechanism? EQ-2 To what extent were the Project objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Specifically, have behaviors leading to deforestation decreased? Are there any differential results across groups of beneficiaries? If so, where and why?/ Probe for livestock, beekeeping, fruit production, cash crop production, etc. Probe for differences between women/men; youth; other vulnerable groups What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? What has been done to mitigate the factors that negatively influence the implementation of interventions? What has delayed the establishment of CRDD, and what has been done to overcome these difficulties? What are the successes and failures encountered in the process of facilitating processing and market access for local products? Explain. What has been done so far by the Project to improve farmers’ access to credit? To what extent did the URP foster partnerships with the private and public sectors to facilitate community access to credit, external funding, or contributions as appropriate? Are there any differences between partnerships established between the public and private sectors? What types of collaboration do the Environment and Agriculture Ministries have with the Project to support local management of adaptation to the effects of climate change? EQ-3 Was an assessment undertaken to analyze youth and gender specific issues during the design / initial phase? If yes, to what extent does the Project take into account the main issues identified? If no, what resources or evidence has the Project drawn upon to inform its gender and youth approaches (setting targets and activity design/implementation)? To what extent were youth and gender taken into consideration during implementation? How were youth/women’s groups targeted/selected? How were the activities designed to be relevant to youth and women? Did URP promote resilience activities that affect men and women differently? To what extent were youth/women able to access credit? To what extent was women’s leadership in the community enhanced? To what extent were gender and youth aspects included in the sub-grantees’ agreements? How are youth/gender integrated into the URP’s MEL plans? To what extent have youth/women participated in capacity-building activities? How does the Project and its sub-awardees follow up with training participants to track that they are applying the knowledge they gained? Are there sufficient human/financial resources to ensure that youth/gender are integrated into the URP? What other resources may be needed? EQ-4 To what extent did the URP promote income-generating reforestation or resilience activities? How do you monitor this activity and know that the activities are really income-generating? Howe do you monitor whether these activities have actually resulted in decreased deforestation? To what extent did the URP build the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub-watershed plans beyond the life of the Project? Probe: Is there a capacity assessment? a capacity building plan? Is the Project working on changing the composition of the sub-watershed management committees? Why? To what extent did the URP succeed in establishing partnerships with the private sector? Probe: What are the challenges the Project faces in establishing partnerships with the private sector? What are the factors that allowed the Project to establish partnerships with ANCASA, RCCL, FERRE? What about SISALCO? How do you know that the awareness raising/training activities that the Project is implementing are leading to behavior change? How does the Project work with central authorities and in the field to ensure the sustainability of interventions? 67 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV What types of collaboration does the Ministry have with the Project to support local management of adaptation to the effects of climate change? What are the policies/laws that need to be put in place to achieve sustainability? To what extent did the URP support laws, policies, regulations, or standards (plans) addressing sustainable landscapes? Probe: Can you elaborate on the Municipal Decree in Bahon? How it was formulated, what technical assistance was provided, a copy of the decree, etc.? 68 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – Project Sub-awardees General 1. Date: 2. Location: 3. Respondent Name: Respondent Title: Sex of Respondent: 4. Respondent Organization: 5. Level of experience/knowledge of the Reforestation Project (low, medium, high): 6. Interviewer/note-taker names: Questions related to the EQs EQ-1 Has your organization worked with CBOs? If yes, which ones? How did you work with them? What are the advantages of working with CBOs? Any difficulties working with CBOs? Has there been any delays in the implementation of the Project? If yes, what is the reason? How often do you send an MEL report to the Project? Do you have any difficulties in doing it? And sending it on time? Are you aware of USAID requirements for data disaggregation or other reporting around gender equality and women’s empowerment? If yes, please describe if and how you report on these. EQ-2 Did you participate in any training offered by the Project? What were the topics? How do you qualify these training courses? (very useful / useful / not very useful) Why? Who generally participates in these trainings from your organization? What improved management practices or technologies did the Project promote? Why were they chosen? Did the farmers change the way they farm (practices or technologies)? If yes, in which way? Specifically, have behaviors leading to deforestation decreased? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project? To what extent were the Project’s objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Are there any differential results across groups of beneficiaries (women/men)? If so, where and why? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? What has the Project done so far to improve farmers’ access to credit? Who provided the credit? Are there differences in how men and women are able to access credit If yes, why are there any differences and what are they? To what extent did the Project promote income-generating reforestation or resilience activities? In your opinion, were the crops or practices promoted appropriate for the area and the situation of the farmers? EQ-3 69 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV To what extent were youth and gender taken into consideration during implementation? How were youth/women’s groups targeted/selected? How were the activities designed to be relevant to youth and women? Did the Project promote activities that affect men and women differently? If yes, please provide examples. From your perspective, did the Project contribute to enhancing women’s leadership in the community? If yes, how? To what extent have youth/women participated in capacity-building activities? What has been the outcomes? In your organization, is there someone who is dedicated to work on youth/gender issues? Did the Project train anyone from your organization on gender issues? If yes, did you change the way you target and implement your activities? Please give an example. EQ-4 Are you promoting income-generating activities? What type? How do you know if they are successful? Do you think these income-generating activities would stop stakeholders from cutting trees? If yes, please explain. If no, please suggest what should be done differently. Have you done any training with farmers? Awareness-raising activities? What type? How do you know that these activities are leading to a change in behavior? Can you give an example? Is there a difference between women and men? And youth in terms of willingness to adapt and change behavior? Do you think when the Project comes to an end the farmers will continue doing what they have learned? If no, why not? Please explain. How does the Project work with central authorities and in the field to ensure the sustainability of interventions? 70 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – Government of Haiti General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs EQ-1 What kind of collaboration has the Ministry/municipality had with the Reforestation Project implemented by Chemonics? To what extent has the Project been able to provide training / technical assistance? How do you qualify these training courses? (very useful / useful / not very useful) Why? What do you think is the primary value added for farmers of these improved management practices or technologies? Has your organization worked with CBOs? If yes, which ones? How did you work with them? What are the advantages of working with CBOs? Any difficulties working with CBOs? Has there been any delays in the implementation of the Project? If yes, what is the reason? If it is a municipality who is doing direct investment and has signed a MOU with the Project: Do you send the Project a MEL report? Do you have any difficulties in doing it? And sending it on time? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project (per type of implementation mechanism)? To what extent do the implementation mechanisms allow partnership with CBOs? (in-kind grants, fixed amount awards, short-term technical assistance, contracts, and direct investment activities)? EQ-2 What is your perspective on the effectiveness of the approach used by the Reforestation Project to improve the resilience of beneficiaries? To what extent did the URP promote income-generating reforestation or resilience activities? To what extent were the Project objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Are there any differential results across groups of beneficiaries? If so, where and why? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? What has the Project done so far to improve farmers’ access to credit? To what extent did the URP foster partnerships with the private and public sectors to facilitate community access to credit, external funding, or contributions as appropriate? Is there any data on deforestation in the Project areas that you can share? Are you monitoring deforestation levels in these areas? If so, when will you have data and will it be made available? 71 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV EQ-3 What policy, economic, or other social factors exist that support gender equality or women’s empowerment in reforestation/natural resource management/resilience activities? What policy, economic, or other social barrier exist that inhibit gender equality or women’s empowerment in reforestation activities? EQ-4 15. Do you think when the Project comes to an end the farmers will continue doing what they have learned? If no, why not? Please explain. 16. How does the Project work with central authorities and in the field to ensure the sustainability of interventions? 17. To what extent did the Project build the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub-watershed plans beyond the life of the Project? 18. To what extent did the URP support laws, policies, regulations, or standards (plans) addressing sustainable landscapes? Probe: Can you elaborate on the Municipal Decree in Bahon? How it was formulated, what technical assistance was provided, a copy of the decree, etc. Are there any laws/policies/plans that the Project should support to ensure its sustainability? 19. How would you suggest improving the design and the implementation of approaches and technologies like those promoted by the Reforestation Project? 20. What types of collaboration does the Ministry/Municipality have with the Project to support local management of adaptation to the effects of climate change? 72 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide - Universities General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs EQ-1 What improved management practices or technologies does the Reforestation Project promote? What do you think is the primary value added for farmers of these improved management practices or technologies? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project? EQ-2 What is your perspective on the effectiveness of the approach used by the Reforestation Project to improve the resilience of beneficiaries? Are the crops and practices promoted appropriate? How has your institution’s collaboration with the Reforestation Project helped meet the needs of landowners in the Project implementation areas? To what extent are the Project objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Specifically, are behaviors that result in deforestation changing or likely to change? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? What are the obstacles and enabling factors generally encountered in the process of facilitating the transformation and market access of local products? Explain. EQ-3 Are there any policy, social, economic, or other barriers/enabling environment with regard to women/youth meaningful engagement in reforestation and building resilience? How should the Project take these into account in its implementation? To what extent were youth and gender taken into consideration during design and delivery of training? Probes: How are youth/women’s groups targeted/selected? How were the training activities designed to be relevant to youth and women? To what extent have youth/women participated in capacity-building activities? EQ-4 73 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV To what extent did the URP build the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub-watershed plans beyond the life of the Project? Probe: What types of capacity-building activities do you think the Project should deliver in the future? To what extent did the URP support laws, policies, regulations, or standards (plans) addressing sustainable landscapes? In your opinion, what types of collaboration should the Ministries of Environment and Agriculture have with the Project to support local management of adaptation to the effects of climate change? 74 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – Private Sector General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs Do you have a partnership with the Project? If yes, why did you decide to enter this partnership? How was it established? If not, why not? What are the advantages for your company to working with the Project? Are there any disadvantages? Anything that needs to be improved? How have you/your business worked with the Reforestation Project? Probe Did the Project help you with expanding your production capacity through new equipment? expanding of your distribution capacity? Finding new consumers of your goods? Increased revenue, sales, or better￾trained employees? EQ-2 What improved management practices or technologies does the Reforestation Project promote? Are these crops and practices appropriate? What do you think is the primary value added for both female and male farmers of these improved management practices or technologies? What is your perspective on the effectiveness of the approach used by the Project to improve the resilience of beneficiaries (female, male, and youth)? How has your institution’s collaboration with the Project helped meet the needs of female and male landowners in the Project implementation areas? To what extent are the Project objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Specifically, are behaviors resulting in deforestation changing or likely to change? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? Are there any differences between women, men and youth? EQ-3 To what extent were youth and gender taken into consideration during implementation? How are youth/women’s groups targeted/selected? Probe: How were the activities designed to be relevant to youth and women? Did URP promote resilience activities that affect men and women differently? 75 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV To what extent have youth/women participated in capacity-building activities? EQ-4 Are you supporting income-generating activities? What type of technical assistance is being provided in order to ensure that these activities are successful? In your opinion, what types of collaboration should the Ministries of Environment and Agriculture have with the Project to support local management of adaptation to the effects of climate, etc. Do you think that the Project’s interventions, and the benefits your business have accrued through it, will be sustainable in the long-term without another project like the URP? What (if any) type of work-related training or capability building support has your company received through the Project? Probe: How would you rate the training and capacity building support you have received through the Project? (very positive impact, positive impact, no impact, negative impact, very negative impact?) 76 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Interview Guide – Sub-Watershed Committees General Date: Location: Respondent Name: Respondent Title: Sex of Respondent: Respondent Organization: Level of experience/knowledge of the Reforestation Project (low, medium, high): Interviewer/note-taker names: Questions related to the EQs EQ-2 What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project? Are these crops and practices appropriate for the area? What are the major influencing factors, and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience (women, men, and youth)? Specifically, are people still cutting down trees, leading to deforestation? If so, why, and what could be done better to reduce this behavior? What are the obstacles and enabling factors generally encountered in the process of facilitating the transformation and market access of local products? Are there any differences between women and men? Explain. EQ-3 What are the roles of women and men at different levels of the committee? How often do they participate? Are women actively involved in the committee decision making? Are there any suggestions that they have made that were taken on board? Anything to change? Improve? Do you think the Project is able to assist female farmers? Youth? How? Give examples. To what extent were youth and gender taken into consideration during project implementation? Probes: How were the activities designed to be relevant to youth and women? Did URP-promoted resilience activities affect men and women differently? To what extent have youth/women participated in capacity-building activities? How are youth/women’s groups targeted/selected? How often do women participate in the SWMC meetings? What type of role do women/youth play in the committee? Probe: Do they speak regularly in meetings? What type of positions do they hold (administrative, leadership, etc)? Are there any suggestions that women have made that were taken on board? Do they have any recommendations to improve the SWMC? Do they think the Project is able to assist female farmers? Youth? Etc. How? 77 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Probe: What is the Project doing now that is promising or successful in this regard? What should it be doing more of? Less of? Please provide examples. EQ-4 Who are the leaders of the committee? What are the processes and criteria for selecting and replacing leaders? How does social or economic status, age, or level of education affect their chances of becoming a leader? Give examples. What financial and material resources does your committee have for its activities and day-to-day management? Where do the resources come from that the committee needs for its activities and day-to-day management? How did you get them / how did you access them? Please provide examples. For the activities that require new or recurring financial resources, what are your suggestions for mobilizing the necessary financial resources after the end of the Project? To what extent did the URP build the capacity of local authorities and communities to ensure they have the skills and incentives to manage and continue implementing sub-watershed plans beyond the life of the Project? Probe: What types of capacity-building activities do you think the Project should deliver in the future? Do you think the Project should assist in the formulation of specific laws to ensure the sustainability of activities? If yes, which law? Interview Guide - Civil Society Organizations/Farmers’ Associations General 1. Date: 2. Location: 3. Respondent Name: Respondent Title: Sex of Respondent: 4. Respondent Organization: 5. Level of experience/knowledge of the Reforestation Project (low, medium, high): 6. Interviewer/note-taker names: EQ-1 How has your organization worked with the Reforestation Project? Probe: Did the Project help you with: expanding your production capacity through new equipment, expanding your distribution capacity, finding new consumers for your products? Increased revenue, sales, or better￾trained employees? Have you or your organization received support from any other USAID or international donor projects? If so, which one(s)? To what extent has the Project been able to provide training / technical assistance to your CSO and its members? On what topics? How do you qualify these training courses? (very useful / useful / not very useful) Why? Who generally participates in these trainings? What improved management practices or technologies did the Project promote? Are these practices appropriate for the area? What do you think is the primary value added for farmers of these improved management practices or technologies? Are different groups affected differently, or are there differences in their ability to access or take advantage of these practices or technologies? What are the strengths and weaknesses of the improved management practices or technologies promoted by the Project? EQ-2 What is your perspective on the effectiveness of the approach used by the Reforestation Project to improve the resilience of beneficiaries? How has your institution’s collaboration with the Reforestation Project helped meet the needs of female and male landowners in the Project implementation areas? To what extent are the Project’s objectives to improve resilience achieved (through its different interventions), or are expected to be achieved? Specifically, are behaviors leading to deforestation being reduced? Are there any differences between women and men? What are the major influencing factors and how are these factors facilitating/preventing the implementation process toward improving beneficiaries’ resilience? Are there any differences between women and men? And youth? What are the obstacles and enabling factors generally encountered in the process of facilitating the transformation and market access of local products? Explain. EQ-3 To what extent were youth and gender taken into consideration during implementation by your organization? How were youth/women’s groups targeted/selected? How were the activities designed to be relevant to youth and women and other vulnerable populations? Please provide examples. Did the Project promote resilience activities that affect men and women and youth differently? Please provide examples. To what extent was women’s leadership in the community enhanced? To what extent have youth/women participated in capacity-building activities? What type of activities? What were the outcomes? EQ-4 Do you think the farmers will continue to practice what they have learned after the Project ends? If yes, can you give an example? If not, what should be done differently? If you participate in a training, what did you learn? What are you doing differently because of the training? 79 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV Informed Consent for FGD Particpants (Project Beneficiaries) Thank you for coming here today [or for accepting our invitation to have the interview]. First, let me introduce the team: [Each member of the team should introduce himself/herself]. We are here on behalf of Social Impact to conduct research on the “Reforestation” Project, which is being implemented by Chemonics, and funded by USAID. The benefit of this research is to ensure that projects are well designed, and that we capture the feedback of persons like yourself who participate in those projects in order to improve them now and in the future. We would like to ask you a few questions about your activities with the Reforestation Project. Before we begin, you should know that neither this research nor your answers to our questions will in any way determine if a project will be implemented here, continue to be here, or effect your ability to access or receive any services. This interview will take approximately one hour. With your permission, we would like to take notes. Primary notes will not be shared with anyone outside the research team at Social Impact, and your name will be kept confidential to the fullest extent permitted by local law, U.S. law, and USAID policy. Only anonymized notes will be submitted to USAID. Additionally, we will not inform anyone else of your participation in the interview. We hope this makes you feel comfortable to express your ideas freely. Due to the private nature of this research, we ask that all focus group participants agree not to share anything that is discussed with anyone outside of this group once this conversation ends. Nonetheless, there is a risk that other discussion participants will repeat what is shared here today. Remember that you are free to refuse to answer any question. We do not foresee any risks to you for participating in this study, but also anticipate that there will be no direct benefit to you for participating. Your participation in this discussion is voluntary, and so if you do not feel comfortable for any reason, then you are welcome to either not proceed with this research at all or to excuse yourself at any time without any consequences to you. Given the COVID-19 pandemic, there are several reasons you may choose not to participate in the study. If you or someone in your household or workplace has been feeling sick, including having a cough or high temperature in the past two weeks, we would ask you not to participate for your safety and the safety of others. Likewise, if you are not comfortable meeting in person or have concerns about the ability to accommodate safe protocols in in a school (such as, social distanced seating, personal protective equipment, well-ventilated meeting areas, etc.). Participation is completely voluntary. You can choose not to participate for any reason, without any consequence, and you will not be asked to share why you have decided not to participate. Likewise, if you do not feel comfortable the day of the interview for any reason, you can decline to participate or end the interview early without any consequence. Also, please note that due to COVID-19, we will be keeping a log of all interviews including your name and contact information to facilitate contact tracing should any member of the research team become ill so that we would be able to inform you. If you have questions or concerns about the study, feel free to reach out to wantoine@socialimpact.com or +509 3698-3754. For other concerns, complaints, or grievances, please write to irb@socialimpact.com or call at +703-465-1884. Do you have any questions? Do you agree to participate? Thank you! Acknowledgment of consent ______________ Interview Guide – FGD with farmers/associations/cbos General 1. Date: 2. Location: 3. Facilitator/note-taker names: PARTICIPANT # SEX TYPE OF AGRICULTURAL ACTIVITY SUPPORTED BY THE PROJECT (APICULTURE, AGROFORESTRY, LIVESTOCK MANAGEMENT, COFFEE, COCOA, ETC.) NAME OF ASSOCIATION/CBO S/HE BELONGS TO What kind of activities did the Reforestation Project do with you/your organization in the community? In what way have you been supported from the Reforestation Project (training, income-generating activities, marketing, etc.) Based on your own experience, what activities did you do with the Project that helped you the most? Why? What activities have been less helpful, if any? What are you doing differently now, after you have received the support compared to before the start of the Project (specify dates or general year/timeframe)? In what way has the Project helped you in dealing with natural disasters? Flood, droughts, other? How has the Project brought about a change in the participation of women and youth? (decision-making, use of resources, etc.) What was successful? (please provide an example) Are there any unintended consequences? How are women affected through the interventions of this Project? Can woman/youth become a member of your association? If so, are women/youth represented in the management of the association? In your opinion, what other actions should the Project take now so that the activities it begins can continue after funding is completed? What other activities? Why? 81 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX D. DOCUMENTS REVIEWED PROJECT DOCUMENTS USAID Reforestation Project (URP) Attachment B - Program Description, not dated ANNUAL REPORTS USAID REFORESTATION PROJECT FY 18 Annual Progress Report September 1, 2017 – September 30, 2018 USAID REFORESTATION PROJECT FY 19 Annual Progress Report September 1, 2018 – September 30, 2019 USAID REFORESTATION PROJECT Year 3, Quarter 1 Progress Report October 1 – December 31, 2019 USAID REFORESTATION PROJECT Year 3, Quarter 2 Progress Report January 1 – March 31, 2020 USAID REFORESTATION PROJECT FY 2020 Annual Progress Report September 1, 2019 – September 30, 2020 WORK PLANS USAID Reforestation Project Year 1 Annual Work plan, October 1, 2017 – September 30, 2018 USAID Reforestation Project Year 2 Annual Work plan, October 1, 2018 – September 30, 2019 USAID Reforestation Project Year 3 Annual Work plan, October 1, 2019 – September 30, 2020 MONITORING, EVALUATION, AND LEARNING (MEL) PLANS USAID Reforestation Project Monitoring, Evaluation, and Learning (MEL) Plan, Year 1: October 2017- September 2018 USAID Reforestation Project Monitoring, Evaluation, and Learning (MEL) Plan, Year 2: October 2018- September 2019 CONCEPT NOTES Support to ANCASA’s marketing pilot of ackee in Northern Haiti (Phase 1) FERRE Haiti North Caracol USAID/RCCL/Chemonics Activities in Bande du Nord PERFORMANCE NARRATIVES FTF Performance Narratives Report as of 13-Nov-2019 FTF, FY2020 IM / Activity Performance Narrative MEMORANDA OF UNDERSTANDING (MOU) MOU 2019-014 Sub-watershed Management Committee of Perches and the Departmental Division of the Ministry of Agriculture in the North-east to increase tree cover MOU 2019-015 Municipality of Sainte Suzanne and the Departmental Division of the Ministry of Agriculture in the North-east to increase tree cover MOU 20-016 Departmental Division of the Ministry of Agriculture in the North for deworming activities MOU 20-017 Departmental Division of the Ministry of Agriculture in the North-east for deworming activities MOU 20-018 Departmental Division of the Ministry of Agriculture in the North-east to improve livestock and pasture management MOU 20-019 Departmental Division of the Ministry of Agriculture in the North and North-east to improve livestock and pasture management TABLE SECTORIELLE AGRICOLE Compte rendu participation Projet de Reboisement de l’USAID / CHEMONICS, 21 mars 2019 Compte rendu de la participation du projet à la table sectorielle agricole Nord (TSAN), 30 Août 2019 AGREEMENTS WITH PARTNERS FAA #001 CASB (AOR #025) FAA #002 APAPANNE (AOR #013) FAA #003 FoProBiM (AOR #014) SGA #001 DDENE (AOR #011) FAA #004 Kiskeya Vet (AOR #016) FAA #005 Institution Univers (AOR #015) FAA #006 FECCANO FAA #007 Village Planète FAA #008 MSPJ FAA #009 AAF FAA #011 RECOCARNO FAA #013 ODL FAA# 014 CACOPA FAA #015 ANCASA FAA #020 RFEO STD #001 FERRE OTHER DOCUMENTS USAID. Measuring Resilience in USAID, PowerPoint Presentation, not dated. USAID Evaluation Report Requirements, A Mandatory Reference for ADS Chapter 201 83 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX E. LIST OF DATA COLLECTION EVENTS TABLE 13. KIIs NAME DATE STAKEHOLDER CATEGORY AND AFFILIATION Macorel Saint-Elien 13-Apr Implementing Partner -Chemonics Keira Derman 14-Apr Implementing Partner – NCBA CLUSA Hugues Bastien 15-Apr Sub-Awardee – Institution Univers Hector Fabien 15-Apr Academic/Training Institution - CFAIM Deslumar Casimir 16-Apr Sub-Awardee - AAF Olivia Gilmore 16-Apr USAID – Environment Office Eugene Levael 16-Apr Implementing Partner – CIAT Fontescony Joseph 17-Apr Government of Haiti – Ministry of Environment, North Region Rev. Jabnel Esperance 19-Apr Sub-Awardee - CASB Serge Jean Louis 5-May Government of Haiti – Ministry of Environment, North-East Region Kenold Moreau 20-Apr USAID Jean Weiner 20-Apr Sub-Awardee - FiProBiM Obéi Dolcé 20-Apr Sub-Awardee – Village Planète Malcolm Porteus Gonzales 21-Apr Private Sector Company - OTP Pierre-Mary Brutus 21-Apr Government – Ministry of Agriculture Jean Luckner Bonheaur 21-Apr Sub-Awardee - RECOCARNO Jean Charles Audate 22-Apr Sub-Prime Implementing Partner -NCBA CLUSA Surfin Philome 22-Apr Sub-Awardee - ARAPANNE NAME DATE STAKEHOLDER CATEGORY AND AFFILIATION Pierre Doudy 22-Apr Sub-Awardee – Kiskeya Vèt Joseph Jean Claude 23-Apr Sub-Awardee - MSPJ Laguerre Jocelyn 23-Apr Sub-Awardee - ODL Joseph Jean Louis 23-Apr Sub-Awardee - FECCANO Jomanas Charles 26-Apr Sub-Prime Implementing Partner -NCBA CLUSA Ursula Perry 28-Apr Private Sector Company - Antillean Canning S.A. (ANCASA) Stanley Fardin 12-May Implementing Partner -Chemonics TABLE 14. GIs DATE LOCATION STAKEHOLDER GROUP # PARTICIPANTS # MEN # WOMEN 04/14/21 Grande Rivière du Nord Comité SBV de Grande Rivière du NordBahon 5 4 1 04/15/21 Perches Comité de Jassa 3 3 0 04/16/21 Capotille Comité de Massacre 5 3 2 04/15/21 Comité de SBV de Marion 2 2 0 04/18/21 Joli Trou/ Cormier Comité SBV de Grande Rivière du Nord 4 3 1 85 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV TABLE 15. FGDs DATE SUB-WATERSHED ASSOCIATION/GROUP MALES FEMALES ASSOCIATION ACTIVITES Plaine du Nord Paul Ascencio/PAC/VIAHSA Limonade APPWOLIM/ Asosyasyon Pwodikte Let nan Limonad/ Association d’eleveurs 7 1 Ouanaminthe GEDWA/Association d’éleveurs Pour le développement de Ouanaminthe 4 3 Perches Bénéficiaires 3 7 Terrier Rouge KOET Koperativ Elvè Terrier Rouge 8 7 Trou du Nord/Roche Plate AFDRP Assosyasyon Fanm pou Devlopman Roche Plate 1 11 Ferrier AAF/ Association des Apiculteurs de Ferrier 9 5 Caracol FoproBIM/ APDK Asosyasyon Péchè pou devlopman Karakòl 8 1 Fort-Liberte CACOPA 4 3 Madras APM/ Asosyasyon Peche Madras 7 2 Limonade OPPBL Òganizasyon Pechè pou Pwoteksyon Bòdmè Limonade 5 1 Grande Rivière du Nord CAJBC Coopérative Agricole JeanBaptiste Chavannes 6 2 Quartier Morin OTG /Oganizasyon ti Plante 8 1 DATE SUB-WATERSHED ASSOCIATION/GROUP MALES FEMALES ASSOCIATION ACTIVITES Gran Pre Bois de Lance/ Limonade MPDB Mouvman Pwogresis pou Devlopman Bois de Lance 6 2 Plaine du Nord FECCANO 6 2 Limonade PLOMB 8 1 Vallières KODV Komite Oganizasyon pou Devlopman Valyè 6 2 Vallières MPFV Mouvman Pwogresis Fanm Valyè 0 9 Vallieres ADTP 4 2 Jaczil AFVMJ / Asosyasyon Fanm Vanyan Marais salant Jaczil 0 5 87 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX F. URP SUB-AWARDEES TABLE 16. URP SUB-AWARDEES IMPLEMENTING PARTNERS LOCAL COMMUNITY INVOLVEMENT COMMUNES SUB￾WATERS HED OBJECTIVE START DATE END DATE BUDGET USD EXPECTED RESULTS CASB (AOR #025) CBOs: Cooperative Agricole de Terrier-Rouge (CART), Initiative pour le Développement Durable d’Haiti (IDDH), (GAO) Local authorities: ASECs, CASECs, BAC, Mayor, Ministry of Environment representatives Terrier-Rouge N/A Development of the wood energy and beekeeping sectors in Terrier￾Rouge August 2018 August 2020 $241,019 1 - 160,000 seedlings produced and transplanted on 110 ha; 2 - 70 model farmers trained in forage production, including hay and silage; 3 - 5 ha of forage species produced and demonstration plots established to promote forage production and management to other farmers; 4 - 250 farmers with increased understanding of improved agroforestry practices and methods; 5- A beekeeping farm and 15 individual apiaries established for members of the 3 CBO partners, and 35 modern beehives established on 5 ha of woodlot APAPANNE (AOR #013) CBOs: Organisations des Paysans pour le Développement de Lamine (OPDL), Organisation de Recherche et d’Actions Durables pour un Développement Intégré (ORADI), Organisation des Paysans pour le Développement de Chambellan (OPDC), Association des Travailleurs pour le Développement de Welch (ATBW), Mouvement des Jeunes pour le Développement de Capotille (MOJEDEC), Capotille Massacre Support of Reforestation for sustainable management of the environment and the municipality of Capotille Novemb er 2018 May 2020 $183,076 1 - 178,000 tree seedlings planted and cared for; 2 - 200 ha of land with increased tree cover and improved management; 3 - 80 ha of woodlots (species: acacia, oak, white wood, mahogany, cedar, cashew nuts, moringa…); 4 - 116 ha of agroforestry plots (cashew nuts, mango, oak, white wood, mahogany, cedar and short cycle crops, such as, yam and pineapple…); 5 - 4 ha of school orchards (species: cashew, mango, avocado…); 6 - 415 farmers and 500 schoolchildren from two 2) schools trained in caring, monitoring, and protecting planted tree seedlings to improve their survival rates and ensure their growth, and in tree and forest resources management; 7 - 132 farmers (members of 7 community-based organizations) and 26 local authorities representatives and civil protection personnel trained on adapted practices on the management of woodlots and agroforestry; and 8 - 1,158 community members indirectly IMPLEMENTING PARTNERS LOCAL COMMUNITY INVOLVEMENT COMMUNES SUB - WATERS HED OBJECTIVE START DATE END DATE BUDGET USD EXPECTED RESULTS Groupman Paysan Kapoti (OPK), Semeur d’Amitié et de Fraternité de Welch (SAFW), Schools: Ecole Nationale Lamine and Lavi pou Timoun, Local authorities: ASECs, CASECs, Mayors, BAC, Ministry of Environment representatives benefiting from improved natural resources management. FoProBiM (AOR #014) CBOs: Asosyasyon Pechè pou Developman Karacol (APDK), Organizasyon Pechè pou Proteksyon Bodmè Limonade (OPPBL), Asosyasyon Pechè Madras (APM) Local authorities: ASECs, CASECs, Mayors, Ministry of Environment representatives, Police Nationale Haiti (PNH) Caracol, Limonade N/A Rehabilitation of 100 ha of mangroves in the Protected Area of Three Bays on the coastal area of the communes of Caracol and Limonade January 2019 July 2020 $143,245 1- At least 100 ha of mangroves in the 3 Bays National Marine Park (3BNMP) protected area rehabilitated through reforestation activities in approximately 20 ha of clearings scattered within the targeted 100 ha; 2 - At least 40,000 red disaster risk reduction and livelihood improvement; 3 - At least 280 community members, farmers and fishermen involved in the rehabilitation, protection and surveillance of this ecosystem. 4 - mangrove propagules produced in Chabannon (Limonade) and Madras (Caracol) and transplanted in the 3BNMP; 5 - A mangrove management and surveillance system established to reduce the illegal exploitation of the mangrove; 6 - At least 25 people trained in mangrove nurseries establishment, and propagules transplantation and protection; 7 - At least 300 community members more aware of the importance of mangroves 89 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV IMPLEMENTING PARTNERS LOCAL COMMUNITY INVOLVEMENT COMMUNES SUB￾WATERS HED OBJECTIVE START DATE END DATE BUDGET USD EXPECTED RESULTS DDENE (AOR #011) Project partners: Institution Univers, Kiskeya Vet Communities: Marion, Savanne Déclée, Dosmond, Capotille Ouanaminthe Marion, Massacre, Jassa Support to the Propagation Plant and Training Center of Dosmond Decemb er 2018 Novemb er 2020 $185,489 1 - Capacity of the propagation plant and training center in Dosmond reinforced; 2 - 350,000 seedlings produced at the Dosmond nursery; 3 - 40,000 seedlings transplanted in Fort￾Liberté (Dumas section) and Perches (Marion section). Kiskeya Vet (AOR #016) CBOs: Organisation pour le Développement d’Acul des Pins (ODA), Organisation pour le Développement de Bachary d’Acul￾des-Pins (ODDA), Asosyasyon Paysan AkildèPen (APA), Organisasyon Paysan AkildèPen (OPA), Organisasyon Agrikol pou Pwoteksyon Environman Akildèpen (PAA) Local authorities: Mayor, ASECs, CASECs, BAC, Ouanaminthe Jassa Strengthening and improvement of tree cover in the communal section of Acul des Pins, Ouanaminthe March 2019 Decemb er 2020 $143,156 1 - One central nursery of 80,000 tree seedlings established at Bachary; 2 - Two decentralized nurseries of 35,000 seedlings each established at Boket and Laurier; 3 - At least 150,000 fruit trees and forest seedlings produced and transplanted. 4 - 229 ha with increased tree cover 5 - Three nursery management committees and three 3) tree monitoring committees created and functioning in Bachary, Boket, and Laurier; 6 - 551 farmers, including 250 women, have adopted improved agroforestry practices; 7 - 51 participants, of which 21 are members of JSE youth association and 30 of a CBO IMPLEMENTING PARTNERS LOCAL COMMUNITY INVOLVEMENT COMMUNES SUB￾WATERS HED OBJECTIVE START DATE END DATE BUDGET USD EXPECTED RESULTS Institution Univers (AOR #015) Local authorities: Mayor, ASECs, CASECs, BAC, Ministry of Environment representatives Schools: Collège George Muller de Ouanaminthe, Collège de l’Etoile, Centre Classique de Formation, Ouanaminthe Jassa Implementati on of an agroforestry system on 100 ha in Morne Coucou, Ouanaminthe March 2019 Septemb er 2020 $172,206 1 - At least 150 youths trained in tree seedlings production and transplantation; 2 - At least 100 farmers’ families trained in and using improved agroforestry production practices for increased income and food security; 3 - A nursery with a production capacity of at least 55,000 seedlings established; 4 - Two tree orchards of Persian lemons, local lemons, and cashew nuts established; 5 - At least 90,000 tree seedlings transplanted on 100 ha for fruit tree and agroforestry production at Morne Coucou; 6 - At least 90,000 tree seedlings cared for by beneficiary families. FECCANO TBC TBC TBC TBC Novemb er 2019 Novemb er 2021 TBC TBC Village Planète TBC TBC TBC TBC Jan 2020 July 2021 TBC TBC ODL July 2020 February 2022 AAF July 2020 February 2022 RECOCARNO July 2020 February 2022 MSPJ Septemb er 2020 Decemb er 2021 91 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX G. TREE SEEDLINGS PRODUCED AND TRANSPLANTED TABLE 17. SEEDLINGS PRODUCED AND TRANSPLANTED AS OF THE END OF FY20 ORGANIZATIONS/ COMMUNES NUMBER OF SEEDLINGS IN PRODUCTION OR PRODUCED NUMBER OF SEEDLINGS TRANSPLANT ED NUMBER OF BENEFICIARIES NUMBER OF HECTARES COVERED BUDGET (TBC IF AVAILABLE) Subgrants Kiskeya Vèt/Ouanaminthe 300,000 169,705 327 324.1 $105,041.43 DDE￾NE/Ouanaminthe48 350,000 71,037 11 72.3 $85,718.30 CASB/Terrier Rouge 160,000 160,513 189 184.14 $174,820.32 UNIVERS/ Ouanaminthe 100,000 73,587 100 100 $171,896.08 APAPANNE/Capotille 150,000 178,610 260 262.7 $133,527.46 FoProBiM/Caracol, Limonade 50,000 46,777 Not applicable 28.5 $127,605.39 FECCANO 314,500 Not available for FY20 Not available for FY20 Not available for FY20 $366,068.00 VILLAGE PLANETE/Acul du Nord 25,000 15,000 Not applicable 6 $85,459.24 TOTAL Subgrants 1,449,500 715,229 887 977.74 1,250,136.22 Direct Implementation (DI) Limonade 69,159 355,937 408 304.47 305,795.00 Acul du Nord 71,693 267,441 496 191.4 262,363.12 Milot – Perches du Bonnet 31,208 176,806 329 202.6 133,837.20 Milot – Bonnet a l’Eveque 11,054 90,996 129 104.4 96,542.83 Quartier Morin 20,675 49,798 103 75.6 114,908.42 Perches 126,336 121,424 279 209.0 165,491.72 Ste Suzanne 21,241 58,030 81 146.0 95,545.07 TOTAL DI 351,366 1,120,432 1,825 1,233.47 1,174,483.36 GRAND TOTAL 1,80,866 1,835,661 2,712 2,211.21 2,424,619.58 ANNEX H. TRAININGS CONDUCTED IN 2018-2020 Following is a draft table that the ET compiled based on available information in the Annual and Quarterly Reports that were shared with the team. The information needs to be verified and completed. TABLE 18. TRAININGS CONDUCTED IN 2018-2020 YEAR TOPIC NUMBER OF PARTICIPANTS NUMBER OF FEMALE PARTICIPANTS PERCENTAGE FEMALE PARTICIPANTS IMPLEMENTATION MECHANISM (G OR DI) OR NAME OF ORGANIZATION OFFERING TRAINING 201849 6 sessions on how to respond to Project RFAs and APS and adequately fill the application form 122 representatives of local organizations and institutions 20 16 Project staff Two-day visit of CASB senior management to the Montrouis and Bas-Boën CRDDs n/a n/a /a Direct investment A series of NRM training programs in 12 communes 489 participants n/a 36 5 sessions on gender and youth inclusion issues 52 n/a n/a Field visit to established woodlots in Ennery, Artibonite department 30 farmers and field technicians (11 from the North and 19 from the North-East) 10 33 Three-day training program on survey techniques 43 young professionals n/a n/a 2019 Training in forage management and silage production 250 livestock farmers 97 39 Direct investment Training on air flow control in the carbonization process 24 charcoal producers n/a 42 Project selected Faculté des Sciences Agronomiques, Halieutiques et Agroalimentaire de Limonade and The training will target more than 600 municipal council members, CASECs and Université of Limonade and CFAIM Grant 49 Annual Report 2018. 93 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV YEAR TOPIC NUMBER OF PARTICIPANTS NUMBER OF FEMALE PARTICIPANTS PERCENTAGE FEMALE PARTICIPANTS IMPLEMENTATION MECHANISM (G OR DI) OR NAME OF ORGANIZATION OFFERING TRAINING the Centre de Formation en Aménagement Intégré des Mornes (CFAIM) to implement a training program on the Haitian Rural Code. ASECs from 19 communes. Rural Code training program organized by the ASEC Coordinator of Haut Maribahoux 53 CBO participants 25 47 Impact of training Training on the importance of mangroves in the ecosystems Members of three local CBOs: Asosiyasyon Pechè pou Devlòpman Karakòl (APDK), Oganizasyon Pechè pou Pwoteksyon Bodmè Limonad (OPPBL), and Asosyasyon Pechè Madras (APM) n/a n/a Training on laws and regulations pertaining to forest/tree cover, water resources, and livestock management 67 Mayors and General Directors of municipal administrations, 155 CASECs members, 291 ASECs members, and 64 public servants from the municipal administrations n/a n/a Université Limonade and CFAIM (Grant) A three-day workshop on integrated pest management in Ouanaminthe 21 participants representing grantees, partners, and staff operating in Region 2 n/a n/a Training on improved agriculture management practices (EG3.2- 24) 512 n/a n/a Grant Number of people trained in sustainable landscapes (Indicator EG13.1) 1,132 n/a n/a Grant Improved agroforestry and woodlots management (EG13.1) 450 n/a n/a APAPANNE Grant Exchange visit in the commune of 20 n/a n/a Direct investment YEAR TOPIC NUMBER OF PARTICIPANTS NUMBER OF FEMALE PARTICIPANTS PERCENTAGE FEMALE PARTICIPANTS IMPLEMENTATION MECHANISM (G OR DI) OR NAME OF ORGANIZATION OFFERING TRAINING Milot at the Emari Botanical Garden with 15 farmers, direct beneficiaries of the sub-awards and five representatives of the sub￾awardees Sub-awardees and their members have benefitted from various technical training on agroforestry systems and woodlot management, nursery and tree management, natural resource management, seasonal crop production, and pest management. Apiculturists were trained on beekeeping practices, care for melliferous plants, and honey collection for commercializatio n. MEL staff trained partners’ and D.I.’s field agents on: 1. GPS and/or smartphone handling as data collection tools; 2. understanding and filling in the data collection forms; and 3. the use of an Excel database to store, secure and analyze data. 12 n/a n/a Project 202050 Modern beekeeping techniques 24 members of 3 CBOs (AAF, MCAD and CACOPA) 7 29 CASB Grant Good hygiene practices and 61 members of 2 CBOs (RFEO and CAJBC) 53 87 50 Annual Report 2020 (draft) 95 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV YEAR TOPIC NUMBER OF PARTICIPANTS NUMBER OF FEMALE PARTICIPANTS PERCENTAGE FEMALE PARTICIPANTS IMPLEMENTATION MECHANISM (G OR DI) OR NAME OF ORGANIZATION OFFERING TRAINING marketing techniques Good environmental practices, production techniques and fodder conservation 249 102 41 Refresher training for veterinary agents in animal health 80 19 24 DI (?) with the Ministry of Agriculture and Natural Resources Seedling production and nursery management 235 6 3 Integrated pest management 25 3 12 ANNEX I. WOMEN’S ORGANIZATIONS ASSESSED IN FY2018 TABLE 19. WOMEN'S ORGANIZATION ASSESSED IN FY2018 WOMEN’S ASSOCIATION ZONE AREA OF INTERVENTION # MEMBERS Rasanbleman Fanm Vanyan Limonad-RAFAVAL Limonade Livestock, agricultural production, and processing of cocoa milk and coffee 500 Asosiyasyon Fanm Limonad pou Devlopman Pwodiksyon Agrikol ak Atizana-AFLIDEPA Limonade Crafts, agricultural production, and fruit processing 500 Asosiyation Fanm Gabart Levaillant Dondon Agricultural production, marketing, and processing of coffee, cassava, and rice 100 Asosiyasyon Fanm Vanyan Dondon Agricultural production, marketing, and processing of coffee, cassava, and rice 100 Asosiyasyon Fanm Agrikòl Devlopman Faeton Phaeton Apiculture and protection of mangroves 80 Organisation des femmes marais salant à Jakzil Jaczil Production of honey and salt 100 Organisation des femmes étoiles de petit-anse Cap Haïtien Tree plantation, Jadin lakou, transformation of cacao and peanuts 100 Total 1,480 Note: RAFAVAL and AFLIDEPA have their own physical facilities with workshops. Some of their members had received training for their activities. The others are rather weak institutionally and technically. 97 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV ANNEX J. EXAMPLE OF INFORMATION NEEDED REGARDING SEX-DISAGGREGATED NUMBER OF PARTICIPANTS IN TRAININGS TABLE 20. TRAININGS CONDUCTED DURING FY2020 Q3 TRAINING SUBJECT COMMUNE(S) # OF TRAINEES # OF MEN # OF WOMEN ORGANIZATION THAT DELIVERED TRAINING Short-cycle crop production Bay of Acul 40 30 10 Mangrove management and climate change Bay of Acul 90 62 28 Agroforestry system management Sainte Suzanne 40 21 19 Agrosylvopastoral system management Terrier Rouge, Trou du Nord, Limonade, Grande Rivière du Nord, Milot 105 82 23 Source: Q3 FY2020 ANNEX K. SIGNED DISCLOSURE OF CONFLICT OF INTEREST FORMS FOR ALL EVALUATION TEAM MEMBERS 99 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV