1 Emergency Food Security and Recovery Program for the vulnerable people affected by the residual effect of Hurricane Irma in the North-East and Central departments of Haiti USAID EFSP Final Evaluation Report Submitted by: Amisial LEDIX, Economist-Statistician/Monitoring & Evaluation Specialist In collaboration with: Amos MONTREUIL JEAN, Agronomist/Food Security and Nutrition Specialist December 1, 2020 2 Table of Contents Acknowledgements 3 List of Tables, Figures and Graphs 4 List of Acronyms 6 Executive Summary 7 1) Background 10 2) Final Evaluation Objectives 12 3) Methodology 12 4) Constraints and Limitations of the Study 22 5) Analysis of the Final Evaluation’s Findings 23 5.1 General Characteristics of Beneficiary Households 23 5.2 Analysis of Project Relevance 29 5.3 Analysis of Project Effectiveness 37 5.4 Analysis of Project Efficiency 46 5.5 Analysis of Project Impact 49 5.6 Analysis of Project Sustainability 51 5.7 Analysis of Project Linkages and Exit Strategies 53 5.8 Analysis of the Local Market in the Intervention Areas 55 6) Conclusion and Recommendations 60 7) Bibliographical References 64 8) Appendix I: FCS Indicator 65 9) Appendix II: HHS Indicator 66 10) Appendix III: Reduced Coping Strategy Index (rCSI) 67 11) Appendix IV: Data Collection Tools (in separate files) 69 3 Acknowledgements The team of consultants in charge of the present study wishes to express their gratitude, in a special manner, to all those who contributed to the completion of this work. Despite the many challenges and difficulties encountered, we were able to succeed through your active and effective participation. First, we wish to thank the Monitoring and Evaluation team, within the World Vision Haiti Central Office, for their open and fruitful collaboration. Interacting with our interlocutors was essential to the successful completion of the consultation. The support provided for the use of tablets, a few scales and measuring poles, and the reference documentation made available to us made our work much easier. Our gratitude, then, goes to the EFSP project manager with whom we also interacted during the scope of work meeting for this study. We also thank World Vision’s Regional Manager in the North for his support in lending us the measuring poles, and the project’s Monitoring and Evaluation key person for their respective support in providing the necessary information to facilitate the progress of field operations. We would also like to thank the enumerators and supervisors who went to the field to collect data from beneficiaries in the project’s various intervention communities. The consulting team remains convinced that the study’s findings and recommendations will effectively serve the EFSP project team in addressing the challenges and achieving expected results on behalf of the beneficiaries living in the organization’s areas of intervention. 4 List of Tables, Figures and Graphs Table 1: Breakdown of the number of beneficiary households to be surveyed by commune............................................................................................................................................ 18 Table 2: Estimated sub-population of breastfeeding women beneficiaries by commune 19 Table 3: Breakdown of the sample of breastfeeding women to be surveyed by commune in the Northeast ................................................................................................................................ 19 Table 4: Breakdown of Recipients Sample with Children 0-59 Months of Age to be Surveyed.............................................................................................................................................. 20 Table 5: Number of Key Interviews by Department................................................................ 21 Table 6: Distribution of Respondents by Department by Housing Type (n = 1358)......... 27 Table 8: Distribution of respondents by department according to the second activity of the head of household...................................................................................................................... 28 Table 9: Main source of water for household domestic use.................................................... 31 Table 10: On the management and maintenance of water points ....................................... 32 Table 11: Distribution of beneficiary households by department according to the water conservation method used at home............................................................................................. 34 Table 12: Level of satisfaction by activity in which respondent took part .......................... 37 Table 13: Dietary Diversity Score by Department ................................................................... 38 Table 14: Food consumption score for project beneficiaries by department...................................... 39 Table 15: Hunger score for project beneficiaries by department ........................................................ 40 Table 16: Reduced Coping Strategies Index (rCSI) .............................................................................. 41 Table 17: Percentage of households where adults and children eat at least 2 meals a day (adults and children), by age .................................................................................................................................... 42 Table 18: Percentage of food use by type (household consumption, sale, exchange, livestock feed) by department............................................................................................................................................ 42 Table 19: Percentage of households reporting having received information to cope with economic shocks for the intervention areas and its communes........................................................................... 43 Table 20: Major shocks that caused hunger in percentage................................................................... 43 Table 21: Prevalence of acute malnutrition by department .................................................. 45 Table 22: Key indicators measured, and their progress compared to targets................... 45 Table 23: Comparison of baseline and final evaluation results .............................................. 50 Table 24: Level of Satisfaction with Criteria Achievement................................................................... 54 Figure 1 : Age pyramid of the communes where the EFSP Centre project is active. Error! Bookmark not defined. Figure 2 : Age pyramid of the intervention communes of the EFSP Nord-Est project....................Error! Bookmark not defined. Graph 3: Distribution of Respondents by Department by Gender (n=1,258) .................... 24 Graph 4: Distribution of Respondents by Department by Household Status (n=1,258).. 24 Graph 5: Distribution of heads of household by department according to their education level (n=1,258).................................................................................................................................... 25 Graph 6: Distribution of heads of household by department according to marital status (n=1,258)............................................................................................................................................. 26 Graph 7: Distribution of respondents by department according to whether or not there is at least one orphan in the household................................................................................................................ 26 Graph 8: Distribution of respondents by department according to whether or not there is a pregnant woman in the household ........................................................................................ 26 5 Graph 9: Distribution of respondents by department according to whether or not there is at least one person with a disability in the household.......................................................... 26 Graph 10: Distribution of respondents by department by monthly household income group.................................................................................................................................................... 28 Graph 11: Distribution of respondents by department by first annual household expenditure item .............................................................................................................................. 29 Graph 13: Distribution of beneficiary households by department according to whether or not they received messages about newborn care in the health centers (n=618)......... 31 Graph 14: Distribution of beneficiary households according to whether they participate in training provided by mother leaders in the community..................................................... 31 Graph 15: Distribution of beneficiary households by department according to the amount of water available for use per day........................................................................................................................ 32 Graph 16: Distribution of beneficiary households by department according to the walking time (round trip) for water collection.......................................................................... 32 Graph 17: Distribution of beneficiary households by department according to whether or not they usually treat water at home ..................................................................................... 33 Graph 18: Distribution of beneficiary households by department according to the fact that they do not usually treat water at home............................................................................ 33 Graph 19: Distribution of beneficiary households by department according to the water treatment method used .................................................................................................................. 34 Graph 20: Distribution of beneficiary households by department according to whether or not they received handwashing messages ............................................................................. 35 Graph 21: Range for reduced coping strategies index (rCSI)................................................. 41 Graph 22: Prevalence of mothers who practice exclusive breastfeeding among infants six months of age or less ................................................................................................................. 44 Graph 23: Distribution of respondents according to their perception of products prices on the vouchers compared to prices for the same products at other sellers .................... 56 Chart 24. Distribution of respondents by who decided when to redeem and what products to get . 57 Graph 25: Distribution of respondents by use of products after voucher redemption.... 57 Graph 26: Distribution of respondents according to their satisfaction with the quality of food products after the voucher was redeemed ....................................................................... 58 Graph 27. Distribution of respondents by how they would like to receive assistance for a similar project.................................................................................................................................... 59 6 List of Acronyms ASEC Assemblée de la Section Communale CASEC Conseil d’Administration de la Section Communale CEPAM Center for Promoting Breastfeeding (Centre pour la promotion de l’allaitement maternel) CNSA Coordination Nationale de la Sécurité Alimentaire COVID-19 Coronavirus Disease 2019 DINEPA Direction Nationale de l’Eau Potable (National Water Authority) DPC Civil Protection Office (Direction de la protection civile) EFSP Emergency Food Security Program FCS Food Consumption Score FEWSNET Famine Early Warning Systems Network FFP Food for Peace FNM Female No Adult Male F&M Female and Adult Male GAM Global Acute Malnutrition GDP Gross Domestic Product GPS Global Positioning System HDDS Household Dietary Diversity Score HHS Household Hunger Scale HNO Humanitarian Needs Overview HTG Haitian Gourde IPC Integrated Phase Classification LA Local Authorities MAST Ministry of Social Affairs and Labor (Ministère des Affaires Sociales et du Travail) M&E Monitoring and Evaluation M4P Making Markets Work for the Poor NGO Non-Governmental Organization OECD Organization for Economic Cooperation and Development PDM Post-Distribution Monitoring rCSI Reduced Coping Strategies Index S4T Savings for Transformation SOP Standard Operational Procedures SPSS Statistical Package for the Social Sciences SRS One-stage Simple Random Sample TED Home Water Treatment (Traitement d’eau à domicile) TEPAC WASH Communal Technician (Technicien d’eau potable et assainissement communal) USAID U.S. Agency for International Development UVT Unconditional Voucher Transfer VFW Voucher for Work WV World Vision WVI World Vision International 7 Executive Summary The Emergency Food Security Program (EFSP), implemented by World Vision in six communes within the Northeast Department (Fort-Liberté, Carice, Mombin Crochu, Sainte Suzanne, Ferrier and Vallières) and two communes in the Central Department (Cerca-la-Source and Cerca Carvajal), seeks to provide immediate access to food to communities in crisis and emergency situations (IPC 3 and 4) and help communities recover from the residual effects of Hurricane Irma. For over 18 months this program has been implemented with USAID’s financial support, targeting more than 6,700 vulnerable households (33,500 people) who needed help mitigating food insecurity in times of crisis due to reduced harvests and livelihoods. In relation to initial objectives, the program included five main intervention areas: Distribution of food vouchers, Nutrition, Drinking Water Infrastructure, Agriculture, Financial support (Training and S4T). As program interventions have come to an end, the present evaluation was conducted to assess the achievement of the goal, objectives, and outcomes and review the means by which they were achieved. For this work, a double quantitative and qualitative approach was used in collecting the required information for the evaluation. 678 beneficiaries per department and other project stakeholders (project managers, government entities, community leaders, etc.) were interviewed. The quantitative survey was conducted among program beneficiary households and was focused on reviewing their situation using the phase classification indicators in terms of food security, and reviewing the situation of the markets serving these households. The qualitative survey targeted key project stakeholders whose interviews were to be used to validate the initial information collected. In addition to this series of interviews, there are more evaluation questions based on OECD criteria. Overall, the findings of this evaluation show that :  There are more people from the 5-19-year-olds in the population structure of the project’s beneficiary households and the female population is much more represented in the Northeast than in the Center, respectively with 71% and 42% of beneficiaries.  Most respondents from the Northeast are women, while the reverse is true in the Central Department. Overall, there are more households headed by adult women and men (F&M) in this survey.  In both departments, most beneficiaries have no education at all (43% for the Northeast and 32% for the Center) or have not completed their primary education (31% for the Northeast and 48% for the Center).  Agriculture and trade are the main sources of income for most beneficiary households and, in most cases, these households earn a monthly income of about 5,500 HTG or less.  Poor households are more inclined to spend their scarce economic resources in food. Taking into account the identified issues and analyzing the outcomes from the five intervention areas, the program’s relevance becomes obvious. From a strategic perspective, the nutrition component’s interventions are in line with the National Nutrition Policy (NNP) that seeks to improve the nutritional situation of communities based on a preventive approach to all forms of malnutrition. Outcomes show that the activities from this component have reached a large 8 number of beneficiaries who were facing this problem. At least 90% of respondents indicated having received messages related to newborn care, and at least 40% of respondents (women) state they attend trainings conducted by mother-leaders in their community. During the implementation period, the Haitian Gourde rapidly lost value and inflation rose steadily. To deal with this situation, monthly adjustments were made to voucher values, in line with the increase in food prices in the local markets. In this activity, the most vulnerable segments of the population, consisting of women leading single-parent households, are the main beneficiaries. According to outcomes from the infrastructure component, at least 60% of beneficiary households in the Central Department have access to an improved water source, compared to 35% of beneficiary households in the Northeast who use uncollected water sources. Training and awareness activities on TED and handwashing have reached a large number of beneficiaries, since (59% in the Northeast and 71% in the Center) most of them reported treating water at home (with chlorine and Aquatabs) and more than 80% reported having received handwashing messages. After the interventions, about 50% of beneficiaries can now use sufficient water and have easy access to water, according to DINEPA standards. The outcomes show that these interventions are in line with the program’s goal of increasing or restoring access to safe drinking water for communities whose water systems were damaged after Hurricane Irma. In the agriculture component, actions were aligned with the National Adaptation Action Plan (NAPA) and directed towards reducing vulnerability to climate shocks and post-harvest losses, two major issues faced by farmers in the program’s intervention areas. To that effect, training and technical support on best land conservation practices were provided to smallholder beneficiaries in target communities. In addition, farmers also received training in post-harvest management. The setting up of credit unions is also a relevant element in strengthening the households affected by Hurricane Irma. Through these groups, households were able to save, in spite of their small income, thanks to the distribution of food vouchers. Thus, they were able to set up a small trade or strengthen an existing business activity thanks to the loan contracted in the community groups. Through analyzing the effectiveness of program interventions, we saw the progress of key indicators against their targets. Only 3 of them did not reach 50% of the targets! However, in our interview with the project’s M&E team, we found out that 4 output indicators did not reach 50% of their targets, namely the “number of stone lines rehabilitated”, “number of check dams rehabilitated”, “number of soil dams rehabilitated”, “number of IEC materials distributed”. Overall, this implies that almost all project indicators have reached at least 50% of their target. The efficiency of the project was assessed by comparing the cost of component activities to their effectiveness. In this analysis, the food voucher, nutrition, agriculture and financial support (training and S4T) components were taken into account. According to the analysis carried out, the program has had efficiency concerns, especially in terms of meeting deadlines with vendors and, in agriculture, due to a lack of human resources to monitor activities. Since we had no financial report, we could not conduct further analysis on the program’s efficiency. 9 The program was able to reach all the targeted beneficiaries, and significant changes were observed in beneficiary households and communities, such as: improvement in the beneficiaries’ food insecurity and living conditions, improvement in the diet of nursing mothers, savings opportunities, and opportunities to diversify households’ business activities through savings groups. These changes are an evidence of the program’s impact on communities and households. The adoption of a strategy based on the project’s ownership by beneficiaries is a key element that can ensure the sustainability of project interventions. The building of infrastructure, training on farming techniques, and the setting up of credit unions are sustainable activities that will bring benefits to the communities, beyond the project’s duration. The involvement of local government actors in the implementation of activities is also a guarantee of sustainability for most project interventions. However, the exit strategy was not adequately implemented because the links with system partners (the sectors related to the project) are not clearly seen in the implementation process. The additional market research conducted to enrich the assessment confirmed that all major markets in the region were operating. Contracted vendors did not highlight any challenge during the selection process, and also reported that the WV team provided critical advice that would enable them to meet the defined selection criteria (such as guidelines on licensing their small businesses). The project played no role in the fluctuation of product prices. Community leaders talked about complaints from non-program vendors about the loss of customers to registered vendors. The amount of money coming in benefits the communities, but the money flow and transactions only benefits a smaller number of vendors. The WV team emphasized, however, that an inclusive and participatory approach was used to engage vendors in the program, using clear selection criteria. Considering this assessment’s findings, it becomes necessary to review the intervention strategy in a future program, especially in the food security and farmers’ capacity building components, so that activities can have greater impact. 10 1) Background In 2018, Haiti had a per capita gross domestic product (GDP) of $870 and a human development index ranked 168th out of 189 countries, for the same year (World Bank1 , 2019). More than 6 million Haitians live below the poverty line of less than US$2.41 per day, and more than 2.5 million live below the extreme poverty line of US$1.23 per day. Political instability has severely hampered Haiti’s economic and social development in recent months, and the country has experienced a fast currency depreciation (around 30 percent) followed by a fast appreciation without much change in economic fundamentals, high levels of inflation (close to 20 percent), and a GDP contraction (projected at 0.5 percent) in fiscal year 2019. In addition to political and economic insecurity, Haiti’s vulnerability to recurring climate shocks has increased over the past five years. The country remains highly vulnerable to natural hazards, mainly hurricanes, floods, and earthquakes, and more than 96 percent of the population is at risk due to chronic environmental degradation, poor infrastructure, and erosion-prone farming practices, among others. Natural and man-made shocks exacerbate Haiti’s many development challenges and underlying poverty drivers. One of the latest major natural disasters, Category 4 Hurricane Matthew, that hit Haiti in 2016, killed approximately 600 people, affected 2.1 million people and caused extensive damage in the southern peninsula and northwestern part of the country. It caused a 32 percent loss of the country’s GDP. Prior to that, in 2010, an earthquake killed approximately 230,000 people, displaced 1.5 million people, and caused a loss of 120 percent of the GDP2 . As reported by FEWSNET and CNSA, crop shortages and livelihood disruptions caused by hurricane Irma and recent heavy rains have led parts of Haiti’s northeastern and central departments into a crisis situation (IPC 3) that continued until May 2018. Damage to infrastructure, homes and livelihoods caused by hurricanes Matthew and Irma was estimated at US$2.7 billion nationwide, including US$573.5 million for the agricultural sector3 . The spring harvest, which normally accounts for half of the country’s annual agricultural yield, fell due to recurrent droughts (from 2016 to 2018) that have affected almost all departments and consequently worsened the already precarious food security situation. In November 2019, the National Food Security Coordination (CNSA) highlighted a significant increase in the food basket’s value with monthly and annual rates of 14% and 40% respectively. Markets in the south (Jérémie, Cayes, Fond des Nègres, Jacmel, and Croix des Bossales) were the most affected, mainly due to the rise in grain prices (corn, rice, and flour). An estimated 430,468 people (367,038 in the Northeast and 63,430 in the Center) are still under shock. “The strike of hurricane Irma and the heavy rains that followed in the Northeast hampered farming activities, especially in the lowland areas. Excessive soil moisture in Ouanaminthe, Ferrier and Fort Liberté prevented farmers from planting winter crops. As a result, there were no 1 See the revised Haiti Country Partnership Framework for the FY16-19 period (English) http://documents.worldbank.org/curated/en/673911467986337757/Haiti-Country-partnership-framework-for-the-period￾FY16-FY19 2 See the World Bank document: Haiti, providing opportunities for all Haitians 3 FAO Haiti: Hurricane Matthew Situation Report, 2017 11 harvests, except in the mountain areas of communes such as Sainte Suzanne, Mont Organisé, Carice and Vallières, where crop yields were still below average as expenses dropped due to reduced economic resources with local farmers”. On the other hand, dry periods in the Upper Plateau, mainly in the communes of Thomassique, Cerca-la-Source and Cerca Carvajal, disrupted winter crops, resulting in poor or even non-existent harvests, especially for beans, which are one of the main cash crops produced at this time of year. According to the CNSA and FEWSNET surveys, very few farmers were able to do the winter plantings, indicating that below-average harvests are expected. FEWSNET predicts low yields due to reduced investment by reason of reduced economic resources4 . According to the October 2019 IPC analysis, focusing on the March to June 2020 period, 12% (1,203,000 people) of the total population will be in an emergency phase and 28% (2,898,000 people) in crisis, which represents about 40% of the population (4.10 million people) facing food insecurity. Both in the current and forecast period, the situation seems to be worse in rural areas with 38% and 42% of the population in phase 3, respectively, compared to 28% and 31% in urban areas. The drought led to a 12% drop in agricultural production in the Northwest, the Upper Artibonite, the Northern and Southeastern border regions, and the mountainous interior of the South and Nippes departments, which significantly reduced food access for many of Haiti’s poorest households. Many rural areas are particularly vulnerable due to a combination of risks related to natural disasters (drought, hurricanes, landslides) and current rates of food insecurity. This is the situational context that led World Vision to propose this Emergency Food Security Program (EFSP) to provide immediate access to food to IPC 3 communities and to help them recover from the residual effects of hurricane Irma. This 18-month program received funding from USAID and reached the Northeast and Central Plateau departments. Nearly 6,700 vulnerable households (33,500 people) were targeted with the goal of alleviating food insecurity in times of crisis due to reduced livelihood harvests. This program involved six communes in the Northeast (Fort-Liberté, Carice, Mombin Crochu, Sainte Suzanne, Ferrier, and Vallières) and two communes in the Central Plateau (Cerca-la-Source and Cerca Carvajal) that were in crisis and emergency situations (IPC3 and 4). Program activities included : ● Unconditional electronic voucher transfer (UVT) ● Conditional voucher transfer-Food Voucher for Work (VFW) ● Essential complementary activities: agriculture, livelihoods, savings for transformation (S4T) and nutrition. The present work was carried out as part of the program’s final evaluation, which took stock of the main findings. 4 CNSA/MARNDR bulletin 16_ footbasket_securite_alimentary-Jan-2018 12 2) Final Evaluation Objectives The overall objective of this final evaluation is to analyze the level of achievement of the goal, objectives, and outcomes of the EFSP program and how these were done. Specifically, the final evaluation considers the following objectives: ● Evaluate program’s achievements with respect to goal, objectives, outcomes and targets. ● Assess program’s effects on local markets and certain interest groups (women and men; youth population; boys and girls, etc.). ● Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program’s outcomes. ● Identify best practices, lessons learned, strengths and challenges in the program’s design, including the logical framework and implementation to achieve project’s achievements. ● Recommended strategies for other projects or new interventions. 3) Methodology To achieve the objectives of this final evaluation, a dual quantitative and qualitative approach was considered. This approach combined the collection of project-related secondary data and primary data among project partners and participants in the intervention communities of the North-East and Central departments. The evaluation questions mentioned in the ToRs are based on OECD criteria. The following tables clarify the approach for addressing the different questions according to the Relevance￾Effectiveness-Efficiency-Impact-Sustainability criteria: Criteria Evaluative Questions Data sources/Collection methods Relevance What are the opinions of stakeholders on the nature and quality of implementation? Under what circumstances and/or in what contexts would the program be replicable, or could it be scaled up? Were relevant MAST officials involved? Has the project implementation strategy been adjusted to take into account the realities on the ground? If so, in what way? Were program activities and outputs consistent with the expected impacts and effects? Was the program consistent with the needs and priorities of the most vulnerable and targeted PLWs? What lessons have been learned? -Quantitative survey of beneficiaries -Key interviews with project manager and/or project component managers -Key interviews with project’s key partners (MAST, AL, ...) 13 Efficiency Did the project have adequate and appropriate resources (human, financial and capital) for its implementation? If there was a lack/problem of resources/capacity, how was it resolved? Assess the communication structure in place and its effectiveness in supporting program implementation. Were quality control and accountability measures in place and consistently applied during the review, approval, funding disbursement, monitoring, and reporting phases? Do recipient comments indicate widespread instances where funds (vouchers) were taxed or stolen, or where receiving a voucher represented a protection risk? -Key interviews with project manager and/or project component managers -Key interviews with female and male beneficiaries -Consulting Project financial reports -Key interviews with project’s key partners (MAST, AL, ...) Effectiveness To what extent were the objectives achieved? What were the main factors that did or did not influence the achievement of the objectives? Did the M&E system provide appropriate and reliable quality information to measure the planned indicators? How effective was the project model in terms of design, relevance, management and accountability? How effective was the program in terms of implementation (coordination, cooperation, effectiveness, standardization)? To what extent were strategies used to carry out project activities? Have humanitarian standards been met and humanitarian principles complied with (SPHERE, HAPs, Codes of conduct)? What measures were taken to reduce the negative effects? -Quantitative survey of beneficiaries -Key Interviews with Project Manager and/or Project Component Managers/M&E Manager -Key interviews with project’s key partners of the project (MAST, AL,) -Consulting periodic M&E tools/reports Criteria Evaluative Questions Data sources/Collection methods Impact Did the project reach the expected number of beneficiaries and territorial coverage? To what extent has the project contributed to reducing beneficiaries’ vulnerability the level? Did the response reduce future vulnerabilities? What are the unintended positive and negative impacts of project implementation? To what extent do the interventions improve the condition of the affected communities? How satisfied are the communities with the response? Did the program require more time from the women? -Quantitative beneficiary surveys -Key interviews with Project Manager and/or Project Component Managers/M&E Manager -Key interviews with project’s key partners (MAST, AL, ...) -Consulting Beneficiary Monitoring Database 14 What do men and other household members (in-laws, etc.) think about women’s participation in the programs? What gender-specific issues were addressed? What positive changes are observed in the lives of the target group as a result of project implementation? Did the voucher project affect the market and context in any way (did the voucher assistance have an impact on inflation? Criteria Evaluative Questions Data sources/collection methods Sustainability To what extent will project benefits continue after donor funding ends? Are positive effects sustainable? To what extent did the project take into account factors that, in experience, have a major influence on sustainability such as economic, ecological, social and cultural aspects? What sustainability drivers are obvious (local ownership, partnership, transformed relationships, household and family resilience)? -Quantitative survey of beneficiaries -Key interviews with project manager and/or project component managers -Key interviews with key project partners (MAST, AL, ...) -Key interviews with beneficiaries Linkages, stratification and exit strategies To what extent did the project take advantage of other USG and non-USG investments in the same area to facilitate linkages with complementary services, overlaying previous investments and implementing exit strategies to minimize reliance on external support? Are there other non-USG investments on which the project has relied in its exit strategy to reduce reliance on external support? To what extent has the project aligned and integrated with the Haitian government’s recent national policy for social protection and promotion? -Consulting government policy document related to the issues addressed by the project -Key Interviews with Project Manager and/or Project Component Managers/M&E Manager -Key interviews with key project partners (MAST, AL, ...) -Key interviews with beneficiaries Quantitative survey A quantitative sampling survey was conducted among project beneficiaries in the Central Plateau intervention communes (Cerca-la-Source and Cerca Carvajal) and Northeast intervention communes 15 (Fort-Liberté, Carice, Mombin Crochu, Sainte Suzanne, Ferrier and Vallières). As mentioned in the ToRs, the same sampling methodology used for the baseline was applied. Thus, a multi-step Proportional Probability by Size (PPS) Cluster Sampling was used. Although beneficiary lists exist, which means according to the FFP protocol, simple random sampling in one step would apply here. However, for the purposes of comparing initial and final indicator values, the same sampling method was used. However, the interviewers had the lists as a reference for accessing households in the program intervention communities. The structured questionnaire used in the project baseline for the collection of quantitative data is used in this study with additions taking into account the evaluative questions based on OECD criteria. The questions take into account the calculation method for project’s key indicators as presented : Key indicators Definition of indicators Data collection method Percentage of targeted households with an acceptable Food Consumption Score (FCS) The frequency-weighted dietary diversity score is calculated from the consumption frequency for the various food groups consumed (Appendix 1) by a household during the 7 days preceding the survey. Quantitative survey of project beneficiary households Prevalence of households with little or no hunger (Household Hunger Scale - HHS) It is essentially a behavioral measure that tends to capture more serious behaviors such as: - Was there never anything to eat in your house? For lack of resources to get food? - Did you or any family member go to bed hungry because there wasn’t enough food? - Did you or any family member go a whole day and night without eating anything because there was not enough food (Appendix II)? Quantitative survey of project beneficiary households Proportion of households consuming at least 6 food groups in the previous month Household Dietary Diversity Scale (HDDS): Dietary diversity represents the number of different foods or food groups consumed in a given reference period. - Similar to the FCS, but usually with a 24-hour recall period with no information on frequency or weighted categorical thresholds - It is an indirect measure of household access to food. Quantitative survey of project beneficiary households Percentage of food use by type (household consumption, sale, exchange, livestock feed) This indicator takes into account food generally available in the household, regardless of its origin and quantity. It analyzes the different modes of use, namely: a. Self-consumption b. Sale c. Exchange d. Livestock feed The priority use of food can inform the economic situation of the household. Quantitative survey of project beneficiary households 16 Key indicators Definition of indicators Method of data collection Percentage of households where adults and children consume at least 2 meals a day (adults and children), by age The average number of meals consumed per day in a household is an indicator of the level of food security or food insecurity. The number of meals in food insecure households is lower than in food secure households. In fact, this indicator records the number of households where adults and children eat at least 2 meals a day. Quantitative survey of project beneficiary households Prevalence of mothers who exclusively breastfeed their infants six months of age or younger This indicator takes into account the proportion of mothers whose children aged six (6) months or less were exclusively breastfed. In the Quantitative Household Survey, a subgroup of questions will be asked of households with mothers who have infants 6 months of age or younger. Reduced Coping Strategies Index (rCSI) rCSI measures behavior: What people do when they don’t have access to enough food by answering the question: What do you do when you don’t have enough to eat and you don’t have enough money to buy food? - Measures the adjustments households make in consumption and livelihoods. These may be changes in consumption, reduced spending, or income growth; - rCSI tends to measure less severe coping behaviors. - rCSI uses the five most common strategies with standardized weights: 1- Focus on less preferred and less expensive foods? 2- Borrow food or rely on the help of a friend or relative? 3- Limit portion size at mealtime? 4- Restrict adult consumption so that young children can eat? 5- Reduce the number of meals consumed per day? (Appendix III) Quantitative household survey Percentage of households reporting greater ability to cope with economic shocks. This indicator takes into account households with information on shocks. The households most vulnerable to risks are those without information. The measurement of this indicator therefore has to do with two (2) essential elements: a. The identification of shocks (floods, wind, drought, plant disease, erosion, hurricanes and others) and the level of participation of each type of shock in food insecurity; b. Information received by households for managing shocks. Quantitative survey of project beneficiary households Prevalence of global acute malnutrition (GAM) To measure GAM, anthropometric measurements (weight-for-height or MUAC) of children aged 0-59 months will be taken to calculate wasting. All children with a weight-for-height Z-score of less than -2 standard deviation and/or edema are classified as having global acute malnutrition. Anthropometric data from quantitative surveys of beneficiaries with children aged 0-59 months. - Calculation of sample size The indicators are measured by department, i.e. for all the communes of a department taken together. Thus, the previously presented indicators will be established for the North- 17 East by considering the 6 project intervention communes and, for the Center, by considering the 2 intervention communes. - Calculation of sample size Since we use the same baseline methodology, i.e. a 2-step Proportional Probability of Size (PPS) Cluster Sampling, to determine the size of the beneficiary sample, the calculation formula is based on the FFP/USAID protocol in that the minimum required number of 678 beneficiaries per department to be surveyed is met for the food security indicators (FCS, HHS, rCSI). The formula for calculating the survey’s sample size is the following: 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = 𝐷𝐷𝑒𝑒𝑒𝑒 ⎣ ⎢ ⎢ ⎡𝑍𝑍1−∝ �2𝑃𝑃�1 − 𝑃𝑃� + 𝑍𝑍1−𝛽𝛽�𝑃𝑃1,𝑒𝑒𝑒𝑒 �1 − 𝑃𝑃1,𝑒𝑒𝑒𝑒 � + 𝑃𝑃2,𝑒𝑒𝑒𝑒 �1 − 𝑃𝑃2,𝑒𝑒𝑒𝑒 � 𝛿𝛿 ⎦ ⎥ ⎥ ⎤ 2 Where 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = is the initial sample size required by surveys for each one of the two-stage points 𝛿𝛿 = 𝑃𝑃1,𝑒𝑒𝑒𝑒 - 𝑃𝑃2,𝑒𝑒𝑒𝑒 = minimum effect size to be achieved over the period specified by both surveys; 𝑷𝑷𝟏𝟏,𝒆𝒆𝒆𝒆𝒆𝒆 = 𝟎𝟎. 𝟒𝟒𝟒𝟒 proportion of the population with an acceptable FCS baseline. FCS is chosen over the other 2 as the one that yielded the largest sample, although insufficient for the measurement of food insecurity indicators. 𝑃𝑃2,𝑒𝑒𝑒𝑒 = 0.69 Proportion of the population with an acceptable FCS in 2nd MDP achieved 𝑃𝑃1,𝑒𝑒𝑒𝑒 = represents a survey estimate of the true proportion of the P1 population at baseline 𝑃𝑃2,𝑒𝑒𝑒𝑒 = represents a survey estimate of the true proportion of the P2 population at the end of the survey. 𝑃𝑃 = 𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒+ 𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒 2 = 0.46+0.69 2 =0.575 𝑍𝑍1−∝ is the value of the normal probability distribution corresponding to a confidence level of 1-𝛽𝛽. For 1-𝛽𝛽 =0.95, the corresponding value is 𝑍𝑍0,95 = 1,64. 𝑍𝑍1−𝛽𝛽 is the value of the normal probability distribution corresponding to a confidence level of 1-.𝛽𝛽. For 1-𝛽𝛽 =0.80, the corresponding value is 𝑍𝑍0,80 = 0,84. 𝐷𝐷𝑒𝑒𝑒𝑒 The estimated Design Effect (DEFF) of the survey is equivalent to 2 here. Hence 18 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = 2 � 1.64 ∗ �2 ∗ 0.575(1 − 0.575) + 0.64 ∗ �0.46(1 − 0.46) + 0.69(1 − 0.69) 0.46 − 0.69 � 2 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = 123 There is indeed a non-response rate (generally set at 1.1, corresponding to 10%, but may change depending on the context). 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 123 ∗ 1.1 = 135 𝑏𝑏𝑏𝑏 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏. As with each department, to measure FCS, HHS, rCSI in this final evaluation, at least 678 units are required, so the sample size per department is adjusted to 678. Thus, overall, 1,356 beneficiary households in the project’s two intervention departments were to be surveyed in this final evaluation. Table 1: Breakdown of the number of beneficiary households to be surveyed by commune Departments Communes Number of households Demographic weight Sample allocation by commune Quantity of surveys carried out Center Cerca-La Source 1,847 63% 427 425 Cerca￾Carvajal 1,086 37% 251 255 Total 2,933 100% 678 680 Northeast Fort-Liberté 229 8% 52 96 Ferrier 506 17% 115 121 Carice 251 8% 57 51 Mombin Crochu 783 26% 178 121 Sainte Suzanne 705 24% 161 178 Vallières 503 17% 115 111 Total 2,977 100% 678 678 Grand Total 5,910 1,356 1,358 For the indicator on the prevalence of mothers who exclusively breastfeed infants six months of age or younger, an attempt was made to identify from the beneficiary list the number of households with mothers who breastfeed children from 0 to 6 months of age. This helped us extract a random sub-sample to have enough sample units for a more valid measurement of this indicator. In the first PDM carried out in August by the project in the 2 departments, the findings showed a difference between the communes because, in Cerca-Carvajal, 14% of households have a pregnant woman, 13% for the commune of Carice, 12% for Vallières, 9% for Cerca-la-Source, 5% for Ferrier, 4% for Mombin Crochu, 3% for Fort-Liberté and 2% for Sainte-Suzanne. Assuming these pregnant women have had live births in the meantime, we estimate the following sub-population of breastfeeding women per commune: 19 Table 2: Estimated sub-population of breastfeeding women beneficiaries by commune Departments Communes Number of households % of households with nursing women Sub￾population of households with nursing women Center Cerca-la-Source 1,847 9% 61 Cerca-Carvajal 1,086 14% 95 Total 2,933 156 Northeast Fort-Liberté 229 3% 20 Ferrier 506 5% 34 Carice 251 13% 88 Mombin Crochu 783 4% 27 Sainte Suzanne 705 2% 14 Vallières 503 12% 81 Total 2,977 264 Grand Total 5,910 420 Considering the number of beneficiaries to be surveyed in the Center department in the 2 communes (427 for Cerca-la-Source and 251 for Cerca-Carvajal, respectively), it was possible to exhaustively survey the estimated number of nursing women within the sample of beneficiaries to be surveyed per commune in this department. However, for the North-East department, the sub-population of nursing women in some communes exceeded the sample of beneficiaries to be surveyed (for example, 57 beneficiaries would be interviewed in Carice while the sub-population of nursing women was estimated at 88 beneficiaries). For that purpose, we determined a sample of nursing women to be surveyed in the Northeast. Hence, with a margin of error set at 7% and the proportion of women practicing exclusive breastfeeding in the Northeast estimated at 54% (project baseline), we determined a number of n=123 breastfeeding women beneficiaries. Hence the breakdown by commune becomes: Table 3: Breakdown of the sample of breastfeeding women to be surveyed by commune in the Northeast Departments Communes Sub-population of households with nursing women Demographic weight Sample allocation Quantity of surveys carried out Center Cerca-la-Source 61 - 61 319 Cerca-Carvajal 95 - 95 148 Total 156 156 467 Northeast Fort-Liberté 20 8% 9 60 Ferrier 34 13% 16 51 Carice 88 33% 41 35 Mombin Crochu 27 10% 13 81 Sainte Suzanne 14 5% 6 53 20 Vallières 81 31% 38 60 Total 264 100% 123 340 We noted that more nursing women were interviewed than the sub-population estimated from the PDM data. This may be explained by the fact that new nursing mothers were added to the list. GAM indicator To measure the GAM (Global Acute Malnutrition Prevalence) indicator, one of the 3 approaches proposed by the FFP protocol is used (approach B). We looked at the official statistics in each of the 2 departments on the proportion of the population of children under 5 years of age: 0.10 in the Northeast and 0.11 in the Center. Data from the PDM carried out in the project confirm an average size of 6 persons per household in the 2 departments. Using these parameters for each of the departments in the USAID sample size calculator5 , we have the respective sizes of children under 5 to consider for the GAM indicator measurement: - n (Northeast) = 1,270 children 0-59 months old - n (Centre) = 1,173 children 0-59 months old These samples of children under 5 years of age were sought within the surveyed household samples. It should be noted that within the same household, more than one child under 5 years of age could be considered. Therefore, the child samples were therefore broken down according to the demographic weight of households by commune. Hence the following table: Table 4: Breakdown of Recipients Sample with Children 0-59 Months of Age to be Surveyed Departments Communes Demographic weight of households Sample allocation of children to be considered by commune Quantity of surveys carried out Center Cerca-la-Source 63% 739 331 Cerca-Carvajal 37% 434 169 Total 100% 1,173 500 Northeast Fort-Liberté 8% 98 63 Ferrier 17% 216 54 Carice 8% 107 39 Mombin Crochu 26% 334 82 Sainte Suzanne 24% 301 73 Vallières 17% 214 61 Total 100% 1,270 372 Grand Total 2,443 872 5 https://www.usaid.gov/documents/1866/population-based-survey-sample-size-calculator 21 More than 50% of children could not be observed in the two departments during the interviewers’ visits for the calculation of GAM. This is therefore a considerable limitation to this study (see details in the Constraints and Limitations of the Study section). In order to quickly get quantitative data, the Ona digital survey platform was used. The survey questionnaire is designed using the xlsform syntax loaded on the platform. Each interviewer was therefore provided with a digital tablet on which the data collection tool was installed. It was therefore possible to follow, in real time, the entire collection process from the Ona platform. The interviewers were also required to enter the GPS coordinates of the locations visited; this reduced the risk of an interviewer filling out the tool him/herself without actually visiting the beneficiary household. Qualitative survey The qualitative approach of the final evaluation is essential not only to complement the quantitative data collected from project beneficiaries, but also to validate the study’s findings through the triangulation of information. It is also important to help better identify the changes induced by project interventions. Thus, individual interviews were addressed to project key actors: Table 5: Number of Key Interviews by Department Key Interviews Center Northeast Total Project team - Agriculture component - - 1 Project team - Food vouchers component - - 1 Project team - VSLA/S4T component - - 1 Project team - M&E component - 1 1 Project team – Manager Local Authorities (CASECs-Mayors) - Community Leaders 7 6 13 Departmental Office – MARNDR 1 1 2 Departmental Office – MAST 2 1 3 CEPAM Manager 1 1 Sellers 6 7 13 Beneficiaries 29 42 71 Other organizations in charge - - 0 Since COVID-19 is still present in Haiti, focus groups with project stakeholders were avoided to prevent the spread of the disease. For this reason, key informant interviews were advocated instead of focus group meetings. 22 Assessing the effects of the program on markets An analysis of the program’s impact on the markets was carried out in a qualitative manner, through interviews with the project’s main wholesalers and retailers, and with beneficiaries through separate key interviews (with men and women as their perceptions may be different). The key issues that were discussed were the following: • Has the market experienced significant variations (in availability, demand, prices, players present) since the program’s inception? And if so, what are the causes? How has the program responded to seasonal price fluctuations, volatility and possible differences? • Have challenges/constraints/distortions that could be attributable to the program been identified? Has information about the program been properly communicated? What market effects/impacts persist beyond the program’s closure? • Is the impact of the program on traders noticeable? Has it helped boost business activities and strengthen the capacity of traders to develop their business? What changes, if any, in marketing strategies/practices were observed among consumers and traders before and after the program? What are the main lessons learned from this program and what are the recommendations for the continuation of similar activities in the context surveyed? Evaluation of the effectiveness and relevance of the transfer modalities The team conducted a qualitative analysis of the effectiveness of transfer modalities that helped the project achieve or not achieve expected objectives. Discussions were conducted with program beneficiaries, stakeholders, the implementation team, wholesalers and retailers. This evaluation considered the following pillars: 1) Relevance: To what extent does the transfer modality used meet beneficiary needs while being gender-sensitive and based on a gender analysis? 2) Effectiveness: To what extent were the project outcomes affected by the chosen transfer modality? 3) Efficiency: Was the transfer modality cost-efficient and implemented in a timely manner? What is the cost-effectiveness of the respective transfer modalities compared to other alternatives? 4) Constraints and Limitations of the Study The data collection operations took place in a difficult context where interviewers had to work under heavy rains. This particularly delayed data collection activities in the Center department. Interviewers had to end working days prematurely because of the long rains. The methodology of the quantitative survey indicated the measurement of the weight and height of children aged 0-59 months to calculate the malnutrition indicator (prevalence of 23 global acute malnutrition (GAM)). This required a heavy logistical burden where interviewers had to carry a scale and a height gauge. In the consultation proposal, we were counting on World Vision and health centers in the intervention communes to support us with these tools, given the difficulty of getting them easily on the market, for the sole use of a survey. Although we were able to access the required number of height gauges for interviewers, this was not the case for scales. This had a negative impact on the survey, especially on the number of children to be observed for the sample calculated for this indicator. Therefore, one scale was used per commune in the North-East department. In the Center, the leadership of our field supervisors helped us borrow some scales from health centers. It was also not possible to identify in the sampling frame, i.e. the lists of beneficiary households, all those with children aged 0-59 months. This also explained this limitation of the study in terms of reaching the children sampled to be considered by department to calculate the GAM. Because the project financial report was not available, the efficiency analysis did not touch on aspects related to the percentage of the budget actually spent versus what was planned. 5) Analysis of the Final Evaluation’s Findings In this section, the findings of the final evaluation are analyzed by first highlighting the general characteristics of the beneficiary households, then, OECD criteria (Relevance-Efficacy￾Efficiency-Impact-Sustainability), links, and exit strategies by answering evaluation questions. 5.1 General Characteristics of Beneficiary Households It is essential to characterize the universe of the study’s surveyed beneficiary households. In order to get an idea of the structure of beneficiary households in the project intervention departments, an age pyramid was designed for each one of them (Figures 1 to 2). The visualization of both graphs reveals almost the same structure for the beneficiary population in the two departments, i.e., very broad at the bottom and very tapered at the top. This is therefore an evidence of two young populations, which then reflects the same situation of the Haitian population in general. The most represented age groups are the 5-19 year old. In both departments, there tends to be a stronger presence of the female population. This reflects the reality of the Haitian population, which has more women than men. 24 Graph 1 : Age pyramid for the intervention communes of the EFSP Center project Graph 2 : Age pyramid for the intervention communes of the EFSP North-East project There is a huge difference between the 2 departments in terms of the gender of respondents to the quantitative survey. Indeed, most respondents in the Northeast are women (71%) while this is the reverse in the Central department, where more than 50% of respondents are men (Graph 1). This difference shows slightly at the household status level where, in the Northeast, there are more female-headed households (FNM), i.e. 26% (Graph 2). Adult female- and male-headed households (F&M) are dominant in this survey, 93% and 64% respectively in the Center and Northeast. Graph 1: Distribution of Respondents by Department by Gender (n=1,258) Graph 2: Distribution of Respondents by Department by Household Status (n=1,258) 15.00% 10.00% 5.00% 0.00% 5.00% 10.00% 15.00% 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85+ F H 10.00% 5.00% 0.00% 5.00% 10.00% 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85+ F H 42% 71% 58% 29% Center Northeast Woman Man 4% 26% 3% 9% 93% 64% Center Northeast Adult female and non-adult male (FNM) Male adult Female non adult (MNF) Adult men and women(F&M) Non-adult child led household (CNA) 25 Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. If we now analyze the education level of project beneficiaries, we identify the same trend in the 2 departments, namely a predominance of “no education” and “incomplete primary education”, for between 30% to 50% of respondents (Graph 3). This corresponds to this known reality in Haiti where a large portion of the population does not always have access to education. It is often this population group that is abandoned in the provinces, especially in rural areas. Today, these rural areas still lack, both in quantity and quality, educational institutions that should help people develop their potential. It can even be argued, from this reality, that this somehow confirms these respondents come from the most vulnerable groups in their communities, with such a low overall level of education. Generally, the lack of economic resources is the reason why parents are unable to provide their children with education. This is a still ongoing situation in Haiti, particularly in rural areas. Graph 3: Distribution of heads of household by department according to their education level (n=1,258) Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. When considering marital status in analyzing the characteristics of project beneficiary households, cohabitation and marriage are the most represented in the 2 departments. This does not contradict the trend in marital status we see at the national level. 24% of respondents with a widow(er) status is a large proportion among beneficiaries in the Northeast (Graph 6). And we see the consistency with the presence of orphans in households in the Northeast, which has slightly more orphans compared to the Central department (Graph 7). 32% 43% 11% 9% 48% 31% 8% 13% 2% 1% Center Northeast Incomplete academic Full academic Incomplete secondary Secondary incomplete Professional Incomplete Primary Full Primary None 26 Graph 4: Distribution of heads of household by department according to marital status (n=1,258) Graph 5: Distribution of respondents by department according to whether or not there is at least one orphan in the household Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. We found virtually no household with pregnant women (Graph 8). 10% or less respondents only mentioned it in the Northeast or Center. This is also the case for the presence of persons with disabilities in beneficiary households, where only less than 15% of respondents reported this, in both departments (Graph 9). This must have been one of the vulnerability criteria for the selection of beneficiaries. This aspect is further analyzed in the evaluation criteria section. Graph 6: Distribution of respondents by department according to whether or not there is a pregnant woman in the household Graph 7: Distribution of respondents by department according to whether or not there is at least one person with a disability in the household Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. 2% 7% 47% 31% 1% 5% 41% 34% 9% 24% Center Northeast Widow(er) Married Divorced/Separ ated Concubinage Single 86% 80% 14% 20% Center Northeast No Yes 89% 92% 11% 8% Center Northeast No Yes 90% 86% 10% 14% Center Northeast No Yes 27 The majority of the households benefiting from the project live in tin-roof housing (Table 6) (38% in the Center and 73% in the Northeast) and simple low-rise houses (33% in the Center and 6% in the Northeast). The type of housing is a good indicator of the household’s level of vulnerability. It is evident here that we do not see the worst types of housing such as cottages and hovels/wooden huts. Table 6: Distribution of Respondents by Department by Housing Type (n = 1358) Type of habitat Center Northeast Grand Total Hovels 4% 2% 3% Other (wooden with tin roof, wooden with mud, ...) 1% 4% 2% Two-story house 0% 1% 1% Simple low-rise house 33% 6% 19% Masonry house 0% 1% 1% Mud house 14% 5% 10% Tin-roof house 38% 73% 56% Hut/Wooden hut 9% 7% 8% Grand Total 100% 100% 100% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Agriculture and trade are the main sources of income for most rural households in Haiti (FEWS NET6 , March 2015). The households benefiting from this World Vision program fall into these same livelihood categories. 85% and 61% of respondents, respectively in the Center and Northeast, reported agriculture as their main activity (Table 7). Trade is the second most important livelihood activity, although in relatively small proportion compared to agriculture. Table 7 : Distribution of respondents by department according to the head of household’s main activity Main activity Department Center Northeast Grand Total Agriculture 85% 61% 73% None 4% 17% 11% Charcoal (production) 0% 4% 2% Trade 7% 11% 9% Breeding 1% 0% 1% Mason 1% 1% 1% Teacher 0% 1% 1% Transportation 1% 1% 1% Other profession (ironwork, cabinet making...) 1% 4% 2% Grand Total 100% 100% 100% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. As a second activity, in addition to trade and agriculture, there is animal breeding, which appears to be very present as a means of livelihood in the program intervention areas (Table 8). This activity has long been considered the “piggy bank” of rural households. 6 Haiti, Rural Livelihoods Profile, FEWS NET, March 2015. 28 Table 7: Distribution of respondents by department according to the second activity of the head of household Second activity Department Center Northeast Grand Total Agriculture 11% 12% 12% No second activity 39% 63% 51% Other profession 2% 2% 2% Charcoal (production) 4% 7% 5% Trade 12% 8% 10% Breeding 28% 6% 17% Mason 1% 1% 1% Professor 1% 1% 1% Transportation 1% 0% 1% Grand Total 100% 100% 100% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. The previously presented economic activities provide monthly income brackets from 1,500 to 5,500 HTG. In the Rural Livelihoods Profile study cited above, the annual income of poor rural households is estimated at 50,000 HTG, or about 4,167 HTG monthly (FEWS NET7 , March 2015). But households in the Northeast show lower income brackets that come close to being unrealistic, less than 1,500 HTG (Graph 10). Given that beneficiary households are supposed to be among the most vulnerable in the project’s intervention areas, this may explain these lower income brackets. Graph 8: Distribution of respondents by department by monthly household income group Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. The trend in expenditure items for beneficiary households is the same in both departments. Three main expenditure items emerged for beneficiary households: food, education (school fees) and health care (Graphs 11 and 12). This is, in fact, the same reality at the national level where the State is unable to protect households from these recurring costs related to 7 Haiti, Rural Livelihoods Profile, FEWS NET, March 2015. 33% 42% 40% 29% 19% 12% 2% 2% 1% 3% 3% 8% 1% 5% Center Northeast More than 11500 Gdes 9500 - 11500 Gdes 7500 - 9500 Gdes 5500 - 7500 Gdes 3500 - 5500 Gdes 1500 - 3500 Gdes 0 - 1500 Gdes 29 household size. Poor households are inclined to spend their meager economic resources on food. Graph 9: Distribution of respondents by department by first annual household expenditure item Graph 12: Distribution of respondents by department by second annual household expenditure item Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. 5.2 Analysis of Project Relevance In this section, the project is analyzed based on the various evaluation questions related to the relevance criterion. The overall idea of this criterion is to see the extent to which the project’s objectives have remained valid while considering the needs or priorities of target communities. In other words, it was essential to look at whether activities and outputs were consistent with expected impacts and effects. The broad lines of planned activities for the project were linked to these five main components, namely : - Distribution of Food Vouchers - Nutrition - Drinking Water Infrastructure - Agriculture - Financial Support (Training and S4T) a) Distribution of Food Vouchers The interview with the project team revealed that while implementing the Distribution of Food Stamps component, everything was set up to reach the most vulnerable. Thus, beneficiaries were selected according to a participatory process, using criteria identified by the communities themselves, namely: • Children under five years of age suffering from malnutrition. • Households with physical disabilities. • Pregnant and lactating women. 1% 2% 0% 9% 9% 0% 1% 87% 85% 3% 3% 0% 0% Center Northeast Other (energy,...) Health Care Food Rent Tuition fees Toiletries and laundry items Agriculture 11% 3% 3% 3% 60% 54% 0% 2% 10% 13% 13% 21% 1% 1% 2% 3% 0% 1% Center Northeast Other (home repairs, etc.) No other activity Clothing Health Care Food Rent Tuition fees Toiletries and laundry items Agriculture 30 • Households headed by elderly people. • Households caring for orphaned children. • Households headed by a woman. • Households that are hit and struggling until exhausted, following hurricane Irma. It was not clarified whether the project had conducted a needs assessment prior to evaluating the voucher value. However, the implementation team made monthly adjustments to vouchers, in line with the increase in food prices in local markets, due to the depreciation of the Gourde and rising inflation. Women heading single-parent households have been one of the project’s and the food voucher component’s main targets to ensure these vulnerable households have access to a reliable food source. b) Nutrition The Nutrition component was implemented by CEPAM (Center for the Promotion of Breastfeeding), a World Vision partner. The interview conducted with a representative from this CEPAM revealed that the identified geographic areas of implementation are those with the highest proportions of vulnerable people affected by hurricane Irma. However, CEPAM states that the WV team conducted local discussions to select the nutrition theme since it was noted there was limited knowledge about good nutritional practices within communities and more specifically with pregnant and lactating women and for cases of malnutrition. Interventions under this component are in line with the National Nutrition Policy (NNP) in the sense that they aim to improve the nutritional situation of communities based on a preventive approach to all forms of malnutrition through the promotion of healthy, nutritious and adequate food (MSPP8 ). According to CEPAM, the communities were well informed of the activities and interventions on nutrition, the mother leader strategy, the expected deliverables and expected behaviors. Indeed, the quantitative survey data confirm that beneficiary households in both intervention departments indicated they received messages related to newborn care from at least 90% of respondents (Graph 13). In addition, between 42% and 47% of respondents (women) said they attended trainings conducted by mother-leaders in their community (Graph 14). 8 https://scalingupnutrition.org/wp-content/uploads/2013/06/Haiti_Plan-Strategique-Nutrition-2013-2018.pdf 31 Graph 10: Distribution of beneficiary households by department according to whether or not they received messages about newborn care in the health centers (n=618) Graph 11: Distribution of beneficiary households according to whether they participate in training provided by mother leaders in the community Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. c) Drinking Water and Sanitation Infrastructure In this component, the project’s objective was to increase or restore access to drinking water for communities whose water systems were damaged after the passage of hurricane Irma. Thus, the quantitative survey showed a much better situation in the Center compared to the Northeast in terms of the percentage of households with access to an improved water source (private connection, private kiosk, public water point, etc.). Nearly 35% of beneficiary households in the Northeast use uncollected water sources (Table 9). Table 8: Main source of water for household domestic use Household water source Department Center Northeast Grand Total Private connection 3% 1% 2% Private water sales kiosk 0% 2% 1% Public water point (DINEPA) 62% 25% 44% Public pump 4% 31% 17% Uncollected water source 29% 35% 32% Impluvium/Rainwater 0% 0,3% 0% Protected wells 1% 2% 2% Unprotected wells 0% 2% 1% Truck/Tanker (Untreated Water) 0% 0,1% 0% River 0% 1% 1% Grand Total 100% 100% 100% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020 According to DINEPA (Direction Nationale d’Eau Potable et Assainissement) standards, access to drinking water also takes into account the amount of water available per person per 10% 9% 90% 91% Center Northeast Yes No 58% 53% 42% 47% Center Northeast Yes No 32 day and the time taken to travel to and from distribution points. Thus, a person is said to have access to a Potable Water Supply/Water Point (PWS/WP) in the sense that he or she can access it in less than 30 minutes round-trip. The system provides at least 15 liters of water per person per day at water points, standpipes and kiosks, and at least 25 liters at private connections. To get water volumes in gallons, simply multiply the liter amount by 0.264 gallons. So, the quantitative data shows that in the Central and Northeast regions, respectively 57% and 45% (Graph 15) of respondents indicate they have between 10 and 20 gallons of water available per day for use. In addition, more than 50% of respondents in the two departments only take 5 to 15 minutes (Graph 16) for the round trip when collecting water. Graph 12: Distribution of beneficiary households by department according to the amount of water available for use per day. Graph 13: Distribution of beneficiary households by department according to the walking time (round trip) for water collection Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. To ensure the sustainability of water service, the project should also work to strengthen the management and maintenance of water points in the communities. The quantitative survey revealed that very few beneficiary households are aware of the creation of water committees (Table 10). Table 9: On the management and maintenance of water points Center Northeast Number of people % Number of people % Aware of the creation of water committees 194 29% 59 9% 24% 35% 57% 45% 19% 21% Center Northeast More than 10 gallons Between 10 and 20 gallons Less than 10 gallons 53% 52% 19% 26% 13% 11% 8% 7% 7% 3% Center Northeast More than 1 hour 45 mns to 1 hr 30 to 45 mns 15 to 30 mns 5 to 15 mn 33 Member of the water committee? 23 3% 7 1% Participating in pump repair activities 36 5% 20 3% Member of the technician groups 7 1% 13 2% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. This component also sought to strengthen the hygiene conditions of the project’s target households by working primarily on home water treatment but also on handwashing. Indeed, in the quantitative survey for this final evaluation, the majority of respondents stated they treat water at home (59% in the Northeast and 71% in the Center (Graph 17)). Those who don’t usually treat water at home say they don’t have water treatment products. In the Northeast and Center, more than half say they already have clean water (Graph 18). Graph 14: Distribution of beneficiary households by department according to whether or not they usually treat water at home Graph 15: Distribution of beneficiary households by department according to the fact that they do not usually treat water at home Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Chlorine and a derivative of this product, Aquatabs, are widely used as home water treatment methods both in the Northeast and Center (Graph 19). In the Center, they use another home water treatment product that is manufactured in Haiti. It should be verified whether there is a production unit for this product in this department; this would ensure its availability in a sustainable way for households. 29% 41% 71% 59% Center Northeast Yes No 51% 9% 47% 77% 1% 9% Center Northeast Clean water Does not know how to treat water No water treatment products Products or processing equipment too expensive Other( discomfort with treated water...) 34 Graph 16: Distribution of beneficiary households by department according to the water treatment method used Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Treating water at home is one thing, but storing to keep it clean is, in many cases, a real challenge for rural households. In this study, the water storage containers reported by beneficiary households were a covered bucket and a gallon in both departments (Table 11). Table 10: Distribution of beneficiary households by department according to the water conservation method used at home Water storage container Center Northeast Number % Number % Covered Drum 101 15% 56 8% Non-Covered Drum 53 8% 15 2% Covered Bucket 585 86% 493 73% Non-Covered Bucket 131 19% 109 16% Gallon 555 82% 527 78% Bowl 47 7% 24 4% Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Under the COVID-19, more than 80 percent of respondents to the quantitative survey reported receiving handwashing messages (Graph 20). The project took advantage of the need brought about by the pandemic to get handwashing messages across. Interviews with local leaders acknowledged that hygiene training and awareness raising have increased community understanding of the essential concepts of good health practices related to water storage and treatment, the use of latrines to reduce the spread of cholera or eradicate it, and handwashing, which helped better appreciate and apply these principles under COVID. 27% 5% 36% 54% 24% 22% 16% 2% 10% Center Northeast Jif/Aquajif Reverse Osmosis Snowshoe Filter Aquatab Boiling water SODIS (in the sun) Chlorine grain Lemon Gadyen dlo 35 Graph 17: Distribution of beneficiary households by department according to whether or not they received handwashing messages Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. d) Agriculture In this project’s agricultural component, training and technical support on best land conservation practices were provided to smallholder beneficiaries in target communities. For the project implementation team, these best practices corresponded to the needs of farmers as the farms in project areas are vulnerable to climate shocks and changes. According to the National Adaptation Action Plan (NAAP), agricultural production in the Center and Northeast is particularly vulnerable to climate shocks, mainly drought, high winds and adverse rainfall, given the significant increase in land aridity in the area. The weather changes, with regard to the seasons’ characteristics and manifestation period, disrupt farmers in their monitoring methodology for better production-related decision-making and make the agricultural system even more vulnerable. Crops are lost and create a deficit in the communities’ ability to meet the food needs of its people (NAAP9 ). Farmers have also received post-harvest management training because, without adequate large-scale storage infrastructure and conditions for the conservation of agricultural commodities, they periodically experience losses ranging from 30% for cereal crops to 50% for fruits and vegetables (University of Florida10). Considering that the farms are small estates of less than 1 hectare, these losses are huge and affect the farmers’ economic life. Therefore, WV, by facilitating these trainings, seed distribution and post-harvest support, supports farmers in adapting their production to recurring climatic hazards. Indeed, the respondents (beneficiaries) stressed the importance of the technical training received to tackle the water deficit affecting, in 2019, the departments of Northeast, North and Northwest (FAO, 202011). 9 https://unfccc.int/resource/docs/napa/hti01f.pdf 10 http://blogs.ifas.ufl.edu/area/files/2018/05/fr-factsheet-postharvest-loss-04-25-18-final.pdf 11 http://www.fao.org/fileadmin/user_upload/FAO-countries/Haiti/docs/Apercu_de_la_réponse_de_la_FAO.pdf 5% 12% 95% 88% Center Northeast Yes No 36 e) Financial support (training and S4T) In this component, the project sought to strengthen community savings groups, which are solidarity-based financial tools based on trust between the people from the intervention localities. According to the program team, the actions related to this component were in line with the needs and priorities of the beneficiaries who were part of the savings groups, meaning that they were able to save part of their meager income among themselves (as we saw earlier, almost all of which is spent on food), since they received a monthly food voucher as part of the food voucher component. The credit aspect worked well in the operation of community savings groups, according to the project team. Beneficiaries were able to either set up a small trade or strengthen an existing business, breaking away from the local system of usurious loans (called ponya in Creole, meaning “stab”) charging monthly interests of 25-50%. This, in many cases, does not offer a significant profit margin to the lenders. On the contrary, within savings groups, interest rates are set by the group through consultation and hardly exceed 5% a month, allowing more flexibility, less pressure and a double return on investment for members. In addition, according to beneficiaries, due to the lack of banking services available nearby, these groups help them save and have their money available in a timely manner for activities ranging from agricultural production, livestock breeding, payment of school fees and care for the sick. But, with the arrival of the COVID-19 pandemic, the process stopped as groups could no longer meet. Overall, one can conclude that the program was relevant, since it was a direct response to support the communities affected by Hurricane Irma, which left tens of thousands of families suffering, mainly in the larger North, and caused partial flooding by runoff in twenty-two (22) communes of the North, North-East, North-West, Center, Artibonite and West departments, bringing about floods in major rivers or marine submersion (Haiti en marche) 12. This is what community leaders and local authorities admitted in interviews conducted with them in the various communes where the project is operating. However, they are not very clear on the process to identify the project’s intervention pillars, since they were not contacted by the program team nor did they take part in participatory needs assessment exercises. However, they emphasized that program interventions and pillars responded to the needs of the vulnerable and Irma-affected populations. Overall, WVI did a good job communicating program’s objectives and implementation strategies with leaders and communities during the start-up phase through meetings, field agents and officers in various public spaces such as churches and local organizations’ premises. Community representatives and local elected officials took part in these start-up meetings, but in the Northeast (specifically in Fort-Liberté and Mombin-Crochu) leaders are advocating for greater inclusion of local leaders and the population. There were not enough information￾sharing meetings conducted during the project’s implementation in the two departments, contrary to the plans, and this considerably limited local actors’ understanding of the project and, specifically, they were unable to share their advice, grievances and recommendations together. 12 https://www.haitienmarche.com/index.php?option=com_content&view=article&id=2001:bilan-encore-partiel-des-degats￾causes-par-le-par-irma-en-haiti&catid=18&Itemid=268 37 According to community and local leaders, the project team did not contact or liaise with other organizations that conducted similar projects in the intervention areas. They even took as an example a CARE-implemented response project to hurricane Irma. This limited the WVI team’s ability to avoid duplication and overlaps, while capitalizing on lessons learned. Thus, during project implementation, one of the coordination-related guiding principles was not applied by WV. According to the OECD, coordination within the overall humanitarian cash response is essential to avoid aid duplication or gaps, ensure information sharing and standardize approaches. Coordination must also happen among all stakeholders, including local and national actors and governments13. Clearly established criteria allowed for the participation of different vulnerable groups in the affected communities, according to the leaders interviewed in Cerca-la-Source. Some stressed the fact that many people with disabilities were not able to participate in the project. They are not clear why. No case of payment for participation in the project was noted by local leaders. Data from the quantitative survey show satisfaction levels ranging from satisfied to very satisfied for operational components such as: agriculture and seed voucher, food voucher, community work in exchange for food vouchers, mother-leader nutrition and Savings and Credit Group (S4T) (Table 12). Table 11: Level of satisfaction by activity in which respondent took part Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. In order to score compliance with this criterion, following the previous analysis, we consider for each criterion this satisfaction scale: 1=Not satisfied at all 2=Not satisfied 3=Moderately satisfied 4=Satisfied 5=Very satisfied. Thus, for relevance, the project got a score of 4: satisfied for this criterion. 5.3 Analysis of Project Effectiveness Analyzing the effectiveness of a project during its evaluation helps see the extent to which project objectives were achieved. For that purpose, impact and effect indicators were analyzed 13 https://www.oecd.org/development/humanitarian-donors/docs/cashbasedresponse.pdf Project activities Number More or less satisfied Satisfied Very satisfied Agriculture and seed voucher Center 56 2% 57% 41% Northeast 103 0% 23% 77% Food voucher Center 298 1% 45% 54% Northeast 423 2% 48% 49% Community work in exchange for a food voucher Center 436 0% 60% 40% Northeast 263 0% 40% 60% Mother Leader-Nutrition Center 10 0% 70% 30% Northeast 57 5% 42% 53% Savings and Credit Group (VSLA/S4T) Center 17 0% 47% 53% Northeast 7 0% 57% 43% 38 through the quantitative survey conducted among beneficiary households, namely: - Household Dietary Diversity Score (HDDS), - Household Food consumption score (FCS), - Household Hunger scale, - Reduced Coping Strategies Index (rSSr/rCSI) - Percentage of households where adults and children eat at least 2 meals a day (adults and children), by age, - Percentage of food utilization by type, - Percentage of households reporting greater ability to cope with economic shocks - Prevalence of mothers who exclusively breastfeed infants six months of age or younger, - Prevalence of global acute malnutrition (GAM) a) Household Dietary Diversity Score Household dietary diversity for this assessment is calculated from the number of different food groups consumed (nine groups in total) by households in project intervention area over the four weeks prior to the survey. According to survey results, over the whole project intervention area (the two departments), nearly 88% of surveyed households are in the high dietary diversity, i.e. 1,193 out of 1,358 households surveyed for 6 or more food groups during the 4 weeks prior to this study. Only 1.6% of them have a low or poorly diversified diet. At the departmental level, the findings related to dietary diversity for surveyed households in the Center are more satisfactory than those in the Northeast. Indeed, in the Center, more than 90% of surveyed households have a diversified diet compared to 84% in the Northeast. However, it should be noted that the percentage of households with average dietary diversity in the Northeast is twice as high as in the Center. In both departments, the fat-based food group is the most consumed, followed by the cereal-based food group, followed by the beverage food group. Table 12: Dietary Diversity Score by Department Indicator: Dietary Diversity Center Northeast All Departments Num14 Den % Num Den % Num Den % Low dietary diversity 5 680 0.70% 17 678 2.5% 22 1358 1.6% Average dietary diversity 48 680 7.1% 95 678 14.0% 143 1358 10.5% High dietary diversity 627 680 92.2% 566 678 83.5% 1193 1358 87.9% Average Dietary Diversity Score 8 7 8 Standard Dietary Diversity Deviation 1.6 1.62 1.60 14 Num=Numerator Den=Denominator 39 Confidence Interval at 95% of the Average [7.88; 8.12] [6.88; 7.12] [7.91;8.09 ] Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. b) Household Food Consumption Score (HCS/FCS) For this study, the household food consumption score takes into account dietary diversity, frequency of food consumption and the relative nutrient intake of different food groups. It is calculated based on the consumption frequency for the different food groups consumed by a household during the 7 days prior to the study. Taking into account the threshold set to interpret the results for this indicator, over the whole project intervention and among the 1,358 surveyed households, 195 or 14.3% of them have a low food consumption, 381 or 28.1% of them have a borderline-acceptable food consumption, and 782 or about 58% of them have an acceptable food consumption. As observed in analyzing the dietary diversity score, the same picture emerges at the departmental level for the household food consumption score. The Center, once again, shows better results than the Northeast. Nearly one-third of surveyed households in the Center have food consumption that is either poor or borderline acceptable, while, in the Northeast, only half of them are in this situation. Table 13: Food consumption score for project beneficiaries by department Indicator: Dietary Diversity Center Northeast All Departments Num15 Den % Num Den % Num Den % Poor food consumption (Percentage of households with FCS of 0-21) 44 680 6.5% 151 678 22.3% 195 1358 14.3% Borderline Food consumption (Percentage of households with FCS of 21.5-35) 183 680 26.9% 198 678 29.2% 381 1358 28.1% Acceptable food consumption (Percentage of households with SCC> 35) 453 680 66.6% 329 678 48.5% 782 1358 57.6% Average Food Consumption Score 43.68 35.82 38.76 Food consumption standard deviation 17.19 16.82 16.96 Confidence interval at 95% of the average [42.39 ; 44.97] [34.55 ; 37.09] [37.86 ; 39.66 ] Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. Situation of beneficiary households in relation to hunger To analyze the situation of households in relation to hunger, three indicators were selected. These are mainly the Household Hunger Index (HHS), survival (coping) strategies-rCSI and whether the household uses their production or not. 15 Num=Numerator Den=Denominator 40 c) Hunger Scale Score/Household Hunger Index (HHI/HHS) This indicator is used to provide information on an important dimension of food security, namely access to sufficient food in the household. Thus, according to the findings of this study, 80% of surveyed households are either severely or moderately hungry in both departments of the project’s target areas. At the departmental level, there is no real difference in the hunger scale score for both departments. However, contrary to the results for the two previous indicators, the Northeast shows better results for this indicator. In the Northeast, 22.3% of surveyed households have access to sufficient food or experience no or mild hunger; however, in the Center, these people represent about 18%. In the Northeast, about 48% experience a situation of severe hunger, while, in the Center, 53% experience severe hunger. Table 14: Hunger score for project beneficiaries by department Indicator: Hunger Scale or Household Hunger Score (HHS) Center Northeast All Departments Num Den % Num Den % Num Den % No or mild hunger in households (scores 0-1) 121 680 17.8% 151 678 22.3% 272 1358 20.0% Moderate household hunger (scores 2-3) 199 680 29.3% 198 678 29.2% 397 1358 29.0% Severe household hunger (scores 4-6) 360 680 52.9% 329 678 48.5% 689 1358 51.0% Average Household Hunger Index 4 5 4 Household Hunger Index standard deviation 2.4 2.43 2.44 Confidence interval at 95% of the average [3.82;4.18] [4.82;5.18] [3.87;4.13] Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. d) Reduced Coping Strategies Index (rCSI) The average score of the reduced coping strategy index calculated is 16.84 and 12.57, for beneficiaries in the Center and Northeast, respectively (Table 16). The higher the index score, the more food insecure the household in question is, meaning that the household uses negative strategies more frequently and/or that the strategies used are more severe compared to a household with a lower score. The maximum score for the index is theoretically 56. The average found for the index for the whole is in the very low rCSI modal class (4-18), i.e. 43% of households at the national level, from the last National Emergency Survey on Food and Nutrition Security (ENUSAN, MARNDR/CNSA, 2019). It is evident that the upper limits of the confidence interval for the estimate of the average calculated by department do not reach 19. More beneficiary households use negative strategies in the Center compared to those in the Northeast. 41 Table 15: Reduced Coping Strategies Index (rCSI) Reduced Coping Strategies Index (rCSI) Center Northeast All Departments Average 16.84 12.57 14.70 Standard deviation 12.84 13.13 12.98 Confidence interval of the average [15.87;17.81] [11,58 ; 13,56] [14.01 ; 15.39] Median 12 12 12 Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. Graph 18: Range for reduced coping strategies index (rCSI) Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. e) Percentage of households where adults and children eat at least 2 meals a day (adults and children), by age The average number of meals a day in a household is considered an indicator of the food security or food insecurity level. The percentage of households with children aged 6-23 months who ate at least two meals a day is 17.8% in the Center compared to 15.8% in the Northeast and 16.8% in the whole project intervention area. For households with children aged 24-59 months, 35.6% of them eat at least two meals a day in the Center, compared to 25.2% in the Northeast, and 30.4% in all target areas. Households with children aged 5 to 18 years who eat at least two meals a day account for 54.7% in both departments (70.3% in the Center and 44.4% in the Northeast). As for adults within households, for the 19 to 59 years age group, Table 17 shows a percentage of 70.3%, 50.0% in the Northeast and 60.2% for those who eat at least two meals a day. Overall, for both children and adults, the percentage of households eating at least two meals a day is 72.9% in the Center versus 49.6% in the Northeast, or 61.3% overall. 41% 25% 33% 51% 25% 23% Center Northeast Less severe(<4) Moderate(4-18) Severe(>=19) 42 Table 16: Percentage of households where adults and children eat at least 2 meals a day (adults and children), by age Number of meals a day by age category Center Northeast Department Set Num Den % Num Den % Num Den % Percentage of children aged (6- 23 months) who eat at least 2 meals a day 121 680 17.8% 107 678 15.8% 228 1358 16.8% Percentage of children aged (24-59 months) who eat at least 2 meals a day 242 680 35.6% 171 678 25.2% 413 1358 30.4% Percentage of children aged (5- 18 years) who eat at least 2 meals a day 478 680 70.3% 301 678 44.4% 779 1358 60.2% Percentage of adults aged (19- 59 years) who eat at least 2 meals a day 478 680 70.3% 339 678 59% 817 1358 60.2% Percentage of adults (60 years and older) who eat at least 2 meals a day 167 680 24.5% 166 678 24.5% 333 1358 24.5% Percentage of children and adults who eat at least 2 meals a day 496 680 72.9% 336 678 49.6% 832 1358 61.3% Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. f) Percentage of food utilization, by type (household consumption, sale, exchange, livestock feed) This indicator helps analyze the different types of food use by surveyed households. Over the whole project area, nearly 100% of households use agricultural production for household consumption, 46.0% of them use it for sale because it helps raise their income, and 4.3% said they use it to feed their livestock. At the departmental level, the trend is almost the same for households using agricultural production for their own consumption, for exchange and to feed their livestock, but it is not the same for sale. In the Center, twice as many households reported selling their agricultural production as in the Northeast department. Table 17: Percentage of food use by type (household consumption, sale, exchange, livestock feed) by department Indicator: Production use Center Northeast All Departments Num Den % Num Den % Num Den % Household consumption 677 680 99.6% 678 678 100.0% 1,355 1,358 99.8% Sale 425 680 62.5% 200 678 29.5% 625 1,358 46.0% Exchange 20 680 2.9% 14 678 2.1% 34 1,358 2.5% Livestock feed 28 680 4.1% 31 678 4.6% 59 1,358 4.3% g) Percentage of households reporting greater ability to cope with economic shocks Among the 680 surveyed households in the Center, only 32 of them (4.7%) reported having received training in shock management, while in the Northeast more households (50) (7.4%) received training. And overall, 82 out of 1,358 or 6.0% have received training in shock 43 management. For those who implemented shock management methods, they represent about 98% overall (100% in the Center versus 96.0% in the Northeast). Table 18: Percentage of households reporting having received information to cope with economic shocks for the intervention areas and its communes Economic Shock Center Northeast All Departments Num Den % Num Den % Num Den % Percentage of households reporting having received information to manage economic shocks 32 680 4.7% 50 678 7.4% 82 1358 6.0% Percentage of households reporting having used information to manage economic shocks 32 32 100.0% 48 50 96.0% 80 82 97.6% Based on household reports, the two main shocks that have caused hunger are drought (73.1%) and the hurricane (16.3%) over the whole target areas. The trend is similar at the departmental level. Table 19: Major shocks that caused hunger in percentage Indicator: Major shocks that caused hunger in percentage Center Northeast All Departments Num Den % Num Den % Num Den % Drought 468 680 68.8% 525 678 77.4% 993 1,358 73.1% Erosion 1 680 0.15% 12 678 1.8% 13 1,358 0.96% Plant disease 29 680 4.3% 15 678 2.2% 44 1,358 3.24% Hurricane 116 680 17.05% 105 678 15.5% 221 1,358 16.3% Landslide 0 680 0% 0 678 0% 0 1,358 0% Other problems 66 680 9.7% 21 678 3.1% 87 1,358 6.4% h) Prevalence of mothers who exclusively breastfeed infants six months of age or younger The prevalence of exclusive breastfeeding among infants aged six months or less is 65.2% in both departments combined. This practice appears to be more prevalent among mothers in the Central department (72.2%) than those in the Northeast (55.6%). 44 Graph 19: Prevalence of mothers who practice exclusive breastfeeding among infants six months of age or less Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. i) Prevalence of global acute malnutrition (GAM) To measure and assess the nutritional status of children, data on their height or length, weight and age were collected so that anthropometric parameters such as height-for-age, weight-for￾height and weight-for-age could be calculated. In this study, the main focus is on low weight￾for-height or wasting, which is considered a measure of acute undernutrition and the consequence of inadequate nutrition over the period just prior to the survey. It measures the body mass in relation to height or length and describes the current nutritional status. Children whose Z-score for weight-for-height is below minus two standard deviations (-2SD) from the median of the reference population are considered lean (wasted) or acutely undernourished. According to the study’s findings, over the whole project intervention area, among the children aged 0-59 months surveyed, 5.4%16 suffer from acute malnutrition. Gender-wise, boys (5.6%) show a slightly higher percentage than girls (5.2%). At the departmental level, the Center stands out considerably from the Northeast with a prevalence of acute malnutrition three times lower (3.3% versus 9.3%). In the Northeast, where the situation seems critical, again for the 0-59 month age group, out of every 100 girls surveyed, about 10 are too lean for their height compared to 8 for boys (Table 21). For the 0-23 months and 24-59 months age groups, there is no real difference in the overall prevalence of acute malnutrition among children. However, at the department level, the findings for the Center seem more acceptable than those for the Northeast. As the prevalence of wasting is influenced by seasonal variations, it is difficult to interpret these variations over time. 16 According to the findings of the latest EMMUS-VI survey (2016-2017), 4% of children aged 0-59 months are wasted, in the 10 departments. The prevalence of wasting in the Center is 3.2%, and the Northeast is 1.5% according to the findings of EMMUS-IV (0-59 months). 72.2% 55.6% 65.2% Center Northeast Department set Figure 20: Prevalence of mothers who exclusively breastfeed infants six months of age or younger 45 Table 20: Prevalence of acute malnutrition by department Indicator: Prevalence of acute malnutrition in children by department (6-59 months) Center Northeast All Departments Num Den % Num Den % Num Den % % of male children under five years (0- 59 months) with a weight/height Z￾score lower than -2 standard deviation and/or edema 10 235 4.3% 10 120 8.3% 20 355 5.6% % of female children under five years (0-59 months) with a weight/height Z￾score lower than -2 standard deviation and/or edema 6 243 2.5% 14 139 10.1% 20 382 5.2% % of children less than 5 years of age (0-59 months) with a weight-for-height Z-score lower than 2 standard deviation and/or edema 16 478 3.3% 24 259 9.3% 40 737 5.4% Percentage of children 0-23 months of age with a weight-for-height Z-score Lower than -2 standard deviation and/or edema 4 175 2.3% 12 127 9.4% 16 302 5.3% Percentage of children 24-59 months of age with a weight-for-height Z-score lower than -2 standard deviation and/or edema 12 303 3.9% 12 132 9.1% 24 435 5.5% Source: EFSP Central and Northeast Final Evaluation Survey, World Vision, September 2020. Data from the quantitative survey therefore helped establish the values for these key project indicators in this final evaluation. We saw the key indicators progress against their targets. Only 3 did not reach 50% of their targets! However, in the interview with the project’s M&E team, they mentioned that 4 output indicators did not reach 50% of their targets, namely the number of stone lines rehabilitated, the number of check dams rehabilitated, the number of earth dams rehabilitated, the number of IEC materials given out. This suggests that, overall, many indicators have reached at least 50% of their target. In the impact analysis section, the values for key indicators are compared with those at the project baseline. Table 21: Key indicators measured, and their progress compared to targets Indicators Final Evaluation Values Center (%) ± CI Northeast (%) ± CI Target % of target met Target % of target met Percentage of targeted households with an acceptable food consumption score (FCS) 66,6 ± 3,55% 75 88,8% 48,50 ± 3,75% 75 64,67% 46 Prevalence of households with mild or no hunger (Household Hunger Scale - HHS) 17,8 ± 2.9% 70 25,43% 10.9 ± 2,35% 70 15,57% Proportion of households eating at least 6 food groups in the previous month 92,2 ± 2.0% 70 131% 83,5 ± 2,8% 70 119% Percentage of food use by type (sale, trade, feeding) 62,5 ± 3.6% 70 89,28% 29,5 ± 3,4% 70 42,14% Percentage of households where adults and children eat at least 2 meals a day (adults and children), age 0-59 months 72,9 ± 3.35% 70 104% 49,6 ± 3.75% 70 70,85% Prevalence of mothers who exclusively breastfeed infants six months of age or younger 72,2 ± 6.9% 60 120% 55,6 ± 9.0% 60 92,67% Reduced coping strategy index (% of households with a less severe or moderate index) 16,84 ± 0.95 (58%) 70% 82,85% 12,6 ± 1.0(74%) 70% 108% Percentage of households reporting greater ability to cope with economic shocks. 4,7 ± 1.6% 60% 8% 7,4 ± 1.95% 60% 12,33% Global prevalence of acute malnutrition (GAM) 3.3 ± 1.65% - - 9.3 ± 3.55% - - Based on the previous analysis of project indicators, it would therefore be tempting to assign a score of 4=Satisfactory on the previously established scale to determine the extent to which the OECD criteria are met in the project. 5.4 Analysis of Project Efficiency To determine the efficiency of the project, we looked at the information available to see how cost-effective activities were in each component. The cost-effectiveness analysis criterion is used wherever possible. We also examined whether the objectives set for the components were achieved on time, based on the resources made available for their implementation. Compliance with provisional/contractual deadlines is also considered. 47 For this analysis, the food vouchers, nutrition, agriculture and financial support (training and S4T) components are considered. a) Food Vouchers WV set a joint team of 4 people on this program component to lead the operational steps and design the vouchers using a system, to design the distribution plan, negotiate prices with the vendors and ensure the quality control of food products. At the time of vouchers’ distribution and redemption, WV set up a distribution committee to support beneficiaries and collect their grievances and an OSDM (One Site Distribution Monitoring) focal point to ensure compliance with humanitarian standards. WV conducted post-distribution monitoring every 3 months and WV’s internal accountability team managed complaints. In addition, the WV team considered humanitarian and security principles in determining distribution points for food vouchers to beneficiaries in order to allow easy access and timely interventions respecting the rights of participants. However, the team felt the need to identify other people in households with reduced mobility to retrieve the vouchers and food and keep them from coming to distribution centers. Waiting times for beneficiaries during voucher distributions and redemption from vendors were reasonable for most surveyed communes. However, delays were recorded, and catch-up sessions were set up to ensure all beneficiaries were served. In the Center during interviews with local authorities, in Cerca-la-Source for example, on many occasions the project team experienced delays and distributions were held in late hours; this posed security risks for beneficiaries, mainly pregnant and nursing women and people with reduced mobility. Communication between the WV team, vendors and beneficiaries was more or less smooth with respect to the planning of voucher distributions, the adjustment of their purchase value and their redemption with vendors. However, misunderstandings were recorded in both departments between beneficiaries and vendors, which led to tensions on the price of products with vendors. In fact, WV carried out quality control measures for food products and monitored prices on local markets, monthly price-fixing meetings were organized with vendors considering the inflation and the depreciation of the Gourde against the dollar, and restitution meetings were held with beneficiaries. In some areas, vendors worked with WV to mobilize and travel to meet beneficiaries in remote areas. Thus, they had to readjust the costs of products without notifying WV, which increased discussions and tensions with vendors. The issue of contractual delay was mainly discussed with a few vendors who pointed out that WV needed more than two weeks to pay, following the presentation and validation of products delivery sheets; which created a challenge, for many of them, to replenish their stocks. This is mainly the case for vendors in Mombin-Crochu, Fort-Liberté and Cerca￾Carvajal. b) Nutrition CEPAM, which was fully in charge of all the activities under this nutrition component, coordinated the selection of mother leaders, nutrition training, and dissemination and awareness campaigns on care and better hygiene practices. Their leaders believe WV made adequate resources available to the project within the desired timeframe to complete the activities. Collaboration and coordination with WV was efficient, as much as the partnership was well defined. WV provided CEPAM with training materials so that they could better 48 understand their role in disseminating information to the population. The project also provided them with cooking materials to continue teaching better cooking practices. The unit cost for infant and young child nutrition counseling ranges from US$5 to US$7 in sub-Saharan Africa (A Framework for Investment in Nutrition, World Bank 2017). It would be interesting to see here the cost to raise the awareness of mother-leaders, especially knowing how much they complain about the accessibility of the areas and transportation costs. To travel from one community to another, people use motorcycles in some communes. Transportation costs can vary from 1,000 to 1,500 HTG (15-20 USD). Since we do not have all the information related to the costs involved, we were unable to make a comparison. These kinds of analysis are important for projects to conduct to see how sustainable these practices are that still create good dynamics in communities. c) Agriculture In the interviews with community leaders and local authorities, they pointed out a lack of human resources in implementing the agricultural component. In fact, the training and plot monitoring (after the planting of seeds) activities were not carried out within the allocated time. This caused considerable delays for example on the availability and subsequent distribution of seeds. Therefore, this was a negative efficiency score! Although WV had community endorsement for the implementation of the project, there were no formal partnerships per se, but short collaborations between WV and local leaders at specific moments in the project, such as during the start-up meetings, mobilization of beneficiaries, managing of security aspects during the distribution of food vouchers and seeds by CASEC and ASEC. So the lack of agricultural staff was a major challenge. The simultaneous management of several farms by the same person resulted in burnouts. This may also lead to a loss of focus and affect the planning. Under this component too, the project is looking for clear data on sowing activities, plot management and subsequent harvesting. For beneficiaries, supporting agricultural production is essential to ensuring food availability within the communities and especially to equipping farmers with smart conservation practices and techniques to adapt production to climate hazards. They claim to be very satisfied and in agreement with the trainings received but think that more efforts can be made under this component. For example: WV could establish community nurseries to multiply fruit and forest seedlings, they could train people on agroforestry, they could manage firewood lots to reduce the cutting of trees and increase biodiversity and ecosystem services. d) Financial support (training and S4T) The necessary human and financial resources were put in place by World Vision for this project component. For local leaders from the 2 project’s intervention areas (departments), the technical trainings on the setting up of new SLA groups and the strengthening of existing groups were a success on which the communities capitalized and with which they continued after the project. Indeed, according to them, the savings and credit groups not only had an economic objective (savings and credit) to help vulnerable people access additional funds for investment in income-generating activities such as agriculture, animal husbandry and small businesses, but also they helped strengthen social ties within the communities. For them, the savings and credit groups helped renew community ties, self-help and family management. 49 In addition, the project team estimated the cost of setting up and strengthening community savings and credit groups at US$420 per unit, and an agent costs US$150 per unit. Through discussions with the Mercy Corps and CARE Haiti specialists in group formation and strengthening, they consider the $400-$500 range as the unit cost to set up and strengthen savings and credit groups when considering the start-up materials to be provided, 2-3 trainings to be given to a 30-member group. They estimate the unit cost for the agent who supports the groups at a US$100-200 range when they consider 3 trainings provided, for example. Based on these figures, it can be said that the project has been financially efficient in this specific component. To summarize, the project has had efficiency concerns especially in terms of meeting deadlines with vendors and in agriculture, due to the lack of human resources for monitoring activities. On the satisfaction scale presented above, it deserves a score of 3=Average satisfaction on the basis of available information. We could not get the project’s financial report to know the percentage of budget expenditures, and which budget item was under or over evaluated. 5.5 Analysis of Project Impact The idea behind this criterion is to establish the level of major informed (reported) and/or observable changes in the living conditions of the populations (or target groups) directly or supposedly related to the project outcomes. According to the M&E team, the project did indeed reach the expected number of beneficiaries and territorial coverage. The project’s interventions virtually led to an overall improvement in key indicators (Table 22). A strong downward difference is noted in the Central Department with the indicator related to exclusive breastfeeding. And yet, everyone knows the project mainly worked with mother￾leaders to act on these aspects. This is probably due to a sampling bias at baseline where the percentage was too high for exclusive breastfeeding, especially when we refer to the national average of 39% (EMMUS, 2016). The difference test explains whether or not there is a statistically significant difference between the values found for the indicators at baseline and the final evaluation. Where the test indicates there is no difference, it is highlighted in red, meaning the values may be different but are not statistically significant to indicate there has been a change in the indicator considered. That way, we may note the result is slightly better in the Center than in the Northeast. 50 Table 22: Comparison of baseline and final evaluation results Indicators Baseline values Final Evaluation Values Difference Test Center (%) ± CI Northeast (%) ± CI Center (%) ± CI Northeast (%) ± CI Center Northeast Percentage of targeted households with an acceptable food consumption score (FCS) 46.0 ± 6.0% 49,9 ± 5.3% 66,6 ± 3.55% p= 0.00< 0.05, there is a difference p= 0.62>0.05, no difference Prevalence of households with mild or no hunger (Household Hunger Scale - HHS) 8.9 ± 3.45% 6.0 ± 2.5% 17,8 ± 2.9% 10.9 ± 2.35% p= 0.002< 0.05, there is difference p= 0.00< 0.05, there is a difference Proportion of households eating at least 6 food groups in the previous month 78,1 ± 5.0% 76,6 ± 4.5% 92,2 ± 2.0% 83,5 ± 2.8% p= 0.000< 0.05, there is a difference p= 0.00< 0.05, there is a difference Percentage of food use by type (sale, trade, feeding) Sale 62,5 ± 5.85% 63,7 ± 5.1% 62,5 ± 3.6% 29,5 ± 3.4% p= 0.5>0.05, no difference p= 1>0.05, no difference Exchange 7,55 ± 3.15% 4,0 ± 2.05% 2,9 ± 1.25% 2,1 ± 1.05% p= 0.99>0.05, no difference p = 0.96>0.05, no difference Livestock feeding 2,3 ± 1.8% 4,2 ± 2.15% 4,1 ± 1.5% 4,6 ± 1.55% p= 0.03< 0.05, there is a difference p = 0.32>0.05, no difference Percentage of households where adults and children eat at least 2 meals a day (adults and children), age 0-59 months 63,6 ± 5.8% 40.0 ± 5.2% 72,9 ± 3.35% 49,6 ± 3.75% p= 0.000< 0.05, there is a difference p= 0.000< 0.05, there is a difference Prevalence of mothers who exclusively breastfeed infants six months of age or younger 87,6 ± 9.65% 53,8 ± 15.45% 72,2 ± 6.9% 55,6 ± 9.0% p= 1>0.05, no difference p = 0.26>0.05, no difference Reduced Coping Strategies Index (rCSI) Average 16,7 ± 0.9 16,7 ± 0.85 16,84 ± 0.95 12,6 ± 1.0 p= 0.84>0.05, no difference p = 0.000< 0.05, there is a difference Standard deviation 10,5 10,9 12,84 13,13 Percentage of households reporting greater ability to cope with economic shocks. 0,9 ± 0.85% 0.0 ± 0.0% 4,7 ± 1.6% 7,4 ± 1.95% p= 0.000< 0.05, there is a difference p = 0.000< 0.05, there is a difference Prevalence of Global Acute Malnutrition (GAM) 10.5 ± 3.6% 20.4 ± 5.1% 3.3 ± 1.65% 9.3 ± 3.55% p= 1>0.05, no difference p= 1>0.05, no difference 51 Beyond the figures shown for the indicators, local leaders recognize that the food insecurity and living conditions of the beneficiaries underwent positive changes since the food vouchers came with support in livelihoods such as agricultural production. In addition, nursing mothers are in a better position to support better nutrition for their offspring and apply better hygiene principles for the whole family. On the other hand, they believe that many unassisted families remain food insecure and their conditions have worsened following COVID-19. Since they were already living in precarious conditions, lockdown restrictions brought these families into an emergency phase of humanitarian assistance. The activities of the savings and credit groups helped families who never had access to financial services to save and access investment funds allowing them to diversify their economic activities. The technical support and agricultural training received helped beneficiaries better approach two windows of agricultural production. Unfortunately, the seeds were given out late, outside the crop calendar, which reduced their productive potential. In addition, the trainings and maintenance of water points and pumps helped the whole communities have access to treated and drinking water, and prevent subsequent cases of cholera. Combined with training on the key aspects of family nutrition, pregnant and lactating women, the WV team believes the distribution of food vouchers helped beneficiaries and vulnerable families better feed themselves by considering the essential categories of a balanced diet. Beneficiaries emphasized that the voucher assistance they received helped their families access food quickly so that they did not need to resort to harmful coping strategies such as sending household members to beg, selling their livestock or productive assets (rCSI <19). Beyond the mother-leaders strategy that encourages and anticipates the inclusive participation of women and girls, project activities had a clear strategy in their implementation to support and strengthen the role and participation of women and girls. For the agricultural training activities, there was no evidence that a gender analysis was conducted to understand the social dynamics and the role of women in the production chain and thus design training that takes these social subtleties into account. As with the other criteria, the previous analysis of the project’s impact led us to consider a score of 4= Satisfied for this, on the satisfaction scale. 5.6 Analysis of Project Sustainability The implementation approach, based on the search for ownership of the project by partners and local communities, is an important factor that can lead to sustainable outcomes. In this approach, two important elements to ensure sustainability are to be highlighted: • The adoption of a clear communication strategy to facilitate understanding and strong involvement of beneficiaries, state and local community structures in both the planning and implementation of activities. • The adoption of a clear exit strategy to facilitate business continuity after the project. Because they were carried out as part of a humanitarian action, the focus of project interventions on activities that can truly meet the needs of beneficiaries is a prerequisite to improve community resilience. They must also be in line with the humanitarian-development nexus. On this point, the orientation of farming activities towards market value chains to ensure production, the construction/rehabilitation of important infrastructure for the 52 community and the strengthening of beneficiaries’ capacities are actions that can ensure sustainability in the project. It can thus be admitted that the implementation strategy is quite correct and has a huge potential to produce sustainable and replicable outcomes in the medium and long term. It is clear that the sanitation work and road section development are sustainable activities that will continue in the communities thanks to the commitments of the community leaders, CASEC and the local municipalities. However, the heavy sanitation work will require local investment funds through the cities (Ministry of Interior), perhaps, or the regional offices of the Ministries of Agriculture and Environment. Water point management committees continue to provide drinking water services in the communities, and mother leaders can join DINEPA structures and MSPP nutrition officers respectively to continue meeting the quality requirements for drinking water treatment and raise the awareness of lactating mothers on the importance of breastfeeding and a balanced diet for the whole family. In addition, due to the transfer of technical agricultural knowledge during training, farmers can continue applying good soil conservation practices to increase their resilience to recurring climate shocks, but they will need to better organize themselves, for example, into groups through the internal agricultural structures of farmers’ organizations to share technical information. The savings and credit groups will continue operating after the project. However, they believe the formation of other groups is a pressing need in the communities, especially in the communal sections. This is the part of the project where there is a higher community ownership! In the meantime, other groups have emerged without the help of the project. This ownership is reflected in the level of involvement of agents who worked without receiving project allowances or money from the savings groups. After receiving the necessary training, the members and agents – since they are also part of the groups – brought the solidarity funding to launch the activities. The project also strengthened the visibility of local authorities and brought them closer to their constituents, especially in the implementation of community work and the distribution of food vouchers, which represent concrete actions to inject funds into the different communities to alleviate the currently prevailing situation. Although food aid in cash or in kind is hardly a sustainable action, it is just a rapid intervention allowing disaster-affected populations to have quick access to food. Local authorities believe that working, at the same time, on reviving livelihood activities (agriculture, livestock, small businesses, etc.) is the best way to ensure better living for communities and, above all, to strengthen their capacity to adapt and transform themselves in the face of these recurring shocks (climatic, social, economic). For that purpose, the project was to support local mechanisms for adaptation to climate shocks through the decentralized DPC branches. In conclusion, the project set into motion dynamics that can continue after its closure, such as savings and credit groups and water point management committees insofar as these are linked to the TEPACs (Techniciens Eau Potable et Assainissement Communal) of the communes. However, some other components will have difficulty staying in the communities. Let’s consider, for example, mother-leaders. To what extent will they be able to be supervised by the ASCPs or other MSPP structures? We have seen in the efficiency section, their financial limitation to go into communities that are difficult to access. For all of this, a score of 53 3=Moderately satisfied can be considered for the assessment of the sustainability criterion in the project’s dynamics. 5.7 Analysis of Project Linkages and Exit Strategies A persistent challenge for projects is to ensure that benefits from interventions are sustained after the projects’ completion. Linkages, especially vertical linkages, such as those between community organizations or individuals and existing institutions in the public or private sector, are essential to the successful progressive transfer of responsibilities previously supported by projects. Project design should include indicators to measure not only impact but also the sustainability of change. Sustainability plans and exit strategies should include clear timelines and benchmarks for progress toward sustainability, separate from project impact indicators. The team analyzed the capacity of WV during the project’s implementation to forge systemic links and partnerships with state structures or organized groups in the communities to support actions and interventions and to continue the dynamics initiated by the project’s many components within the communities: Food aid: WV did not have direct collaboration with MAST to support the internal mechanism for strengthening social safety nets, protection and social promotion. Although WV has used SIMAST through WFP, the evaluation team believes this approach does not, in any way, ensure a transfer of lessons learned to the relevant state structure and especially allows MAST to appropriate data from WV in order to update SIMAST and strengthen their capacity to coordinate with other actors for better use of available resources and avoid duplication in future interventions. This brings the evaluation team to the following conclusion: the approaches included in the project proposal regarding links to social protection mechanisms to target the most vulnerable were not taken into account by the implementation team. While national governments should remain responsible for the design and management of social protection programs, according to the OECD Guidelines for Humanitarian Assistance17, humanitarian actors should play a role in strengthening existing policies and supporting the creation of new social protection systems in areas where such programs do not exist or are underdeveloped. There are opportunities to link humanitarian assistance and social assistance for two main reasons : - An existing social protection system can potentially help provide rapid response to a large number of people. - Engaging with social assistance can help strengthen the State system. Nutritional component: Project interventions are in line with the National Nutrition Policy (NNP) in its objectives of nutrition prevention, improvement of nutrition information systems and nutritional protection in emergency situations. WV has trained on key issues related to the promotion of proper nutrition during the life cycle of men, women and children in the communities, improved infant and young child feeding practices and strengthened the fight against micronutrient deficiencies. However, no vertical links or partnerships with the MSPP or horizontal links with the local agents of the nutrition section of the said Ministry were created. Because of that, mother leaders themselves pointed out a lack of supervision and support, refresher training (continuous capacity building) and resources to continue supporting pregnant and breastfeeding women in their communities with clear objectives. 17 https://www.oecd.org/development/humanitarian-donors/docs/cashbasedresponse.pdf 54 Agricultural component: WV did not use a market system approach such as the M4P methodological framework to make market systems work for the most vulnerable. Farmers were not put into contact with potential sellers and buyers, which does not guarantee a secure market for surplus production (immediate results of input distributions), nor does it provide incentives (additional income accumulation) for farmers to continue applying improved practices and market their crops. Furthermore, WV did not follow the MARNDR pre￾requisites that require any intervention in the seed sector to use the national seed service to ensure good quality and conduct seed adaptability tests before distribution to farmers. This would also help establish links between seed suppliers and the project’s localities and areas of intervention in order to ensure better bargaining power for future orders from the communities through organized groups of farmers (ensuring demand and supply). The internal structures of farmers’ organizations and associations were not intentionally involved in the technical agricultural training. According to community leaders, the agricultural committees within farmers’ associations should be integrated through a Farmer Field Schools methodology and cascade training. These internal structures already exist and work voluntarily around the mission of farmers’ organizations to support communities to adapt to climatic and socio￾economic shocks. In addition, a reduced quota of women’s participation in the trainings was observed (less than 20%), which also reduces the clear transfer of agricultural technical knowledge to ensure full sustainability in the application and subsequent implementation of these practices beyond the project cycle. CNSA, for example, through these observation centers, was not contacted or put in contact with farmers to provide information related to the monitoring of agricultural and food product prices in the main markets, in order to help farmers better negotiate and value their products surplus on the markets. Community activities: Excellent collaboration was observed between WV, local leaders and municipalities around community work. Respondents stressed their importance to communities and persisted on the need for local governments to dedicate future lines in municipal budgets to continue this work. It is important to remember that not all models for achieving sustainability are equally applicable to all technical sectors. On the other hand, the strengthening of community drinking water points is positively noted by the communities and project intervention areas as sustainable. Management committees are better able to continue serving the communities and replace defective pumps and equipment and especially train local technicians able to respond to subsequent technical problems. This will allow for continuity and better management of water points. Of all the above, the satisfaction score is quite low for this analysis criterion, 2=Not satisfied. Indeed, at this level, the links with system partners (the sectors linked to the project) are not very clear. The following table summarizes the satisfaction scores for each criterion: Table 23: Level of Satisfaction with Criteria Achievement Criteria Score awarded* Relevance 4 Effectiveness 4 Efficiency 3 Impact 4 Sustainability 3 Links and exit strategies 2 *1=Not satisfied at all 2=Not satisfied 3=Moderately satisfied 4=Satisfied 5=Very satisfied 55 5.8 Analysis of the Local Market in the Intervention Areas An additional market study was also conducted to enrich the evaluation. Both a quantitative and qualitative approach were used, interviews were conducted with vendors, community leaders, the WV team, and there was a survey of beneficiaries. Market situation and availability of food products The findings of the market assessment confirm that all major markets in the region were functioning adequately during project implementation and were able to absorb any changes in demand caused by cash injections. Markets were accessible to beneficiaries. Discussions with the WV team and vendors revealed that a variety of products were available in sufficient quantities. For the vendors, there was no marked period of shortages for voucher food products since most were imported products. Shortages could be recorded for local products such as certain vegetables and fruits that are marked by well-defined farming periods. For tubers and vegetables, in addition to their farming availability, their acquisition takes place in remote areas through the Madan sara. During the rainy episodes, it was particularly difficult to access these production points. Beneficiaries in both departments confirmed they were able to find all the food products listed on the vouchers at the project’s vendors who made efforts to ensure a constant flow of supplies, especially for local products mentioned earlier (nearly 97% of the surveys). Community leaders, in turn, said they observed no break in the supply of food, wholesalers were always accessible and, in addition, closer links were formed between vendors (especially those who were part of the program) and wholesalers, which opened up informal opportunities for loans on sales to retailers. In order to reduce travel distances, the mobilization of traders was necessary to facilitate a market response in areas where vouchers were distributed in order to trigger a market response in areas where markets were absent (remote communal sections). The mobilization consisted in disseminating information on the potential increase in demand for commodities in order to stimulate commodity inventory planning. Indeed, vendors in both departments expressed a clear understanding of the selection process they went through to join the program: from the publication of the call for interest, surveys, audits, and visits to sales sites to the publication of the final list. Once selected, the vendors took part in meetings with the WV team to better understand the project and become aware of their role in the implementation of the food voucher component. They reported no difficulties during the selection process, and further reported that the WV team provided them with critical advice that would help them meet the defined selection criteria (e.g., guidelines on licensing their small businesses). There were no difficulties in meeting contractual requirements with WV, in payment mechanisms, product prices and voucher values, and the recipient verification system on the vouchers. Product prices For the operational aspect of vouchers, the vendors emphasized how easy it was to identify project beneficiaries and the need to coordinate with WV on a clear distribution schedule. The value of vouchers was adjusted following complaints from beneficiaries about the depreciation of the Gourde, increased inflation and socio-political unrest that impacted local markets and increased the price of basic foodstuffs. According to community leaders, the project played no role in price fluctuation as local markets had sufficient food to meet the increased demand generated by the project. Moreover, the changes in food 56 prices were not a response to increased demand but rather to factors external to the project such as rising inflation, devaluation of the local currency, and socio-political unrest. Indeed, in October 2019, CNSA reported a 14% monthly and 40% annual increase in the nominal cost of the basic food basket (rice, wheat flour, corn, beans, sugar and vegetable oil) with the highest increases recorded in Jérémie (62%), Fond des Nègres, Cayes, Hinche and Croix des Bossales (+41%). And imported products such as rice, vegetable oil, spaghetti, sugar, beans, flour and corn whose prices are strictly volatile compared to variations in the exchange rate, underwent increases in annual rate (2018 to 2019) of +38% for rice, +49% for corn, +53% for wheat flour and 52% for sugar during project implementation (CNSA18). 59% of surveyed beneficiaries in the Center felt that the prices of products displayed on the vouchers were more expensive than on the local market, compared to 41% in the Northeast (Graph 21). As mentioned in the previous sections, WV made monthly adjustments to the price of products on the various markets in line with inflation and the depreciation of the Gourde, and consultations were made with vendors and beneficiaries about the prices set, but there were nevertheless some irregularities. The depreciation rate of the Gourde required a tighter adjustment, at least every 15 days, and some vendors had to travel to areas that were difficult to reach for beneficiaries, which forced them to increase prices to absorb transportation costs and also use inflated values to amortize any additional increase before the next adjustments. However, most local leaders felt that the purchase value of the vouchers should be adjusted to USD 100 to absorb fluctuations. Graph 20: Distribution of respondents according to their perception of products prices on the vouchers compared to prices for the same products at other sellers Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Redemption and use of vouchers According to the project team, WV made internal arrangements to combat voucher fraud. In the beginning of the project, WV used sealed paper coupons with logos and different colors, bar codes, graphics that could prevent their reproduction. As time went by, the paper vouchers were replaced by a digital card (NFC) for beneficiaries and telephones (MPOS) for vendors with a link to the LMMS/EVS system. Information sharing, mobilization and awareness sessions were organized in the communities to reduce any risk of fraud, voucher theft, or exposure (security). Vendors were clearly identified with signs near their sales kiosk and practice sessions were conducted with beneficiaries on the required quality of food products for adequate consumption while maintaining their nutritional value. When redeeming vouchers 18 https://fscluster.org/haiti/document/bulletin-du-panier-alimentaire-de 8% 59% 41% 41% 51% Center Northeast Lower coupon product prices Product prices on higher coupon Same price everywhere 57 at sales kiosks, nearly 98% of surveyed beneficiaries mentioned they had no difficulty redeeming vouchers with project vendors and 80% on average (81% in the Northeast versus 79% in the Center) felt safe to complete such transactions. In addition, special attention was given to understanding the identification process, signature, balance on the vouchers, among others. On average, vendors received 20 voucher-holders per day and needed 10 minutes to complete the voucher redemption process for project beneficiaries, of which 3 minutes were dedicated to filling out the delivery form that had to be sent to WV for subsequent reimbursement. Within the families benefiting from the project, women (72% of respondents in the North East and 63% in the Center) made the decisions on redeeming the vouchers and the choice of products, followed by men. This confirms the program team’s targeting strategy that prioritizes female-headed households and pregnant and lactating women. Not many respondents said that these decisions were made in consultation (Graph 24). The same trend was noted where nearly 65% of respondents identified women as the household member who collected vouchers from distribution centers and kept them before they were redeemed with vendors. Graph 21. Distribution of respondents by who decided when to redeem and what products to get Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. 89% of households in the Center used food products received from the program for internal family consumption, compared to 73% in the Northeast. They also helped other families and neighbors, as nearly 11% in the Center and 26% in the Northeast reported having consumed some of the food and shared the products received (Graph 25). Graph 22: Distribution of respondents by use of products after voucher redemption 63% 72% 17% 18% 18% 5% 1% 2% Center Northeast Other (nephew, cousin, ...) Son/Daughter Both (husband/wife) Husband/Husband Woman/wife 58 Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Quality of products received from vendors For vendors in the Northeast, no complaints were received from beneficiaries on the quality of food products received in exchange for vouchers. For the Center, beneficiaries reported to vendors the poor quality of beans and tubers, which was improved in future deliveries because the vendors changed suppliers. This trend is supported by surveyed beneficiaries, of whom 8% and 1% in the Center and Northeast respectively felt that products were not of good quality. Nearly 35% mentioned they were acceptable (normal quality), and nearly 60% felt that products were of good quality (Graph 26). Graph 23: Distribution of respondents according to their satisfaction with the quality of food products after the voucher was redeemed Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. Impact on the supply chain Community leaders highlighted complaints from non-program vendors who mentioned the loss of customers to registered vendors. Economic opportunities in monetary terms benefit the communities, but the money flow and transactions benefit a smaller number of vendors. The WV team emphasized, however, that an inclusive and participatory approach was used to engage vendors in the program through clear selection criteria. It would be preferable, in these cases, to select as many vendors as possible and assign them a reasonable number of customers (splitting the additional flow of customers). Vendors ensured that the increase in demand and customer flow (traditional customers and project beneficiaries) was manageable and did not affect their capacity in any way, since they 89% 73% 11% 26% Center Northeast Sale of products Storage of products Consumed in part and shared with neighbors Consumed by the family 31% 43% 8% 1% 61% 56% Center Northeast Normal product quality No, the products are not of good quality! The products were of good quality 59 had the goods (agricultural products) to meet this increase in demand, although, in some cases (more than 30 clients per day), they were assisted by a helper whose role was to package the products for beneficiaries. For them, it was mainly difficult to explain to beneficiaries the decrease in the purchase value of vouchers due to the increase in the price of products on the local markets as a direct consequence of the rising exchange rate and inflation. Transportation costs increased and road conditions were not favorable during the rainy seasons to go and buy local foodstuffs such as tubers, fruits and vegetables. WV took more than two weeks to pay, following the presentation and validation of products delivery sheets, which created a challenge, for many of them, to replenish their stocks. This is mainly the case for vendors in Mombin-Crochu, Fort-Liberté and Cerca-Carvajal. In addition, many vendors mentioned delays from the WV team during open market days and the lack of coordination. In fact, they advise that each area should have an established market day for beneficiaries to redeem their vouchers. Beneficiary preference According to findings from the quantitative survey, beneficiaries prefer by far to receive humanitarian assistance in food vouchers, with nearly 60% for both departments, preferring receiving cash assistance. It should be noted that in-kind distribution is not much preferred and cash vouchers (i.e., from previously identified vendors) was not much appreciated (Graph 27). Graph 24. Distribution of respondents by how they would like to receive assistance for a similar project Source: Final evaluation of the EFSP project in the Center and Northeast, World Vision Haiti, October 2020. 58% 62% 2% 4% 26% 27% 1% 6% 11% 1% Center Northeast Food Coupon Cash coupon to be free in my purchases Cash Food products through direct distribution Any modality Other (agricultural project,...) 60 6) Conclusion and Recommendations The overall objective of this final evaluation is to analyze the level of achievement of the goal, objectives, and outcomes of the EFSP program and how these were done. For this purpose, a research combining quantitative and qualitative tools was conducted. 1,358 quantitative surveys were carried out among beneficiaries from the different project components. On the qualitative level, several dozen interviews were conducted not only with key persons in the project but also with beneficiaries, community leaders, local authorities and vendors. Overall, the findings show that the majority of beneficiary households are headed by adult women and men (F&M) in both departments. Their profile is characterized by vulnerability points. For the most part, they have no education or were only able to get an incomplete primary education. They live mainly in tin roof houses which are not always of good quality. Agriculture (including animal husbandry) and trade are their main activities, with monthly incomes of around 5,500 HTG or less. In terms of meeting OECD criteria, data analysis showed that the project was well positioned in terms of relevance, effectiveness and impact insofar as it responded to the priorities of the people in the communities affected by hurricane Irma by largely achieving its objectives and covering the targeted territories. The project is more or less well positioned, considering the aspects of efficiency and sustainability. This is because not all the components were able to get adequate human resources to meet contractual deadlines and there were no total ownership of actions by the communities in all the components. The project is rather weak in terms of linkages and exit strategies when considering the poor involvement of the regional structures of sectoral Ministries related to project activities, such as MARNDR for example. The local market analysis confirmed that all major markets in the region were operating adequately over the life of the project. Vendors expressed a clear understanding of the selection process in which they participated. The project did not contribute to increasing prices in the local market. Unfortunately, one negative effect of the project observed is the market distortion created by draining a large part of the local clientele to contracted vendors. Other vendors outside the project complained about this to local authorities. based on the findings of this study, recommendations were therefore expressed by project activity component: Food Voucher Component • Conduct analysis and monitoring of local markets to reflect fluctuations in food prices in order to anticipate and adjust the monetary values of food vouchers or simply define a lump sum that takes these fluctuations into account. Regularly check exchange rates to ensure that vouchers keep their value. • When a voucher approach is used, ensure that as many merchants as possible participate. This ensures a competitive marketplace. • Monitor food and non-food item prices in the nearby markets to ensure that your project area does not unfairly increase prices. • Strengthen feedback mechanisms for beneficiaries (free numbers, suggestion boxes, volunteers, etc.). • Ensure effective and continuous communication with beneficiaries and stakeholders throughout all phases of the project cycle (design, set-up, planning, 61 implementation, evaluation, monitoring and transition at the end of project). WV staff noted that despite initial outreach efforts and messages, non-beneficiary households became angry at the lack of assistance and directed their anger and aggression at staff throughout program implementation. Although adequate communication and awareness raising can be considered a “best practice” in all humanitarian programs, and particularly in financial assistance programs, care must be taken to maintain consistent, constant, and accurate communication about selection criteria, resource availability, priority of assistance, and recognition of needs at the community level. The regular and repeated transmission of these messages will help reduce misunderstandings and resentment among potentially volatile and aggressive populations. • Reduce security risk by adjusting voucher distribution and redemption cycles according to the context. WV must avoid the risk of congestion and disruption or frustration in the vicinity of project offices or merchant sales areas by providing well￾communicated and established periods for voucher collection and redemption. For example, implement a ten-day cycle, allowing five days for voucher collection in a designated area and five days for voucher use at the vendors’ premises to reduce the risk of crowd formation. Crowding can lead to a lack of understanding of the program on a larger scale, community unrest, and the risk of theft or pressure on vendors. Vouchers distribution and redemption cycles should be determined according to the context and through a clearly defined participatory process. • Support the harmonization between beneficiaries and vendors and strengthen sensitization sessions with beneficiaries and vendors on WV’s internal accountability mechanism and thus help them channel their grievances for further processing, which will help in better planning and service delivery. • Continuously and carefully raising awareness and informing non￾beneficiaries is essential to maintain stability in the implementation area. • Extremely vulnerable populations require additional logistical support for the process of collecting and redeeming food vouchers. Although the project’s beneficiary targeting criteria identified extremely vulnerable people for food assistance, community feedback indicated that the elderly, sick, and disabled should not have to wait in line for long periods of time or deal with the logistics of bringing their food basket to their homes. Future programs should consider special accommodations for these vulnerable groups to avoid excessive physical and mental stress or increased security risk during the voucher redemption process and transportation of purchased food. • WV must anticipate community safety net and social solidarity systems in targeting and allocating vouchers. In all project sites, households frequently shared their food aid with family, friends and/or vulnerable community members. Community solidarity practices should be taken into account in the final determination of the amount and use of vouchers to ensure that food aid is sufficient for households that can engage in such practices. • The project was to support local mechanisms of adaptation to climate shocks through the decentralized branches of the DPC. Food aid in cash or in kind hardly represents sustainable actions, but just rapid interventions allowing disaster￾affected populations to have a quick access to food. Working in parallel to revive livelihood activities (agriculture, livestock, small businesses, etc.) is the best way to ensure better living for communities and, above all, to strengthen community capacity to adapt and transform themselves in the face of these recurring shocks (climatic, social, economic). 62 • The project should take a more holistic approach to food security. It should also focus on issues such as access to food (including equitable access within households), absorption and maintenance. This would require a focused and logical approach aimed at specific changes in attitudes and behaviors. • WV should play a role in strengthening the country’s existing policies and social protection system by using them to target the most vulnerable or by conducting assessments to enrich the MAST database. • WV should integrate a community-based Disaster Risk Reduction (DRR) approach to help the community cope with the stressors and shocks associated with natural hazards. • The project’s logical framework should be reviewed on the basis of a causal analysis. A multidimensional and comprehensive approach targeting the same cohort may be more effective in creating impacts in terms of resilience and food security. • It is recommended that field activities be monitored by dedicated and, to some extent, independent monitoring and evaluation staff. Field monitors will provide management with first-hand information on unilateral reporting of activities. This way, findings are verified and will contribute to transparency and accountability. Nutrition component • Ensure that households not selected also benefit from sensitization sessions on good nutritional and hygiene practices in order to emancipate project impacts and thus improve the negative indicators of the nutritional situation in assisted areas. • The implementation of intervention activities to improve child nutrition is not only the responsibility of the health sector, but also requires the participation of local government, the agricultural sector, women’s and youth associations, farmers’ organizations and other local departments and mass organizations. • As child rearing is not the sole responsibility and task of mothers, the project should have a clear framework to integrate and promote men’s participation in nutrition practices and awareness sessions. • Adopt a decentralized approach for mother-leaders to reduce the need for transportation costs during awareness campaigns and thus ensure the sustainability of these activities beyond the project life cycle. • WV must ensure these implementation partners are able to meet quality requirements and project management standards. Provide better monitoring of nutritional activities under the lead of CEPAM to ensure objectives are achieved on time and with the desired performance. As an implementation partner, CEPAM has no internal monitoring and accountability system. Savings and Credit Community Group Component • Participants in the savings groups would need more support, strengthening, and skill development to manage and support the groups independently. Agricultural section • The distribution of seeds to beneficiaries should be supported by a monitoring and evaluation system using beneficiary monitoring sheets to record the conditions of production, transfer and evolution of successive generations of crops 63 and varieties. The members of the agricultural committees of local farmers’ associations can be trained to ensure the monitoring in this regard. • To ensure existing good conditions for the reception of the distributed seeds such as the existence of ecological fertilizers (manure and compost) whose absence compromised their performance. • Allocate additional human resources to support the various stages of the agricultural component, from logistical planning for seed distribution to post-yield evaluations. • There are no clearly defined exit strategies for the agricultural component beyond program support. WV needs to consider complementing agricultural interventions with broader actions to support local efforts to adapt to climate shocks, for example, providing training to communities on the Civil Protection Department’s early warning system, introduction of smart and climate￾smart seeds, promotion and introduction of eco-friendly agricultural practices, and establishment of firewood lots on steep slopes exceeding 40%. • Integrate and ensure a good collaboration with MARNDR and the Communal Agricultural Offices during the whole project cycle. And above all, ensure that the National Seed Service is involved in the quality control of seeds to be distributed, and conduct germination tests through standard operating procedures (SOPs) that are clearly defined and approved by the WV team. • Unlike other interventions, seed distribution must follow a strict schedule in accordance with the cropping calendar of the intervention areas. WV should strive in the future to respect the planting windows during seed distributions to ensure their full adaptation and the development of their productive potential. • WV should consider improving the productivity of local horticultural activities and also introduce other water-efficient crops such as the peanuts. This will help further diversify the agricultural economy and create a buffer against risks. • Use a cascade training strategy (Training of Trainers) for agricultural training using already existing local structures such as the agricultural coordination of farmers’ organizations. This will ensure adequate transmission of knowledge and thus guarantee sustainability in the application of conservation and post￾harvest management techniques. • Conduct analysis on gender and gender roles in farming and women’s working hours in order to produce adapted training modules that take into account women’s days and hours of availability and thus ensure their full participation and inclusion in training sessions. 64 7) Bibliographical References 1) Humanitarian Needs Overview (HNO), 2019, retrieved from: https://www.humanitarianresponse.info/en/programme-cycle/space/document/2020- humanitarian-needs-overview-templates 2) Haiti, Supply and Market Outlook, Fews Net, September 2019 retrieved from the link: https://reliefweb.int/sites/reliefweb.int/files/resources/Haiti_Perspectives_de_lOffre_e t_du_Marche_Septembre_2019_Juin_2020.pdf 3) UN, Designing Household Survey Samples: Practical Guidelines, 2005. 4) http://documents.worldbank.org/curated/en/673911467986337757/Haiti-Country￾partnership-framework-for-the-period-FY16-FY19 5) World Bank: Haiti, offering opportunities to all Haitians 6) FAO Haiti: Hurricane Matthew Situation Report, 2017 7) Haiti, Rural Livelihoods Profile, FEWS NET, March 2015. 8) An Investment Framework for Nutrition, World Bank 2017 65 8) Appendix I: FCS Indicator These groups include different types of foods such as: Group No. Food Group Weighting 1 Staple food: Corn, rice, sorghum, other grain, roots and tubers (potatoes, yucca, yam, sweet potatoes, large breadfruit, small breadfruit) and plantain 2 2 Legumes: White beans, black beans, red beans, pinto beans, green beans, nuts, peanuts (and other similar foods) 3 3 Vegetables/Leaves: Lyann panye, spinach, chives, cabbage, pumpkin, tomatoes, onions, broccoli, radishes (and all kinds of similar vegetables) 1 4 Fruits: Mango, papaya, guava, apricot, cantaloupe, pineapple, orange, melon, watermelon, quince, cherries, lemon, grapefruit, avocado, banana, apple, plum, tamarind, strawberry, pear (and all kinds of fruits) 1 5 Meat, poultry and offal: goats, pigs, sheep, cows, horses, chickens, turkeys, guinea fowls, pigeons, liver, kidneys, hearts, intestines, offals, brains, (and all other types of meat) Seafood: Fresh fish, salted fish, salted cod, crabs, shrimp, (and all kinds of seafood) 4 6 Milk and dairy: Cow’s milk, powdered milk, canned milk and batch milk, yogurt (and all other similar products) 4 7 Sugar and honey: White sugar, red sugar, honey (and all other similar products) 0.5 8 Oils and fatty products: Vegetable oil, olive oil, butter, shortening, fat (and all other similar products) 0.5 9 Spices/drinks: Coffee, tea, spices (parsley, thyme, garlic, clove), salt, fish powder, creamer 0 These scores were established based on information or experience from other surveys around the world. For the purposes of this survey, the following thresholds were considered: Poor food consumption: score between 0 and 21 Borderline food consumption: score between 21.5 and 35 Acceptable food consumption: score above 35 66 9) Appendix II: HHS Indicator Households will be asked about their experiences of hunger in the last four (4) weeks/30 days preceding the survey. Their answers allow us to classify them into three (3) categories: mild or no hunger, moderate hunger and severe hunger. The first level of the scale is considered an acceptable or normal situation from the food access standpoint. Three (3) frequency answers (Never=0, Seldom or Sometimes=1, Often=2) A score is calculated for each household (summing the three (3) responses), with a minimum possible score of 0 and a maximum possible score of 6. Three (3) categories of hunger are thus defined: a. "No or mild hunger in households" (scores 0-1) b. "Moderate hunger in households" (scores 2-3) c. "Severe household hunger" (scores 4-6) 67 10) Appendix III: Reduced Coping Strategy Index (rCSI) Survival strategies were categorized according to their severity : Category Behavior Weighting Stress Strategy Buying or borrowing food on credit 2 Borrowing money 2 Spending savings 2 Use more casual work than usual 2 Crisis strategy Selling productive assets 3 Removing children from school 3 Reducing health and education expenses 3 Emergency strategy Sending household members to beg 4 Selling the last breeding females 4 Migration of the whole household 4 In the so-called reduced strategy, only five (5) standard elements (standard strategies) are taken into account with their weighting, which tells their severity. No Strategy Weighting 1 Eat cheaper but less preferred food 1 2 Borrow food or money from friends or family 2 3 Reduce portion sizes in meals 1 4 Reduce adult consumption so that children eat more 3 5 Reduce number of meals a day 1 The maximum possible value of the score is 56 since a household uses all 5 strategies over all 7 days. 68 The situation was evaluated according to the index value: Group Index Value Less serious Less than 10 Moderate Between 10 and 19 Serious Between 20 and 29 Very serious More than 30 Source: CNSA, 2017 69 11) Appendix IV: Data Collection Tools (in separate files) 1) M&E Team Interview Guide 2) Interview guide for the person in charge of access to drinking water and hygiene 3) Interview guide for the Agriculture Manager 4) Interview guide for the Food Voucher Distribution Manager 5) Interview guide for the S4T Manager 6) Project Manager Interview Guide 7) Interview guide for salespeople participating in the project 8) Interview Guide for the MARNDR/BAC Manager and Local Representative 9) Interview guide for the head of CEPAM 10)Interview guide for the MAST manager and representative 11)Community Leaders Interview Guide 12)Interview guide for the beneficiaries of each project component 13)Interview guide for managers of other food security programs in the region and/or project target areas. 14) Quantitative questionnaire to evaluate project indicators