Final Assessment of the Emergency Food Security and Recovery Program for Vulnerable Populations Affected by the Residual Effects of Hurricane Irma in the North-East and Central Departments of Haiti USAID EFSP #72DFFP20GR00085 FINAL REPORT Date: October 11th, 2021 Submitted to WV / Haiti CARICONSUL GROUP World Vision International Haïti 1 | Page Table of Contents Acknowledgment.................................................................................................................................... 3 Abbreviations and acronyms .................................................................................................................. 4 List of tables............................................................................................................................................ 5 List of figures........................................................................................................................................... 5 Executive Summary................................................................................................................................. 6 Introduction .......................................................................................................................................... 10 1. General presentation of the Assessment .....................................................................................11 1.1. The goal of the assessment................................................................................................... 11 2. Methods and limits....................................................................................................................... 13 2.1. Collection method................................................................................................................. 13 2.1.1. Quantitative data collection method and analysis.......................................................13 2.1.2. Qualitative data collection method and analysis..........................................................16 3. Results .......................................................................................................................................... 17 3.1. Socio-economic profile of the households surveyed............................................................17 Objective 1. - Evaluate the achievements of the project in terms of its goal, objectives, results, and targets................................................................................................................................................... 19 GOAL 1: Food security for breastfeeding women and vulnerable households is improved ................19 1.1. Proportion of households having consumed at least 6 food groups....................................19 1.2. Food Consumption Score (FCS).............................................................................................20 1.3. Percentage of households where adults and children consume at least 2 meals per day...21 1.4. Domestic Hunger Index (DHI) ............................................................................................... 22 1.5. Percentage of food use ......................................................................................................... 23 1.6. Prevalence of Global Acute Malnutrition (GAM)..................................................................24 1.7. Value of other product/process indicators related to access to food (Effectiveness) .........25 Goal 2. The health and nutritional behavior of breastfeeding women is improved Error! Bookmark not defined. 1.8. Prevalence of exclusive breastfeeding in infants six months of age or younger..................26 GOAL 2: Communities have improved their capacities to cope with economic shocks.......................26 Objective 2.1: Access to sustainable livelihoods increased..............................................................26 1.9. Reduced Copying Strategy Index(rCSI)..................................................................................26 1.10. Value of other performance indicators related to strengthening livelihoods (Effectiveness)................................................................................................................................... 27 1.11. Food security indicators and gender of the head of household.......................................28 1.12. Percentage of achievement of the Higher Results Indicators compared to expectations (Effectiveness and Impact)................................................................................................................ 29 Satisfaction of project beneficiaries ................................................................................................. 29 2 | Page Explanatory factors linked to the achievement or not of results.....................................................30 Contribution of cross-cutting terms ................................................................................................ 31 Monitoring and Evaluation, and Coordination .............................................................................31 Respect for humanitarian standards ............................................................................................31 Objective 2. Evaluate the effects of the project on local markets and its repercussions on certain interest groups ..................................................................................................................................... 32 Influence on the offer....................................................................................................................... 32 Influence on demand ........................................................................................................................ 33 The prices of the products on the market of the intervention municipalities .................................33 Influence on the relationship between market players ...................................................................34 Objective 3. Evaluate the effectiveness and relevance of the transfer modality and complementary interventions in achieving the results of the activities.......................................................................35 Targeting of beneficiaries ................................................................................................................. 35 Food vouchers................................................................................................................................... 35 Technical assistance in Nutrition ...................................................................................................... 36 S4T Structures................................................................................................................................... 36 Objective 4. Identify best practices, lessons learned, strengths, and challenges in the design of activities, among others the logical framework, and in the implementation to obtain the outputs of the project........................................................................................................................................ 37 Partnership........................................................................................................................................ 37 Food vouchers................................................................................................................................... 38 Nutrition and behavior change......................................................................................................... 38 Logical context .................................................................................................................................. 38 Objective 5. Formulate (strategic) recommendations for other projects or new interventions. .....39 ANNEXE ................................................................................................................................................. 41 ANNEXE I. Terms of Reference for the Final Evaluation ...................................................................41 ANNEXE II. Performance Indicators Reference Sheet (PIRS) ...........................................................41 ANNEXE III. Household survey questionnaire / Children .................................................................41 ANNEXE IV. Guide to Qualitative Interviews with Key Informants and............................................41 Guide to Focused Group Discussions................................................................................................ 41 ANNEXE V. Form of consent - qualitative survey..............................................................................41 ANNEXE VI. EFSP Project Performance Indicator Monitoring Table (IPTT) ........Error! Bookmark not defined. ANNEXE VII. Product Price Control File................................................ Error! Bookmark not defined. ANNEXE VIII. EFSP Project Baseline Report ......................................... Error! Bookmark not defined. ANNEXE IX. Distribution Post Distribution Assessment Report........... Error! Bookmark not defined. ANNEXE X. Final project document...................................................... Error! Bookmark not defined. 3 | Page Acknowledgment The Cariconsul Group conducted this final assessment for World Vision Haiti (WVH) which received funding from USAID, through the “Food for Peace” (FFP) office, to implement the 2nd phase of the Emergency Food Security and Recovery Program for Vulnerable Populations Affected by the Residual Effects of Hurricane IRMA in the Departments of Northeast and Center of Haiti. The Cariconsul expresses its gratitude to the leadership of the WVH for their collaboration throughout the process of carrying out the assessment. In particular, we would like to thank the manager of M&E activities of the WVH, Ms. Henriette Nzuma DIAKA, the procurement Officer Ms. Naomie ZEPHIR, Mr. Roselin COLVERT, the M&E Officer assigned to the EFSP project, and field support staff. All of those who contributed either to the design or in the implementation and finalization of the report. The evaluation was coordinated by Mr. Peterson Abnis I FAURE and Mr. Jameson GALA, both co-founders of the Cariconsul. Mr Jameson GALA was the principal statistician￾demographer of the evaluation, as well as the specialist in quality control and database management. Mr. FAURE led and produced the data analysis and report writing, and Agr. Evens CAYEMITTE was our food safety technical advisor. Mr. Alphanor DANIEL, sociologist by training, and Ms. Rachelle MOREAU, nurse, led the qualitative interviews, i.e., group discussions and individual interviews with key informants, including representatives of local government, community leaders and beneficiaries of the project (households, mother leaders). We would also like to thank all our interviewers, supervisors and field coordinator who were able to carry out the methodological protocol of the collection under difficult conditions. 4 | Page Abbreviations and acronyms Acronyms Definitions ASEC Assemblée de la Section Communale ( Communal Section Assembly) ASCP Agents de Santé Communautaire Polyvalent ( Polyvalent Community Health Agent) CASEC Conseil d’administration de la Section Communale ( Communal section administrative committee CEPAM Centre Pour la Promotion de l’allaitement maternel ( Center for Breastfeeding Awareness) CU5 Children Under 5 DDS Dietary Diversity Score DHI Domestic Hunger Index DHS Demographic Health Surveys EBF Exclusive Breastfeeding FFP Food For Peace FGD Focus Group Discussion FCS Food Consumption Score GAM Global Acute Malnutrition HDDS Household Dietary Diversity Score HS Household Survey IPC Consumer Price index KII Key Informant Interview LQAS Lot Quality Assurance Survey MAST Ministère des Affaires sociales et du Travail ( Ministry of social Affairs and Labor) MTPTC Ministères des Travaux Publics Transports et Communications ( Ministry of Public work, Transports and Communication) MSPP Ministère de la Sante Publique et de la Population ( Ministry of Public Health and Population) M&E Monitoring and Evaluation NE North-East PDM Post Distribution Monitoring rCSI Reduced Copying Strategy Index S4T Savings for Transformation USAID United States Aid International Development WFP World Food Program WHO World Health Organization WV/H World Vision Haiti 5 | Page List of tables Table 1. Brief presentation of the values and indicators of the final evaluation against the baseline and the planned targets ................................................................................................9 Table 2. Parameters selected for calculating samples..............................................................15 Table 3. Distribution of the number of households/children 0-59 months investigated by targeted commune....................................................................................................................15 Table 4. Number of qualitative interviews carried out and number of participants................17 Table 5. Sociodemographic and economic characteristics of households by department.......18 Table 6. Comparison of Households having consumed at least 6 food groups per department between the baseline and the final assessment ........................................................................19 Table 7. Comparison of the FCS by the department between the baseline and the final evaluation (Final Ev)................................................................................................................20 Table 8. Comparison of the percentage of households where adults and children consume at least 2 meals per day per department between baseline and final assessment.........................22 Table 9. Comparison of DHI between baseline and final assessment.....................................23 Table 10. Percentage of food use between baseline and final assessment...............................24 Table 11. Comparison of the prevalence of acute malnutrition between baseline and final assessment................................................................................................................................24 Table 12. Level of achievement of output indicators ..............................................................25 Table 13. Comparison between the baseline and the final assessment of the RCSI by department................................................................................................................................27 Table 14. Other indicators related to resilience .......................................................................28 Table 15. Comparison of indicators by sex of the head of household.....................................28 Table 16. Percentage of achievement of targets set for higher-level results indicators...........29 List of figures Figure 1. Baseline comparison - Final assessment of the percentage of households having consumed at least 6 food groups per department.....................................................................20 Figure 2. Baseline comparison - Final assessment of the percentage of households with an acceptable FCS.........................................................................................................................21 Figure 3. Comparison of DHI between baseline and final assessment....................................23 Figure 4. Baseline comparison - Final assessment of the prevalence of global acute malnutrition in children by department....................................................................................25 Figure 5. Comparison of the prevalence of exclusive breastfeeding in children 0-5 months between baseline and final assessment by department ............................................................26 Figure 6. Overall satisfaction level of beneficiaries with the project ......................................30 Figure 7. Evolution of inflation year-on-year April 2019-April 2021.....................................33 6 | Page Executive Summary The World Vision Haiti (WV/H) received, in June 2020, funding from USAID through the office of Food for Peace (FFP) to implement the 2nd phase of the Emergency Food Security and Recovery Program in Favor of Vulnerable Populations Affected by the Residual Effects of Hurricane IRMA in the Departments of Northeast and Centre of Haiti. This project was implemented with a local organization - The Center for the Promotion of Breastfeeding (CEPAM), and supported by a coordination structure that includes state institutions at the department and local levels. EFSP was a 13-month project developed by WV/H, in response to hurricane IRMA, to help eight (8) most affected municipalities in the North-East and Centre departments to strengthen security food and nutrition for households faced with natural shocks and market disruptions linked to Covid-19. To achieve its goal, the project provided targeted households with food vouchers supported by the implementation of complementary activities to increase access to food, reduce malnutrition in children under 5, and improve livelihoods. The project also worked to promote the adoption of good nutrition practices and exclusive breastfeeding for breastfeeding women. The results, conclusions, and recommendations of the evaluation should be used, among other things, to improve future programmatic interventions. The final evaluation of the project was carried out by combining quantitative and qualitative methods. It relied on several sources of data and information from both a household survey in the areas supported by the project, interviews with key informants and focus groups (FGDs) with beneficiaries and other stakeholders, and a review of internal project documents. The household survey was conducted on a total of 761 households and 993 children under 5 years old following a random selection proportional to size. Household survey data was used to calculate and compare standard food security indicators between baseline and final assessment. The household survey data was supplemented by qualitative interviews conducted with 130 participants through 29 KIIs and 18 FGDs conducted with project stakeholders. The assessment team drew on key implementation documents, including indicator monitoring data, baseline, and other project documents to address questions and support. The results of the final evaluation led to the following: Efficiency /Effectiveness/ Relevance The project, for the majority, showed significant improvements across all performance indicators. The project performed effectively in the implementation of activities such as; targeting, distribution of vouchers, the establishment of TIPAs, establishment of mothers' clubs, and nutrition awareness-raising activities. Activities related to improving community infrastructure and training young people in strengthening TIPAs could not be implemented as planned. Concerning the achievement of superior results, the household survey confirmed an improvement of the indicators from the baseline and considerable progress against the defined targets.  The proportion of households having consumed at least 6 food groups improved in the two departments. It increased from 87.9% to 96.2% from baseline. It reached 96.2% of its target. 7 | Page  The overall FCS increased from 38.76% to 41.6% over the lifetime of the project, and the percentage of households with an acceptable FCS increased from 57.6% to 64.5%, and thus reached 83% of its target. This overall increase is attributable to that observed in the NE.  Access to the quantity of food for households has improved considering the two departments. It is mainly influenced by the considerable and significant increase recorded in the NE department which posted an overall increase of 42%.  The structure of the distribution of households according to the degree of hunger is modified. There is a significant improvement in both the percentage of households with moderate hunger and those who continue to experience severe hunger. The prevalence of households with moderate hunger is 64.65% at the end of the project against 23.71% at the baseline across all departments. The proportion of those who were classified as severely hungry households is 18.67% versus 61.92% at baseline. The percentage of households that are little or no hunger has not improved (from 14.36% at baseline to 16.68% at the end of the project).  Households continue to use nearly all of their production for their consumption. The results on the percentage of food use are different depending on the department considered. While the percentage of use for sale has increased in the NE, the Centre has shown a significant decline.  While exclusive breastfeeding improved in both departments, the household survey indicated a non-significant decrease of 0.7% in the prevalence of global malnutrition, from 5.4% at baseline to 4.7% at the end of the project. The project was unable to meet its objective of reducing malnutrition by 3.9% across the two departments.  The rCSI has not improved considering that households continue to use risky strategies to meet their food needs. Overall, the households surveyed indicated on average a Reduced Copying Strategy Index value equivalent to 26.5 at the end of the project against 14.7 at the baseline. This lack of improvement is also observed in the two departments. This result deserves to be further explored for those communities who strongly desire the continuity of the program. In conclusion, the literature review and qualitative interviews made it possible to note that the EFSP-II project was grafted onto national and local plans linked to the objectives of reducing food insecurity. An example of this aspect of the relevance of the project is the use of the MAST SIMAST database. The various consultations carried out to design and implement the intervention involved all relevant actors in the sector (MAST, MSPP, MA WFP, etc.). Efficiency The project implementation approach was essentially based on the use and development of local resources. The various project interventions were carried out in a participatory manner, relying on local partnership and governance structures, whether it concerns the process of targeting beneficiaries, the distribution of food stamps, the execution of behavior change activities related to nutrition, establishment, and strengthening of S4T structures. The various stakeholders did not comment on the non-rational use of the materials delivered (either to CEPAM or to the Leader Mothers) and of the financial resources committed during the implementation of the project. By way of illustration, the project chose to use the CEPAM, in this case, the AENs of the MSPP to supervise the ASCPs and the mother leaders to promote the adoption of good nutritional practices for breastfeeding women and children under 5 years old. 8 | Page This approach made it possible to reach a large proportion of targeted households while optimizing the use of the limited human and material resources available for the implementation of the project. Another example is the establishment and strengthening of S4T groups which are generally autonomous structures that use very little financial resources from the project. Finally, the food voucher transfer system (conditional and unconditional), grafted onto the existence of local resources (already existing local vendors, local distribution station, etc.) made it possible to reach all of the targeted households throughout the project while minimizing logistical costs (transport, training, etc.) and having an immediate direct impact on their food situation. Impact The EFSP-II project had an immediate but fragile impact on the municipalities of intervention. Interventions related to the transfer of food vouchers targeted deserving beneficiaries and contributed to an immediate significant improvement in their food situation. Complementary interventions, although they are hampered by logistical and cultural constraints, have supported the progress made in certain food security indicators. The logistical constraints are those expressed by CEPAM which wanted to have more materials and equipment support (balance, kitchen utensils, hygiene kits, etc.) for the proper functioning of mothers' clubs (Ti FWAYE); the mother leaders also expressed a need for supervision and adequate materials (MUAC, balance, hygiene kits, additional transport costs). The cultural constraints refer to ancestral practices in the targeted communities (Ex: Lok), which to a certain extent limited the adoption of good nutrition promoted by the project. Almost all of the indicators reached a significant percentage of expected values. The FCS, DHI, and HDDS reached 64.5%, 64.6%, and 96.2% of their targets, respectively. The expected percentage for households with severe hunger exceeded its target (it fell to 18.6% against 31% expected). Awareness raising activities have helped to improve the prevalence of exclusive breastfeeding. However, DHI was associated with little or no hunger and the rCSI showed no improvement. The expected percentage of food use related to consumption remains largely unsatisfactory despite the improvement seen from baseline. Contributing and limiting factors linked to the results • The causal explanations of the improvements observed are based on the participatory implementation strategy of the project which includes the use of the expertise as well as the proximity and governance structures of the intervention municipalities. The limiting factors are related to the scale and the systemic nature of the food insecurity problem; the overall social and macroeconomic conditions of the country. • The participatory implementation approach focused on the exploitation of local human and material resources, contributed to proper targeting, and facilitated the participation and ownership of the project by communities and local governance structures. • The provision of direct support, in this case, food vouchers, contributed, although considered partial, to improving the availability in quantity and quality of food at the target household level. • S4Ts and mothers' clubs have been effective strategies to support food voucher transfer activities. These structures remain fragile if they do not receive additional support, such 9 | Page as continuous technical assistance (training) on the management of these groups and the necessary equipment (funds). • The overall economic context marked by rising inflation has limited the extent of progress made and is a factor in sustainably reducing vulnerability. • The EFSP project was an emergency project that aimed to restore food security to targeted communities in the continuing adversity of natural shocks. The implementation strategy did not focus on creating or strengthening the livelihoods of target populations sustainably. Almost all of the participants, particularly the CASECs, affirm that the real supports needed to go beyond the provision of food vouchers and the establishment of S4T structures. The project did not contribute to strengthening local production structures, especially in the field of agriculture. This feeling expressed suggests that the effects of the project would not be sustainable as desired by many stakeholders. Table 1. Brief presentation of the values and indicators of the final evaluation against the baseline and the planned targets Indicators Baseline (LB) Final Ev. Planned target Desired tendency Actual tendency Comments % Of targeted households with an acceptable food consumption score 57.6% 64.5% 77.6% + Increased Reached 83.1% of the planned target Prevalence of households with little or no hunger (domestic hunger index 14.36% 16.68% ND - Has deteriorated Lost 2.32 against BL Prevalence of households suffering from severe hunger 61.92% 18.67% 31% - Improved Significantly exceeded the planned target Proportion of households having consumed at least 6 food groups. 87.9% 96.2% 100% + Improved Reached 96.2% of target Percentage of food utilization, disaggregated by type Total 99.8% 88.9% 18.15% - Improved over BL Achieved 20.4% of target % Of households where adults and children consume at least 2 meals per day 61.3% 70.5% 86.3% + Improved over BL Reached 81.7% of target Prevalence of exclusive breastfeeding in infants six months of age or younger 65.2% 72.7% 75% + Improved over BL N/A Reduced Copying Strategy Index 14.7 26.5 ND - Has deteriorated N/A Prevalence of Global Acute Malnutrition 5.4 % 4.7% 3.9% - Improved (Not significant) Reached 82.9% of target Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 10 | Page Introduction Context One of the poorest countries in the Western Hemisphere, Haiti, remains prone to natural disasters and vulnerable to price spikes in the global food market. Drought conditions, political instability, and deteriorating economic conditions have resulted in food insecurity across the country. Drought conditions in parts of Haiti continue to limit livelihood opportunities, including agricultural and livestock activities and exacerbate acute food insecurity. According to the IPC analysis carried out in October 2019, communities in parts of the Northeast (NE) and Central departments of Haiti will face crisis and emergency food insecurity (IPC phases 3 and 4) from March to June 2020. The IPC analysis attributes the situation to crop production deficits resulting from sporadic rainfall and livelihood disruptions caused by socio-political unrest. Forty percent of the population of the NE department (146,814 people) and the CP department (283,040 people) will be in crisis and emergency at the height of the lean season. The situation is exacerbated by the rise in the prices of basic foodstuffs, supported by inflation (nearly 20%), and by socio-political unrest which has considerably reduced access to foodstuffs at the market for the poorest households and eroded their productive assets. The persistence of the El Niño phenomenon during the first half of 2019 worsened the drought situation in many parts of the country, causing a considerable drop (12% less than last year) in the agricultural production of main crops, especially cereals (maize, rice, and sorghum) and beans/peas. Poorer households that have exhausted their initial coping strategies will face severe consumption deficits during the next lean season, as income from agricultural labor will not be sufficient to cover food expenses due to high prices. Indicators collected during the national nutrition and food security emergency survey showed that around 20% of the population has a poor food consumption score and 31% a limited score. In Haiti, a higher proportion of children in rural areas (24%) are malnourished compared to urban areas (18%); 30% of rural children in CP and 21% of children in NE are malnourished. Poor feeding practices and infections are also important factors, as a significant number of mothers still lack basic knowledge of exclusive breastfeeding (EBF) and nutritional practices. The current FFP-funded Emergency Food Security Project (EFSP) in the Northeast/Center has trained mother leaders and mothers' clubs to strengthen knowledge on nutrition and EBF. Although current statistics on this nutrition knowledge gap are not available, a 2016 survey by World Vision (WV) found that 37% of mothers do not exclusively breastfeed in the first six months. This situation increases the risk of infections and nutritional deficiencies due to unsafe eating practices. Since 429,854 people (85,970 HH) in NE and Center departments are classified as IPC 3, immediate and life-saving food assistance is needed to enable them to meet their immediate food needs and prevent implementation of additional harmful coping strategies. In addition, the current danger of the COVID-19 pandemic on the weak Haitian health system threatens to affect food security substantially and negatively in Haiti. According to the results of a SMART survey carried out in January 2020 in Haiti by the Ministry of Public Health and Population (MSPP), global acute malnutrition (GAM) is on the increase. At the national level, in all departments, chronic malnutrition stands at 22.7%, slightly more than the 22% of the Demographic and Health Survey (DHS) VI. In the Northeast, the prevalence of GAM in children under five (CU5), which was around 4% according to DHS VI (2016-2017), has increased to 5.5%. In addition, out of the ten departments of the country, the 11 | Page North-East has the highest prevalence of severe acute malnutrition (SAM), i.e., 1.3%, some areas of the North-East registering a prevalence of SAM = 3.1% (55% above the WHO emergency threshold). Trends in SAM levels in Haiti indicate that, according to the results of various surveys, since 2000, the prevalence of malnutrition as measured by anthropometric weight and height indices has decreased overall. This decline mainly occurred between 2005- 2006 and 2012. Since 2012, there has been virtually no change. Between 2000 and 2005-2006, levels of malnutrition remained stable or increased: the prevalence of stunted growth remained almost the same (29%) and began to decrease from 2005-2006, from 29% to 22% in 2012. Since then, there has been no change (22%). The prevalence of emaciation doubled between 2000 and 2005-2006, from 5% to 10%, and then decreased significantly to reach 5% in 2012 and 4% in 2016-2017. According to a Lot quality assurance survey (LQAS) conducted by WV in March 2019 and focused on maternal and child health, the proportion of households with children aged 6 to 23 months with minimal dietary diversity (7 groups or more) in Cerca La Source is 43.5%, while in Cerca Cajaval it is 41.9%. This raises a legitimate concern about the nutritional status of children under 5 living in the two departments. As a result, the WV, together with a partner specializing in nutrition, focused on a multi-sectoral approach to support the reduction of acute malnutrition in the two targeted departments using a community￾managed approach. Objectives of the Project This EFSP-II project aims to strengthen the food and nutrition security of 5,085 vulnerable households in the light of ongoing shocks due to drought, floods, market disruptions, and Covid 19, through the provision of unconditional voucher transfers and essential complementary activities. The EFSP-II project worked in six municipalities in the North-East department (Fort Liberté, Ferrier, Vallières, Mombin Crochu, Sainte Suzanne, and Carice) and two municipalities in the northern part of the Center department (Cerca La Source and Cerca Carvajal). The project will prioritize hard-to-reach and most vulnerable rural groups. WV proposed to target 5,085 households (25,425 individuals), representing around 30% of the most vulnerable rural population in the targeted communes in NE and Center. 1. General presentation of the Assessment 1.1. The goal of the assessment This report presents the results and recommendations of the final assessment of the emergency food security and recovery program implemented in favor of vulnerable populations affected by the residual effects of Hurricane IRMA in the northern -East and Central departments of Haiti. The purpose of this final assessment was to determine the extent to which the project achieved its expected results, provide evidence of the possible effects of the project's interventions on the fixed objectives and make recommendations to WV to improve its future interventions, particularly in the area of food safety. This assessment was conducted from August 14, 2021, to September 30, 2021. 1.2. Specific Objectives (SO) and evaluation questions This section reproduces the specific objectives (SOs), or evaluation questions formulated in the terms of reference (see Annex I) of the EFSP project. These evaluation objectives are broadly aligned with the final performance evaluation criteria questions that are commonly investigated. OS. 1. Evaluate the achievements of the project to its goal, objectives, results, and targets. 12 | Page OS.2. Evaluate the effects of the project on local markets and its repercussions on certain interest groups (women and men, young people, boys, and girls, etc.). OS.3. Evaluate the effectiveness and relevance of the modality, transfers, and complementary interventions in achieving the results of the activities. OS.4. Identify best practices, lessons learned, strengths, and challenges in the design of activities, among others, the logical framework, and in the implementation to obtain the outputs of the project. OS.5. Formulate (strategic) recommendations for other projects or new interventions. This final evaluation should make it possible to assess the progress made by measuring at the end of the interventions the value of the results indicators calculated from the baseline of the project. Outcome indicators include the rate of acute malnutrition in children 0-59 months (GAM), the food consumption score (FCS), the domestic hunger index (DHI), the household dietary diversity Score (HDDS), exclusive breastfeeding (EBF), and the Reduced Copying Strategy Index (rCSI). In conclusion, a list of dimensions and relative questions should also be explored and responded to by the evaluation. Relevance  What are the opinions of interested parties on the nature and quality of project implementation?  Have government officials been involved?  Has the project implementation strategy been adjusted to adapt to realities on the field? If yes, how? Profitability  Does the project have sufficient and appropriate human resources (human, financial, and capital) for its implementation?  In case there were deficiencies/resource problems/capabilities, how were they remedied?  Have quality control and responsibility measures been put in place and systematically implemented during the review, approval, disbursement of funds, monitoring, and reporting phases? Efficiency  To what extent have the objectives been realized?  What were the main factors that affected objectives?  Has the monitoring and evaluation system provided quality, appropriate and reliable information when measuring planned indicators?  To what extent has the project model been effective in terms of design, adequacy, management, and accountability? 13 | Page  To what extent has the project been effective in terms of program implementation (coordination, cooperation, effectiveness, standardization)?  To what extent are the strategies used to carry out the planned activities appropriate?  Have humanitarian standards been respected and the humanitarian principles followed? (Sphere, HAP, Codes of Conduct)? Impact  To what extent has the project helped to reduce the level of vulnerability of beneficiaries? 2. Methods and limits 2.1. Collection method The final evaluation of the EFSP-Phase II project has been conducted using a mixed search method that includes the use of primary data and data generated by the project's M&E system. Primary data comes from a survey conducted with project beneficiary households and a series of qualitative interviews with key stakeholders and informants. The household survey (HS) fundamentally provided the data for measuring eight (8) outcome indicators and report on potential changes by the project on its target groups. Qualitative interviews have a dual purpose: 1) provide additional elements on understanding the context related to the results of the quantitative survey; 2) Get the perception of stakeholders on issues such as relevance, effectiveness, efficiency, impact, challenges, and lessons learned. The final evaluation also exploits secondary quantitative data and information arising from the project's M&E system, documents, and follow-up reports in its exercise to achieve evidence￾based conclusions and recommendations. 2.1.1. Quantitative data collection method and analysis The household survey tool was used to collect the quantitative data needed to measure outcome indicators on a representative sample of the intervention areas selected by the EFSP project. While the baseline had measured for the pre-selected households the status of performance indicators before the implementation of the interventions, this final assessment provides, using Student T tests, an assessment of the possible change registered at the level of these same indicators at the end of the project after thirteen (13) months of implementation. Indeed, this final assessment makes it possible to detect real significant changes in the food situation of households (including children under 5 years old) during the project implementation period. However, it does not constitute in itself an impact assessment capable of allowing us to assign the results obtained only to the interventions of the project. This final assessment has adopted the same procedure for selecting household samples and children from 0-59 months used for the baseline as well as the questionnaire, with the removal of certain sections considered irrelevant. The qualitative interview guides have been developed to answer certain questions relating to the evaluation of certain specific criteria and objectives (relevance, efficiency, the effect of the project on local markets, etc.). Sampling The samples are selected following a probabilistic method and proportional to the size to ensure some statistical power capable of providing reliable estimates of the results indicators. Sample sizes for households and children from 0-59 months have been calculated using a formula for 14 | Page detecting possible differences in performance indicators, setting a 5% error margin. and a 95% confidence interval. 10% estimated anticipated non-response rates have been set to maintain the desired level of precision. In the end, the project investigated 761 households and 993 children from 0-59 months in the 8 communes of the two departments (North-East and Centre) of project intervention. These sizes remain in the predetermined confidence intervals required for the desired level of precision (+/- 10 of the non-response rate). To determine the size of the sample for the household survey (including that for measuring the GAM related to children from 0-59 months), the calculation formula was inspired by the FFP / USAID protocol so that the required minimum sizes (678 households and 1080 for children of 0-59 months) for the observation and analysis units are respected for the calculation of food security indicators (FCS, DHI, rCSI) and the one on the Acute Malnutrition (GAM). The formula for calculating the sample size for the survey is as follows: 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = 𝑫𝑫𝒆𝒆𝒆𝒆𝒆𝒆 ⎣ ⎢ ⎢ ⎡𝑍𝑍1−∝ �2𝑃𝑃�1 − 𝑃𝑃� + 𝑍𝑍1−𝛽𝛽�𝑃𝑃1,𝑒𝑒𝑒𝑒 �1 − 𝑃𝑃1,𝑒𝑒𝑒𝑒 � + 𝑃𝑃2,𝑒𝑒𝑒𝑒 �1 − 𝑃𝑃2,𝑒𝑒𝑒𝑒 � 𝛿𝛿 ⎦ ⎥ ⎥ ⎤ 2 Or • 𝑛𝑛𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 = is the initial sample size required by the surveys for each of the two-stroke points • 𝛿𝛿 = 𝑃𝑃1,𝑒𝑒𝑒𝑒 - 𝑃𝑃2,𝑒𝑒𝑒𝑒 = minimum size of the effect to be achieved over the period specified by both surveys • 𝑃𝑃1,𝑒𝑒𝑒𝑒 = represents an investigation estimate of the true proportion of the P1 population initially • 𝑃𝑃2,𝑒𝑒𝑒𝑒 = represents an investigation estimate of the true proportion of the P2 population at the end 𝑃𝑃 = 𝑃𝑃1,𝑒𝑒𝑒𝑒 + 𝑃𝑃2,𝑒𝑒𝑒𝑒 2 • 𝑍𝑍1−∝ is the value of normal probability distribution corresponding to a confidence level 1-β. For 1-β = 0.95, the corresponding value is Z_0,95 = 1.64. • 𝑍𝑍1−𝛽𝛽 is the value of normal probability distribution corresponding to a confidence level 1-β • For 1-β = 0.80, the corresponding value is z_0.80 = 0.84. • 𝑫𝑫𝒆𝒆𝒆𝒆𝒆𝒆 : This is the estimated plan effect (DEF) of the investigation. • There is indeed a non-response rate (usually set at 1.1 corresponding to 10% but may change depending on the context) => table reference shown in the TDRS. In summary, the parameters selected for the calculation of the samples are specified as follows: 15 | Page Table 2. Parameters selected for calculating samples Parameters of the formula of the samples The values considered for the calculation of the FCS, DHI, rCSI, HDDS indicator measurement The values considered for the calculation of the sample relating to the measurement of the GAM indicator 𝑷𝑷𝟏𝟏,𝒆𝒆𝒆𝒆𝒆𝒆 0.5 (Basic value) 0.15 (Basic value) 𝑷𝑷𝟐𝟐,𝒆𝒆𝒆𝒆𝒆𝒆 0.4 0.10 𝒁𝒁𝟏𝟏−∝ 1.64 1.64 𝒁𝒁𝟏𝟏−𝜷𝜷 0.84 0.84 𝑫𝑫𝒆𝒆𝒆𝒆𝒆𝒆 2 2 Initial size (𝒏𝒏𝒊𝒊 𝒊𝒊 𝒊𝒊𝒊𝒊𝒊𝒊 ) 610 610 Adjustment related to the anticipated response rate 10% 10% Final size 678 1080 The number of beneficiary households and the number of children under the age of 5 to investigate through the municipality of intervention is specified in Table 2. Since the household sample is constructed according to a draw proportional to the size, the sample of children under 5, automatically follows the same logic of selection. Thus, the distribution of the number of children under 5 to investigate per municipality is made under the assumption that each household has an average of 1,59 children under 5 years. This hypothetical average is obtained by the ratio between the calculated minimum size of the household sample and that of children under 5 years. (1080/678). Table 3. Distribution of the number of households/children 0-59 months investigated by targeted commune Department/ Target communes # Of households to investigate # Of households surveyed Target children from 0-59 months # Of children from 0-59 months whose anthropometric measurements have been taken Northeast 359 409 568 559 Carice 73 80 116 123 Ferrier 19 19 29 28 Fort-Liberté 25 48 39 41 Mombin Crochu 94 96 150 141 Sainte Suzanne 84 89 133 116 Vallières 64 77 101 110 Center 320 352 511 438 Cerca Carvajal 161 159 256 213 Cerca-la-Source 159 193 255 225 Data analysis The data collected during the household survey is used to assess the changes that have occurred in the eight (8) project results indicators. Descriptive analyses are performed on the data to provide context for the calculated indicators and to answer specific questions. The data were compared between the baseline and the final assessment by the department to report on the 16 | Page progress or lack of progress observed. To this end, Student's T test of comparison of proportion or means was carried out to judge the significance of the observed differences. Ratios were calculated to assess the effectiveness/efficiency of the project. Quantitative analyzes are combined with qualitative interviews to meet the objectives of the evaluation. 2.1.2. Qualitative data collection method and analysis The following methods were used in the qualitative part of the assessment: • Literature review of the project document, product price monitoring data at the local market level, performance indicator data, post-distribution report of food vouchers. • FGDs conducted with key groups, beneficiaries, or other stakeholders (beneficiaries, mother leaders, young people, members of S4T). • Qualitative interviews with key informants such as mother leaders, vendors, CEPAM staff, representatives of local state institutions (mayors, CASEC, ASEC), project staff. Qualitative data is obtained from qualitative interviews carried out following a logic of reasoned choice and maximum diversification of participants. The qualitative approach was used in parallel and in a complementary manner to the household survey to address certain evaluation questions related to the relevance, effectiveness, efficiency, impact, identification of challenges, and lessons. Qualitative data also makes it possible to explore other themes linked to the implementation strategy such as the selection of beneficiaries, the inclusion and participation of state structures in interventions, gender considerations, and compliance to humanitarian standards. Qualitative data is recorded with the consent of the participants, otherwise collected on paper for appropriate processing and analysis. The responses of the participants are transcribed succinctly in a matrix to facilitate their use and consolidation. Content analysis is carried out on the material collected to determine the trends, nuances, and variations between the speeches of the different groups to draw coherent and representative conclusions of the participants on the different themes under study. Focus Group Discussions (FGDs) Eighteen (18) FGDs were conducted in the two departments (Centre and NE) in the 8 communes of intervention. Target groups were identified with the assistance of WV/H field workers and community leaders. In total, one hundred and one (101) participants participated in the FGDs. All the FGDs took place in the places determined by the choices made by the participants and had an average of 5 people follow the preventive measures related to Covid￾19. All participants were informed of the nature and purpose of the exercise and gave their consent before the FGDs were held, which were recorded using tablets to facilitate operation. Table 3 summarizes the number of FGDs carried out by category of participants. Key Informant Interviews (KIIs) A total of twenty-nine (29) KIIs were carried out with key people who played a significant role in the implementation of the project. These people consist of project implementation staff (voucher distribution manager, M&E officer), staff of implementation partners (CEPAM agent, etc.), and representatives of local governance structures and community associations (mayors, ASEC, leaders, etc.). The qualitative interviews made it possible to collect information and perspectives related to the achievement or not of results, efficiency, relevance, contributing factors associated with the results, targeting, and the identification of challenges and lessons. 17 | Page Table 4. Number of qualitative interviews carried out and number of participants TYPE OF INTERVIEW / PARTICIPANTS NUMBER ACHIEVED NUMBER OF MEN NUMBER OF WOMEN Centre NE Centre NE Centre NE TOTAL EIIS 8 21 6 13 2 8 Project staff (M&E) 1 1 - - Voucher distribution manager 1 1 1 1 0 1 State representatives (mayors, casec, etc.) 2 6 2 5 0 1 ascp/aen (POLYVALENT COMMUNITY HEALTH WORKERS) 2 5 1 0 1 4 Staff cepam (CENTER FOR BREASTFEEDING Promotion) 1 - - - 1 - Community leaders 2 6 2 5 1 Vendors 1 2 1 1 TOTAL FGD 5 13 11 24 21 45 Beneficiaries heads of households 2 6 7 14 7 18 Mothers leaders / mother's club 1 3 0 0 6 15 S4T members 1 1 1 2 5 3 youth 1 3 3 8 3 9 Literature review The assessment made use of the project documents, food price control file, post-distribution evaluation report, and performance indicator monitoring data as an integral part of the analysis to address, among other things, questions that relate to effectiveness, the influence of the project on local markets, relevance and efficiency. These internal project documents also served as additional sources to compare and better understand the qualitative and quantitative data resulting from the exercise of primary data collection through the household survey. 3. Results 3.1. Socio-economic profile of the households surveyed Disaggregated by department, the socio-demographic and economic profile of the households surveyed is almost identical. Household survey data indicated that the surveyed households, headed predominantly by women, are generally of large population size and relatively very low income. The heads of households have a relatively low level of education and are more than 80% in some form of union or cohabitation (formal or free married or living with). It is important to note that almost half of the heads of households surveyed are aged 50 and over. These socioeconomic data seem to be consistent with the logic of the intervention of the EFSP￾phase II project, on which the factors for targeting households and intervention communities are based on economic vulnerability and food insecurity. In summary:  In total, 64.9% of household heads are women, with respectively 59.9% for the Center and 69.2% for the NE; 18 | Page  Household heads are generally more represented (25%) in the 50-59 age category; in the Center, it is the 40-49 age group that includes the most heads of households surveyed;  More than 80% of household heads live in union (married or common-law);  53.9% of household heads say they have not had the chance to receive formal education;  More than two-thirds of households estimate their monthly income at less than 5,000 gourdes;  Households are made up of an average of 7 people. Table 5. Sociodemographic and economic characteristics of households by department Socio-demographic and economic variables Total Department Center North-East Gender of household head Men 35.1% 40.1% 30.8% Women 64.9% 59.9% 69.2% Age 18-29 10.2% 8.5% 11.7% 30-39 18.4% 14.8% 21.5% 40-49 24.4% 27.8% 21.5% 50-59 25.1% 26.7% 23.7% 60 and up 21.8% 22.2% 21.5% Marital status Married 42.6% 36.9% 47.4% Live with 42.9% 33% 37.6% Single 2.6% 3.7% 3.2% Divorced 0.3% 0.5% 0.4% Separated 3.4% 4.6% 4.1% Widow 13.9% 10.8% 12.2% Education None 53.9% 59.7% 48.9% Literate 1.3% 2.0% 0.7% Primary 28.8% 22.2% 34.5% Secondary 15.1% 15.3% 14.9% Higher 0.9% 0.9% 1.0% Reported income class 5000 and less 82.4% 81.5% 83.1% 5001-15000 16.6% 17% 16.1% 15001-25000 0.7% 0.9% 0.5% 25001 and more 0.4% 0.6% 0.2% Average size 7.1 7.2 7.17 % Of households with children under 5 77.9% 71.8% 83.1% % Of households with children aged 0-6 months 11.6% 9.7% 13.2% Source: Final evaluation of the EFSP project, Centre and NE departments, WV/H, Sep 2021 19 | Page Objective 1. - Evaluate the achievements of the project in terms of its goal, objectives, results, and targets. The level of achievement of the expected project results was assessed by a specific objective or component. For each specific objective, the associated performance indicators were calculated using data from the Household Survey (HS) and data taken from the project performance indicator monitoring table appropriately. Analysis of data from qualitative interviews adds additional and contextual information to the results. This section of the report also makes it possible to assess the effectiveness/efficiency of the project as it provides the indicator data at different levels of the project results chain to the objectives set. Some results obtained for the higher-level indicators of the results framework also help in the analysis of the impact of the project. A comparison of the results obtained with those of the baseline was carried out to reflect the improvements or lack of improvements observed. GOAL 1: Food security for breastfeeding women and vulnerable households is improved Objective 1: Access to diversified, nutritious, and quality food for breastfeeding women and households is improved 1.1. Proportion of households having consumed at least 6 food groups The proportion of households that consumed at least 6 food groups increased significantly from 87.9% to 96.2% (p-value = 0.00 <0.05), according to the Household survey data. The mean dietary diversity score (DDS) indicated that household access to more diversified and better￾quality food underwent a significant improvement (p-value = 0.0008 <0.05) at the end of the project. This improvement in household dietary diversity was observed in the two departments. The proportion of households that consumed at least 6 food groups was evaluated according to the sex of the head of household and no significant difference (p-value = 0.4> 0.05) was obtained (see Table 14). Table 6. Comparison of Households having consumed at least 6 food groups per department between the baseline and the final assessment Indicator Total p￾value Centre p￾value Northeast p￾value Baseline Final Ev Baseline Final Ev Baseline Final Ev % Of households that consumed at least 6 food groups Average diversity score 87.9% 96.2 0.00 92.2% 98% 0.000 1 83.5% 94.6 % 0.00 Average 8 8.21 0.000 8 8 8.53 0.00 7 7.9 0.00 Standard deviation 1.6 1.25 N/A 1.6 0.98 N/A 1.62 1.39 N/A Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 (NA = Not applicable) 20 | Page Figure 1. Baseline comparison - Final assessment of the percentage of households having consumed at least 6 food groups per department 1.2. Food Consumption Score (FCS) The household survey data showed an improvement in the overall average household food consumption score and particularly for the percentage of households with an acceptable consumption score. The overall FCS increased from 38.76% to 41.6% over the lifetime of the project, and the percentage of households with an acceptable FCS increased from 57.6% to 64.5%. This improvement is especially observed at the level of the department of Northeast. The percentage of households with an acceptable FCS has slightly decreased for the department (Center). However, there is no evidence to confirm that this observed difference in the percentage of households with an acceptable FCS between the baseline and the final assessment for the Center is real, the significance test performed indicated. In general, the percentage of households with a low (poor) FCS has improved in the two departments (not significant for the Center). Table 6 illustrates the improvements observed at the level of the different categories of the value of the FCS. The Food Consumption Score (FCS) is a composite score based on dietary diversity, food frequency, and the relative nutritional importance of different food groups. It is an indirect indicator of diet. To calculate the FCS, the consumption frequencies are added and multiplied by the standardized weight of the food group. Households are then classified into three groups based on their weighted scores (poor, borderline, or acceptable) using World Food Program recommended cutoffs (or specific country-approved cutoffs). Table 7. Comparison of the FCS by the department between the baseline and the final evaluation (Final Ev) Indicator Total Center Northeast Baseline Final Ev p-value Baseline Final Ev p￾valu e Baseline Final Ev p￾value % Of households with an FCS between 0-21 14.3 8.2 % 0.00 6.5 5.7 0.61 22.3 10.3 0.00 % Of households with an FCS between 21.5-35 28.1 27.4 % 0.72 26.9 29 0.48 29.2 25.9 0.24 87.9% 92.2% 83.5% 96.2% 98.0% 94.6% Total Centre Nord-Est Percentage of households that consumed at least 6 food groups Ligne de base Ev. Finale 21 | Page % Of households with an FCS> 35 (acceptable) 57.6 64.5 % 0.0017 66.6 65.3 0.68 48.5 63.8 0.00 Average FCS 38.76 41.6 0.005 43.68 41.27 0.01 6 35.82 41.86 0.00 Median ND 40.0 NA ND 40.0 NA ND 40.0 NA Standard deviation 16.96 15.38 NA 17.19 14.15 NA 16.82 16.18 NA Confidence interval [37.86 ; 39.66 ] [40.5 ; 42.69] NA [42.39 ; 44.97] [39.7 ; 42.7] NA [34.55 ; 37.09] [40.2 ; 43.4] NA Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 (ND = Not available) Figure 2. Baseline comparison - Final assessment of the percentage of households with an acceptable FCS 1.3. Percentage of households where adults and children consume at least 2 meals per day Food insecure households generally consume fewer meals per day than those who are not. Access to a certain number of daily meals in the household helps to account for the level of food insecurity. The quantitative survey revealed that households in the project intervention areas improved their access to food in quantity. The percentage of households where adults and children consume at least two meals per day has undergone a marked increase overall, from 61.3% at the baseline to 70.5% at the end of the project. This overall improvement in household access to the quantity of food is mainly influenced by the considerable increase recorded in the NE department which posted a global progression of 42% (from 49.6% to 70.5%). While no overall improvement was obtained for the Center, this indicator showed an increase for all age categories considered except for adults aged 19 to 59 who represent the age category with the most people in the household. However, it is important to note that there is not enough evidence to confirm that the observed difference in the overall percentage recorded for the Center is significant (p-value = 0.42> 0.01 (and 0.05)). 57.6% 66.6% 48.5% 64.5% 65.3% 63.8% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% Total Centre Nord-Est Household with an acceptable FCS (in%) Ligne de base Ev. Finale 22 | Page Table 8. Comparison of the percentage of households where adults and children consume at least 2 meals per day per department between baseline and final assessment Indicator Total Center Northeast Baselin e Fin. Ev p-value Baseline Fin Ev p￾value Baseline Fin Ev p￾value % Of households where children and adults consume at least 2 meals per day 61.3 70.5 0.00 72.9 70.6 0.42 49.6 70.5 0.00 % Of households where children 6-23 months consume at least 2 meals per day 16.8 79.7 ND1 17.8 91.5 ND 15.8 73.5 ND % Of households where children aged 24-59 months consume at least 2 meals per day 30.4 76.5 ND 35.6 77.6 ND 25.2 75.7 ND % Of households where 5-18-year-olds consume at least 2 meals per day 57.4 74.5 ND 70.3 77.4 ND 44.4 72.2 ND % Of households where adults aged 19-59 consume at least 2 meals per day 60.2 70.5 ND 70.3 70.1 ND 50.0 70.8 ND % Of households where adults aged 60 and over￾consume at least 2 meals per day 24.5 72.1 ND 24.5 79.2 ND 24.5 66.8 ND Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 1.4. Domestic Hunger Index (DHI) The average score of the domestic hunger index dropped considerably at the end of the intervention considering all the departments (Center and Northeast). This drop recorded in the household hunger index is explained by a change in the structure of household distribution according to the degree of hunger reported by households. This change is observed in the two departments and results in a higher concentration of households in the category of those having a moderate level of hunger, and a lower percentage of the ones that have displayed severe hunger. However, the percentage of households that do not or barely suffer from hunger has not decreased regardless of the department considered. However, it is essential to note that this observed difference is not significant. Note that the DHI is a food deprivation scale that measures the percentage of households suffering from hunger. This indicator is measured by investigating the presence and frequency of the following household events over the past month: 1) No food at all in the house; 2) At least one member of the household went hungry; 3) A member of the household went all day and all night without eating. A standard scoring system was applied to arrive at an overall score varying between 0 and 6 which is classified into 3 categories: 1) 0-1 corresponds to absence or low level of hunger; 2) moderate hunger level = 2-3 and 3) severe hunger = 4-6. (See Annex II for more details on the indicator calculation method). 1 Not Determined 23 | Page Table 9. Comparison of DHI between baseline and final assessment Indicator Total p￾value Center p￾value Northeast p￾Baseline Fin Ev Baseline Fin Ev Baseline value Fin EV Prevalence of households with little or no hunger 14.36% 16.68% 0.151 17.8% 20.7% 0.25 10.9% 13.2% 0.256 Prevalence of households with moderate hunger 23.71% 64.65% 0.00 29.3% 74.1% 0.00 18.1% 56.5% 0.00 Prevalence of households with severe hunger 61.92% 18.67% 0.00 52.9% 5.1% 0.00 70.9% 30.3% 0.00 Average 4 2.79 0.0 4 2.38 0.0 5 3.13 0.00 Median ND 3.0 NA ND 3.0 NA ND 3.0 NA Standard deviation 2.44 1.51 NA 2.4 1.22 NA 2.43 1.64 NA Confidence interval [3.87;4. 13] [2.68 ; 2.89] NA [3.82;4.1 8] [2.25 ; 2.51] NA [4.82;5. 18] [2.98 ; 3.29] NA Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 Figure 3. Comparison of DHI between baseline and final assessment 1.5. Percentage of food use The percentage of households that claimed to have used their production for consumption fell by 10% across all departments, from 99.8% at baseline to 88.9% at the end of the project. Barter appears to have increased and the use of feed for livestock has declined slightly. While sales present a different situation depending on the department, it remains unchanged (not significant) overall. Indeed, more households (who practice agriculture) reported that they sold part of their production at the end of the project (46.3%) compared to the baseline (29.5%) for the NE. The opposite situation was observed for the Center, which went from 62.5% to 46.2%. Barter decreased in the NE and significantly increased in the Center (from 2.9% to 10.7%). The table below shows the significant or unobserved variations in the different types of food use between baseline and final assessment. 14.4% 17.8% 10.9% 16.7% 20.7% 13.2% Total Centre Nord-Est Households with little or no hunger (in%) Ligne de base Ev. Finale 23.7% 29.3% 18.1% 64.6% 74.1% 56.5% Total Centre Nord-Est Households with moderate hunger(in %) Ligne de base Ev.finale 61.9% 52.9% 70.9% 18.6% 5.1% 30.3% Total Centre Nord-Est Household with severe hunger (in%) Ligne de base Ev.finale 24 | Page Table 10. Percentage of food use between baseline and final assessment Indicator Total p￾value Center p￾value Northeast p￾Baseline Fin Ev Baseline Fin Ev Baseline Fin Ev value Consumption 99.8% 88.9 % 0.00 99.6% 77.6% 0.00 100 % 99.7% 0.14 sales 46% 46.1% 0.96 62.5% 46.2% 0.00 29.5% 46.3% 0.00 Barter 2.5% 5.5% 0.00 2.9% 10.7% 0.00 2.1% 0.6% 0.09 Feed the cattle 4.3% 2.4% 0.03 4.1% 2.7% 0.26 4.6% 2.2% 0.06 Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 1.6. Prevalence of Global Acute Malnutrition (GAM) The household survey anthropometric data for children under 5 in the project intervention communities showed a 0.7% (non-significant) decrease in the prevalence of global malnutrition, from 5.4% at baseline to 4.7% at the end of the project. The project was unable to meet its objective of reducing malnutrition by 3.9% in all of the two departments. (See table 10). The departments each posted a different result. Improvements in the prevalence of acute malnutrition were observed at the NE department level for all age and sex categories considered, except for children from 0 to 23 months where the observed difference is not significant. The Center has shown a deterioration in the acute malnutrition situation for almost all age and gender categories. This in part is due to the project targeting communes in the Center which are in very remote areas and the agricultural sector is facing challenges (drought) related to climate change, as reported by local representatives and beneficiaries. The mother leaders reported that remote communities, because of their lack of accessibility, have been weakly exposed to complementary activities, in particular, awareness raising activities that promote good nutritional practices for children, pregnant and breastfeeding women. In addition, Households’ survey data indicated for the Center department a higher percentage of head of household with a relatively low level of education (nearly 60 of heads of household with no level of education against 53% in the NE), which often constitutes a limiting factor of behavior changes. Table 11. Comparison of the prevalence of acute malnutrition between baseline and final assessment Indicator Total p￾value Center p￾value Northeast p￾value l.b Fin Ev l.b Fin Ev l.b Fin Ev % Of female children under five years of age (0-59 months) with a weight￾for-height Z score less than -2 standard deviation and / or with edema 5.2 4.4 0.54 2.5 8.9 0.002 10.1 1.0 0.00 % Of male children under five years of age (0-59 months) with a weight￾for-height Z score less than -2 standard deviation and / or with edema 5.6 5.1 0.62 4.3 9.2 0.03 8.3 1.5 0.00 Percentage of children with a weight￾for-height Z score below 2 standard deviation and / or with edema less than five years old (0-59 months) 5.4 4.7 0.5 3.3 9.1 0.00 9.3 1.3 0.00 25 | Page Percentage of children 0-23 months with a weight-for-height Z score below standard deviation -2 and / or who have edema 5.3 3.1 0.19 2.3 5.0 0.17 9.4 1.7 0.05 Percentage of children aged 24 to 59 months with a weight-for-height Z score below standard deviation -2 and / or who have edema 5.5 5.4 0.88 3.9 10.6 0.00 9.1 1.1 0.00 Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 Figure 4. Baseline comparison - Final assessment of the prevalence of global acute malnutrition in children by department 1.7. Value of other product/process indicators related to access to food (Effectiveness) M&E data showed that the EFSP-II project was successful in implementing activities and achieving the immediate (very short-term) results associated with them. Almost all of the targets for product or process indicators were 100% met. Some targets have even been exceeded (the training of mother leaders). The interventions carried out at the level of this component were particularly intended for women (breastfeeding women, pregnant women, female head of household) and their households. For children under 5, interventions related to the promotion and adoption of good nutritional practices have reached more girls than boys. Table 12. Level of achievement of output indicators Indicators Total planned target Total number reached % Women % Achieved # Of individuals who participated (received services) in the EFSP-II project 5085 5085 62.1 100% Number of vouchers planned and distributed 45765 45101 N/A 98.55% Number of vouchers redeemed 457657 45101 N/A 98.55% Number of beneficiary households having received food vouchers for Work 45765 40636 62.1 98.79% Number of beneficiaries of unconditional vouchers 4590 4586 N/A 99.9% 5.4% 3.3% 9.3% 4.7% 9.1% 1.3% 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0% 7.0% 8.0% 9.0% 10.0% Total Centre Nord-Est Global acute malnutrition in children 0-59 months(in %) Ligne de base Ev.finale 26 | Page Number of pregnant women who received specific nutrition services 1500 1101 N/A 73.40% Number of children under 5 who received specific nutrition services 8500 8843 57.7% 104.04% Number of Leader Mothers' clubs created 36 36 N/A 100% Number of mother leaders trained 690 542 100% 78.55% Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 1.8. Prevalence of exclusive breastfeeding in infants six months of age or younger Household survey data from the final assessment indicate a significant increase in breastfeeding mothers who practice exclusive breastfeeding in both regions. The prevalence of exclusive breastfeeding increased by 11.5%, from 65.2% at baseline to 72.7% at the end of the project. This increase in prevalence is much stronger in the Center Department (gained 18.9 points) than in the Northeast (gained 5.5 points). Figure 5. Comparison of the prevalence of exclusive breastfeeding in children 0-5 months between baseline and final assessment by department GOAL 2: Communities have improved their capacities to cope with economic shocks Objective 2.1: Access to sustainable livelihoods increased 1.9. Reduced Copying Strategy Index(rCSI) Despite the improvements observed in the mean food consumption score and the domestic hunger index in the targeted households, the households reported that they continue to use several coping strategies with a higher frequency to meet their food needs. The rCSI at the end of the draft is 26.5 versus 14.7 at baseline. No department showed improvement in the rCSI. Investigating the factors that would be at the basis of this result is the one that can be explained by the adoption of strategic behavior of households related to their perception of the evaluation exercise. Households have consistently expressed satisfaction with the intervention, especially the food stamps, and have insisted that they want the program to continue. They continue to express their vulnerability to attract more assistance. Thus, this result could be affected by 65.2% 72.7% 72.2% 91.1% 55.6% 61.1% Ligne de base Ev. fin Ligne de base Ev. fin Ligne de base Ev. fin Total Centre Nord-Est 27 | Page responses that reflect the desire to see the program continue in the intervention areas much more than the actual impact of the project. “The project is coming to an end; I can only regret it. If God could work a miracle for the project to continue, I would be so happy. With the school re-openings, I am lost” (FGD S4T, NE) “…. This is the end of the project, of course. But I wish it hadn't. The small pot of rice is sold for 50 gourdes ...” (FGD S4T, NE) "For what the organization did for us, we do not have the words to thank it, and I have a lot of satisfaction and if the aid ends today, we do not know what we are going to do" (FGD Beneficiaries, Center) The reduced coping strategy index is a behavior indicator linked to the household food situation. The highest theoretical value of the index is 56. A higher score for a household indicates that this one uses the strategies considered as drastic or negative with a higher frequency to try to meet its food needs than another household with a lower score. There is no conventional classification threshold for this index. Table 13. Comparison between the baseline and the final assessment of the RCSI by department Indicator Total p￾value Center p￾value Northeast p￾Baseline Fin Ev Baseline Fin Ev Baseline value Fin Ev Reduced index of coping strategies (mean score) 14.7 26.5 0.00 16.84 26.1 0.00 12.57 26.9 0.00 Standard deviation 12.98 9.05 N/A 12.84 10.34 N/A 13.13 7.75 N/A Median 12 25 N/A 12 25 N/A 12 26 N/A Confidence interval Z [25.89 ; 27.1] N/A [15.87 ; 17.81] [25 ; 27.1] N/A [11,58 ; 13,56] [26.1 ; 27.6] N/A Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 1.10. Value of other performance indicators related to strengthening livelihoods (Effectiveness) The performance indicator monitoring table provided the data below for the indicators associated with strengthening livelihoods until June 2021. Certain results expectations related to interventions in community infrastructure and technical supervision of young people aimed at support S4T groups have not yet been accomplished. The constraints relating to the execution of community infrastructures are linked to the ineffectiveness of the collaboration with the Ministry of Public Works, Transport and Communication (MTPTC) which was unable to respond in time to the various project requests relating to the execution of the action plan for the duration of the project. Political shocks were also constraining factors in the implementation of these activities, and in particular, those related to the training of young people who should themselves serve as technical mentors for S4T groups to support their sustainability. 28 | Page “We have to carry out construction work of some culverts in certain communities in conjunction with the MTPTC. We had several meetings with the leaders, but without success…. "(Interviewed, project staff) "We planned to do training sessions for young people during the last 4 and 5 months of the implementation of the project, but the assassination of the President as well as the associated events prevented the execution of certain plans" (Interviewed, project staff). Table 14. Other indicators related to resilience Indicators Target % Women % Achieved Percentage of beneficiaries reporting an improvement in income. ND ND 91% Percentage of S4T groups that are functioning normally 46 N/A 100 Number of infrastructures rehabilitated/created ND ND 0 Number of S4T groups established 46 N/A 100% Number of community workers trained to manage S4T groups 260 0 0 Number of young people to manage S4T groups 65 0 0 Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 1.11. Food security indicators and gender of the head of household Food security-related outcome indicators such as FCS, DHI, HDDS, and rCSI were assessed based on the gender of the household head to test for the existence of an association relationship. The various indicators (except the rCSI) have been transformed into dichotomous nominal variables (ex, if the FCS is acceptable, the household is classified as "yes", and "no" if not) to facilitate the application of the Chi-square test. The results of the test indicated that the indicators are not significantly related to the sex of the head of household (all p-values are> 0.01), i.e., that we cannot clearly state that the sex of the head of household influences the value of the calculated indicators. This statement is also true for the rCSI, where no significant difference was revealed by Student's T test. Table 15. Comparison of indicators by sex of the head of household Indicators Male-led household Household headed by a woman P-value Total % Of targeted households with an acceptable FCS 68.5% 64.8% 0.2 64.5% Prevalence of households with little or no hunger 19.5% 15.2% 0.1 16.7% Proportion of households having consumed at least 6 food groups (HDDS) 95.5% 96.6% 0.4 96.2% Reduced Copying Strategy Index(rCSI) 26.4 26.6 0.7 26.5 Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 29 | Page 1.12. Percentage of achievement of the Higher Results Indicators compared to expectations (Effectiveness and Impact) The values of the results indicators at the top level of the results framework such as FCS, DHI, HDDS, rCSI, GAM, calculated from the Households survey were compared to the expected values of the project to assess them. Household survey results indicated that almost all indicators reached a significant percentage of expected values. The FCS, DHI, and HDDS reached 64.5%, 64.6%, and 96.2% of their targets, respectively. The expected percentage for households with severe hunger exceeded its target (it fell to 18.6 from the expected 31%). However, the target set for the percentage of food use related to consumption remains largely unattainable despite the improvement seen from baseline. The level of achievement of the rCSI indicators, the DHI associated with little or no hunger. Table 16. Percentage of achievement of targets set for higher-level results indicators Indicators target achieved % Completed % Of targeted households with an acceptable food consumption score 77.6% 64.5% 83.1% Prevalence of households with little or no hunger(domestic hunger index) ND 64.65% Prevalence of households suffering from severe hunger 31% 18.67% 100%, Exceeded target Proportion of households that consumed at least 6 food groups. 100% 96.2% 96.2% Percentage of food utilization, disaggregated by type Total 18.15% 88.9% 489.8% Exceeded target % Of households where adults and children consume at least 2 meals per day 86.3% 70.5% 81.7% Prevalence of exclusive breastfeeding in infants six months of age or younger 75% 72.7% 96.9% Reduced survival strategy index 14.7 26.5 Has deteriorated Prevalence of Global Acute Malnutrition 3.9% 4.7 82.9% (improved) Source: Final evaluation of the EFSP project, Centre and NE departments, WV / H, Sep 2021 Satisfaction of project beneficiaries Finally, the level of satisfaction of all the beneficiaries of the project was assessed as an additional element in the analysis of the project’s effectiveness. Almost all (over 95%) of the recipients of project services expressed their satisfaction with the project, with 75.9% saying they were very satisfied. Dissatisfaction, although poorly reported (around 1.5%), is related to irregularities (significant delay, lack of discipline in waiting lines) in the food voucher distribution process. 30 | Page Figure 6. Overall satisfaction level of beneficiaries with the project Explanatory factors linked to the achievement of results Among the many factors influencing the achievements of the project cited in the qualitative interviews (EIIs and GDFs), several key themes emerged. Some of these themes represent decisions internal to the project and certain external factors beyond the control of the project. The most frequently cited theme revolved around questions relating to the interventions and the project implementation strategy. Thus, the influence of choices in project design and management was mentioned. Finally, external factors linked to macroeconomic conditions were cited as factors limiting the ability of the project to achieve certain objectives. The recurring factors at the interview level, both in the Center department and the NE are summarized as follows.  The participatory implementation approach emphasized the exploitation of local human and material resources. Here, the participants, particularly the implementing partner and the local governance structures (CEPAM, Town Hall, project staff), especially argued that the participation of the town hall, the use of CEPAM and the ASCP network, mothers' clubs have been an effective approach to improving certain food security indicators. Targeting is the result of an institutional collaboration involving MAST and town halls.  The provision of direct support, in this case, food vouchers, contributed, although considered partial, to improving the availability in quantity and quality of food at the target household level.  On the other hand, the overall economic context marked by rising inflation has limited the extent of progress made and is a limiting factor in sustainably reducing vulnerability.  The EFSP-II project was an emergency project that aimed to restore food security to targeted communities in the face of continued adversity from natural shocks. The implementation strategy did not focus on creating or strengthening the livelihoods of target populations sustainably. Almost all of the participants, particularly the CASECs, affirm that the real supports needed to go beyond the provision of food vouchers and the establishment of S4T structures. The project did not contribute to strengthening local production structures, especially in the field of agriculture. This feeling expressed suggests that the effects of the project cannot be sustainable as many stakeholders’ wish. 1.4 0.8 21.9 75.9 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 Pas satisfait Assez satisfait Satisfait Très satisfait Niveau de satisfaction Distribution in% of all households according to their level of satisfaction with the project 31 | Page Contribution of cross-cutting terms Monitoring, Evaluation and Coordination The EFSP project was supported by a M&E officer who received assistance from other technical implementation staff both at the level of the WV/H and its implementing partners, and also contractors recruited to carry out collection tasks for specific activities, such as PDMs. The project has sufficient human and financial resources to implement its robust M&E plan for collecting and measuring performance indicators regularly. Data quality assessments are performed by the M&E Officer to ensure the validity of the data. Despite the challenges encountered with certain partners (CEPAM for example) in the collection and reporting process, the information obtained from the interviews suggests the existence of good coordination between the WV/H and its implementing partners in the implementation of the M&E plan. The EFSP project uses the Survey CTO platform as a flexible performance indicator data management tool to report on progress in the implementation of activities. This platform also helped to ensure a certain quality in the collection, processing, and analysis of data. Alongside the Survey CTO platform, the project conducted post-distribution monitoring surveys to ensure beneficiary satisfaction and the quality of the process related to compliance with gender and safety principles and to take corrective measures required to the observed deviations. A response complaints mechanism has also been implemented as an integral part of the M&E plan. A hotline was available to facilitate community intermediation with the project where beneficiaries could voice their grievances and get a response. All of these information production mechanisms have sometimes been shared in coordination meetings and help facilitate the adoption of new guidelines to improve implementation. “… Meetings were held regularly to communicate the new directives…”. (Interviewed, Project staff) Even though the M&E unit was an essential support for the implementation and management of the intervention, the overall context of insecurity affected the respect for carrying out certain M&E activities which limited the ability of the management team to make decisions promptly. Respect for humanitarian standards The Assessment team also explored from the qualitative interviews an important aspect of the effectiveness of the project which focuses on the project's compliance with humanitarian standards related to respect for the dignity of the person, inclusion, and the participation of state structures at the central and local level, the protection and safety of beneficiaries, the inclusion of gender, to understand whether the intervention was appropriate, adapted, participatory and timely. The comments collected from beneficiaries, state authorities, and community leaders made it possible to conclude that the interventions (Vouchers, Awareness, and S4T) were carried out with respect for the beneficiaries and did not practice any gender￾related distinction. It has been claimed that the food voucher transfer system is appreciated by the beneficiaries in the vast majority. Besides the irregularities in the distribution (delay in the distribution), they wish to have more distribution points, the payment of the vouchers which are mentioned as factors of slight frustration, the beneficiaries did not express any concerns relating to their dignity and security. 32 | Page Although community leaders and state representatives at the town hall level wish to participate in the design of the project, they recognized that the project gave them a place of influence in the implementation and in particular in the targeting process. The coordination structure made it possible to maintain some communication with key players (CEPAM, Town Hall, etc.) in the implementation. Representatives of state structures and the implementing partner affirmed that the project was a necessity for the intervention communes. Regarding gender mainstreaming, although women are strongly represented across all components of the project due to the nature of the project and local socio-cultural realities, men have also received project services. However, the project will have to ensure better integration of young people in the project and address structural constraints (difficult access areas) in favor of other categories of people with reduced mobility to facilitate their effective participation. Objective 2. Evaluate the effects of the project on local markets and its repercussions on certain interest groups The analysis of the project's effects on the markets of the intervention communities considered the following four (4) dimensions: the influence on supply, the influence on demand for certain products, price variation, and relationships between the actors (beneficiaries/buyers, selected sellers, unselected sellers, large suppliers,) who interact in the market. The data on which this analysis is based comes from qualitative interviews with key informants and price monitoring data from the project. Interviews conducted with beneficiaries, vendors, mayors, and community leaders made it possible to suggest that food vouchers had an immediate impact on the dynamics of local markets insofar as they constitute an increase in effective demand of some food items for local vendors. This directly results in increased sales and profit figures for sellers. Overall, the project has not had any significant change in the structure of the supply (number of sellers, integration of new products), demand, and prices of food products in the municipalities of intervention. Influence on the offer The EFSP-II project did not modify the structure of the food product supply at the level of the local markets of the intervention communities, both in the number of sellers in the communal markets and their location, as well as in the products they offer. Indeed, the selection of local vendors was based on local realities by using almost the same distribution of vendors over the geographic area of the municipalities of intervention. The consumption basket offered by the food voucher provided by the project falls within the range of food products already available on the communal markets. The basket consisted of imported products as well as local products whose supply did not suffer from significant scarcity during the project implementation period. For some local products, the markets have recorded seasonal disruptions which are generally attributable to the dynamics of agricultural production in the targeted regions (period of drought or torrential rain, change of crops depending on the season, etc.) and are not associated with the increased demand for certain products resulting from the distribution of vouchers. For imported products, certain situations of product scarcity were rather linked to the Covid￾19 pandemic and the consequences of recurring socio-political disturbances which reduced the mobility of certain sellers or limited their access to certain supply markets. However, these irregularities were not of a nature and magnitude to disrupt the operating system of food vouchers, say the sellers who confirm that in the majority of cases they manage to ensure the availability of products before the coupon redemption period. 33 | Page Influence on demand The EFSP project stimulates the demand for the food products which constitute the consumption basket of households that the coupon enables them to acquire. This increase in demand for food was not, however, likely to cause pressure that vendors could not respond to. It was noted that the project made no distinction of origin of food production and that the consumer basket encouraged consisted of traditional food products that reflect household consumption habits. A recurring statement recorded with vendors is that related to the fact that a significant number of households tend to ask for other products, generally non-food, not appearing on the list of products promoted by the project. A state representative wants the project to place particular emphasis on promoting local products to stimulate agricultural production and improve the income of local growers. "I was always available to provide the beneficiaries with the products requested because I always looked for the products before the delivery date, we were always notified well in advance of the date of delivery of the food products to the beneficiaries" "The beneficiaries very often asked for products that are not on the product lists: for example, there are people who wanted to have sugar, soap that was not products available on the list" (Interviewee, Carice salesman) The prices of the products on the market of the intervention municipalities The stimulation of demand for food products linked to the transfer of food stamps to households did not increase the general price level of these products on the markets of the intervention municipalities. According to the mayors and community leaders, the inflation observed during the project implementation period is rather a reflection of the overall macroeconomic conditions of the country. Indeed, the economy's headline inflation rate grew year-on-year, from 22.34% ((144-117.7) * 100)) in April 2020 and from 15.97% in April 2021, according to figures provided by the IHSI. In the Northern region (which includes the NE) and the cross￾sectional region (which includes the Center), the annual changes in year-on-year inflation are 16.2% and 17.3% respectively. Data from the price control project confirmed the almost total increase in all the products that make up the household consumption basket, except for beans, which posted an overall drop in price over the period from September 2020 to July 2021 in the two departments (see Annex Price file). Figure 7. Evolution of inflation year-on-year April 2019-April 2021 117.7 144 167 0 20 40 60 80 100 120 140 160 180 Avril 2019 Avril 2020 Avril 2021 Evolution of the global CPI in annual slide 34 | Page Source: BRH (https://www.brh.ht/wp-content/uploads/ipc.pdf)There is an increase in prices, however circumstantial and partial, which results from the irregularities recorded in the operations of the food stamp operating system. Some sellers admitted to having increased the price of the products by a few gourdes to compensate for the shortfall generated by the delay in the payment of food stamps. This circumstantial increase in the prices of certain products slightly reduced the number of products in the beneficiary's regular consumption basket. It was not the project that influenced the market price, it is preferably the other way around because sometimes the market sells a bag of spaghetti for thirty (30) gourdes. Me, sometimes, I would go two months without receiving my paycheck from World Vision, I have no choice but to sell the same bag of spaghetti for (35) gourdes. After all, I am a businessman” (Interviewee, Carice salesman) Generally speaking, in the country, the price of a product can go from one level to another ... and sometimes it is not the people’s behavior, rather the political situation of the country that has caused this. (Local state representative, PC) Influence on the relationship between market players The analysis of the influence of the project on the links between market players, in this case, selected sellers, unselected sellers, wholesalers, and voucher recipients, is carried out by considering the different relationships in question. For sellers and wholesalers, retail sellers said they have more flexibility from wholesalers where they are used to sourcing. They obtained more significant credits from wholesalers with more flexible payment terms. The project does not appear to have created any conflict and tension between the network of project vendors and, between the project and local vendors who were unable to participate in the project. A practice that has been frequently cited is that which describes a certain consensus between the selected sellers where they distribute the customers of a given intervention zone in a more or less equitable way to meet not only the demand but also to guarantee more or less balanced mutual gains. Another example points out that other unselected vendors are used to supporting selected vendors on coupon redemption days to enable them to serve beneficiaries more quickly. “Us, three vendors, stand together. We have 146 people to serve. What we do, divide the 146 by 3 to make the distributions. On the contrary, sometimes we find other merchants in the community who are not in the project who come to help us " (Interviewee, Vendor Ferrier) In conclusion, the EFSP project was not at the origin of the creation of conflicts or did not exacerbate existing conflicts at the level of the communes of intervention. Conversely, the comments collected rather referred to the creation or strengthening of links between the members of the communities and this aspect seems to be the result of the project implementation approach focused on the mobilization of local resources and the establishment of community structures. An example of a link-building structure is the S4T which brings together several community members around mutual aid and economic strengthening structures. Although S4Ts is not new in most of the intervention communes, the project has helped to strengthen them technically and financially. 35 | Page Objective 3. Evaluate the effectiveness and relevance of the transfer modality and complementary interventions in achieving the results of the activities. The household survey showed a significant improvement in almost all of the project performance indicators both at the level of the achievement of products and at the level of the transformation of activities into results over the medium or more or less long term. The factors contributing to the observed results were analyzed in the light of the targeting process, the management of the voucher transfer process and the contribution of the complementary interventions carried out. This section of the report answers questions relating to the efficiency of the project as it describes how the various project activities were carried out to produce results achieved given the resources available and the duration of the project. Targeting of beneficiaries The use of an institutional base of MAST coupled with a community approach that includes the participation of local state representatives has been widely recognized as an effective means of selecting beneficiaries. The governance structures, in this case, the town halls, participated in the identification and adaptation of the potential beneficiaries list pre-identified by the project. This type of selection was understood and judged fairly by most of the respondents. The beneficiaries selected for the most part meet the targeting criteria defined by the EFSP project. The recurring exclusion factors mentioned are linked to a large extent to the fact that the project should include more households and very vulnerable communities, generally very remote in the intervention areas. Thus, the various respondents, in particular the mayors, are unanimous in recognizing that the intervention should cover more deserving households. It was noted that some extremely vulnerable households and communities in terms of food, due to their difficult geographic accessibility, could not be exposed to the intervention and therefore excluded from the project. Food vouchers The transfer of food vouchers was judged to be an effective means of acting directly on the food situation of households in the municipalities of intervention. The beneficiaries all converge on expressing their satisfaction with the choice of households, the monetary value of the coupon, and how the process of distributing and exchanging (or redeeming) the coupons are managed. Indeed, even though households have recognized that many deserving families have not been affected by the project, they are unanimous in believing that the coupon is awarded to very vulnerable people in the communities. The relevance of targeting is considered to be one of the factors associated with the improvements observed in certain indicators such as the FCS, the DHI, the HDDS, and the percentage of households where children and adults consume at least two meals per day. Although it was not applied systematically throughout the project implementation period, the adjustment of the coupon value to compensate for the inflationary effects resulting from the overall macroeconomic conditions was an adaptation of the execution strategy which contributed to the improvements observed in food security indicators. Thus, the price monitoring mechanisms and the monitoring of grievances through the structures of formal or non-formal meetings have constituted potential data sources for the making of certain 36 | Page management decisions related to the modality and operation of the coupon system that has gone from a traditional procedure to using technology to improve performance. At the start of the project, WV used sealed paper coupons with logos and different colors, bar codes, graphics to prevent their reproduction. Over time, paper coupons were replaced by a digital card (NFC) for beneficiaries and telephones (MPOS) for sellers with a link to the LMMS/EVS system. (Baseline, October 2020) Technical assistance in Nutrition WV, together with the (Center for Maternal Breastfeeding) (CEPAM), implemented a very participatory and pragmatic approach to raise awareness of the adoption of good nutritional and hygienic practices for breastfeeding women, infants, and young children. With the assistance of nutrition officers (AEN), the establishment of mothers' clubs was recognized as an effective learning structure for mother leaders and an important lever in sensitizing households to adopt practices. Despite the logistical challenges, such as the lack of appropriate equipment (balance, MUAC, etc.) and the absence of adequate transport costs, the mother leaders affirmed that the existence of this structure made it possible to reach vulnerable households in very remote areas who needed to be exposed to these kinds of nutrition and hygiene messages and knowledge. However, it is important to note that the adoption of exclusive breastfeeding is hampered by cultural resistance and the vulnerable economic conditions of some households. “… The project has helped people, especially in the nutritional context. When you do the "TI FWAYE" exercise, which consists of helping people to eat healthily with the means that CEPAM has set up. It helps them to avoid dealing with malnourished children.” (AEN, NE). S4T Structures The addition of S4T structures as a complementary component of the project, commonly called TIPA in the intervention communes, was considered a contributing factor to the success of the improvement observed in certain food security indicators such as the FCS, the hunger and diversity index. These structures constitute an important recourse for certain households who wanted to address circumstantial shocks (payment of schooling, illness, etc.) and a source of loan to support or undertake a small income-generating activity. It appears that food recovery interventions have a better chance of success when combined with TIPAs. They are cited in the majority of cases as a component of the project that is more likely to be sustainable and continue to have an impact on the lives of many households even after the end of the project. TIPA structures nevertheless presented risks of total dissolution in the sense that the size of certain TIPA groups was considerably reduced at the end of the project. "I have to tell you that the TIPA question is a real part of the answer to the problems that households face. This is an opportunity they have to save a little and meet other needs especially with the issue of the loan.” (Local state representative, NE) "... other advantages that I can add, sometimes, for the children to go to school we would have to sell a goat but now we can acquire a loan in TIPA to manage these things and pay them back afterward," it is very useful in that sense.” (FGD, S4T Cerca La Source). “We were reduced to only 6 people. One way of saying that some people are starting to lose interest.” (FGD, S4T, Center,). 37 | Page In conclusion, the project implementation approach was essentially based on the use and development of local resources. The various project interventions were carried out in a participatory manner, relying on local partnership and governance structures, whether it concerns the process of targeting beneficiaries, the distribution of food stamps, the execution of behavior change activities related to nutrition, establishment, and strengthening of S4T structures. By way of illustration, the project chose to use CEPAM, in this case, the MSPP AENs to supervise ASCPs and lead mothers to promote the adoption of good nutritional practices for breastfeeding women and children less than 5 years old. This approach made it possible to reach a large proportion of targeted households while optimizing the use of the limited human and material resources available for the implementation of the project. Another example is the establishment and strengthening of S4T groups which are generally autonomous structures that use very little financial resources from the project. Finally, the food voucher transfer system (conditional and unconditional), grafted onto the existence of local resources (already existing local vendors, local distribution stations, etc.) made it possible to reach all of the targeted households over time of the project by minimizing logistical costs (transport, training, etc.) and having an immediate direct impact on their food situation. Objective 4. Identify best practices, lessons learned, strengths, and challenges in the design of activities, among others the logical framework, and in the implementation to obtain the outputs of the project. The identification of best practices, lessons learned, strengths and challenges in the activities, the logical framework, and the implementation is supported by the various qualitative interviews and the literature review. The essential elements that emerged were classified appropriately by component or cross-cutting term of the project. Partnership • Participatory verification and targeting processes were important for household ownership and identification processes. Thus, the selection of beneficiaries as the result of the use of the SIMAST database and the participation of town halls officials not only facilitated fair targeting but also favored the ownership and collaboration of the project with local governance structures. However, it is important to ensure a balance in the leadership of the targeting process to limit the risks of inappropriate inclusion. • Mayors and community leaders have expressed their concern to be able to participate in the development of these types of food security interventions that they consider to be a priority for their community. • The use and enhancement of local resources, in this case, CEPAM and the ASCP network, in the implementation of activities facilitated the implementation of nutrition activities. 38 | Page Food vouchers • It is good practice to use local vendors as food suppliers. This approach has direct positive effects on local markets which translate into increased income for merchants and farmers. • Faced with the extent of the economic vulnerability of households in the intervention communes, food vouchers, although they constitute a significant contribution in reducing household food insecurity, have not been able to contribute to building resilience and the autonomy of households to meet their own food needs. • Monitoring the prices of local products to support decisions to adjust the value of the coupon was a good practice of the project. A more regular adjustment of the value of the coupon according to the variation of inflation at the local level could contribute to more significant improvements at the level of food security indicators such as FCS, HDDS, and rCSI, DHI. This adjustment is even of more importance as many households said they shared their food with other households in dire need. Nutrition and behavior change • The use of effective techniques, such as Ti FWAYE, in the process of sensitizing and disseminating nutrition messages for breastfeeding women, women heads of households, as well as learning-by-doing methodologies, has produced definite and lasting results in the change of behavior of some breastfeeding women and the reduction of global acute malnutrition. • The challenges which limited the scope of the results are rather related to the insufficient funds of the project to ensure wider coverage of the sensitization interventions. According to CEPAM and the Mothers Leaders, the lack of appropriate resources has reduced the reach and effectiveness of outreach activities. • The extreme level of household vulnerability in some communities poses a challenge to the adoption of exclusive breastfeeding. Logical context • It was recognized by all stakeholders that the integration of S4T structures and nutrition training and awareness activities was an essential support to the food stamp transfer component. • According to some mayors, the EFSP project, as designed, would not be a sustainable response to the problem of food insecurity in the targeted areas. A more holistic approach that devotes much more investment to the agricultural sector would have helped to produce more sustainable results. • The lack of significant support in strengthening the empowerment of women through the promotion and creation of small, structured businesses was cited as a limiting factor in improving the resilience of households in the face of recurrent shocks and food insecurity. • The project appears not to have been able to implement its approach to integrating young people as planned. There was no active and systematic involvement of young people in project interventions, whether, at the level of targeting, the food stamp distribution process or the establishment of technical supervision of S4T structures. 39 | Page This clear approach to integrating young people could have a positive effect on the sustainability of the project, particularly concerning the sustainability of TIPA and the achievements in terms of behavior change linked to nutrition. Objective 5. Formulate (strategic) recommendations for other projects or new interventions. Conclusion and recommendations Conclusion The results of the final evaluation showed that the project has made significant improvement in almost all of its food security indicators. The project has reached its expected outcome indicators targets at nearly 100%, except for the rCSI which has declined. Although most stakeholders and project partners believe that households have improved their food insecurity to some extent, they are unanimous in recognizing that the magnitude of the food insecurity problem in these intervention communes which are constantly exposed to recurrent natural shocks, requires more substantial investments and a holistic approach that acts on institutional strengthening and the development of local agricultural production capacities. The time required for sustainable improvement in food insecurity was cited as a challenge to ensure a more significant and lasting improvement in the project outcomes. Qualitative interviews conducted with beneficiaries’ households, sellers and local government officials indicated that the project, despite its effects on stimulating local food demand, had no inflationist effect on local markets. The inflation observed on local markets during the life of the project, as reported by the project internal monitoring process, is associated by the macroeconomic conditions of the country. In addition, the project was not the cause of any social conflict or did not exacerbate any existing conflict between market actors or community groups. Conversely, the project contributed to improve stock management practices and collaboration between local market vendors. Finally, the project implementation strategy used a participatory and community-based approach which allowed it to use and build on existing community resources. Although the project relevance, effectiveness and efficiency have been revealed by both the HS data and qualitative interviews, its sustainability has been recognized as a serious challenge in the face of the severity of the food insecurity issue in the target communities. Recommendations Based on the evaluation findings, the suggested recommendations to improve the impact of other projects and future interventions are formulated as follow: • The end of the transfer of food vouchers (conditional and unconditional) was cited by the project beneficiary households as a challenge to continue to access the types of diverse and nutritious foods allowed by the project, even if they acknowledged having acquired some knowledge of good nutritional practices. Thus, these target communes continue to need additional food security interventions which will use more transformative and more sustainable approaches which involve local productive capacities development, in particular, the agricultural production and small and medium business creation. Local state authorities, as well as beneficiary households, believe that 40 | Page natural shocks and stagnation in agricultural production are likely to continue and that these types of interventions should place more emphasis on the development of the agricultural sector and the creation of other income generating activities such as the promotion of small businesses to help the communities of intervention to face food insecurity in a sustainable manner. • The problem of global acute malnutrition remains a concern, especially in the center department, particularly in remote areas, and will require more prevention, awareness, treatment, and nutrition interventions. • Local behavior change structures will require better supervision in terms of equipment and tools necessary to carry out behavior change activities related to nutrition and exclusive breastfeeding. • Local state authorities believe that the project should put more focus on the agricultural sector to enable it to face the challenges of climate change and adopt a supportive approach that encourages people to build their resilience and reduce their reliance on humanitarian assistance. 41 | Page ANNEXE ANNEXE I. Terms of Reference for the Final Evaluation WVHaiti_EFSP NECP2_Final Evaluat ANNEXE II. Performance Indicators Reference Sheet (PIRS) ANNEXE II - Document de referenc ANNEXE III. Household survey questionnaire / Children ANNEXE III - Questionnaire d’enqu ANNEXE IV. Guide to Qualitative Interviews with Key Informants and Guide to Focused Group Discussions ANNEXE IV - Guide_Entretien&Focu Guide_Entretien_S&E. docx ANNEXE V. Form of consent - qualitative survey ANNEXE V - Forme de consentement - en